Proper controls in ecological experiments

It's an essential feature of experimental design, and often not as straightforward as you think...

Experimental controls (image created in leonardo.ai)

Following up on my rather popular recent rant on pseudoreplication, I here share some thoughts about proper controls in ecological experiments. This is an equally important topic (and incidentally this was also completely botched in the infuriatingly terrible preprint we recently read in lab meeting).

A common misconception about controls is to do nothing, to leave the experimental unit alone, to not add anything for example. Nothing could be further from the truth. Sure, sometimes this is adequate; but very commonly it isn’t. Those of you who do experiments will know.

Take our global change field experiment in Berlin as an example. We had installed rain-out shelter roofs to have a drought treatment, and so needed to install soil dividers around the plots to prevent water from flowing laterally back into the plots from the surrounding area. In this case it would have been a fatal flaw to not also install these soil dividers in the control plots, since installing them had induced a disturbance. A small point you think? Oh no, it was very clear how at least during the first weeks of the experiment vegetation was much more lush around the edges, stemming from the soil disturbance and the ensuing nutrient mineralization. Of course, this increased the effort in this experiment tremendously, it was awful to install these soil dividers in all 154 plots, and we broke every shovel and spade of any institute in the general vicinity doing so.

This illustrates the important point of controls: to account for all effects of the treatment that were not intended. That’s sometimes straightforward and sometimes rather complicated.

Here’s a complicated example for you from the world of mycorrhiza. So we add mycorrhizal inoculum to our treated pots and none to the control? Far from it, because the inoculum contains carrier material (often soil or some other substrate), the microbes therein, and also nutrients. So in order to have a real control for the mycorrhizal inoculation, we need to add autoclaved inoculum to the control (this accounts for the nutrients and whatever the carrier material is, also including roots in most cases), and then we need to add a microbial wash (a microbial suspension) created from the live inoculum, in order to add also the microbial community contained in the inoculum, other than the AM fungi themselves. In other words, the control is easily two to three times the amount of work and effort than the treatment, which is comparatively simple.

This is not atypical, often the control is tricky, requires a lot of thought and is more work than the actual treatment. And designing the control forces you to think through what you are actually testing. Of course it would also be fine to add inoculum to the treated pots, and nothing to the control pots in the example discussed above. But then you are asking a question about ‘adding this inoculum’ (the entire thing with the AM fungi, the microbes and the carrier material), and definitely not a question about adding the mycorrhizal fungi. There are many examples like this. The key is: make sure the controls match your actual scientific question.

Especially when working with novel factors, devising controls can be rather challenging. For example, when you add microplastic fibers to a soil, you are clearly adding microplastic, but also a fiber-shaped object. Would adding any kind of fiber shaped object have similar effects? What’s the solution: adding different fiber-shaped objects of similar size that are not plastic. Not that easy, we tried this several times and pretty much failed to find good controls (so has everybody else as far as I’m aware). Our solution: add different fibers made from different polymers. And also use the same polymer in different shapes. In the end, perhaps this is not so critical, because your question can very well be: what does this material do (including its shape) when we add it to soils. But as a field matures, and questions become more sophisticated, the experimental design also needs to mature and account for different properties of your treatment.

Another example: when we mix in the plastic materials into the soil in the treatment, we inevitably create a disturbance in these soils, by the act of mixing. So it was very important to realize, right at the beginning of doing this work, that we also needed to mix the controls for exactly the same amount of time, and with the same amount of energy, even though we’re not even mixing anything in.

Thinking about this is actually fun. Designing experiments really never gets old, there is always something new to think about. And very often the novelty is also in the control.

Clearly, you can also get this terribly, terribly wrong. And then your study either means nothing (worst case) or it is severely limited in its meaning, because maybe you actually did an experiment on a different question than the one you had in mind.

Do you have any interesting cases to share with respect to coming up with good controls? The ones above are what I use in my courses. If you do, please share in the comments, and — as always — looking forward to hear about what you think in general about this as well.