Discover more from The Ecological Mind: ecology, research, creativity
Pseudoreplication in ecological research, and how to avoid it
Let's eradicate pseudoreplication - a rant! :)
Recently we read a preprint of a paper on a topic I am very interested in, and went through quite a range of emotions from eager anticipation and nervousness (oh my this is totally what we’re trying to do) to relief (this is a totally worthless study, it tells us nothing), to frustration (I cannot unsee it now that I’ve read it) to sadness (this makes everyone worse off, the authors, us who have read it, and the public, because of course this has been picked up by the media).
One should not write posts in anger, or maybe it’s cathartic to do so. Anyway, I confess to being incensed about papers, such as this one, with blatant pseudo-replication. Why? Well, because it is a serious experimental design error that makes a study completely worthless if as in this case, the study is actually non-replicated; but also because we often — with enormous effort — try to avoid such mistakes. And it seems some people just take the easier route because they don’t know any better. It’s always substantially easier to pseudoreplicate than to do something right. Not that we’re perfect, but I think we’ve been primed to look for such mistakes in our own work.
So, here just a little experimental design refresher. An experimental unit is the entity (pot, microcosm, other thing) to which a treatment is applied. The experimental unit is the (true) replicate. This is the simple rule. Admittedly, sometimes it is not as trivial to see what the experimental unit really is. So let’s say you’re doing an experiment on warming and you’re using incubators. In that case, and in all other atmospheric treatments, the experimental unit is the entity to which this treatment is applied, and in this case it is the incubator, which you set to a certain temperature. If you have 20 Petri dishes inside that incubator, your replication is still one, and not 20. Why is this? Because you are applying the treatment (a certain temperature) at the level of that incubator. If that temperature control is faulty (or anything else happens to this incubator, like maybe it has slightly higher relative humidity, or it runs slightly irregularly, or anything it all), it affects all things inside of it (meaning they are not independent). And thus, you’re confusing this potentially random noise that affected the incubator with your treatment. So if you pretend that you had a sample size of 20 (the number of Petri dishes in your one incubator), you would be committing this error of pseudoreplication. What you really need is 20 incubators, or any other number of true replicates, or another means of delivering that heating treatment. This is why we have over 20 incubators; and why, in other experiments where we need higher levels of replication, we apply the heating treatment individually to experimental units using about 100 separate temperature controllers. This is of course vastly greater effort and cost!), but it is also the correct way of doing it.
As I mention above, this is a problem that arises for all atmospheric treatments, and there are many other cases of this. During my PhD I saw an experiment (that I wanted to collaborate with) that applied elevated carbon dioxide. But it had one greenhouse with elevated, and one greenhouse with ambient levels. One greenhouse, I still remember even though this is some 30 years ago, had a huge tree shading it. It was so obviously terribly wrong. But inside they had hundreds of pots with plants….The list of pseudoreplication examples I saw during my career is very, very long. It could fill pages.
This error comes in different levels of severity, even though it always remains an error. For example, in this preprint we just read, they really only had one experimental unit per treatment (they even wrote it exactly like this). Which means this cannot be saved, there are no replicates. If they had used several independent “items” (like chambers) in their experiment, and had committed pseudoreplication by treating things inside them as independent, then we could still re-calculate the real values to be used in statistics. So it can be “repaired” after the fact. In this particular case, no.
This is why I teach this even in basic ecology to all undergrads at my university. And still, papers or preprints with this fundamental error come out all the time; like the one we just read. It’s infuriating.
So if you do experiments yourself, or read papers, and then write about them, please know about pseudoreplication. Thanks and have a fantastic day!
(end rant)
(sorry)
It can be real bad. There are some great publications about this problem.
I think it's not as clear cut as you say.
Common garden experiments for example. If you randomly sourced 100 plants per latitudinal zone across a range of latitudes to see if their were phenotype response differences in a high and low carbon environments in your glass houses, or across latitudes. The variability between individuals within each source region is of interest and within each treatment.