Failure in research: if you do experimental work you're going to experience failure
Experiments failing really sucks, but we rarely talk about it
If you do experimental work, sooner or later your experiment will fail. If you do experiments, you will know. But we rarely talk about it, because we like to share our success stories.
Failure really sucks, it can be very discouraging and it is a waste of resources and time, and usually means also quite a setback. But this is only really true for one kind of failure: let’s call it the bad failures.
This means there could also be good failures. What are those?
Good failures. Those happen on the path towards trying something new, something creative, something untested, something risky. As you explore something very new, or something technically challenging, or something that in some way departs from the ordinary, the probability of failure is higher than when doing more straightforward work. While these kinds of failures still sting — and they can be painful when they involve a lot of work, time and effort — these failures are happening while you explore something new, and thus from those you will inevitably learn something that will help you move towards the goal. Sometimes a failed experiment, after scrutinizing what might have happened, can even give rise to new ideas. This has definitely happened to me, but not very often — but this is possible. So it is always worth looking at an experiment from different angles before giving up on it.
In a nutshell, good failures offer opportunities to learn. Now on the other hand, there are also bad failures.
Bad failures. These are failures that do not come with a good opportunity to learn. Something has been overlooked, for example, and a design flaw that has been realized after the fact makes results invalid. These kind of failures are often avoidable, for example by discussing experiments before they are being done in a protected space, such as during a lab meeting. In our lab, we have weekly meeting to provide a forum just for such exchanges, to minimize the probability of an experiment being set up that suffers from design flaws (such as missing essential controls, or the experiment is not addressing the actual goal or question). If an experiment with a design flaw is executed, and then the flaw is realized after the fact, the opportunity to learn came too late, and now time and resources have been wasted. Another brand of failure is equipment failure of some kind (for example contamination, environmental controls of a growth chamber malfunctioning, or a critical piece of equipment not working). These do happen, and they’re nobody’s fault, but they offer little opportunity to learn while ruining your experiment.
We can try to minimize bad failures, but they will still sometimes happen. How about good failures? We should encourage them, but of course we need to make sure they don’t take over. When trying new things out — whenever possible — we should make sure that we minimize the risk of catastrophic failure, like a very large, time-consuming experiment failing. Instead, it is better to have smaller, exploratory experiments fail.
What actually is failure? Design flaws and equipment failure aside, the most common failure is perhaps that we see no effects of treatments or that a treatment itself did not work. This is not the same. An example of the latter, it can be that a microbial inoculum was ineffective or that a warming treatment did not actually work. This means that the intended perturbation did not actually take place. The former (no effect of treatment ) means that no effect was apparent, even though there was no hint that the treatment per se (like microbial inoculation) was not operative. This could happen for any number of reasons, most commonly there was insufficient statistical power, because of too few replicates, too much variability or low effect sizes. There is an infinite number of ways this can occur. Such ‘negative’ effects, i.e. there is no apparent effect, are difficult to interpret, because it is never clear if they were due to insufficient power or due to the actual absence of effects. Other examples of failure are that the results cannot be repeated, meaning that every time the experiment is done the results differ, and it is not clear why (likely there is some unknown factor that is not controlled).
What can be done? Failures will always happen, and they are painful. And we typically don’t talk about them. People want to hear about successes, and they want to share their victories, not their failures. When failure happens, it’s good to get some form of closure, maybe to burn the results among friends, and to allow yourself to mourn about the loss. Take a break, eat some ice cream. For the ‘good’ failures, it is a great idea to share them; this requires a bit of a shift in how we view failure, namely as steps towards success.
I also made two videos about failure, perhaps you find them interesting:
The first one talks about good and bad failures.
And the second one talks about how we can encourage the good, creative failures.
What’s your take on failures? how do you deal with them? do you share failures in the lab? Let me know your thoughts in the comments, please! Thanks.