A few days ago, the following tweet crossed my Facebook feed, and I felt it worth writing about it in more detail here:
Part of the reason the pandemic is so bad in US is because people who are never involved with scientific method are seeing Science unfold in real-time. Errors, evidence, & changing methodology happen 100%, but in public eye, seen as "they have no idea what they're talking about."
— Alex Martin (@SidewalkSciGuy) July 16, 2020
Laws are like sausages, it is better not to see them being made.
–Otto von Bismarck
Most science writing I’ve seen relates to finished products, which is to say, completed experiments with actual results. That isn’t to say that written communications are all about “settled” science; it’s more that scientists are often reluctant to communicate about work in progress. In some cases this makes sense. There are several places in the scientific method when the outcome can go off the rails. First, here’s a graphical representation of the method:
Source: ScienceBuddies.org
- Ask a question/Do background research: If you’re new to your field, you might have a question that has already been asked and answered. You might be told that the question you’re trying to answer is stupid (pretty certain that happened to me in high school). Or you might be asking a question there is no way to prove in a laboratory setting.
- Construct a hypothesis: It is entirely possible that the theory you suggest for explaining why X happens is flawed. You might have overlooked something. Your original observation(s) might be flawed. Also, steps farther down in the process could cause your hypothesis to change.
- Test with an experiment: Experiments have a list of requirements to be considered valid for providing results that back up your hypothesis. There has to be a way to test (“control”) for only one, specific aspect or variable of the natural process you’re trying to uncover. Your equipment has to be clean, in good repair, well calibrated, and standardized (i.e., using common tools, measurements, materials, and measurement methods available in the scientific community) so that it can be replicated by other scientists seeking to confirm or disprove your hypothesis.
- Procedure working?: In the process of your scientific “sausage making,” you might discover that your equipment is not working, requiring you to go back and make certain that everything is functioning as it should.
- Analyze Data and Draw Conclusions: In the biology world, data could take the form of hundreds of “runs” (repetitions of a physical, chemical, or biological process) to create a statistically significant sample. Your computer could come back with nonsense results. Your analysis method might be flawed. Or the process you were testing results in no particular pattern. Or the results are purely inconclusive. Or the results directly contradict your original hypothesis. In the end, assuming you’ve done everything right to the best of your ability, you still have to explain what happened.
Science consists of thousands of individuals making observations, asking questions, conducting experiments, and analyzing results. That process is messy, like most human activities, made even more so by minor differences in procedures, sampling sizes, analysis methods, as well as assumptions, accidents, and errors.
I’ve observed a somewhat similar process in the aerospace engineering world. The public is used to seeing finished products that work. However, NASA faced similar problems early in the Space Age (as depicted in The Right Stuff). In this century, SpaceX accumulated multiple failures before it managed to land and reuse the first stage of their Falcon rockets and is now experiencing different failures as it tests their Starship vehicle, which is designed to be a reusable interplanetary rocket–something that’s never been done before. It can be disturbing to people dreaming of flying to the Moon or Mars to watch test article after test article explode on the stand, but again, this is part of the learning process.
The Process of Science in the Age of COVID-19
Normally, that ongoing cycle of hypothesis-experiment-analysis is the typical process human beings have used for the last 200 years to determine systematicallyhow our universe works. The general public doesn’t see the messiness of the sausage making; all they see are the completed papers published in Nature, Science, Scientific American, or some prestigious, discipline-specific journal. The general public also doesn’t get to see the process of “peer review,” which is where scientific articles are farmed out to other eminent experts in the field for critical analysis or general-purpose nitpicking.
The bottom line is that any scientific hypothesis has endured a lot of behind-the-scenes messiness, which most people never see, before the scientist reaches the “publish or perish” stage. If it’s published, the scientific (and maybe the broader) world sees only the finished product, and it appears in a neatly formatted, more or less well-packaged report. Non-scientists might be under the impression that the practice of science is that neat and organized, but it’s not.
And here we come to the problem with COVID-19.
Scientists are continuing to practice their discipline as they have before, albeit now under a great deal more pressure and scrutiny because people are dying and the virology and medical communities are still trying to figure out how the novel coronavirus works, let alone how to stop it. Meanwhile, policy makers who are informed by the peer-reviewed scientific literature are trying to make public health decisions in real time as researchers are still learning themselves.
Recommendations
During my time at NASA, I really wished the agency had been more open about sharing the little behind-the-scenes problems, challenges, and dramas that occur during a rocket launch. Just listening to some of the mission control channels during a flight test was illuminating because I was listening to people solve problems in real time.
And really, in engineering just as in science, the most interesting parts of the disciplines are when people are finding unexpected issues, figuring out whey they happen, and solving the problems. That is what makes them fields interesting enough to make careers. Most of the time, it’s not the nomenclature or the math that moves people.
The fields of science and engineering are not just about the finished products or proven results, though that is mostly what we see. In my mind, that’s a communications failure, and it gives a false impression of omniscience that ought to be avoided. We need to see more communications during and about the process before the final result. It will require reorienting how we communicate about and relate to science. I don’t see either of those as a bad thing. It might teach all of us–from technicians to engineers to scientists to technical writers to policy makers–a little more humility.