From an early age, we’re taught/trained to pass tests. It’s an unfortunate word educators use for knowledge assessments because it can warp our attitudes toward tests in the scientific or engineering worlds. Experiments and hardware tests can have different definitions of failure that don’t (necessarily) reflect on the person(s) performing it.

I never lose. I either win or learn.”
― Nelson Mandela

“I have not failed. I’ve just found 10,000 ways that won’t work.”
― Thomas Edison

Most of my experience with tests (outside of academia) has been with engineering. In that world, the above quotations from Nelson Mandela and Thomas Edison more accurately reflect how engineers view test “failures.”

Some tests, for example, deliberately test a structure or material until it fails, such as by stretching a graphene fiber or an airplane wing until they break. Thomas Edison claimed to have tested at least 6,000 materials for the incandescent light bulb filament before he got the light intensity and duration that he wanted. This recurring testing is a regular and central part of engineering, especially when the material or product is being designed for high-stress or safety-critical operations.

A rocket engine manager I worked with at NASA Marshall Space Flight Center once told an audience that he planned to test several engines to the failure point. Actually he said “I plan to blow up a couple engines in testing,” which did not go over well with the head office because the public would perceive engines “blowing up” as a failure, but that’s a public affairs perception, not an engineering perspective. However, agency or public caution aside, rocket engines or entire rockets can and do explode in the process of performance testing. This can happen for multiple reasons, from a design or manufacturing defect to a test mechanism failure to an “unknown unknown” failure in the hardware that comes to light in the process of performing normal or beyond-normal operations.

That brings me to the event inspiring today’s post. Yesterday, SpaceX performed another flight test of the upper stage of its Starship rocket.

Video credit: SpaceX

During the six-and-a-half-minute flight, the rocket ascended to 10 kilometers (6.2 miles), purposely turning off two of its three engines to allow it to fall back to Earth. It flipped over to a horizontal position so that the vehicle’s fins and 50-meter (160-foot) fuselage could use air resistance to slow the vehicle from what would be space-rated speeds; reactivated the two dormant engines to rotate back to vertical; and landed vertically some distance away, albeit at an angle, and with some fire occurring around the engines. Approximately eight minutes after touchdown, the rocket exploded.

Video credit: Space.com

The question then becomes: “Was this test a failure?”

Someone looking to find faults could point to a few obvious issues:

• While a “soft” landing in that it touched down vertically, the rocket stood at an angle, either because some of the landing legs did not deploy or the vehicle came down hard and broke a couple.
• There was a fire at the base of the rocket by the engines, which likely led to the explosion.

However, if you ask SpaceX and a lot of rocket people, the answer is in fact no for several reasons:

• This was the third time the rocket had proven its “belly flop” approach to slowing down within the atmosphere, providing additional data that that process works.
• After three attempts, this was the first Starship prototype to soft-land vertically. The two previous rockets (SN8 and SN9) crashed and exploded when they reached the ground.

A complex engineering test like this has multiple objectives, including safe liftoff and ascent to in-flight shutdown, controlled rotation to horizontal, safe engine restart in flight, and a safe, vertical soft landing. Looked at from that list, the only “failure” might be the safe soft landing. However, everything else worked and the explosion occurred several minutes after its relatively soft landing. SpaceX has learned how to master its belly-flop deceleration and bring the rocket back to vertical after a couple of hard landings. They captured data throughout the process to show them what was working and what wasn’t. And they know what they need to work on for the next test vehicle, SN11.

Differences in the Scientific World

When scientists perform experiments, they sometimes have a lot more factors to check if they do not get the results they expected. Scientists, after all, are trying to understand something unknown about the natural world. They will go through their data to see if it makes sense. If it does not, they’ll start looking for anomalies and outcomes that might be explained by improper equipment setup or handling. Were the test tubes clean? Was the oven or centrifuge operating within the correct range? Were their optics (microscope, telescope) clean to the appropriate levels? Were the instruments calibrated properly?

Whether or not any or all of these items turns out to show a defect in performance, the scientist(s) might have to repeat the experiment multiple times, each time reviewing the data to see if they get the same result. If the equipment was handled within the desired precision and tolerances, then the investigators have to go back and review all of their data sets and come with a theory that explains the unusual results. Science fiction author Isaac Asimov is quoted as saying, “The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’, but ‘That’s funny…'”

Why Does This Matter to Technical Writers?

You might not be working in aerospace or microbiology, but if you are working with scientists or engineers, you will likely have to write about experimental or test results. If you are on the reporting or public affairs side of the operation and your test/experiment is in under public or management scrutiny, you’ll likely have to help the principal investigator(s) explain the results, including whether a particular unexpected result amounts to a “failure.”

That will be when you need to remember and paraphrase Nelson Mandela: “We never fail. We either win or learn.” That is a helpful bit of advice in the experimental and test world. However, if you get an F on a college exam, that’s still on you, even if you do learn from the experience.

Copyright secured by Digiprove © 2021 Bart Leahy