← All stories

What a promising result really tells you

Five conversations about the checks between an interesting answer and a useful conclusion.

When Leslie Pack Kaelbling was a graduate student, she wanted a small robot to drive without crashing. She gave it a goal that rewarded avoiding collisions. After moving around, the robot stopped.

Problem solved—at least according to that rule. The robot was not crashing. It was not getting anywhere either.

Kaelbling added a requirement to keep moving. The robot found another way to satisfy the instructions without doing what she intended: it spun in place. In her Voices in STEM conversation, the MIT researcher used the episode to explain something important about robotics. A machine can produce a valid result for the problem it was given while missing the purpose the human had in mind.

That is part of what makes science more interesting—and more demanding—than simply getting an answer. You have to decide whether the answer is useful, what it establishes and which part of the problem you might have described incorrectly.

More feedback. But more learning?

Victoria Dean, a computer science lecturer at Princeton, described a student-led project using a language model to help teaching assistants prepare written feedback. The assistants assigned the numerical scores themselves; the software suggested comments explaining the marking.

The project produced more feedback for students. But that left another question: did students perform better afterward?

Dean explained that the sample was too small to draw a firm conclusion about improved learning. The result said something about the feedback process. It did not yet establish that students understood the material better.

For anyone using an AI study tool, the distinction is practical. More explanations, more notes and more completed questions can feel productive. But what would show that you can now solve a similar problem without the help?

Was the star really new?

While observing galaxies in Chile in 1997, astronomer Duilia de Mello noticed an unexpected object on her screen. She first checked whether she was looking at the wrong part of the sky. She investigated the object's position and observed it more closely.

De Mello later recalled asking colleagues how they could be sure she had made a discovery. Even when someone congratulated her, she wanted to know whether the object had already been recorded. She requested a catalogue. Only after checking did she feel confident about reporting what she had seen.

The discovery of supernova SN 1997D is recorded in published research on the supernova, which cites the original discovery report. De Mello also describes it in a first-person NASA feature. The particular conversation with the astronomers is her own recollection. Both details matter: a striking observation drew attention, and verification gave it meaning.

A treatment that showed promise—and then didn't

A result becomes even more consequential when people are waiting for a medicine. In her interview, Angela Russell, a professor of medicinal chemistry at Oxford, described work on a potential treatment for Duchenne muscular dystrophy.

The drug, ezutromid, reached a clinical trial. Early findings looked encouraging. But those effects were not seen at the end of the study, and development stopped. The trial sponsor’s published results announcement reports that the study did not meet its primary or secondary endpoints. Russell’s research account at Oxford also describes the lack of sustained efficacy.

Russell described investigating what had happened and pursuing further compounds. The later laboratory work she discussed was not evidence that a replacement treatment already worked in patients. A trial that does not deliver lasting benefit remains an unsuccessful trial—even when studying it can make the next attempt better informed.

Sometimes science isn't the reason a project ends

Biochemist Yulia Matiuhin recalled a different disappointment. A project she had worked on for six years was closed for economic reasons, although she believed its science was promising. The decision affected how she approached the relevance and financial realities of research.

Her account is not another story of an experiment disproving an idea. A company can stop a project for reasons that differ from whether a scientific hypothesis was correct. Researchers working in industry may have to think about both questions.

The question after the result

These five women are not describing one kind of success or failure. Kaelbling needed a better definition of the goal. Dean needed stronger evidence of the learning outcome. De Mello checked whether an observation really was new. Russell faced a clinical result that did not last. Matiuhin encountered an economic decision about a project.

For a student choosing whether she might enjoy research, that is a more revealing picture of the work than an image of constant breakthroughs. Curiosity matters. So does being willing to ask an unglamorous question: What, exactly, does this result tell us—and what still needs checking?