Absence as Evidence: The Epistemology of Things That Aren’t There

In December 1892, The Strand Magazine published a Sherlock Holmes story called “Silver Blaze,” in which a prize racehorse vanishes from a Dartmoor stable and its trainer is found dead nearby. Inspector Gregory, doing his procedural best, asks Holmes whether there is any point to which he wishes to draw attention. Holmes, in the line that would outlive almost everything else Conan Doyle wrote, replies: “To the curious incident of the dog in the night-time.” Gregory protests that the dog did nothing in the night-time. Holmes says: “That was the curious incident.”

Nothing happened. That was the clue.

The watchdog had been on the premises. A horse-thief had taken Silver Blaze in the small hours. The dog did not bark. From that, Holmes deduces that the thief was someone the dog knew — and the case unravels from there, ending with the trainer himself as the unlucky abductor. The story is famous in mystery circles for its construction, but it is famous outside mystery circles for something more general: it is the cleanest illustration in popular literature of the idea that the absence of an expected signal is itself a signal. The thing that did not happen carried more information than any of the things that did.

This essay is about that. It is about reading what is missing — in cosmology, in medicine, in scientific journals, in your own product analytics — and about how easily we forget to look.

The Maxim Most People Have Backwards

You have heard the saying “absence of evidence is not evidence of absence.” It is often deployed by people defending a fringe claim — just because we don’t have evidence of bigfoot doesn’t mean bigfoot doesn’t exist — and at first pass it sounds like rigorous epistemology. It is in fact wrong, or at least seriously incomplete, and the formal philosophical correction has been around for years. Christopher Stephens, in a 2011 paper in Informal Logic, worked through the Bayesian arithmetic carefully and showed that absence of evidence is evidence of absence whenever you would have expected, conditional on the hypothesis being true, to have seen evidence by now.

The fix is small but it changes everything. Absence is evidence proportional to the visibility you would have expected. If a creature is the size of a grizzly bear and lives in temperate North American forests crawling with hikers, hunters, hikers’ iPhones, and trail cameras, the absence of a single confirmed body or skeleton in a hundred years of looking is very strong evidence. If the creature is the size of a beetle and lives in the soil of a single canyon nobody has surveyed, the absence of evidence is weak evidence. The dog that did not bark was strong evidence precisely because that dog always barked at strangers. Holmes was doing Bayes informally and getting it exactly right.

This matters in software the same way it matters in mystery novels. When you ship a new feature and your dashboards show no spike in errors, that is evidence the feature is working — but only if you would have expected the errors to surface in your dashboards within the window you are watching. Most error dashboards have visibility gaps you have never measured. The silence is real; the conclusion you draw from it depends on whether anyone would have heard a bark.

Neptune and Dark Matter

The most famous demonstrations of absence-as-evidence in science are two stories about gravity. In the 1840s, Uranus was misbehaving. Its observed orbit kept drifting away from the predicted one, in ways that Newtonian mechanics could not explain unless something else was tugging on it. The French mathematician Urbain Le Verrier sat down and did the calculation in reverse: if Newton was right, then for Uranus to be in the position it actually occupied, there had to be another planet of a particular mass at a particular point in the sky. On September 23, 1846, the German astronomer Johann Galle pointed a telescope at the coordinates Le Verrier had sent him and found Neptune within a degree of the predicted spot, on his first night of looking. The English astronomer John Couch Adams had done much of the same math independently. Neptune was discovered by inference from an absence — the absence of a known cause for Uranus’s wobble.

The other story is harder to picture. In 1933, the Swiss-American astrophysicist Fritz Zwicky was studying the Coma Cluster of galaxies and noticed that the galaxies were moving far too fast for the visible matter in the cluster to hold them gravitationally together. He coined the term dunkle Materie — dark matter — for the unseen mass that had to be there. Nearly a century later, the consensus inference is that something like 85% of the matter in the universe is matter we cannot see and have never directly detected. Every dark matter search experiment running today — XENONnT, LUX-ZEPLIN, the various axion haloscopes — is, in effect, looking for what is missing from our model of the visible universe.

If you want to feel the strangeness of this, sit with the number. Most of the universe is something we have never directly observed. The single largest claim in modern physics is built on absence. The math does not work without it. The galaxies fly apart without it. The microwave background acoustic peaks do not line up without it. The thing we are most sure of about the universe is the existence of something nobody has ever held in their hand.

The Drowned Believers

There is an older formulation of the same epistemology, told as a story by Cicero in De Natura Deorum and resurrected by Nassim Nicholas Taleb in The Black Swan. Diagoras of Melos, a fifth-century BC philosopher remembered as Diagoras the Atheist, was shown a temple full of painted tablets — votive offerings from sailors who had prayed during shipwrecks and survived. The argument the tablets were meant to make was obvious. Look: prayer works. Diagoras asked the question that has echoed for two and a half thousand years: Where are the pictures of those who prayed and drowned?

Taleb called this silent evidence, and the point he kept hammering at is that the visible world is a survivorship-biased sample. We see the buildings that did not collapse, the founders who became billionaires, the strategies that worked, the medications that didn’t kill the patient. The buildings that collapsed are rubble. The failed founders run delivery routes. The strategies that didn’t work are not on the slide deck. The medications that killed somebody are in a sealed legal file. The temple holds only the tablets of the survivors. The graveyard does not contribute its statistics.

If you are a product person reading this, the practical translation is: your customer feedback channel is a tablet wall. You hear from the customers who churned but had the energy to write a complaint, and from the customers who loved you so much they wrote a testimonial. The ones in between — the ones who quietly stopped using the product, the ones who churned without a word, the ones who never converted — are the drowned believers, and your roadmap is built almost entirely from the survivors.

The File Drawer

The cleanest scientific version of silent evidence has a name: the file drawer problem. The phrase comes from Robert Rosenthal’s 1979 paper in Psychological Bulletin, which formalized something working scientists had grumbled about for decades. If you run an experiment and get a positive result, you write it up and submit it to a journal. If you run the same experiment and get a null result, the paper is harder to publish, less rewarded by your department, less cited, and ends up in a literal or metaphorical file drawer. Across a discipline, this generates a published literature that is systematically biased toward positive findings, because the negatives evaporated.

The consequences of this are not subtle. A 2016 eLife paper by Nissen and colleagues (“Publication bias and the canonization of false facts,” DOI: 10.7554/eLife.21451) modeled the dynamics formally and showed that, when only a fraction of negative results are published, false positives can become canonized as fact — accepted as established findings in textbooks and meta-analyses despite never having been true. The published literature is a tablet wall. The replication crisis in psychology, in cancer biology, in economics, is partly a story about what was missing from the file drawers nobody opened.

The lesson generalizes immediately to engineering. Your post-mortem document is the survivor’s testimony of an outage that ended. The outages that almost happened but quietly resolved themselves before anyone wrote them up — the near-miss the on-call engineer never escalated, the latency spike that decayed before the alert fired — are absent from your incident database, and your engineering reliability picture is correspondingly wrong by a margin you have no way to measure unless you go looking. Some of the best site-reliability organizations now do exactly that: they write near-miss reports, in the same way commercial aviation does. They are filling in the file drawer on purpose.

The Inverse Problem — When AI Sees What Isn’t There

There is a version of this story that runs in the opposite direction. Humans are absence-blind: we struggle to see what is missing because the cognitive machinery preferentially registers presence. Large language models have the opposite failure mode. When a language model is asked about something it does not know — a paper that does not exist, a function that was never written, a quote that was never said — the model frequently produces an answer anyway, complete with names, dates, and authoritative tone. We call this hallucination, and what it is, in epistemological terms, is the inverse of absence-blindness. Where the human eye glides over the gap, the model fills the gap with confident fiction.

The two failures meet at the same root: poor calibration between what you have observed and what you have warrant to be confident about. A human who has not seen the negative results assumes the canon is the truth. A model that has not seen the right answer produces a plausible one. The first is silence misread as confirmation. The second is silence overwritten by invention. Both produce the same downstream symptom — a confidently wrong picture of the world — and both require the same fix, which is to take the missing data seriously enough to either find it or refuse to speak past it.

Designers of AI systems are starting to build this in explicitly. Some recent agent benchmarks score an agent more highly when it says “I cannot determine this from the information I have” than when it confidently produces a wrong answer. This is, at the level of system design, a vote in favor of epistemic humility as an engineering specification. The hardest part is not the technical one. The hardest part is that the easy version of the product is the one that always answers, and the customer who has not yet been burned will, on average, prefer the easy version. Reading absence costs you something. It is also what separates a system you can deploy from one that will eventually embarrass you in front of a regulator.

The Trained Absence-Readers

Some professions are organized around reading absences. Radiologists spend years learning to notice the shadow that ought to be there and isn’t, the contour that doesn’t match the contralateral side, the lymph node that has gone quiet. Infectious disease physicians learn to suspect a culprit from the absence of bacterial growth on a particular medium. Demographers read population pyramids for missing cohorts — the men who didn’t return from a war, the births that never happened in a famine year. Forensic accountants chase the transactions that ought to appear in the ledger and don’t. Cryptographers, when they audit a protocol, look for the assumption that is silently missing from the threat model.

What unites these crafts is that they treat absence as a positive object of attention. The absence has a name. It has a shape. It has expected dimensions. You can teach a junior radiologist to look at the right place on the scan; you can teach a junior auditor to ask why a particular cash flow doesn’t appear; you can teach a junior threat modeler to ask which assumption is implicit and unstated. In every case, what you are teaching is a kind of inverted vision — the discipline of asking, before you accept what is in front of you, what ought to be in front of you but isn’t.

This is a skill, not a personality trait, and the encouraging thing about that is that it is teachable. You can build it into how your team reviews code, how you read incident reports, how you evaluate vendor claims, how you sanity-check your own dashboards. The bad news is that it does not happen by default. The default is to read what is on the page. The trained reader reads what is not on the page.

What Are You Not Seeing Right Now

If there is a single takeaway here for a developer or a tech leader, it is this. Once a week, sit with one of the data sources you rely on — your error log, your conversion funnel, your customer interview notes, your model evaluations — and ask, explicitly: what would I expect to see in this if the picture I have is wrong, and would I see it? If the answer is no, that is the most important thing on your dashboard, and it is, by construction, not on your dashboard.

The history of consequential mistakes in software is full of cases where the missing signal was the load-bearing one. The crashes that didn’t get reported because the telemetry pipeline dropped them. The cohort that churned without a survey because the survey only fired on uninstall. The model error that didn’t appear in eval because the eval set didn’t include the data that triggered it. In each of these, somebody trained to read absences would have asked the right question first.

The dog that did not bark is everywhere. It is in your funnel, your tests, your incidents, your model evals, your science, and — if you are honest about how the universe is composed — most of the matter around you. The skill is to stop trusting silence at face value and start asking, every time, what kind of bark you would have expected to hear.

That is the discipline. And once you have it, you do not lose it.

Sources: Conan Doyle, “Silver Blaze,” The Strand Magazine, December 1892; Stephens, “A Bayesian Approach to Absent Evidence Reasoning,” Informal Logic 31(1), 2011; Le Verrier & Galle’s Neptune discovery, September 23, 1846; Zwicky, on the missing mass of the Coma Cluster, Helvetica Physica Acta 6, 1933 (“dunkle Materie”); current cosmological consensus on dark-matter mass fraction (~85% of matter); Cicero, De Natura Deorum, Book III, on Diagoras of Melos; Taleb, The Black Swan (Random House, 2007), on silent evidence; Rosenthal, “The ‘File Drawer Problem’ and Tolerance for Null Results,” Psychological Bulletin 86(3), 1979; Nissen, Magidson, Gross & Bergstrom, “Publication bias and the canonization of false facts,” eLife 5:e21451, 2016 (DOI: 10.7554/eLife.21451).