The MH370 search did not find the airplane. It produced more high-resolution data about the deep southern Indian Ocean than the previous century of intentional ocean mapping combined — at 7–20× lower cost than dedicated programs. The pattern is everywhere once you see it, and you can build for it on purpose.
On 7 March 2014, Malaysia Airlines Flight 370 disappeared over the Indian Ocean with 239 people aboard. The search for the wreckage became the largest and most expensive in aviation history. Over five years, a Dutch contractor named Fugro and a Texas company called Ocean Infinity dragged towed sonar arrays back and forth across 279,000 square kilometers of seafloor — an area larger than the United Kingdom. The cost ran past $200 million.
They did not find the airplane.
What they found instead was the seafloor itself. Underwater mountains taller than the Alps. Volcanic ridges no human had ever mapped. Sediment patterns suggesting current systems oceanographers had only modeled. The bathymetric data from the MH370 search produced more high-resolution information about the deep southern Indian Ocean than the previous century of intentional ocean mapping combined. At roughly $717 per square kilometer, it cost between 7 and 20 times less than dedicated ocean-mapping programs, which typically run $5,000 to $15,000 per square kilometer. The plane was missing. The ocean, it turned out, was also missing — nobody had ever looked at it that closely. The search instrument's resolution was so high that anything passing under it became a discovery.
This is the pattern I want you to recognize. I have come to think of it as the serendipity engine.
The conventional story about serendipity is that it is luck — happy accident, fortune favoring the prepared mind, the dropped petri dish that turned out to contain penicillin. The conventional story is wrong, or at least incomplete. A 2025 paper in Scientometrics analyzed over 750 major scientific discoveries and reached a different conclusion: discoveries commonly labeled serendipitous are actually sparked by the first use of a powerful new tool that makes the unexpected observation possible. The microscope was built to see cells; it found bacteria. The telescope was built to see planets; it found galaxies. The MH370 sonar was built to find debris; it found an ocean.
Serendipity, on this reading, is not luck. It is the mathematical consequence of search resolution exceeding target specificity. When you can see in 10-meter detail across a 279,000-square-kilometer area, everything 10 meters or larger across that area becomes a finding. Most of those findings are not what you were looking for. Most of them turn out to matter more than what you were looking for.
The corollary is uncomfortable for institutional planners: the most productive thing you can do is build search instruments whose resolution exceeds the specificity of any single target you are searching for. The resolution makes the serendipity inevitable. The mission becomes a pretext for the instrument, and the instrument outlasts the mission.
This essay walks through four cases that illustrate the pattern, then offers an operational framework. The framework is one developers and tech leaders can apply directly — because the serendipity engine isn't a metaphor. It's how most useful work actually happens, and once you see the pattern you can deliberately build for it.
In late 1943, a team led by Tommy Flowers at the Post Office Research Station in Dollis Hill, north London, completed an electronic machine they called Colossus. Its job was specific and urgent: to break the Lorenz cipher used by the German High Command. By February 1944, Colossus was running at Bletchley Park, processing 5,000 characters per second — fast enough to crack messages in hours instead of weeks. By the end of the war, ten Colossi were operating.
The machine was secret. It was dismantled and most of its drawings burned in 1945. The Official Secrets Act kept Colossus classified for thirty years.
But the people who built it were not secret. They went home. They went to universities. They took with them the working knowledge — not yet articulated as a theory, but absolutely working in their hands — that you could build a digital electronic machine that performed general computation by switching valves in patterns dictated by a stored program. Maurice Wilkes at Cambridge built EDSAC. Alan Turing went to the National Physical Laboratory and proposed the ACE. Manchester built the Mark 1. The first commercial computer (Ferranti Mark 1) shipped in 1951.
The mission was cryptanalysis. The lateral yield was the entire field of digital computing. The world we live in — every smartphone, every cloud server, every line of code you write — descends from a machine built to do something else.
This is the serendipity engine's second motif, after resolution: infrastructure outlives the question. Once you have built the instrument, the instrument has a life beyond its original mission. The instrument knows things you did not ask it to learn. The people who built it know things you did not ask them to discover. When the original problem is solved or abandoned, the instrument and the people remain — and they go on doing what the instrument and people can do.
For developers, the practical reading is this: when you build a tool to solve a specific problem, you are simultaneously building a tool that can be used for problems you have not yet imagined. The first tool I built in my career was a parser for a single client's invoice format. Three years later, that parser had been generalized and was running across forty-three different customer formats. The original client had churned. The parser was still earning revenue, processing invoices for clients who didn't exist when it was written.
In April 2018, California investigators arrested Joseph DeAngelo for the Golden State Killer murders — a series of crimes spanning 1976 to 1986 that had been cold for over thirty years. The arrest was made possible by GEDmatch, a free genealogy database where hobbyists upload their DNA results to find relatives. Investigators uploaded crime-scene DNA, matched it to distant cousins of DeAngelo, built a family tree, and zeroed in on him.
What had been built? A genealogy hobby site. What was found? A serial killer the FBI had failed to identify across four decades.
The investigators who pioneered this approach — CeCe Moore, Barbara Rae-Venter, and others — went on to formalize a discipline now called Investigative Genetic Genealogy. By mid-2024, the technique had been used to identify 293 perpetrators across 621 cases, including a 1956 murder in Montana that had been cold for 65 years. The oldest unsolved murder in the United States, the 1881 Hauck case, was solved in 2024 by the same method.
The genealogy database was, in serendipity-engine terms, a search instrument whose resolution exceeded the specificity of its original target. People upload DNA to find their great-grandparents. Most of them never find what they were looking for — the database doesn't go back that far, the relatives haven't tested, the trail is cold. Those “failed” searches produced, in aggregate, a database dense enough to triangulate any DNA-leaving criminal in the United States to within a few branches of their family tree.
Nobody designed GEDmatch to solve murders. The hobbyists who built it would not, individually, have had the resources or expertise to build a forensic identification system. But the search infrastructure they collectively built — millions of partial pedigrees, distant-cousin matches, autosomal DNA records — turned out to be exactly that system. The mission was personal heritage. The lateral yield was solved homicides.
When you upload data to anything, you should ask the same question: what is the lateral yield of this aggregated database that nobody is talking about? In most cases the answer is “nothing important.” In a small but consistent percentage of cases, the answer is “an entire new discipline that nobody asked for.”
In 1965, Barnett Rosenberg, a biophysicist at Michigan State University, set up an experiment to see how electric fields affected the growth of E. coli bacteria. He used platinum electrodes because platinum is inert — it should pass current without contributing chemically to the system. He turned on the field. The bacteria stopped dividing. Instead of splitting normally, they grew into long filaments, up to 300 times their normal length.
Rosenberg's first hypothesis was about the electric field. The first hypothesis was wrong. The bacteria's behavior was being caused not by the field but by a chemical compound. The platinum electrodes, supposedly inert, were producing trace amounts of a platinum-ammonia-chloride complex in solution. This compound — cisplatin — was inhibiting cell division.
Rosenberg's next move is the one that matters. He could have noted the contaminant, switched to gold electrodes, and continued studying electric fields. Instead, he asked: what else does this compound do? The answer, eventually, was: it stops cancer cells from dividing too. Cisplatin became one of the most important chemotherapy drugs in history, used today against testicular, ovarian, bladder, head and neck, and lung cancers. Survival rates for testicular cancer went from roughly 10% to over 95% in the decades after cisplatin's introduction.
The story is taught as accidental discovery, and at one level it is. But look at the structure. Rosenberg's experimental apparatus had a resolution — precision measurement of cell division under controlled conditions — that was orthogonal to the target he was studying. The target (electric fields on cell division) turned out to be irrelevant. The methodology (precise quantification of how anything affects cell division) was exactly the methodology you'd want for screening anti-cancer compounds. He had built a serendipity engine and pointed it at the wrong thing. When the wrong thing turned out to be the right thing, the engine made the discovery visible.
The reading for tech leaders: when your team builds a measurement or evaluation harness — a CI pipeline, a logging system, a metrics dashboard — the harness has uses beyond what you built it for. The harness is your serendipity engine for engineering. It will surface failures and patterns nobody designed it to surface. Honor it. Don't dismantle it when the original problem is solved.
The Large Hadron Collider at CERN cost roughly $4.75 billion to build and runs at about $1 billion a year to operate. Its headline mission was finding the Higgs boson, accomplished in July 2012. Since then, the LHC has been searching for “new physics” — phenomena beyond the Standard Model that would point to deeper structure in the laws of nature. As of 2025, the LHC has not definitively found new physics. It has produced “unusual energy patterns linked to hidden particles” that may eventually point to it; the CMS collaboration's 2024 Physical Review Letters paper on Soft Unclustered Energy Patterns is a strong candidate.
By the narrow criterion of its primary mission, the LHC is the most expensive scientific failure in human history. By any reasonable accounting, it is one of the most successful instruments ever built.
While searching for the Higgs, CERN scientists invented the World Wide Web. Tim Berners-Lee proposed it in 1989 to help LHC researchers share documents. The Web's economic value now exceeds the LHC's total construction cost roughly five-thousand-fold per year. While operating the LHC, CERN developed superconducting magnet technology now used in MRI machines and fusion reactors. Detector technology developed for the LHC's CMS and ATLAS experiments now appears in medical imaging, security scanners, and semiconductor manufacturing. Grid computing developed for LHC data — 50 petabytes a year — is the architecture underlying modern cloud platforms.
CERN is, in financial terms, the world's most expensive serendipity engine. It was built to find one particle. It found the particle. It continues running because the instrument's resolution exceeds any single target — and the “peripheral data” from a collider operating at 13 TeV is, almost by definition, the frontier of human knowledge. Anything you can do at that energy that you cannot do anywhere else is, at some level, a discovery.
The principle generalizes. The most productive infrastructure in any field is the infrastructure whose resolution is so high that even its peripheral output is more valuable than the dedicated output of less ambitious instruments.
If the serendipity engine is real — if the resolution thesis holds — then it is possible to deliberately design for serendipity rather than waiting for luck. A 2025 paper by Smith and Hilbolling, Demystifying Serendipity: How Mundane Practices Enable the Identification and Pursuit of Unexpected Opportunities, identifies four practice bundles that distinguish people who repeatedly catch serendipitous findings from people who let them slip past.
Deep-diving. The first bundle is immersive domain expertise sufficient to recognize when something is anomalous. The MH370 oceanographers knew what a normal seamount looks like, so they recognized when the seafloor was producing unusual patterns. The Bletchley codebreakers had to understand the Lorenz cipher in depth before they could see what Colossus was actually doing. Without deep-diving, anomalies look like noise.
Listening in. The second bundle is peripheral attention to signals outside the main search beam. The platinum-contamination signal in Rosenberg's experiment was not what he was watching for. He was watching for electric-field effects. Listening in is the discipline of noticing the other thing — the unexpected pattern in the logs, the user behavior the metrics weren't designed to track, the customer support ticket that doesn't fit any existing category.
Connecting. The third bundle is mapping anomalies from one domain onto frameworks from another. Tim Berners-Lee's connection of hypertext (a literary-information-science concept from Ted Nelson) to TCP/IP (a networking protocol) to the LHC researchers' document-sharing problem (a particle-physics workflow problem) is the canonical example. None of those domains was, on its own, equipped to invent the Web. The connection was.
Implementing. The fourth bundle is converting the connection into actionable output. This is where most attempts at engineered serendipity fail. The connection is recognized; the implementation is deferred; the moment passes; the institutional memory degrades. Rosenberg's pivot from “electric fields aren't what's happening here” to “let's screen this compound on cancer cells” was an act of implementation as much as insight. The implementation is where the lateral yield gets converted into the next stage of work.
The practical operating manual for the serendipity engine is: invest deeply, attend broadly, connect liberally, implement decisively. The four together are the discipline that lets a search for a missing plane turn into an ocean-mapping program, a cryptanalysis project turn into the computer, a hobby database turn into forensic genetic genealogy, a study of electric fields turn into a cancer drug, and a quest for the Higgs turn into the global communications infrastructure.
Here is the takeaway I want you to keep. The conventional wisdom about searches is that the goal is to find the thing you are looking for. The deeper truth is that the goal of a high-quality search is to fail in informative ways. The search for MH370 failed; the data is treasure. The search for new physics at CERN has mostly failed; the lateral output is the rest of modern technology. Most genealogy searches at GEDmatch fail at their personal mission; the aggregated database has solved more murders than the FBI in some categories.
When you sit down to build something — a tool, a service, a research program, a database, a measurement harness — ask not only “will this solve the problem I'm pointing it at?” Ask: “what is the resolution of this instrument? What will become visible at that resolution that I'm not currently searching for? Who else might use this resolution for something I haven't imagined?”
The serendipity engine is the answer to those questions. The infrastructure outlives the question. The resolution exceeds the target. The mission becomes a pretext for the instrument, and the instrument becomes the thing.
The MH370 search ended without finding the airplane. The undersea volcanoes are still there, mapped now, named, studied by oceanographers who never met the families of Flight 370. The volcanoes were always there. The search made them visible. That is the entire pattern, and once you see it, you'll see it everywhere — in your code, in your products, in your career. The thing you couldn't find taught you what you didn't know to look for, and the looking turned out to be the work.
Build instruments whose resolution exceeds your target. The everything-else will come.
Sources: Fugro and Ocean Infinity public reporting on the MH370 underwater search and resulting bathymetric dataset (Geoscience Australia / CSIRO releases, 2017–2020). Scientometrics, 2025, analysis of serendipitous discoveries linked to first-use-of-tool. Copeland, Colossus: The Secrets of Bletchley Park's Codebreaking Computers, Oxford UP. CeCe Moore and Barbara Rae-Venter case histories; Investigative Genetic Genealogy case-count summaries through mid-2024; 1881 Hauck case resolution, 2024. Rosenberg et al., Nature, 1969, on cisplatin's discovery. CERN public reporting on LHC construction and operating costs; CMS collaboration, Physical Review Letters, 2024, on Soft Unclustered Energy Patterns. Berners-Lee, “Information Management: A Proposal,” CERN, 1989. Smith & Hilbolling, Demystifying Serendipity, 2025.
Build a serendipity engine for your agents.
Chain of Consciousness is a high-resolution provenance instrument for agent behavior: a hash-linked, append-only chain that records every action, tool call, and decision with cryptographic verifiability. The original mission is “prove what an agent did.” The lateral yield, once you have it running, is the same shape as the MH370 dataset — a recording dense enough that questions you didn't ask at instrument-build-time become answerable. Drift detection. Capability discovery. Post-incident reconstruction. Cross-agent comparison. The resolution exceeds the original target.
Install: pip install chain-of-consciousness or npm install chain-of-consciousness
Hosted Chain of Consciousness · Verify a provenance chain · Follow a claim through its evidence