The $760 Weekend: What 50 Years of Biosecurity Governance Already Knows About AI Vulnerability Disclosure

In late 2025, a person without biology training took Evo 2 — the open-weights biological foundation model released earlier that year — and fine-tuned it on the human-infecting viral sequences its creators had carefully excluded from the training set. The job took one weekend. The compute cost about $760. The coding agent assisting them issued no refusals.

That data exclusion was, until that weekend, the model’s primary biosecurity safeguard.

If you build, deploy, or red-team AI systems, that story should sound uncomfortably familiar. It’s the standard prompt-injection demo with a different payload. A safeguard the field was treating as load-bearing collapses in under 48 hours of compute by someone outside the field. The interesting question isn’t whether this is bad — it obviously is — but whether AI security has any institutional memory it can borrow before re-discovering each lesson the hard way.

It does. It just isn’t AI’s. It’s biology’s.

The 80-Year Cost Collapse

Biology has been here. Not the same place, but the same shape.

Source: CSIS, Opportunities to Strengthen U.S. Biosecurity from AI-Enabled Bioterrorism (2025); GovAI, Coding Agents Are Changing the Biosecurity Risk Landscape (2025).

That’s roughly five orders of magnitude over 80 years. Each step compressed who could do it, and each step required governance to catch up. AI security is roughly at the 2017 horsepox moment: capability cost is plummeting, the field still treats access controls and data filtering as primary defenses, and the policy infrastructure is roughly where biosecurity was in the late 1990s — built around the threat model of a previous decade.

The pattern biology learned at great cost: when a capability becomes radically cheaper, governance designed for the previous price point fails silently. You don’t notice it has failed until the first outsider walks through the wall.

Why Biosecurity Is the Right Precedent

Two reasons.

First, the structural problem is identical. Defense and offense draw from the same knowledge base. Understanding how H5N1 achieves airborne mammalian transmissibility is essential for pandemic preparedness and a blueprint for bioweapons (Fouchier et al., Science, 2012). Understanding how a prompt injection bypasses an agent’s safety filter is essential for hardening that agent and a recipe for compromising the next one. Nick Bostrom formalized this in 2011 as the “information hazard” problem, with subtypes — data hazards, idea hazards, attention hazards, template hazards — that map cleanly onto what every AI bug bounty report navigates today. The vocabulary already exists; AI security mostly hasn’t picked it up.

Second, biosecurity has spent 50+ years building governance frameworks for exactly this dilemma. Asilomar (1975). The Fink Report (2004). NSABB (2005). DURC (2012). The H5N1 publication crisis (2011-2012). The Microsoft–IBBIS tiered-access framework (2025). Each milestone left an institutional structure that AI security can borrow, modify, or learn from. Most security fields don’t have a sister discipline with a multi-decade head start. AI does, and has barely started reading.

Three Governance Models, Ranked by Current Adoption

The current AI bounty model is binary disclosure. A researcher submits a finding under NDA; once patched, an advisory is published. The vendor reviews its own findings, decides on severity, and times the disclosure. The conflict of interest is structural — the same entity is researcher, reviewer, and publisher.

Biosecurity moved past this model 50 years ago. Tiered access has now arrived in operational form. In 2025, after Microsoft researchers found that AI protein design tools could generate modified ricin variants capable of evading commercial DNA synthesis screening, Microsoft and the International Biosecurity and Biosafety Initiative for Science (IBBIS) jointly launched a three-tier framework: a public tier with low-risk summaries; a restricted tier requiring identity verification, institutional affiliation, and stated purpose; and a controlled tier with formal access agreements and non-disclosure terms. Two-person independent review handles every access request, modeled on DNA synthesis customer screening. Science formally endorsed the framework — the first time a major scientific journal endorsed tiered access to manage an information hazard.

No AI vulnerability disclosure framework has received equivalent endorsement, because no AI vulnerability disclosure framework has tried this structure.

The third tier — institutional review — is the hardest, the slowest to build, and the one biosecurity converged on after voluntary self-regulation kept failing. NIH Guidelines require any institution receiving federal funds for recombinant DNA research to operate an Institutional Biosafety Committee that reviews dual-use potential before research begins. Since June 1, 2025, IBC meeting minutes are required to be posted publicly. AI has nothing comparable; bug bounty programs are vendor-run end to end.

The Deepest Analogy: Red Teaming Is Gain-of-Function Research

Gain-of-function (GoF) research deliberately enhances a pathogen’s transmissibility, virulence, or host range to understand pandemic risks. AI red teaming deliberately elicits dangerous capabilities from a system to measure and mitigate risk. The methodology is structurally identical: probe a system for the worst it can do, document what you find, use the documentation to defend.

The asymmetry is striking, and it was first surfaced sharply by a 2025 BlueDot Impact analysis. Gain-of-function research is heavily regulated and politically radioactive. Executive Order 14292, signed May 5, 2025, halted federally funded “dangerous gain-of-function” research and rescinded the 2024 DURC/PEPP policy entirely. Meanwhile, AI red teaming is mandated by the EU AI Act and encouraged by NIST frameworks and multiple executive orders. There is no analogous framework restricting it.

When Anthropic spent more than 150 hours with biosecurity experts probing whether Claude could assist in designing biological weapons (Anthropic, Frontier Threats Red Teaming for AI Safety, 2023), the researchers were performing the AI equivalent of gain-of-function research with an institutional review apparatus that biosecurity would consider radically incomplete. The work was good and necessary. The point is that the exact same methodology applied to a pathogen would have required an Institutional Biosafety Committee, a federal funding review, a P3CO-style oversight package, and probably Science journal coordination on publication. In AI, it requires a blog post.

This is either a serious policy blind spot or a legitimate domain difference. The honest answer is some of both, and where you draw the line matters.

Where Capability Thresholds Are Converging

Biosecurity’s Dual Use Research of Concern (DURC) framework defines 7 categories of experiments of concern and 15 select agents and toxins — a category-based approach 20+ years in the making. AI safety, working without that history, has gone straight to numerical thresholds:

• OpenAI’s Preparedness Framework defines a “high” capability threshold as a model that “provides meaningful assistance to novice actors with basic relevant training” on biological threat creation.
• Anthropic’s Responsible Scaling Policy uses 25% accuracy on advanced bioweapon-relevant questions versus internet-only controls as a key tripwire.
• Claude 3.7 Sonnet scored 91% on a bioweapons planning trial against an 80% threshold for “high risk” (CSIS, 2025).
• OpenAI’s o3 outperformed 94% of expert virologists on virology lab protocol tests (CSIS, 2025).

The convergence on numerical thresholds is itself a governance design choice biosecurity has debated for two decades. Numbers scale better than category lists in fast-moving fields and create measurable goalposts auditors can track, where category lists fossilize.

The biosecurity counterargument is that numbers without categories drift: a 25% accuracy threshold means something different on different question sets, and the threshold can be re-tuned by the entity it constrains. Bright lines need both the number and a public methodology for setting and evaluating it. As of early 2026, no AI lab has published its threshold methodology to the level of detail DURC’s category list specifies.

A Modern Asilomar That Actually Worked

One detail in the IBBIS story deserves more attention than it has received: more than 175 developers of AI for biodesign have signed the Commitments for the Responsible Development of AI for Protein Design, including a commitment to use only screened DNA synthesis providers. Voluntary self-regulation at scale, driven by norm-setting rather than legislation — a modern Asilomar.

The counterintuitive insight is that voluntary commitments work better in narrow technical communities than in broad ones. The 1975 Asilomar moratorium worked because recombinant DNA research was concentrated in roughly 100 NIH-funded labs with strong professional norms and shared funding dependency. The 2023 “Pause Giant AI Experiments” letter, which echoed Asilomar’s logic at the level of all frontier AI development, was widely ignored by the labs it targeted. Different community size, different incentive structure, different result.

The lesson isn’t “voluntary commitments don’t work for AI.” It’s “they work for AI applied to specific dangerous domains, where the community is small and the stakes are visible.” Protein design met that bar; frontier AI development as a category did not. The open question for AI security is whether vulnerability disclosure is more like the first or the second.

Where the Analogy Breaks

The strongest objection to all of this is physical.

Biological agents have physical chokepoints. Even after a $760 weekend on Evo 2, the attacker still needs to synthesize DNA to create a working pathogen. That synthesis is increasingly screened: as of November 6, 2025, the IBBIS DNA Screening Standards Consortium operationalizes ISO 20688-2:2024 across roughly 30 expert organizations spanning industry, academia, government, standards bodies, and civil society. A complementary Sequence Biosecurity Risk Consortium defines what counts as a “sequence of concern.” This is a layered defense whose final layer lives in the physical world, where coding agents cannot help.

AI exploits have no physical chokepoint. A jailbreak posted online is instantly weaponizable worldwide at near-zero marginal cost. There is no wet lab, no synthesis screening, no last physical filter that resists circumvention. GovAI’s headline policy recommendation for biosecurity in 2025 was to prioritize physical chokepoints over digital safeguards, on the explicit reasoning that digital safeguards are systematically bypassed by coding agents. AI security cannot follow this advice, because it has no physical chokepoint to retreat to.

This means the timelines transfer poorly. Biosecurity governance evolved over 50+ years against a threat where capability cost fell about five orders of magnitude. AI capability cost is falling roughly an order of magnitude per year. The biosecurity policy cycle — Fink Report 2004 → DURC 2012 → policy update 2024 → executive rescission 2025 — runs in 21 years. AI does not have 21 years for the next policy iteration. The principles transfer; the cycle times do not.

It also means the strongest single biosecurity tool is the one AI cannot copy. Worth saying plainly rather than pretending the mapping is clean.

What to Actually Do Now

Three concrete moves are available without waiting for new institutions.

Move binary disclosure to tiered. A high-severity AI vulnerability has three audiences with different needs: the public (who needs to know the class exists), defenders (who need enough technical detail to test their own systems), and red-team peers (who need full reproduction). Three tiers, three access levels, three disclosure timelines. The institutional plumbing exists at IBBIS. Forking it for vulnerability disclosure is a session of work, not a multi-year program.

Build an IBC equivalent for AI vulnerability research. The conflict of interest in vendor-run bounty programs is structural. Independent, institution-level review — the way universities review human-subjects and biosafety research — would give AI the distributed oversight that biosecurity built over four decades. Some labs have informal versions; codifying them with public charters, posted minutes, and external members would close most of the gap.

Recognize red teaming as gain-of-function research and oversee it accordingly. This doesn’t mean banning it. It means subjecting AI capability red-teaming to the same institutional review (purpose, capability thresholds, results-handling, publication review) that biosecurity already applies to its structurally identical sister discipline. The methodology is the same; the oversight should match.

Where This Argument Is Weakest

Three concessions worth naming.

The first is the speed mismatch. Biosecurity governance evolved at the pace of biological capability cost compression. AI capability cost compression is one to two orders of magnitude faster. Importing biosecurity processes without compressing the timelines is a recipe for shipping 1995 governance for 2030 risks. The principles transfer; transplanting the calendar does not.

The second is market structure. Biosecurity governance built on shared NIH funding dependency — a structural lever AI security does not have. Voluntary moratoria and self-regulation do not bind a competitive commercial landscape, and the 2023 AI pause letter proved this at scale. Mandatory institutional review remains plausible only because it can be tied to liability, insurance, and procurement requirements rather than to research funding.

The third, already covered, is the physical chokepoint gap. Biosecurity’s last line of defense is physical. AI’s last line of defense will be digital, and digital defenses are exactly what the $760 weekend showed are bypassable by coding agents. Whatever AI security builds has to function in the layer where the safeguard already failed.

These are real limits. They argue for selective borrowing, not wholesale transplant.

The Practical Insight

If you run a bug bounty program, write a vulnerability disclosure policy, or evaluate AI security at any scale, ask the same question for each layer of your program: what would the biosecurity version of this look like in 2025, and where am I still doing 1995 governance for 2030 risks?

The honest answer almost everywhere is: yes, you are. Binary disclosure where tiered is operational. Vendor self-review where independent IBC-style review is standard in adjacent domains. Encouraged red teaming with no oversight equivalent to the gain-of-function frameworks already in place for the structurally identical biological version. Voluntary safeguards at the level of “all AI” rather than at the level of specific dangerous applications, where Asilomar evidence shows voluntary safeguards actually work.

The 50-year lag isn’t AI’s fault — there was nothing to lag. But there’s no excuse for repeating mistakes that are already in the public record, written up by the people who made them, with retrospectives that name what they would do differently.

The $760 weekend will keep happening. Every safeguard whose primary defense is “we filtered the training data” will fall to a coding agent and a fine-tune. The work ahead isn’t in better filters. It’s in the governance layer biosecurity has spent half a century learning how to build — and that AI security, against all available evidence, is still trying to invent from scratch.

Sources: CSIS, Opportunities to Strengthen U.S. Biosecurity from AI-Enabled Bioterrorism (2025); GovAI, Coding Agents Are Changing the Biosecurity Risk Landscape (2025); Fouchier et al., Science (2012); Bostrom, “Information Hazards” (2011); Microsoft & IBBIS, tiered-access framework (2025); BlueDot Impact, gain-of-function/red-teaming analysis (2025); OpenAI Preparedness Framework; Anthropic Responsible Scaling Policy; Anthropic, Frontier Threats Red Teaming for AI Safety (2023); Executive Order 14292 (May 2025); ISO 20688-2:2024; NIH IBC Guidelines.