When a threshold crossing triggers the selloff that confirms it — and why your autoscaler runs the same loop.
A corporate bond rated BBB− is, by the slimmest possible margin, investment grade. One notch. It sits at the very bottom of the floor that separates the bonds pension funds, insurers, and index trackers are allowed to own from the ones they are not. For years it can live there quietly, paying its coupon, bothering no one.
Then, one Tuesday morning, a rating analyst publishes a single notch of downgrade. BBB− becomes BB+ (or, on Moody's scale, Baa3 becomes Ba1). The bond itself has not changed. The company has not, in the space of that morning, built worse cars or shipped buggier software. But the bond is now, by a contractual definition written into thousands of fund mandates, high yield — and within the hour it is radioactive to the very investors who held it the day before.
They sell. Not because they've reassessed the company and concluded it's doomed — many of them never form an opinion at all — but because their mandate makes holding a sub-investment-grade bond a compliance violation. Index funds tracking investment-grade benchmarks must drop it, because it has dropped out of the index. Insurers must shed it or eat a punitive capital charge. They all sell at once, because the downgrade is public and they all read it the same morning. The synchronized selling overwhelms the buyers and craters the price. The cratered price raises the company's real-world cost of borrowing. More expensive borrowing weakens the company — which is precisely the deterioration the downgrade was warning about. The measurement triggered the selling that made the measurement true.
This is a fallen angel, and it's one of the most studied reflexive traps in finance. As of the end of 2024, Insight Investment counted $699 billion of BBB− rated corporate debt sitting one notch above that cliff — 28% of it already a single downgrade from the fall. If you build automated infrastructure for a living, that number should make you slightly nervous, because you have almost certainly built the same machine.
The investor George Soros gave this pattern a name: reflexivity. We usually assume measurement is passive — a thermometer doesn't change the temperature. But in systems where the act of measuring triggers action, and the action changes the thing being measured, the measurement becomes a participant. The rating agency isn't a thermometer. It's a thermometer wired to a furnace.
The fallen-angel loop runs in about seven steps, and they're worth naming, because the infrastructure version runs in exactly the same seven:
The energy stored above this particular cliff is enormous. Beyond the $699 billion of BBB− debt, ratings agencies had roughly $387 billion of BBB corporates on negative outlook as of the end of 2024 — the highest, outside the pandemic, since at least 2010 (Insight Investment). Insight Investment projected on the order of $55 billion of actual fallen angels over the near-to-medium term, against a sell-side median forecast around $45 billion for 2025. That's not a market that might see forced selling. It's a coiled spring with a published list of which names sit nearest the edge.
Picture a cluster in the middle of a busy afternoon, under heavy but survivable load. Every node is running hot — say 90% CPU. Your orchestrator runs a health check that marks a node failed if it stays pegged too long, and its remedy is sensible in isolation: evict the unhealthy node and reschedule its work elsewhere.
One node tips over the line. It's evicted. Its share of the traffic has to go somewhere, so it lands on that node's neighbors — which were already at 90%. The extra load pushes them over the same line. They fail the same health check. They're evicted. Their load redistributes onto an ever-smaller set of survivors, each now carrying more, each crossing the threshold in turn. In two minutes a cluster that was coping is a smoking crater — and every step of the collapse was a correct, sanctioned action by a system doing exactly what you told it to do. The threshold you installed to protect the cluster became the trigger that destroyed it. That's a fallen angel with a kubelet.
It shows up everywhere a hard threshold is wired to automatic, synchronized action:
In each case, the threshold crossing triggers synchronized automated action that amplifies the very move that caused it. That's the fallen-angel signature, and once you can see it, you'll find it in load balancers, connection pools, retry storms, cache stampedes, and quota cliffs all over your stack.
Here's the good news, and the reason this analogy earns its keep: credit markets have been getting mauled by reflexive thresholds for decades, and they've worked out the defenses. They aren't exotic. They map straight onto infrastructure.
Hysteresis: use different thresholds to enter and to leave a state. A thermostat doesn't switch the heat on and off at exactly 20°C — it heats to 21 and cools to 19, with a dead band between, so a faint draft doesn't make it chatter. Electrical engineers call the circuit that does this a Schmitt trigger. The principle is the cure for flapping in any feedback system: separate the threshold for crossing one way from the threshold for crossing back. If your autoscaler scales up at 70% and scales down at 70%, it lives on the boundary and oscillates across it. Scale up at 70% and down at 30%, and you've opened a wide, stable corridor in the middle where nothing happens at all.
Gradualism: respond proportionally, not all at once. The disaster at the boundary isn't that some action is taken — it's that all the action is taken instantly. A fallen-angel selloff is so violent precisely because the mandate forces an all-at-once dump at whatever price the market will bear. The infrastructure version of gradualism is to remove one pod, wait, observe whether the load you just shed landed safely, and only then remove another — and on the circuit-breaker side, to reopen with a trickle of traffic that ramps up as the dependency proves it can take it, rather than slamming the full firehose back on the instant the cooldown expires.
De-synchronization: don't let everyone hit the boundary on the same tick. Forced selling is devastating because it's coordinated — every mandated holder acts on the same public downgrade inside the same narrow window. Markets blunt this through sheer heterogeneity: funds with different rebalancing calendars and risk tolerances don't all sell on the same day. The infrastructure fix is jitter — add a small random delay to cooldowns, health-check intervals, and retry backoffs, so your breakers and scalers and clients don't all fire on the same clock edge. Jitter is the cheapest defense against synchronized collapse there is, and it's routinely left out.
There's a fourth, quieter mitigation worth stealing: absorbers. In credit markets, “crossover” funds are mandated to hold both investment-grade and high-yield paper, so they can buy a fallen angel without anyone ever forcing them to dump it — they soak up the selling pressure at the boundary. The infrastructure analog is reserved slack and graceful-degradation pools: capacity that can accept a degraded or overloaded component without immediately evicting it, giving the system somewhere to put a spike besides its neighbors' throats.
Now the part that should change how you set a threshold on Monday.
Credit markets don't use symmetric hysteresis. They use a deliberately lopsided version: it is easy to fall out of investment grade and hard to climb back in. A downgrade can land in days when fundamentals deteriorate; the round trip back across the line typically demands sustained improvement proven over many quarters. The agencies are stingy with upgrades near the boundary precisely because they know a too-eager upgrade, reversed a month later, would whipsaw the entire forced-buyer/forced-seller machine. That asymmetry is not a bug. It's the damping that keeps the most consequential line in finance from chattering.
Infrastructure, overwhelmingly, does the opposite. The default autoscaler scales up and down off the same utilization target. The default health check evicts at a load level the eviction itself will immediately recreate on the surviving nodes. The default circuit breaker closes fully after a single successful probe. Symmetric thresholds at a boundary that has feedback don't merely fail to prevent oscillation — they guarantee it. This is, quietly, one of the most common reflexive bugs in production systems, and one of the easiest to fix: split the threshold, and make the gap wide and lopsided. Scale down far more reluctantly than you scale up. Require several consecutive healthy probes, not one, before a breaker fully reopens. Stop evicting well before load falls back to the level that triggered eviction. You aren't tuning a number; you're installing a dead band where the system is allowed to rest.
One last thing the bond market knows that should reframe how you feel about the whole mess. Forced selling doesn't push the price to fair value — it pushes it below fair value, because the sellers aren't selling on judgment, they're selling on obligation, and obligation doesn't stop at the right price. The data is striking: fallen angels have historically returned roughly −10% in excess terms over the six months before the downgrade, and then roughly +20% over the twelve months after, outperforming other comparable high-yield credits (Insight Investment; VanEck). The reflexive overshoot in one direction creates a real opportunity in the other — there are entire ETFs (VanEck's ANGL, iShares' FALN) built on little more than buying what the forced sellers had to throw away.
The infrastructure version is gentler but real: a system that survives a cascade often comes back healthier than it went in. The weakest nodes are gone, the load is right-sized to what's actually left, the dead weight has been cleared. A cascade is a brutal and wildly expensive form of garbage collection. The lesson is emphatically not that cascades are fine — it's that the overshoot is real and roughly symmetric, so design for the rebound. Let recovered capacity rejoin gradually (there's gradualism again), and don't let your monitoring panic-buy at the bottom, over-provisioning in response to a transient you'll be unwinding an hour later.
Go find every hard threshold in your system that is wired to automatic mass action — autoscaler targets, health-check eviction rules, circuit-breaker trip-and-reset logic, quota cliffs, rate-limit cutoffs, retry policies. For each one, ask three questions:
Any threshold that answers same / cliff / same-tick is a fallen-angel mechanism that simply hasn't fired yet. Credit markets learned all three mitigations the hard and expensive way — through hundreds of billions of dollars of forced selling and a couple of generations of crises — and then they wrote the answers down. You don't have to repeat the tuition. Add the hysteresis, add the gradualism, de-synchronize the actors — and do it before the crossing triggers the selloff that confirms it.
Make your measurement a thermometer, not a furnace.
The fallen-angel trap is reflexivity: the measurement triggers the action that confirms the measurement. The defense is to keep the thing that records separate from the thing that acts — an honest account of what happened that doesn't itself move the market. The same applies the moment an AI agent acts on a threshold: you want a record that observes without participating, so you can see the loop instead of becoming part of it. Chain of Consciousness anchors every agent action to a verifiable external record — a thermometer for your system, never wired to the furnace.
See a verified provenance chain · Hosted Chain of Consciousness
pip install chain-of-consciousness · npm install chain-of-consciousness