Why your best cache TTL, retry policy, and agent autonomy are all "intermediate" — and the hundred-year-old method for finding the peak without climbing it blind.
Here is a graph a chemist drew more than fifty years ago. On the x-axis: how tightly a metal surface grips a hydrogen atom. On the y-axis: how fast that metal splits water into hydrogen gas. The points do not climb steadily as the grip gets stronger. They climb, peak, and fall — an inverted parabola, a mountain seen head-on. Platinum sits near the summit. Gold, which barely holds hydrogen at all, languishes on the left slope. Tungsten, which clamps down so hard the hydrogen never leaves, sits on the right. The chemists call it a volcano plot, and the shape is so reliable it is now a design tool: tell me one number — the binding strength — and I will tell you, without running your reaction, roughly how good your catalyst will be.
Now here is a graph a platform engineer drew last quarter. On the x-axis: cache TTL, the number of seconds a cached value is allowed to live. On the y-axis: effective throughput, some blend of hit rate and freshness. The points climb, peak, and fall. Too short, and nothing stays cached long enough to be reused — every request stampedes the origin. Too long, and you serve stale data until someone files a bug. The good TTL is in the middle. Same volcano. Different mountain.
The chemist has had a name for this shape since 1912, a graph for it since the 1960s, and — this is the part worth your attention — a method for finding the peak without climbing it blind. The engineer, most of the time, just nudges the number up and down and watches the dashboard. We are leaving a hundred years of borrowed insight on the table.
In 1912 the French chemist Paul Sabatier shared the Nobel Prize in Chemistry with Victor Grignard, in Sabatier's case for showing that finely divided nickel could catalyze the hydrogenation of organic compounds. Out of that work came the idea that now carries his name. The Sabatier principle says a catalyst works best when its interaction with the reacting molecules is just right — neither too strong nor too weak.
The logic is almost embarrassingly simple once you see it, and it has two failure modes, not one. A catalyst's whole job is to grab a molecule, hold it through a transformation, and then let the product go. If the surface binds too weakly, the molecule never sticks around long enough to react — the reaction never starts. If it binds too strongly, the molecule reacts fine but the product clings to the surface and never desorbs — the catalyst clogs with its own success and goes silent. Chemists call that second mode poisoning. Maximum activity lives in the narrow band where binding is strong enough to engage and weak enough to release.
Plot reaction rate against binding strength and that trade-off draws itself as the volcano — a shape usually credited to the Soviet chemist Alexander Balandin, who formalized the volcano curve in the mid-twentieth century. The summit is the Sabatier optimum. The two slopes are the two ways to fail, and they fail for opposite reasons: the left slope is under-engagement (nothing happens), the right slope is over-engagement (nothing finishes).
The hydrogen evolution reaction is the textbook case. Plot the rate of hydrogen production against the metal–hydrogen bond strength and platinum sits near the apex, which is a large part of why platinum is the expensive heart of so many electrolyzers and fuel cells. Industrial ammonia synthesis — the Haber–Bosch process that, over a promoted iron catalyst at 200–300 atmospheres, fixes the nitrogen that feeds roughly half the people alive — lives on the same curve. Iron sits near the nitrogen-binding optimum: metals that hold nitrogen too weakly can't crack the brutally stable N≡N triple bond to begin with, and metals that hold it too strongly build stable nitrides that won't give the nitrogen back. Iron is the compromise that fixes a meaningful share of the world's food supply and burns one to two percent of global energy doing it. The stakes of getting the binding "just right" are not academic.
Once the volcano is in your eye, you start seeing it everywhere in systems work — and not as a loose metaphor. The mechanism is identical: a knob that controls how tightly your system engages with something, with failure at both ends for opposite reasons.
| Knob | Too weak (left slope) | Too strong (right slope) |
|---|---|---|
| Cache TTL | nothing persists; hit rate collapses, origin stampeded | stale values cling and get served — never desorbs |
| Retry aggressiveness | give up too soon; transient blips become permanent errors | retry storm buries a wobbling downstream — poisoning |
| Explore / exploit | sample endlessly, commit to nothing, no compounding | lock onto a local optimum, miss the better ridge over |
| LLM temperature | locked to the mode; repetitive, creatively inert | coherence dissolves into word salad |
| Agent autonomy | every step waits on approval; human is the bottleneck | flash-crash: spirals at machine speed, no one to break the grip |
Cache TTL is the cleanest case. Too short is weak binding: values don't persist long enough to be reused, hit rates collapse, and the origin takes the full load it was supposed to be shielded from. Too long is strong binding: stale values cling to the cache and get served long after they should have been released — the product never desorbs. The good TTL is the band where data stays long enough to be useful and leaves before it lies.
Retry aggressiveness has the same anatomy. Too conservative — give up after one failed attempt — and you abandon requests that a single retry would have saved; transient blips become permanent errors because you never engaged hard enough to recover. Too aggressive — retry hard, retry often, no backoff — and you get the retry storm that every distributed-systems team learns to fear: a wobbling downstream service gets buried under a thundering herd of retries and collapses, poisoned by the very traffic meant to route around its trouble. Amazon's guidance on timeouts, retries, and exponential backoff with jitter is, read through this lens, an entire engineering document about finding the middle of a volcano.
Exploration versus exploitation — the dilemma at the heart of reinforcement learning, A/B testing, and any system that must both use what it knows and discover what it doesn't — is volcano-shaped, and the management literature has measured the curve: studies of firms find an inverted-U relationship between the exploration–exploitation balance and performance. Pure exploitation is strong binding: you lock onto the best thing you've found and never discover the better thing one ridge over, trapped in a local optimum. Pure exploration is weak binding: you sample endlessly, commit to nothing, and never convert discovery into compounding returns. The payoff peaks in the middle, where you exploit what works while spending a disciplined slice of effort probing for what might work better.
LLM temperature is a volcano you can feel in an afternoon of prompting. Near zero, the model is locked to the single most probable next token — deterministic, repetitive, safe, and creatively inert; the output is bound so tightly to the mode that nothing new ever releases. Crank it toward the ceiling and coherence dissolves into word salad; nothing binds long enough to form a sustained thought. Useful, surprising-but-coherent generation lives in the broad middle band, the exact place practitioners learn to sit by feel.
Agent autonomy — the one that matters most as software starts taking actions on our behalf — is the same shape. Grant too little and the human becomes the bottleneck: the agent engages so weakly that every step waits on an approval click, and human latency dominates the whole loop. Grant too much and you get the flash-crash failure mode: the agent binds so hard to its own plan that it spirals, compounding a wrong turn at machine speed with no human in the loop to break the grip. The productive band is the one where the agent handles the routine and releases to a human at the exceptions — engages, completes, hands back.
Five knobs, one mountain. And in every case the instinct that wrecks teams is the belief that one extreme is virtuous — that the tightest coupling, the most aggressive retry, the most autonomous agent must be the best, if only you could afford it. The volcano says no. The strongest binder is never the best catalyst. The summit is in the middle, and both shores are wrong.
If the essay stopped here it would be a tidy metaphor and not much else. The reason catalysis is worth borrowing from is that chemists do not just know the curve is a volcano — they have a method for finding the peak without testing everything, and that method is the part engineering keeps reinventing badly.
Here is what a catalyst designer does not do: synthesize ten thousand candidate materials, run ten thousand reactions, and keep the best. That would be hill-climbing blind, and the search space is astronomically large. Instead, beginning with the surface-science program of Gerhard Ertl and the computational chemistry of Jens Nørskov and collaborators, the field found that you can predict where a catalyst lands on the volcano from a single cheap descriptor — usually the adsorption energy of one key intermediate. Through what are called Brønsted–Evans–Polanyi relations, that one binding energy scales linearly with the activation energies that actually set the rate. So you compute one number for each candidate surface, drop it onto the volcano, and read off the predicted activity. Modern catalyst screening runs exactly this loop at scale: machine-learning models estimate that one descriptor across thousands of hypothetical alloys, and only the handful near the predicted peak ever get made in a lab. One measurement stands in for ten thousand experiments.
That is the move. Not "the optimum is in the middle" — everyone half-knows that. The move is: find the one cheap descriptor that tells you which slope you're standing on, and steer by it, instead of nudging the knob and squinting at the dashboard.
For each engineering volcano, the descriptor is sitting right there, usually already in your telemetry:
In every row, the descriptor is cheaper to measure than the outcome is to optimize, and — this is the whole point — it tells you not just that you're off-peak but which direction the peak is. That is the difference between navigation and flailing.
The volcano's most practical gift is that its two slopes fail for genuinely different reasons, and the fixes point in opposite directions. Under-engagement on the left: nothing sticks, nothing starts, the reaction never happens — cache misses, abandoned retries, an agent waiting on permission. Over-engagement on the right: things start but never finish or never let go — stale data, retry storms, an agent that won't surrender a bad decision.
This matters because the cost of misdiagnosis is not zero, it is backward. If your retries are failing and you wrongly conclude you're being too timid, you turn the knob up — straight into a storm that takes the downstream service down hard. If your cache is serving stale data and you wrongly read it as a hit-rate problem, you lengthen the TTL and make the staleness worse. On a volcano, moving the wrong way doesn't just stall progress; it accelerates the failure you already have. So the first question is never "more or less?" It is "which slope am I on?" — and the descriptor answers it.
Two honest caveats keep this from becoming a magic formula, and both are useful.
First, the peak moves with context. Different reactions have different optimal binding energies; there is no universal "best catalyst," only the best one for this reaction under these conditions. Likewise there is no universal best TTL, retry budget, or temperature — only the best one for this workload, this downstream, this task. The Sabatier principle never hands you the answer. It hands you the shape of the answer (always a volcano) and the method for finding the peak (steer by the descriptor). That is more durable than any single tuned value, because when the workload shifts, the method still works and the old number doesn't.
Second — and this is the frontier that should make you ambitious — the static peak isn't even the ceiling. A recent line of work on dynamic, or "resonant," catalysis shows that if you oscillate the binding energy rather than holding it fixed at the Sabatier optimum, you can push the reaction rate above the static volcano's summit: bind strongly for a beat to drive the reaction, then weaken to release the product, then bind again. The best fixed setting is beaten by a setting that moves. The engineering echo is immediate and underused: the best fixed TTL is beaten by an adaptive one that lengthens under read-heavy load and shortens after writes; the best fixed temperature, by one that runs hot to brainstorm and cold to commit; the best fixed autonomy, by an agent that earns slack on routine work and surrenders it near anything irreversible. The static peak is where you should start. It is not where you have to stop.
The next time you are tuning a parameter and find yourself torn between "more" and "less" — more caching, more retries, more autonomy, more creativity — stop and draw the volcano. Three moves, in order.
Paul Sabatier worked this out for molecules on metal a century ago, and an entire industry now screens catalysts by computing one binding energy and reading a graph. Your cache, your retry loop, your agents, and your prompts are sitting on the same mountain. You don't have to climb it blind. Find the descriptor, locate yourself, and move toward the middle — because the strongest grip was never the answer, and neither was the loosest.
Agent autonomy is the volcano with the highest stakes.
Of the five knobs, autonomy is the one where the right slope — an agent that binds too hard to its own plan and spirals — does real damage at machine speed. Tuning it safely needs more than a number: you need to know which agent acted, what it consumed, and whether it has earned the slack — the descriptor that tells you to hand back to a human before the irreversible step. That's what the Agent Trust Stack provides: verifiable identity, provenance, and portable reputation, so you can grant autonomy on what's proven safe and pull it back at the exceptions.
pip install agent-trust-stack · npm install agent-trust-stack
See it in action →