At one request every ten seconds, an AI agent logs 10,000 interactions in under 28 hours. The research on human expertise and on agents says that count predicts little. Agents improved where something outside them graded the work and the grade was kept.
Peter Norvig's essay "Teach Yourself Programming in Ten Years" opens with a search. He asked Amazon for books published since 2000 with "teach," "yourself" and "hours" in the title, and got back 512 of them. "Of the top ten, nine are programming books (the other is about bookkeeping)." The essay has been answering those titles since 2001. In twenty-four hours, Norvig writes, "you won't have time to write several significant programs, and learn from your successes and failures with them."
The rush has a new customer now, and this one skips the book. Put an AI agent behind a queue that hands it a request every ten seconds, and it logs ten thousand interactions in under twenty-eight hours. That is Malcolm Gladwell's famous number, reached in about a day, with requests where Gladwell counted hours. It is tempting to read that count as experience, and experience as learning. Read closely, Norvig's essay says the count, by itself, tells you very little. The research on human expertise since then backs him up, and the research on agents says it more sharply.
People remember the title. The working sentence of the essay is about what fills the decade: "The key is deliberative practice: not just doing it again and again, but challenging yourself with a task that is just beyond your current ability, trying it, analyzing your performance while and after doing it, and correcting any mistakes. Then repeat. And repeat again."
Further down, in his recipe for becoming a programmer, Norvig restates the idea in the language of the expertise research he draws on. The first quotation is a warning: "the maximal level of performance for individuals in a given domain is not attained automatically as a function of extended experience." The second is a specification: "the most effective learning requires a well-defined task with an appropriate difficulty level for the particular individual, informative feedback, and opportunities for repetition and corrections of errors."
Read those the way an engineer reads a requirements document. There are four requirements: a task at the right difficulty, feedback that carries information, repetition, and correction of errors. Accumulated experience is the one thing the first quotation rules out as sufficient. It is also the one thing a deployed agent piles up fastest.
The ten thousand has its own history. In 1993, Anders Ericsson, Ralf Krampe and Clemens Tesch-Römer published a study of violin students at a music academy in Berlin. The best of them had practiced an average of ten thousand hours by the age of twenty. In 2008, Gladwell's Outliers turned that average into "the ten-thousand-hour rule."
In 2016, Ericsson and the science writer Robert Pool took the rule apart in an excerpt from their book Peak. "First, there is nothing special or magical about ten thousand hours," they wrote. The same students had averaged "approximately seventy-four hundred hours" by eighteen, and "Half of the ten violinists in that group hadn't actually accumulated ten thousand hours at that age."
Their sharper correction concerns Gladwell's other example, the Beatles in Hamburg. Gladwell counted some twelve hundred performances of up to eight hours each, close to ten thousand hours. Ericsson and Pool point to Mark Lewisohn's 2013 biography Tune In, which puts the total nearer eleven hundred hours of playing. Then comes the sentence this essay is built on: "performing isn't the same thing as practice." An hour on stage, "where the focus is on delivering the best possible performance at the time," differs from an hour of practice "designed to address certain weaknesses and make certain improvements."
That distinction is the trouble with counting an agent's interactions. Nearly everything a deployed agent does is performance. Each request asks for the best possible answer now. No one designed the request to expose a weakness, and unless someone builds the loop, nothing tells the agent afterward whether its answer was right. Ten thousand requests like that are ten thousand nights in Hamburg.
There is a harder correction, and agents need it more than musicians do. In 2014, Brooke Macnamara, David Hambrick and Frederick Oswald pooled 88 studies of deliberate practice and performance, covering 11,135 participants. Practice explained 26% of the variance in performance in games, 21% in music, 18% in sports, 4% in education and less than 1% in professions, where the estimate was not statistically significant. Their summary is blunt: "deliberate practice is important, but not as important as has been argued."
Two findings from that paper fit agents uncomfortably well. The first is predictability. Practice explained 24% of the variance for highly predictable activities, such as running, and 4% for activities low in predictability, such as handling an aviation emergency. An agent fielding open-ended requests lives much nearer the emergency than the running track. The second is the authors' guess about why professions scored so low: "One possibility is that deliberate practice is less well defined in these domains." At work, practice and performance share the same hours, and nothing separates one from the other unless someone sets it up.
Even the founding study has softened. When Macnamara and Megha Maitra repeated the violin study in 2019 with a double-blind procedure, they reported: "We did not replicate the core finding that accumulated amounts of deliberate practice corresponded to each skill level." The best violinists estimated about 22.7 hours a week of solo practice, and the merely good ones about 30.7. Practice still mattered, explaining 26% of the variance in their sample against 48% in the original.
Put together, the human evidence is mostly about what not to count. Hours are a weak predictor, weakest where the work is unpredictable and least defined, and the most famous result shrank when it was rerun. For people, that is about where the evidence stops, because nobody can rerun a violinist's childhood with the teacher taken out. With agents, you can.
In 2023, researchers from Google DeepMind and the University of Illinois tested whether large language models could improve their answers by reviewing them, with nothing outside the model to check against. The title of their paper, presented at ICLR 2024, gives the result: "Large Language Models Cannot Self-Correct Reasoning Yet." In their words, "LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction." That is Norvig's step of analyzing your performance, run with no one available to say what the mistakes were.
The same paper went back to earlier reports of successful self-correction and found where the gains came from: "the improvements in these studies result from using oracle labels to guide the self-correction process, and the improvements vanish when oracle labels are not available." One of the studies it named was Reflexion, a 2023 paper on agents that learn from their own reflections. In Reflexion's reasoning experiments, each attempt was scored by exact match against the dataset's answer, and the agent was told whether it passed. Its reflections were real. Its teacher was the answer key.
That reads like a debunking. It is more useful as a specification, because Reflexion's mechanism is sound. Its agents "verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials." The reflection is where the correction lives. The open question is where the feedback comes from, and in Reflexion's coding experiments it came from somewhere better than an answer key.
For each problem, the agent wrote up to six unit tests of its own and ran its code against them in a real interpreter. With that loop, the paper reports 91% pass@1 on the HumanEval benchmark, against 80% for GPT-4, the previous state of the art. The tests were the agent's own, and the paper tabulates how often they passed broken code or failed working code. The pass or fail still came from running the program, and an interpreter is indifferent to how confident the agent feels.
Other systems make the same shape easier to see. ExpeL, accepted at AAAI-24, lets an agent gather experience on "a collection of training tasks" and write down what it learns as natural-language insights, and its authors report "a consistent enhancement in its performance as it accumulates experiences." Each training task ends in success or failure, and that outcome is what turns an episode into experience. Voyager, an agent that plays Minecraft, reads almost like Norvig's recipe written in code. It has "an automatic curriculum that maximizes exploration," which hands the agent "progressively harder tasks" suited to its current state. It has "an ever-growing skill library of executable code," where solved problems are kept. And it refines its programs with prompts that incorporate "environment feedback, execution errors, and self-verification." Against the previous best method, it collected 3.3 times as many unique items and reached key tech-tree milestones up to 15.3 times faster.
Then there is the literal ten thousand, or close to it. A 2024 study of many-shot in-context learning put up to 8,192 examples into a single prompt for Google's Gemini 1.5 Pro. "Going from few-shot to many-shot, we observe significant performance gains," the authors wrote, and many-shot learning proved "effective at overriding pretraining biases." When human-written solutions ran short, they had the model write its own and kept only the ones that reached the right answer: they "select rationales that obtain the correct final answer." Showing the model unsolved problems alone also helped on some tasks, but the authors found the graded variant "more broadly effective."
The pattern across these papers is consistent. Where an agent improved with experience, something outside it was grading the attempts, and the grade was kept where the next attempt could use it. Where nothing outside graded the work, the self-correction study found no reliable improvement, and sometimes a decline.
Norvig's recipe carries over to agents with very little lost.
A task just beyond current ability. A production queue is whatever arrives, and the agent already handles most of it. Keep a set of tasks at the edge of what it gets right, the ones it fails some of the time, and run it there on purpose, apart from the traffic. Voyager's curriculum does this automatically. A team can do it with a spreadsheet. We looked at the same edge from the training side in The Zone of Proximal Development, where learners, human and machine, learn fastest when they fail about 15% of the time.
Feedback from outside the agent. Unit tests, an environment that reports success, an exact answer, a reviewer, a user's correction. The self-correction result is the warning label: a model that grades itself with nothing to check against does not reliably improve, and sometimes gets worse. The other side of that result is in Test-Time Compute, Five Ways: DeepSeek's R1 learned to check and correct its own reasoning while it was rewarded for getting things right.
Corrections that are kept and read. Reflexion's memory buffer, ExpeL's insights and Voyager's skill library each put the lesson where the next attempt will look. A correction written to a log that no later run retrieves does nothing for the next attempt.
Other people's programs. "Talk with other programmers; read other programs. This is more important than any book or training course," Norvig writes. For an agent, that means worked examples from people who know the domain, and the many-shot results suggest they keep paying off long past the handful a few-shot prompt holds.
A different number on the dashboard. Stop reporting interactions handled. Report how many were graded by something other than the agent, how many were corrected, and where the corrections went.
Norvig quotes the photographer Henri Cartier-Bresson: "Your first 10,000 photographs are your worst." Then he adds a dry aside: "(He didn't anticipate that with digital cameras, some people can reach that mark in a week.)" For anyone building agents, that aside is the whole story in miniature. The count has become almost free. A photographer with a digital camera can reach ten thousand frames in a week, an agent can reach ten thousand requests in about a day, and neither number says what was learned.
There is a hopeful half, and it comes from Hamburg. Ericsson and Pool grant that the Beatles did improve on those stages, because they played the same songs night after night and got feedback "both from the crowd and themselves." Performance starts to work like practice when the reaction comes back. An agent's crowd is the user who rewrites its answer, the ticket that reopens, the test that fails a week later. Each of those is a grade, if someone captures it and puts it where the next attempt will read it.
So the next time a dashboard, or a vendor, tells you an agent has handled ten thousand interactions, ask the question Norvig's recipe implies: how many of them was it ever told it got wrong? The ten thousand will arrive on their own. The loop is the part you build.
Chain of Consciousness keeps a tamper-evident, dated record of what your agents learned, decided and created, in hash-linked entries. Log each grade and each correction as an entry, and “where the corrections went” has an answer you can check.
pip install chain-of-consciousness npm install chain-of-consciousness
Or start without installing anything: Hosted Chain of Consciousness.