The pile, sorted
Most of this week is people measuring what AI does to other people’s work. Mercor’s CPAs averaged ~37% on month-end close tasks that Claude Opus 5 aced in under 10 minutes for 49× less money; Buzzard’s mathematicians are going through the stages of grief over AI proving hard theorems; and Matt Hodges has the studies showing an LLM can talk people out of conspiracy theories. Two of those are hopeful and one is a bill, and the benchmark is the one with the caveats attached - the tasks were built to stump models and stripped of everything that makes the job a job.
Then there’s the money. Bernstein and Cummings say the build-out can’t pay for itself; I think their headline number assumes the spending continues whatever happens to revenue. That’s the same argument as the build-out numbers last week, from the other side. Cloudflare’s Clef is the small version of the same question - the benchmarks say it beats Jev, Hailey’s one task says otherwise, and the honest answer is that it’s one person’s task.
The PID post is the odd one out, and also the one I’d most like someone to pick up: AI risk is a noisy derivative with no outer loop to put the answer in.
Through-lines from earlier weeks
- Clef is the Jev idea from Week of September 14 arriving from a second company, and the first outside results are already coming in.
- The bubble argument picks up the build-out and price-of-thought numbers from Week of September 21 .
To grieve or not to grieve
Kevin Buzzard walks through how mathematicians are taking AI proving hard theorems, using the five stages of grief. Denial is the Association for Human Mathematics, with Peter Scholze saying he’ll “die on that hill”; bargaining is a committee of senior mathematicians asking the labs to stop testing hard problems on proprietary models; depression is a PhD student whose stuck lemma got one-shotted by ChatGPT, wondering what the point is. Then he says he’s an optimist anyway.
His case: leaning on “it’s about human understanding” is a risky place to retreat to if machines are soon explaining proofs as well as finding them, and he’s frank that he’s in it for the theorem tally. Today’s machines are good at problem solving and haven’t shown much at theory building or choosing what to prove next, and finite resources mean they stop somewhere short of infinity. He also wants OpenAI to publish its pile of unreleased results rather than sit on them.
Maths is just first. What the mathematicians are working through - what’s left for humans to do, and how to get there from here - is coming for everyone else soon.
Matt Hodges on LLMs talking people out of conspiracy theories
There's growing experimental evidence¹˒²˒³˒⁴ that LLMs are unusually good at talking people out of entrenched false beliefs and conspiracy theories. ¹ pubmed.ncbi.nlm.nih.gov/39264999/ ² pubmed.ncbi.nlm.nih.gov/41179846/ ³ sciety.org/articles/act... ⁴ connecticut.adl.org/resources/re...
— Matt Hodges (@matthodges.bsky.social) 2026-10-03T19:00:34.427Z
Matt quote-posts an anecdote — someone watching relatives finally accept that the NRA, OAN and Newsmax were lying to them, with ChatGPT as one of the reasons — and backs it with studies. The big one is a Science paper : 2,190 conspiracy believers had a personalised, evidence-based chat with GPT-4 Turbo, and their belief dropped ~20%, still holding two months later. A follow-up in PNAS Nexus (N = 955) found it made no difference whether people thought they were talking to an AI or a human expert — the effect comes from the quality of the argument.
That’s the part I find hopeful. X and Facebook spread this stuff at scale; a patient thing that answers the specific claim you actually believe is a plausible counterweight. The replies have the obvious worry, and it’s a fair one: if persuasion works that well, it presumably works in either direction.
Max Woolf on an AI-made music video
The post is a video — a complete Europop music video, song and visuals both, made by an AI agent working from basically a single prompt and a pile of tools. The creator told it to imagine it was recording the music video from a Windows XP desktop, and let it use Midjourney videos as assets alongside Seedance. Max’s follow-up credits the original to anabology on X.
What makes it more than a stunt is that the whole thing is documented. The creator released the prompts and the assets, with an explanation of how it was made, in a Google Drive folder — which is how you find out how much of the result came from the prompt and how much from the tools wired up around it.
The replies are worth a scroll: from “wtf i love her” to “I am now worried about AI safety” to “this is the good slop (complimentary)”, with one person pointing out that meta-textual music about the weirdness of generative AI is about the only subject where the generated version is more fitting than a handmade one.
Human Baselines for Benchmarks: AI Now Outperforms Junior Accountants
Mercor hired 12 licensed CPAs — ~5.5 years’ experience on average — to do four month-end close tasks from their APEX-Accounting benchmark: dig through a company’s working files, find the right numbers, do the math, hand back a table. The accountants averaged ~37%, with most attempts taking 30–180 minutes. Claude Opus 5 scored 100% on all 20 attempts, each in under 10 minutes, at $0.21 per rubric criterion met vs $10.35 for the humans at median wage — 49× cheaper. Eighteen months ago no model cleared the human average.
None of this is much of a surprise to me — AI has already saved me thousands in professional services bills. Mercor are upfront about the limits: the tasks were built to stump models, so they’re full of compounding traps where one missed number tanks the score (the task authors expected juniors to land around 30%), and the setup stripped out coworkers, client context and most of what the actual job involves.
Introducing Clef: our open-source decision models, and new RL fine-tuning platform
Cloudflare has released Clef and Clef-flash, Apache-2.0 decision models in the same mould as Typesafe’s Jev: give them some state and a typed schema of questions, get back calibrated probabilities rather than generated text. They implement the Jev/System One API, so they’re a drop-in replacement. On Cloudflare’s runs Clef beats Jev on 8 of 10 benchmarks and 3 of 4 of Typesafe’s own workflow evals — but accuracy was never really Jev’s problem. Latency is: median 524ms for Jev, 209ms for Clef, and 39ms for Clef-flash, which stays within ~5 points of Jev on Typesafe’s workflow suite while being ~13× faster.
They get there by not generating anything — a frozen Qwen does a single prefill pass and a small routing head scores the valid schema choices in parallel. These are Cloudflare’s numbers, and Jev still wins When2Call and BRIGHT.
The first outside results aren’t as kind. Hailey ran Clef-flash against Jev on her own task:
at least with the flash version of this, no real benefit compared to jev. in fact, in situations where jev and clef disagree in my task, jev is right the majority of the time
— hailey (@hailey.at) 2026-10-02T08:34:06.613Z
She also says it hedges a lot more — less sure of itself, and still wrong on the hard ones. When someone pointed out that you can at least run Clef locally, she replied:
yes, but still at a much lower req/s
— hailey (@hailey.at) 2026-10-02T14:47:10.692Z
That’s throughput, not latency — and Cloudflare’s headline is latency. Both can be true: 39ms for one call doesn’t mean the box keeps up once you push a lot of requests through it. So far Clef looks weaker than its benchmarks suggest, with the caveat that this is one person’s task, on the flash model only.
Government as a PID controller
The ones that work look like cascade control. Slow outer loop sets the target & basically never changes. Fast inner loop picks the means & gets yelled at by voters.
— Craig Hughes (@craig.rungie.com) 2026-09-27T16:30:42.827Z
Tim Kellogg asked which organisations are actually built to think long-term — successfully, with evidence — and I answered in control theory. P-heavy government fails two ways: crank the gain and you oscillate (the IRA in ‘22, gutted in ‘25); keep it low and you get steady-state error — the crisis fades, the push fades, the problem stays half-solved.
The ones that work look like cascade control: a slow outer loop sets the target and basically never changes, and a fast inner loop picks the means and gets yelled at by voters. Norway’s oil fund and the UK’s legally binding carbon budgets have that shape, and so does the Swiss debt brake:
Swiss debt brake. 85% of voters put it in the constitution in 2001. Spending cap flexes with the business cycle & any overspend goes in a "compensation account" that has to be paid back later. Literally an I term. Swiss federal debt/GDP 25% → 13.5% over 18 years to 2019.
— Craig Hughes (@craig.rungie.com) 2026-09-27T16:30:42.830Z
Obviously this bites for AI growth control and safety. D is the hard term — it amplifies noise, so everyone filters it hard, and usually they’re right to. AI risk is pure D: a noisy derivative, with no outer loop to put the answer in.
AI Companies Are In a Race Against Time
Jared Bernstein and Ryan Cummings re-up their AI-bubble call with a DCF model of six hyperscalers. Since 2024 they count ~$1.2T of AI capex against $277B of AI revenue — just paying that back takes $2.4–3.8T over six years, 13–45× today’s incremental AI revenue. Add planned spend of $5.7T over 2026–30 and the bill becomes $13.1–18.7T of revenue in the next ten years.
I argued with it on Bluesky — mainly that the headline number presupposes the revenue keeps growing:
Their scariest number assumes these firms spend all $5.7T planned for 2026-30 no matter what. They won't - if AI revenue disappoints, they slow the build-out. So $13.1-$18.7 trillion in revenue required is already assuming that...revenue shows up in the meantime to sustain investment rate.
— Craig Hughes (@craig.rungie.com) 2026-09-27T19:59:14.755Z
Revenue is also moving faster than they imply — Anthropic’s run rate went ~$9B → $65B in 7 months — and their harshest case counts zero for AI making search cheaper or ad targeting better unless a company reports it as a separate line. They’re right that some of the fastest-growing revenue is circular (labs spend it on compute from hyperscalers that own stakes in them). But Google, Microsoft, Meta & Amazon mostly pay for capex out of cash flow, so a miss there means write-downs. The stress is at the levered edge — CoreWeave’s CDS is ~855bp, Oracle’s at BBB-.