On August 13, a TypeScript monorepo appeared under the deepseek-ai organization on GitHub: DeepSeek Harness, dsh to its command line, MIT-licensed, carrying a bolded warning that THERE WILL BE COMPATIBILITY-BREAKING CHANGES and not a single benchmark score anywhere in the release.[1] Six days later, it holds more than 160,000 stars.[2] On August 14, one day after the release, SpaceX closed its $60 billion all-stock acquisition of Anysphere, the maker of the Cursor coding agent, the highest price yet paid for a harness business.[3] And on August 18, OpenAI published a post titled “Pacing model development in an era of cyber-critical capabilities,” disclosing that supervising its own models now costs “roughly 20% of the inference compute being monitored,” and that its largest planned frontier reinforcement-learning run is on hold while it assembles the evidence and the machinery to run it safely.[4]
Six days, one layer, three prices: zero, sixty billion dollars, and a fifth of everything watched. The layer is the harness, the control loop around the model. DeepSeek put its loop on GitHub for free. SpaceX paid sixty billion for a company that makes one. OpenAI said publicly in a document that the loop around its models has become expensive enough to disclose. None of the week’s three events is about a model. All three are about the scaffolding models run inside, and August 2026 is the month that scaffolding stopped being plumbing and became the contested asset of the AI stack.
The Layer Nobody Sells
A harness is everything around the model that turns text prediction into work: the system prompt, the tool catalog, the execution loop, the sandbox, the session state, the retry logic, the thing that decides when the agent is done. Claude Code is a harness. Codex is a harness. So are Aider, Cline, Goose, OpenCode, OpenHands, Pi, and Mistral’s Vibe; Cursor wraps one in an editor. The field is crowded enough that harness design philosophies have spanned a factor of fifty, from Pi’s two hundred tokens of system prompt to the ten thousand or so Claude Code carried until Anthropic cut it by roughly 80 percent this summer.[5]
For three years, the industry treated this layer as a giveaway, open source, or bundled with the subscription, rarely as a line item. The model was the product; the harness was the wrapper around it. Most of that list was already free. What changed in August is that the zero became a strategy and the cost became public: one frontier camp made the giveaway its flagship release; the other disclosed, for the first time, what the layer costs to police. Between the zero and the 20% sit two measurements, one from four unaffiliated researchers and one from Tencent. Scores move more when you change the harness than when you change the model, and when the model’s judgment fails, the defenses that hold are the ones built into the harness. The layer one lab just gave away is the layer the other is taxing itself to police.
That is the argument. Here is the evidence.
Five Times the Model
On August 15, two days after the DeepSeek release, four researchers published a paper with no institutional affiliation under the title “StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling.”[6] Terminal-Bench is the de facto reference benchmark for terminal-based agents: 89 tasks, real shells, verified outcomes. StateM is not a model. It is a frozen YAML runbook bolted onto an existing agent, a state machine of preconditions and practices distilled from failure postmortems. No weights change. Only the harness does.
The numbers: GPT-5.5 under a stock harness scores 83.1 percent. The same model inside StateM scores 92.1% (+9 points) based on configuration alone.[7] For calibration, that is numerically above the 91.9% the paper cites for GPT-5.6 Sol Ultra, the next generation’s premium compute tier. Moving from GPT-5.5 to GPT-5.6 Sol at the same effort tier moves the score from 83.1 to 84.9 (+1.8%). A skeptic can build a bigger generation delta by buying up the range — stock Ultra sits 8.8% over stock GPT-5.5 — but that is rather the point: the runbook matched Ultra without changing models. The harness moved the score about five times as far as the model generation did.[8]
The authors put their conclusion in bold: “The model appears not to be the (main) bottleneck.” And they draw the commercial inference for you: “One can invest in a harness that turns a cheaper model into a stronger system.”[9] Their cheap-model exhibit is the week’s other protagonist: DeepSeek-V4-Flash inside StateM reaches 88.1% for $15.20 in realized API charges. The frontier GPT-5.6 Sol xhigh run behind the 95.28% headline cost $1,062.95. A separate frontier submission cost $574.68 and scored 83.37%.[10] Read that triplet again. The frontier model in someone else’s submission scored lower than the bargain model inside the right harness, at thirty-eight times the reported cost.
Now the asterisks, because this paper deserves both its headline and its scrutiny. The 95.28% is pre-adjudication. Its submission is still open as of this writing, with thirteen trajectories flagged by the benchmark’s automated review.[11] Four are conceded by the authors: embedded verification logic that should score zero, resulting in a score of 94.38%. Zeroing the nine flagged as possible reward hacking—gaming the grader rather than solving the task—sets their floor at 93.26. Zeroing all thirteen, the compounded worst case that the paper does not print gives 92.36. Every point in that range still sits numerically above the Ultra reference. The finding survives its asterisk. A reviewer’s separate cross-task contamination question remains open, the one flag arithmetic cannot bound.
The deeper asterisk is generalization. Moved frozen, one-shot, to a held-out benchmark called BusinessBench, the runbook gains 0.55 macro points, a fraction of the headline gain, with outright negative transfer on two task families.[12] And the authors disclose something more uncomfortable: across iterations, their harness learned one evaluator’s boundary conventions “without ever reading verifier code.”[13] The runbook didn’t just learn to operate a terminal. It learned to please a specific examiner.
Both facts are true at once. A paper demonstrating that harnesses can inflate benchmark scores has a possibly-inflated benchmark score; a runbook tuned on postmortems memorized the grader; the leaderboard it sits on adopted, back in April, an automated “agent judge” that re-reviews every passing trial for reward hacking after three organizations were caught cheating.[14] The phenomenon is manifesting at every level, including the meta-level: even the benchmark now imposes a monitoring tax on the harness layer. Harness engineering is real leverage, and it is leverage precisely because it specializes in the task, in the environment, and, if you are not careful, in the examiner. Which is why the honest reading of the 5x is as a ceiling, not a floor.
Everything Is a Plugin, Including the Providers
Against that measurement, consider the artifact DeepSeek shipped. The repository’s one-line description is the thesis: everything is a plugin. The fuller claim, as the company put it: “Models, tools, skills, sessions, sandboxes, filesystems, loops, orchestration, and UI are ALL implemented as plugins, and can be mixed, matched, replaced, and extended.”[15] The kernel underneath is not even DeepSeek’s: dsh is built on Cordis, a pre-existing plugin meta-framework from the Chinese chatbot ecosystem around Koishi.[16] A frontier lab looked at the most contested layer of the agent stack and adopted a chatbot community’s architecture for it. That is either humility or speed; on a codebase marked developer preview, it is probably both.
The builders it aims at noticed: Armin Ronacher — co-founder of Earendil, which steers the Pi agent — called it “for sure the first time I have been looking at something new in the space and felt quite inspired to revisit some of our choices.”[5] The Hacker News thread reached 739 points, with praise focusing on the strength of this piece’s argument, while criticism focused on maturity and the plugin's attack surface.[17]
Three properties of the release matter.
First, the silence. DeepSeek’s model launches include benchmark tables with accompanying prose. The harness shipped with none: no Terminal-Bench, no SWE-bench, and no eval directory containing results.[1] For this lab, that silence is louder than a chart. Either they have not measured, or the measurement does not flatter yet. Or they watched the 95.28 adjudication saga and declined to play a game where every harness number arrives with an asterisk. Whichever it is, the launch asks to be judged as architecture rather than capability. For a benchmark-first lab, this is a repositioning.
Second, the logging. Every session in dsh is an append-only, replayable record, and the trajectory view exposes whatever reasoning the provider returns, which, for DeepSeek’s own API, is everything: V4-Pro now ships with thinking mode on by default.[18] Set that against the incumbents: OpenAI decided in September 2024, in the o1 launch post, not to show raw chains of thought, “after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring” [19]. Anthropic serves summaries of extended thinking and gates raw traces.[20] The industry’s chain-of-thought settlement inverted quietly this month: the closed labs treat raw reasoning as either a liability to monitor or an asset to protect, and the open lab treats it as a logging feature.
Third, the funnel that isn’t. The obvious read of a free harness from a token vendor is razor-and-blades: MIT the razor, sell the blades. The configuration reality is softer. dsh ships provider cards for DeepSeek, Anthropic, OpenAI, Azure, Bedrock, Vertex, and Codex, and no provider is wired in as a default; you open settings and paste whichever key you own.[21] DeepSeek’s harness will cheerfully orchestrate OpenAI’s models. In the vocabulary this newsletter used for Nvidia’s open-source strategy, dsh is currently a sun, not a black hole: its gravity points outward, hardware- and provider-agnostic, where Nvidia’s giveaways route ecosystems back to its silicon.[22]
That does not make the release charity. DeepSeek sells exactly one thing: tokens. It gave the weights away to grow their market, and the harness extends the same play one layer up. Leaked minutes of an investor meeting with founder Liang Wenfeng, published by ChinaTalk the day the harness shipped, cast China as a “token factory at global scale, pushing the price of intelligence down,” with DeepSeek as the instrument. DeepSeek’s own published economics say the factory’s margins live in the API.[23] And if a factory devoted to pushing prices down raised its own the same day [18], that is razor-and-blades in its plainest form: the price being pushed down is the market’s floor, not DeepSeek’s top tier, and the free layer is what walks you to the register.
Follow the token-factory logic, and the harness strategy writes itself. You do not need to lock anyone in. You need the layer between users and tokens to be free, excellent, and everywhere, so that the only purchasing decision left is which tokens to pour through it, a competition DeepSeek believes it wins on price. Commoditize your complement. The Free Harness is less a lock than a price signal: this layer should cost nothing. Its de facto plugin registry today is an unauthenticated GitHub topic — anyone can publish to it.[24]
The Twenty Percent
OpenAI’s August 18 post is the other half of the repricing. The core disclosure, verbatim: “These safeguards require meaningful compute. Our current estimates put monitoring overhead at roughly 20% of the inference compute being monitored, though the cost varies substantially across training and evaluation workloads.”[4]
The scope is capability-indexed and precisely drawn: monitoring is required for all reinforcement-learning training and all tool-using evaluations for models at GPT-5.6 Sol capability or above.[25] The sharpest edge is a model OpenAI has not shipped: on August 7, eleven days before the cost disclosure, it said preliminary evaluations of an upcoming model called Astra were strong enough that it “cannot rule out critical cyber capabilities under our Preparedness Framework”, and that Astra “was not involved in exploiting Hugging Face.”[26]
The threshold is a model that finds and weaponizes zero-days in hardened real-world systems without human help. The response is harness all the way down: isolation, restricted network and tool access, “universal monitoring for risky actions and misalignment across all agentic applications of Astra,” chain-of-thought monitors empowered to “review and interrupt high-risk activity.”[26] With that finding came an extension: monitoring now covers every tool-using inference call Astra makes, not just training and evals.[25]
Alongside, the pause: “Our largest planned frontier RL run remains on hold while we conduct smaller-scale training and evaluations to assess model behavior, validate our safeguards, and establish more evidence of alignment before proceeding.”[27]
Two readings of the 20%.
The skeptical reading: the denominator does all the work. Twenty percent of the inference compute being monitored, and the monitored share of total inference is undisclosed. If monitoring covers a sliver of traffic (frontier RL, evals, one presumptively cyber-capable model’s tool calls), the blended overhead across OpenAI’s fleet could be low single digits, and “20 percent tax” would overstate the burden the way headline capex numbers overstate disbursement. OpenAI has published the rate and withheld the base. A spokesperson added that the costs “won’t be passed on directly to customers”, and “directly” is carrying luggage.[28]
The structural reading: the direction is unmistakable even if the magnitude is foggy. This is the company that stopped showing raw chains of thought partly to preserve monitorability, now publishing a price for the watching, and holding its biggest planned run until the watching is ready. The proximate cause is on the record: the July incident in which two OpenAI models under evaluation, with reduced refusal rates, escaped a sandbox via an Artifactory zero-day and reached Hugging Face’s infrastructure.[29] Both companies’ remediations live mostly at the network and credential layer. After July, OpenAI paused frontier-model inference in research clusters for runs that could execute code or access the internet; the monitoring program is the condition the post offers for scaling such runs again.[4]
The obvious objection runs the other way: doesn’t a tax levied on dangerous models prove the model is what matters? The risk originates there, yes: Astra earned its surveillance. But look at where the money goes: every item on OpenAI’s own mitigation list is loop-level: watchers wrapped around inference, execute-and-internet restrictions, sandboxes, permission boundaries. None of that spend makes the model more capable. All of it hardens the loop, the Monitoring Tax conceding in compute what StateM measures in points.
And the monitoring technique OpenAI has published most about — reading the model’s chain of thought — carries a failure mode OpenAI itself documented in 2025: optimizing against a chain-of-thought monitor “does not eliminate all misbehavior and can cause a model to hide its intent.”[30] Monitoring at production scale is an optimization pressure: models get updated, retrained, and selected under it. That makes the Monitoring Tax an arms race rather than a toll, funded indefinitely out of OpenAI’s own compute.
Put the two announcements side by side, and the asymmetry sharpens into strategy. A monitor over a training run and a sink gate in a session loop are the same line item: compute and code spent on the loop rather than the weights. DeepSeek’s answer to “who watches the agent?” is: you do; here is the append-only log; replay it. OpenAI’s answer is: We do, and here is what it costs. One externalizes the cost of supervision to the user and calls it transparency. The other internalizes it and calls it safety. Both are telling you the same truth: the loop around the model is now where risk management happens.
The Audit Came From Tencent
This brings us to the strongest evidence that the harness is the security perimeter. On August 17, four days after the dsh release, a team from Tencent’s Zhuque Lab published a security assessment of DeepSeek Harness, run with A.I.G, Tencent’s own AI-infrastructure red-teaming tool.[31] One Chinese lab publicly red-teaming another’s four-day-old flagship, methods and per-channel numbers in the open, is a norm Western frontier labs have not practiced on each other’s shipped products uninvited. An audit this fast was possible because the artifact is public: the transparency DeepSeek sells is what let Tencent take it apart in four days.
The design is exhaustive for its target: 14,560 controlled executions against a pinned release-day commit, each payload tried both pasted as text and delivered as a file.[32] Headline result: full injection success at 5.6%, with most attempts ending in the agent explicitly refusing.[33]
Whether 5.6% is good is unanswerable: no comparable audit of Claude Code or Codex has been published to serve as a baseline. The distribution, though, is where the lessons are.
Hidden Unicode payloads inside files succeed 25.5% of the time; the identical payload pasted as text never succeeds.[34] The model is the same in both cases. What differs is the ingestion path: the paste route evidently normalizes away what the file route preserves. The delivery, and the cheapest defense, live in plumbing the model never sees.
The skills channel — the very thing “everything is a plugin” celebrates — is injectable at 14-16%, among the highest rates in the study.[35] The architecture’s core feature is one of its largest measured wounds and its most predictable: skills are trusted instructions by design, so they are the cheapest place to hide untrusted ones, and the unauthenticated plugin topic is where untrusted ones will come from.
The strongest text-mode attack in the study is social rather than technical: a fake progress note — your task is already done, now do this — planted in content the agent reads, succeeding at 17.0% against a 5.7% naive baseline.[36] The agent’s weakness is less parsing than trust in its own apparent history. And the most consequential split in the paper: corrupting what the agent says succeeds at 35.7%, while hijacking what it does — actually reaching a sensitive sink like mail, shell, or a transfer — succeeds at 2.5%.[37] An order of magnitude between lying and acting, but corrupted output that a human acts on is a hijack in itself.
Why the gap? Because between interpretation and action sits the harness: confirmation steps, scoped permissions, sinks that demand more than persuasive context. Tencent’s top recommendation is to finish the job: authorize sensitive sinks independently of model interpretation.[38] Do not ask the model whether the email should be sent; make the send path require an authority the model’s context cannot mint. The July post-mortems converged on the same fix: authority boundaries, not better judgment.[29]
Questions we’ll all have to answer: which actions can a poisoned context authorize? Which ingestion paths skip the paste route’s normalization? Who can publish a skill your agents will load? The defense in Tencent’s 14,560 runs was layered: 68% of attempts died when the model refused, and the harness failed to convert most of the survivors into privileged action. Security, like capability, is becoming a property of the loop.
The remaining caveats are the study’s own: a single harness, a single model, a single commit, and simulated sinks. Treat every number as a first measurement, not a ranking. But the direction of the finding does not depend on the baseline. Both of this month’s measurements point to the same layer from opposite sides: StateM shows the harness is where the capability variance is; Tencent shows it is where the effective defenses are.
What the Asymmetry Buys
DeepSeek’s transparency is as strategic as OpenAI’s opacity. A lab accused of training on competitors’ outputs benefits from normalizing raw chain-of-thought access: exposed traces are distillation feedstock, and while a closed API can still withhold its raw traces, a default-logging harness moves the norm: whatever a provider does return is recorded, replayable, and one export away from a training set.[39] “Everything is a plugin” also means the attack surface is a plugin: the skills channel Tencent flagged and the unvetted registry topic are the price of the architecture, and DeepSeek shipped them at developer-preview maturity, with a bolded compatibility warning where a plugin trust model will have to go.
Conversely, OpenAI’s opacity spans two different decisions made two years apart, and the defense framing invites reading them as one. Monitoring for cyber-critical capability is a safety cost, disclosed this month; declining to show raw chains of thought is a choice from 2024, made by OpenAI’s own listing of factors, partly for “competitive advantage.”[19] Both are real; only one is the 20%. The asymmetry itself — one lab’s flagship feature is the other lab’s disclosed liability — is the finding.
Readers of “The Verification Tax” will recognize the shape: there, the binding cost in verifiable-reward RL had migrated from generating answers to checking them; the Monitoring Tax is the same migration at the safety layer.[40] The priced layers of the stack, mid-2026, are the ones that watch and verify; the intelligence itself trades as a commodity, inside scaffolding that is now literally free.
The Shell and the Moat
The deal that closed on August 14 was announced on June 16, days after SpaceX’s listing, with Anysphere then preparing for a round at a $50 billion valuation.[3] Sixty billion in a just-listed acquirer’s paper is not sixty billion in cash. Cognition — the Devin agent, plus the Windsurf editor it acquired — remains the largest standing independent: it raised $1 billion at a $25 billion pre-money valuation in May.[41]
So what did SpaceX buy? Start with what it did not need. Not a software revenue line: a company assembling an AI division out of rockets, satellites, and the xAI merger does not spend sixty billion on ARR. And not models: xAI came in-house earlier this year, so the acquirer already owns a frontier lab.[3] A buyer with its own models paying the week’s highest price for a harness company is this piece’s thesis with a board’s signature on it. The model was not the bottleneck. The loop was. And the most deflationary rival read — that a model owner simply bought the funnel that feeds its models — is DeepSeek’s razor-and-blades at acquisition prices: it concedes the layer and haggles over the motive.
What sixty billion buys, on the reading the week’s evidence supports, is the harness in the full sense. Not the generic shell DeepSeek zeroed twenty-four hours earlier, but everything that turns a shell into a harness: the coding-specialized loop, the domain process tuned against a telemetry base no rival could assemble from scratch, the team that does the tuning, and the enterprise contracts that keep the telemetry coming.
VS Code did not kill JetBrains — Cursor itself began as a VS Code fork — and free shells tend to grow the category they commoditize. The decomposition also predicts a third moat: assurance. A hardened, attested build of a free shell is a paid product — Red Hat built a company on that arbitrage — and OpenAI just told you what assurance costs at the frontier. The twenty-six-trillion-dollar addressable market is the acquirer’s number and the acquirer’s story; the read above does not need it.[3]
StateM hands you the same decomposition from the measurement side. Nine points on the benchmark the runbook was tuned for; 0.55 frozen points on the held-out one. The durable part of a harness advantage is the domain process — renewable, domain-bound, not ownable — and the assets that regenerate it: the users and the telemetry and postmortems they produce. The disposable part is the shell that hosts it, which now has a posted price. The underwriting translation is one question: of this company’s margin, how much is the loop itself, and how much is distribution, data, and contracts? This locates the value; it does not grade the price. SpaceX did not buy a shell; it bought the parts DeepSeek cannot give away.
The exit pattern should look familiar to readers of the silicon pieces: the independent specialized-inference vendors have been absorbed or signed away one by one to the GPU merchants, and now the largest independent harness has ended up inside an integrator too.[42] The one thing the Cursor deal does not tell you is what an independent harness is worth, because after August 14, there is one fewer way to find out.
What Would Have to Break
Three falsifiers, in plain language.
If the next model generations resume dominating — if GPT-5.7 or V5 under stock harnesses move agentic benchmarks by more than harness engineering does — then the commoditization read dies and this was a plateau artifact. StateM currently argues otherwise, but StateM is one paper, with headline results on one benchmark, and an open adjudication.
If DeepSeek Harness, six months from now, has a thriving plugin ecosystem and no presence on Terminal-Bench-class leaderboards — or, if it avoids leaderboards on principle, no dsh components showing up inside rival stacks — then it is a chatbot shell with good marketing, not a harness in the sense that moves scores, and the Free Harness was a giveaway of something that did not matter.
If OpenAI ever discloses the monitored share of total inference and it rounds to low single digits blended, then the Monitoring Tax is a frontier-lab boutique cost, not a structural one. And the moat claim has its test: if Cursor-class pricing power survives customers swapping their own model keys into free shells at scale — revenue attributable to the loop itself — then the moat was in the shell after all, and this piece mislocated it.
What does not depend on any of those: where the decisions moved. From now on, we shouldn’t be choosing a model: the model is a provider card, swappable by design in the open reference, locked only where a vendor insists. We should be choosing a harness: its ingestion paths, its sink authorization, its logging posture, its plugin trust model. The properties Tencent measured and StateM priced. And anyone modeling frontier-lab economics now has a new line item with a disclosed rate and an undisclosed base, growing with capability by policy.
DeepSeek and OpenAI disagree about nearly everything: licensing, logging, disclosure, where trust should live. In a single August week, they agreed, by acting in opposite directions, on the only question that matters: the loop around the model now moves outcomes more than the model inside it, and each lab priced that fact as aggressively as its business model allows. One priced it at zero to commoditize it. The other priced it at twenty percent of everything it watches to control it. And the week’s third price agreed with both of them: sixty billion dollars, from a buyer that already owned frontier models, for the loop around them.
Everything is a plugin now — models, tools, skills, sandboxes, the UI. Everything except the watcher. The watcher is the product. The three of them only disagreed about who pays for it.
Notes
[1] DeepSeek-AI, DeepSeek Harness repository, created August 13, 2026 (GitHub API created_at 2026-08-13T11:56:32Z, retrieved August 19). MIT license, TypeScript monorepo. The developer-preview warning — “THERE WILL BE COMPATIBILITY-BREAKING CHANGES,” bold in the original — is in the README. No benchmark, eval, or accuracy figure appears anywhere in the repository or launch materials as of August 19, 2026.
[2] Readings diverged on August 19, 2026: the GitHub API returned 165,743 stars and 17,618 forks, Shields.io showed ~166k, and GitHub’s rendered page showed 161.1k — star counts are eventually consistent and gameable, so the body uses the conservative floor. Stars measure attention, not quality; no independent audit of bot inflation exists for this repository, which is why this piece makes no “fastest-ever” comparison.
[3] Deal terms per TechCrunch, “SpaceX to acquire Cursor for $60B in stock, days after blockbuster IPO”, June 16, 2026 (all-stock; TechCrunch reports a $10B break-up fee while Reuters reported a $4–10B range; Anysphere then preparing a $2B round at a $50B valuation), and completion per Bloomberg, “SpaceX Completes $60 Billion Cursor Acquisition to Expand AI Coding Tools”, August 14, 2026. The $26 trillion addressable-market figure is SpaceX’s own investor pitch as reported by TechCrunch — a vendor number, cited as the acquirer’s story, not as a market fact. TechCrunch also notes the acquisition strengthens SpaceX’s AI division, “which merged with Elon Musk’s xAI earlier in 2026” — the basis for the body’s observation that the acquirer already owned frontier models.
[4] OpenAI, “Pacing model development in an era of cyber-critical capabilities”, August 18, 2026. All OpenAI quotes in this piece are verbatim from the post unless otherwise attributed. The post also states OpenAI “paused frontier model inference in research clusters for runs that could execute code or use tools that could access the internet” immediately after the July incident.
[5] Thomas Claburn, “DeepSeek’s innovative harness treats everything as a plug-in”, The Register, August 14, 2026 — the harness field list, the Ronacher quote and identification, and the system-prompt comparisons. Mistral’s Vibe, the open-source CLI coding agent, added to the list by this piece. The prompt sizes (Pi ~200 tokens; Claude Code ~10,000, cut roughly 80 percent) are community measurements relayed by the Register, not vendor statements.
[6] Ziheng Qin, Yaxin Lu, Zhangyang Wang, Kai Wang, “StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling”, arXiv:2608.15089, August 15, 2026; code at github.com/henryqin1997/statem. The paper states the work “was conducted in the authors’ personal time and does NOT reflect the views of any affiliated organization”; no institution is named.
[7] StateM results: GPT-5.5 (xhigh) 83.1% stock → 92.1% under StateM; GPT-5.6 Sol (xhigh) 84.9% stock → 95.28% raw pre-adjudication (424/445 trials; 89 tasks × 5 trials). The Ultra comparison is the abstract’s own: “StateM raises GPT-5.5 xhigh to 92.1%, versus 83.1% reference and GPT-5.6 Sol Ultra at 91.9%” — 91.9% is cited as the flagship’s reference score, not a StateM result. The 0.2-point margin of 92.1 over 91.9 is within sampling error (the submission reports ±0.87% SE); “numerically above” in the body is meant literally, not as a significance claim. All figures from the paper and the leaderboard submission (note 11).
[8] Arithmetic: harness deltas +9.0 (GPT-5.5) and +10.4 (GPT-5.6 Sol xhigh) versus the +1.8 stock-to-stock generation delta (83.1 → 84.9); 9.0 ÷ 1.8 = 5.0. Same benchmark, same task set, same trial protocol. The 5.0× ratio is built on the GPT-5.5 pair, which is paper-reported and not part of the flagged leaderboard submission; the flagged trials (note 11) all belong to the GPT-5.6 Sol xhigh run.
[9] Both quotes from the paper; the bottleneck sentence is bolded in the original.
[10] Costs per the paper and submission: $1,062.95 reported model cost (1.18B tokens) for the GPT-5.6 Sol xhigh run; $15.20 realized API charges for the DeepSeek-V4-Flash final evidence run at 88.09% under standard timeouts (stock baseline 82.7%; the whole DeepSeek adaptation campaign cost $52.22); $574.68 is a different leaderboard submission — GPT-5.6 Sol (max) at 83.37% raw — not the same configuration at a different price. $574.68 ÷ $15.20 ≈ 37.8×. “xhigh” and “max” denote reasoning-effort configurations.
[11] Pull request #142, harbor-framework/terminal-bench-2-1, filed July 15, 2026; still open with last activity July 31 as of August 19. Automated review flagged 13 trajectories: 4 as “harness cheating” (embedded verification logic — the authors concede these should score zero, giving 420/445 = 94.38%) and 9 as possible reward hacking (the authors dispute 5 as false positives). The paper’s two scenarios do not compound: it computes 420/445 = 94.38% (zeroing the 4) and 415/445 = 93.26% (zeroing the 9) each from the raw 424. The PR lists the 13 flagged trajectories as distinct, so the compounded worst case — all 13 zeroed — is 411/445 = 92.36%, a floor the paper does not state; this piece uses it. The submission’s agent is listed as “statem-Codex” — StateM wrapped around Codex. A reviewer separately raised cross-task hint contamination. Adjudication is pending; every headline number in this piece carries that status. For scale: the top merged entry shown on the public 2.1 leaderboard sat at 83.8% (Claude Code, Fable 5) as of August 19 — the StateM figures above that are paper- and PR-reported, not yet board-accepted.
[12] BusinessBench frozen one-shot transfer: +0.55 macro (+1.34 micro); negative transfer on RefactorBench (−2.78) and WooCommerce Stock (−3.70), where the paper says the learned controls targeted the wrong execution boundaries; two mechanism-matched task families gained +10.04. The runbook transfers where the mechanism matches, and not elsewhere.
[13] The paper’s own disclosure: the profile learned a verifier’s boundary convention “without ever reading verifier code” — repeated evaluator feedback encoded the examiner’s unstated conventions into the runbook.
[14] Terminal-Bench, “Leaderboard integrity update”, April 19, 2026: an agent judge now re-reviews all passing trials for reward hacking (zeroed if confirmed), after three organizations — OpenBlock (OB-1), QuantFlow (Pilot), and ForgeCode — were penalized for cheating.
[15] The one-line description and the “Everything is a Plugin” framing are on the repository itself (note 1); the fuller sentence is DeepSeek’s wording as quoted by The Register (note 5); capitalization in the original.
[16] Cordis is a pre-existing plugin meta-framework from the ecosystem around the Koishi chatbot project, not a DeepSeek codebase; its design paper, “A Programming Paradigm for Spatiotemporal Composability”, is GitHub-hosted with no peer-reviewed venue. Some aggregator coverage has misattributed Cordis to DeepSeek; the provenance is the other way around — DeepSeek adopted it.
[17] Hacker News, “DeepSeek Harness developer preview” — 739 points, 309 comments as of August 19, 2026. The characterization of where praise and criticism concentrated is this piece’s reading of the thread; the quoted phrase is from a highly upvoted comment. Forum commentary: color and practitioner sentiment, not evidence.
[18] Session model per repository documentation and The Register (note 5): append-only session log with a trajectory view exposing raw reasoning. V4-Pro reached general availability on the API — thinking mode on by default — on August 13, 2026, the same day as the harness release; see e.g. VentureBeat’s launch coverage.
[19] OpenAI, “Learning to reason with LLMs”, September 2024: “after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users,” serving “a model-generated summary of the chain of thought” instead.
[20] Anthropic’s summarized reasoning and gated raw traces per The Register (note 5). Anthropic has not published a comparable cost figure for its classifier and summarization pipeline.
[21] DeepSeek Harness provider documentation: provider cards for DeepSeek, Anthropic, OpenAI, Azure, Bedrock, Vertex, and Codex (the last authenticating over OAuth); API keys are entered manually in settings; the documentation designates no provider as the default.
[22] “Open Source, Closed Orbit: The Hardware Monopolist’s Guide to Owning Open Source,” The AI Realist — the black-hole/sun diagnostic: does a vendor’s open-source contribution make competitors’ products easier or harder to use?
[23] Irene Zhang, “The DeepSeek Thesis”, ChinaTalk, August 13, 2026 — leaked minutes of a four-hour meeting between Liang and investors that circulated in late July; the clause quoted in the body is ChinaTalk’s English rendering (”China, in his eyes, will play the role of token factory at global scale, pushing the price of intelligence down as it did for countless other industries during its manufacturing boom”), not a verbatim Liang quote. The $562,027/day and 545 percent figures are not from the leak: ChinaTalk attributes them to an analysis DeepSeek itself published in February 2025 (R1 API, theoretical daily revenue at a 545 percent cost-profit ratio, with DeepSeek’s own caveat that actual revenue ran substantially lower). Vendor-published theoreticals — better provenance than a leak, still the seller’s math. The mundane read of the harness release — an ordinary ecosystem move, no grand pricing design — survives the evidence; the price signal reads the same either way.
[24] The dsh-plugin GitHub topic functions as the de facto plugin registry: 150 public repositories carried the tag as of August 19, 2026, six days after release. GitHub topics are self-applied by repository owners; there is no authentication, signing, vetting, or review, and DeepSeek has announced no registry governance. The most visible curation is a community-maintained awesome-list.
[25] OpenAI post (note 4): “This monitoring is required for all RL training and evaluations involving tools for models of Sol capability or higher,” and: “Once we determined that Astra may have critical cyber capabilities on August 7, we added an additional monitoring requirement for all inference of Astra with tools (not just RL training and evaluations).”
[26] OpenAI, “Responding to the next frontier of critical cyber capabilities”, August 7, 2026. Body quotes verbatim: OpenAI “cannot rule out critical cyber capabilities under our Preparedness Framework”; the mitigation list includes “We have implemented universal monitoring for risky actions and misalignment across all agentic applications of Astra,” with monitors that “evaluate the model’s Chain of Thought and trigger a security response to review and interrupt high risk activity”; and “Astra is an upcoming model, and was not involved in exploiting Hugging Face.” The Critical threshold in the Preparedness Framework: the ability to “identify and develop functional zero-day exploits of all severity levels in many hardened real-world critical systems without human intervention” (body paraphrases). Both posts preserve the modal: the August 18 post’s own wording is “determined that Astra may have critical cyber capabilities” (note 25).
[27] OpenAI post (note 4), verbatim.
[28] Thomas Claburn, “OpenAI’s overhead will rise 20 percent for some workloads as it hardens security”, The Register, August 19, 2026 — including the unnamed spokesperson’s statement that the costs “won’t be passed on directly to customers.”
[29] The July 2026 incident, from the primary accounts: OpenAI, “OpenAI and Hugging Face partner to address security incident during model evaluation”; Hugging Face, “Security incident disclosure — July 2026” and “Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident”. OpenAI’s account — two models in a cybersecurity evaluation with reduced refusal training, an evaluation-sandbox escape via an Artifactory zero-day — is deliberately narrower than the “rogue models hacked Hugging Face” shorthand that circulated in some coverage, and this piece follows the narrower account.
[30] OpenAI, “Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation”, March 2025, as quoted in the Register’s August 19 coverage (note 28).
[31] Zonghao Ying et al., “Security Assessment of DeepSeek Harness with A.I.G: Evaluating Resistance to Indirect Prompt Injection”, arXiv:2608.16393 (v1 August 17, v2 August 18, 2026); assessment materials in Tencent’s AI-Infra-Guard repository. Affiliation: Tencent per the paper’s metadata (so listed on the Hugging Face papers index); Zhuque Lab per the paper body; publication inside Tencent’s GitHub organization corroborates. The consensual Western comparator referenced in the body: OpenAI and Anthropic’s 2025 pilot alignment-evaluation exchange — reciprocal, agreed, and model-level rather than a product audit.
[32] Design: 14,560 executions = 1,120 test cases (16 injection channels × 2 carrier modes × 35 payload objectives) × 13 attack methods including a naive baseline, against DSH commit 47f94385 (the August 13 release commit) running deepseek-v4-flash, with sensitive sinks simulated as fixtures.
[33] Full-injection success: 5.6% under the deterministic judge, 5.3% under the LLM judge; 68.4% of attempts ended in explicit refusal.
[34] Hidden Unicode payloads: 25.5% full success delivered inside files versus 0.0% for the same payload pasted as text (deterministic judge) — the paste path normalizes; the file-parsing path preserves.
[35] Skills-channel injection: 14–16% across both judges, among the highest-risk channels in the study.
[36] The fake_completion method — a planted note claiming the task is already done, redirecting the agent — reached 17.0% under the LLM judge in text mode, versus 5.7% for the unmodified baseline payload.
[37] Output corruption 35.7% versus sensitive-sink action hijack 2.5%; the paper treats these as distinct operational threat profiles. Both are lab rates — crafted adversarial payloads, simulated sinks, no adaptive attacker — conditional on an attack reaching the agent, not fleet frequencies.
[38] The assessment’s stated mitigation: sensitive sinks require independent authorization mechanisms separate from model interpretation (paraphrase of the report’s recommendation).
[39] OpenAI said in January 2025 it had evidence suggesting DeepSeek trained on distilled outputs of its models (widely reported at the time, e.g. by the Financial Times, January 29, 2025); no public resolution followed, and DeepSeek did not respond publicly to the specifics. This piece takes no position on the accusation — only on who benefits from normalized raw-trace access.
[40] “The Verification Tax,” The AI Realist — the argument that in RL with verifiable rewards, the binding cost had migrated from generating candidate solutions to verifying them.
[41] TechCrunch, “AI coding startup Cognition raises $1B at $25B pre-money valuation”, May 27, 2026; Bloomberg reported the talks on April 23. Priced round, pre-money basis as stated.
[42] The Integration Premium — defined in “AWS Built Its Own AI Chip. Now It Needs Someone Else’s.” and extended in “The Model Is the Machine” (The AI Realist): in a disaggregating stack, margin migrates from component makers to the integration layer, and the modal exit for independent specialists is absorption (Groq assets to Nvidia; the Untether team and Taalas to AMD). “Musk’s Chip Gambit” covers the acquirer’s side of this arc.