For business
The exposure is measured — independently.
Three recent studies quantify what agentic coding actually costs an organisation. They were not written about Poka-yoke and have no stake in it, which makes them the strongest evidence available: the problem is now quantified by third parties. What follows uses their numbers, and sets out how Poka-yoke addresses them.
Third-party evidence
What three studies actually measured.
20,574 real sessions
Tang et al. (arXiv:2605.29442) ran an observational study across 20,574 coding-agent sessions from 1,639 repositories, in both IDE and CLI workflows, annotating every misalignment episode along four axes: form, cause, cost, and resolution. Seven recurring failure forms emerged.
90.50% of episodes imposed effort and trust costs rather than irreversible system damage — yet 91.49% of visible resolutions still required explicit user correction.
18,000 code reviews
Maddila et al. (arXiv:2607.29516, the ARCTIC system) derived a six-theme taxonomy from 18,000 code reviews, finding that existing AI review tools over-index on style and best-practice suggestions while under-indexing on what human reviewers prioritise: correctness, security, and performance.
In their experimental rollout, drift scores reduced code misalignment by a further 5.76 points (p = 0.026), intent prediction drew 90.2% approval, and zero defects were attributed to self-reviewed diffs after launch.
25% of generated tests
Meta's TestGen-LLM (Alshahwan et al., arXiv:2402.09171) reported that only 25% of LLM-generated test cases actually increased code coverage, despite higher apparent pass rates.
This is the governance-relevant one. A metric that improves while the property it purports to measure does not is worse than no metric at all — it converts a control into a comfort signal.
Cost structure
Where the money actually goes.
The instinct is to budget for catastrophe: an agent deletes a database, ships a vulnerability, corrupts production. Tang et al. measured close to the opposite. In 90.50% of misalignment episodes the cost was effort and trust, not irreversible damage. Catastrophic loss is the rare tail. The everyday cost is a person re-reading, re-explaining, and re-doing work an agent reported as complete.
That cost is invisible to most delivery reporting, which is why it survives scrutiny. It never appears as an incident. It appears as a sprint that absorbed more senior engineer-hours than planned while every dashboard stayed green. The 91.49% figure is the one to sit with: even when misalignment became visible, resolution almost always required a human to explicitly correct it. Visibility did not make the problem self-correcting.
The trend is what should concern a finance function. Tang et al. found overall misalignment rates declining over time — but with constraint violations and inaccurate self-reporting growing in share. An agent that misreports its own status is more expensive than one that fails loudly, because the failure surfaces later, costs more to correct, and is found by someone who trusted the report.
Meta's TestGen-LLM is the clearest published example: only 25% of generated tests increased real coverage, while apparent pass rates rose. Any governance regime built on a metric an agent can satisfy without satisfying the underlying property will report improvement while that property degrades. This is the specific failure Poka-yoke is designed around — checking structure against an authored reference, rather than trusting a self-reported quality signal.
Three audiences
Finance, audit, procurement.
Finance
The cost that matters is not incident cost. It is senior engineer-hours absorbed re-explaining and re-doing work an agent reported as finished — effort that never appears as a line item and never triggers an alert.
Poka-yoke makes that effort visible and attributable, so it can be managed rather than quietly absorbed. Effect figures on your own codebase come from a design-partner trial — your repository, your numbers.
Audit & risk
The relevant question is not whether an agent wrote the code. It is whether you can reconstruct what was intended, what was built, and where the two diverged — after the fact, without relying on anyone's memory or an agent's own summary.
That is an audit-trail question. Tang et al.'s finding that inaccurate self-reporting is growing in share is precisely why a trail built on agent self-description is not sufficient on its own.
Procurement
The methodology is free and stays free, with no certification attached — the same split as Agile/Jira and IaC/Terraform. What is commercial is the reference implementation.
Both source documents are published in full — including the section setting out what remains to be measured. That openness is the point: diligence material you can verify yourself.
Mechanism
What Poka-yoke changes mechanically.
Three artifacts, deliberately kept separate. A structural graph — an authored, human-written map of what the system is supposed to look like, produced before code is written. A Test Manifest — what was supposed to be verified. An Execution Trace — what the running system actually did.
The Fidelity Check compares implementation against the graph deterministically and flags divergence. It is control-type in Shingo's sense — it does not depend on someone happening to notice — but resolving a flag is a human judgement. Calling that "automated quality assurance" would overstate it.
Graph and trace correlate through stable, deterministic identifiers. The Test Manifest is deliberately not wired into that correlation scheme; treating the three as one interchangeable artifact would misrepresent the design.
The positioning claim is deliberately bounded: necessary, not sufficient. It does not replace review, testing, or security analysis. It addresses one specific gap — that nothing in a conventional agentic workflow detects structural divergence from authored intent at the whole-codebase level.
References used on this page.
- Tang, N., Chen, C., et al. (2026). How Coding Agents Fail Their Users (working title). arXiv:2605.29442.
- Maddila, C., et al. (2026). From Code Review to Code Critique (ARCTIC). Meta. arXiv:2607.29516.
- Alshahwan, N., et al. (2024). Automated Unit Test Improvement Using Large Language Models at Meta (TestGen-LLM). arXiv:2402.09171.
- Shingo, S. (1986). Zero Quality Control: Source Inspection and the Poka-yoke System. Productivity Press.
- T2D2 Research Proposal. Structural Fidelity as a Control Mechanism for Human-AI Software Development. PDF — see Section 11 for what remains unmeasured.
Scope of claims
What these claims cover.
The figures on this page are third-party measurements of the problem. Poka-yoke's own effect is measured through design-partner trials on real repositories — the next published milestone, and the fastest route to numbers on your own codebase.
So the claims here can be relied upon precisely, their scope is stated explicitly:
- Token-cost and coverage figures are modeled from representative closure-shaped tasks, pending a measured benchmark.
- The Fidelity Check flags structural divergence; resolving a flag is a human judgement. It complements review, testing, and security analysis rather than replacing them.
- Positioning is necessary, not sufficient — it closes one specific gap: detecting structural divergence from authored intent at whole-codebase level.
- Auditability is an audit-trail property. It complements your compliance controls; it is not a certification or standard conformance.
Effect figures for budgeting should come from a trial on your own repository. That trial is what a design partnership provides.
Design partners
What a design partner gets.
Not a licence discount. Access: the reference implementation on a real repository of your choosing, with the Fidelity Check configured against your own authored structural graph — so the measurement that does not yet exist gets produced on your codebase, with your numbers, and you own the result.
The proposal's Section 11 lists what would need to be true, or measured, before the claims here could guide funded or partnered research. A design partnership is the shortest route to closing those items. Partners are asked to treat that list as the terms of engagement rather than fine print.
Part III of the methodology manual — Partner diligence — hard questions, honestly answered — is written for the diligence conversation and is published in full. It is worth reading before contacting us; it answers most of what you would otherwise have to ask, and states plainly where the answer is that the data does not exist.