Anti-Goodhart Lab

Every metric corrodes, but that does not mean the only option is to stop measuring. Since any metric will be gamed, design an evaluation where gaming costs more than it is worth. In C02 you learned to use the seven levers to estimate how badly an evaluation will distort, in G03 how held-out test sets and dynamic rotation keep the optimization loop permanently behind, in G05 the four fallbacks of strategyproof mechanism design, plus the trick with 43x experimental backing: hide the scoring. Assembling them into one working machine matters, because no single piece is enough.

Nail the ceiling to the wall first. You saw it in G05: the Gibbard-Satterthwaite theorem proves that in the general case no aggregation rule is both effective and fully manipulation-proof; the machine learning version from Skalse et al. in 2022 points the same way, that outside trivial cases any pair of imperfectly aligned rewards is almost certain to be gameable. So "build an ungameable eval" is mathematically a dead end. Past that dead end, the only feasible goals are the four fallbacks with provable costs: restrict the domain, accept approximation, change the structure, raise the cost of gaming. What remains is to assemble those four into a usable layered defense.

There is only one key regularity in the sandbox: no single defense stops everything, and it is the combination of layers that pushes the cost of gaming past worth it. Each layer blocks one specific route.

• Held-out sets and dynamic rotation block item overfitting and contamination (follows G03, Y02, Y03): keep the optimization loop from ever seeing or catching up with your item pool. An honest caveat, also from G03: hiding items buys time, not immunity. A 2026 cross-benchmark study found no statistical difference between the saturation rates of private and public test sets.
• Paired counterweight metrics block dimensional narrowing and style-driven score inflation (follows B09, Y08): several metrics pulling against each other are hard to satisfy at once, and it also eases surrogation (B07). Its ceiling comes from B10: as long as true value is high-dimensional and implicit, any finite-dimensional weighted combination leaves a residual alignment gap.
• Decoupling eval from optimization blocks climbing straight up against the boundary (follows G02): the eval is only for diagnosis and does not directly drive model changes. A metric is a tin opener, not a dial.
• Random human spot checks block "whatever cannot be measured gets ignored" (follows the McNamara fallacy in B05): deliberately look at dimensions the rubric does not cover, against the line that if it cannot be measured it does not exist.
• Acknowledging capacity constraints blocks real crowding-out (follows B09): do not put all the optimization pressure on one narrow eval. Model capacity is finite, and what goes to the measured dimension is drawn from somewhere else.
• Scoring logic invisible to the evaluated (follows G05): METR measured in 2025 that the same model cheats 43 times as often in environments where it can see the scoring function as where it cannot. This one has a microfoundation: in the principal-agent setting of B12, strategic opacity is an optimal mechanism derivable from first-order conditions, not administrative laziness.

Stack the six and you find they are not alternatives but complements: held-out treats "memorizing the items," pairing treats "training only one side," decoupling treats "climbing the score," human review treats "ignoring what is not measured," pressure limits treat "draining everywhere else," and hidden scoring treats "drilling into the scoring function." Drop a layer and you open a route. Which is why single-point defenses always fail. Gaming squeezes through whichever seam is unguarded.

Take paired counterweights concretely. Assess an agent on task success rate alone and it will learn to hit high success by the laziest route with the worst side effects. Pair it with an opposing metric that gets worse when it takes shortcuts, such as harmful side-effect rate, resource consumption, damage to unrelated state, or hallucination rate, and gaming success rate alone immediately shows up on the paired item. Andrew Grove made the point in one line back in 1983: measure breakfasts served, not eggs cooked. The trick of pairing is not adding a few more KPIs, it is picking opposites that are genuinely orthogonal, where the cheats do not carry over. If one cheat can raise both metrics at once, the counterweight is fake.

Take random human review concretely. Studying the English NHS, Bevan and Hood named a common failure they called synecdoche: the part that is measured gets treated as the whole, and what is not measured is treated as nonexistent. The remedy is not to measure every dimension, which cannot be done, but to make it credible that anything might at any moment be pulled out for close reading. Credibility itself is what creates the deterrent, and its scale far exceeds the cost of the actual sampling, which tax research calls the scarecrow effect. The key is that checks land on the seams most open to exploitation and stay unpredictable. If the subject cannot work out which round will be opened up, the expected payoff of targeted cheating collapses.

One open question: the four fallbacks each carry a provable cost, so is there a unified "manipulability budget" framework that puts the cost of conflict of interest, the degree of approximate strategyproofness, and the overhead of hidden items and audits into one calculation of how much cheating I am willing to tolerate and how much I am willing to pay, and optimizes them together? Not yet. It is the last piece needed to collapse the whole green defensive line from G01 to G05 into a single computable objective.

The one-line takeaway: no single layer stops gaming; an eval that holds up is several layers stacked, each blocking one route, and the cheapest first step is to align interests.