Strategic Classification

Every assessor runs into the same situation: the moment you publish the scoring rule, the people being scored start remaking themselves to fit it. Credit scores reward opening more cards, so people open cards instead of saving; resume filters reward keywords, so applicants pile keywords into resumes. In 2016 Hardt, Megiddo, Papadimitriou and Wootters turned this situation into a formal research area called strategic classification. It yields one very hard design theorem: to make an assessment incentivise real ability rather than performance, you must first solve a causal problem, and there is no shortcut.

The framework first. A classifier is an automated decision rule: input a person's features (income, debt, number of cards) and output a decision (lend or refuse). Strategic classification models it as a sequential game (the term is a Stackelberg game): you publicly commit to the rule first, the other side sees the rule, and as long as the cost of remaking themselves is below the payoff of landing on the good side, they remake themselves. The original paper proves that a naive classifier which pretends the other side will not move degrades sharply under this game, and it also supplies a robust classifier that anticipates the move and defends accordingly. This is Goodhart inside game theory: the thing measured is no longer a static sample, it is a strategic opponent.

This and the principal-agent problem in B12 are two scenes on one stage: there the problem is an agent gaming the contract terms, here the problem is a scored party gaming the scoring formula. Vendor manoeuvres around benchmarks are a direct instance: tuning the "eval scaffold" (system prompts, answer templates, answer extraction methods, all peripheral settings that leave the model itself untouched), privately testing many variants and publishing only the best, specialising for particular question types. The same model's score can drift several percentage points on peripheral tuning alone. The more finely the scoring standard is published, the larger the surface available for performance.

Step three's theorem is the crucial one, so here it is again in plain words. First separate two kinds of "remaking yourself to fit the rule": gaming changes only the signal and not the substance, like temporarily churning transactions to make the statements look good while repayment capacity is unchanged; improvement is genuinely getting better, like paying off the debt. On the surface of the data the two look almost identical, both appearing as "the features changed and the prediction followed". The difference is underneath: improvement changed the thing that actually determines the outcome, while gaming changed only a stand-in that correlates with the outcome without causing it.

Miller, Milli and Hardt proved in 2020 that to design a scoring rule which incentivises only improvement, you must know which features are causes and which are merely correlates, and that is mathematically equivalent to solving a non-trivial causal inference problem. Causal inference, plainly, is answering "if I intervene on this quantity, will the outcome follow", and it is notoriously harder than finding correlations. The force of the theorem is that anti-gaming has no general recipe, and there is no patch of the form "add some anti-counterfeiting measures"; your scoring can only defend as far as your understanding of what causes what.

The corollary lands directly on feature selection. Anchor the score to causal features (the task was actually completed, the debt was actually repaid), and the anchor itself is the very thing that determines the outcome: here the signal equals the substance, with no separate surface to manipulate. The only way to change the signal is therefore to change the substance itself, so the costs of gaming and improvement merge. Once the costs merge, the game has no cheaper bypass left and is forced into improvement. Anchor the score to proxy features (how the statements look, the format of the output, whether it announced "done") and you are subsidising performance. Keeping the scoring details secret is a stopgap: it lowers the other side's optimisation efficiency without changing the causal status of the features, and one leak zeroes it.

The fairness ledger has to be kept too. Milli et al. worked it out: raising the bar to the game-theoretic optimum to prevent gaming does not make the cost vanish, it transfers it to the people being classified, and the groups for whom changing features is most expensive (often the ones with the fewest resources) bear the most. Anti-gaming tightening is regressive, like a tax levied downwards. Every tradeoff a score designer makes between "being fooled" and "harming the innocent" has distributional consequences.

Back to Y01: overfitting says "you look at the data repeatedly and the data's noise seeps into your model"; strategic classification says the reverse, "the data looks at you repeatedly and your rule seeps into the data". Put together, measurer and measured shape one another, which is the game-theoretic way of writing the two-way loop that keeps recurring through the yellow branch.

One open question: when the follower's response speed outpaces the leader's rule updates, how much binding force is left in the "commit first, respond second" ordering of the game? A model under evaluation can now read your scoring criteria on the spot and then answer, which is no longer the applicant of the 2016 framework who had to pay a cost and remake himself slowly.

The one-line takeaway: publish the scoring rule and both people and models will remake themselves to fit it; to make that remaking real progress rather than performance, you have to know which features actually cause the outcome, and there is no shortcut.