Optimizer's Curse

One of the four types in B08 says the true level of whatever gets selected is necessarily below its reading: the regressional type. It is the most counterintuitive member of the whole family. It needs no cheats, no bad measure, not even anyone making a mistake. Even if every estimate is unbiased (on average neither too high nor too low), the mere act of "pick the one with the highest estimate" has disappointment already written into the mathematics. Its name is the optimizer's curse, given a clean form in 2006 by the decision scientists Smith and Winkler. Picking the highest rated of three takeaway restaurants, picking the top scoring candidate from a round of interviews, picking the checkpoint with the best eval score out of N: every one of these sets it off.

Smith and Winkler nail the mechanism with a toy model you can write in three lines (section 2.1 of the paper). Take three candidate options whose true values are all zero, meaning there is no "better one" at all; your estimate of each is unbiased with error following a standard normal distribution. Now pick the highest estimate. The expected maximum of three standard normals is about 0.85. The one you picked is exactly the true value of zero plus this maximum noise, taken as its estimate. So the option you choose will on average disappoint you by 0.85 standard deviations: all three are identical and yet you are convinced you found a gem. Raise the candidates to 4 and the disappointment rises to 103% of the estimate's standard deviation; raise it to 10 and it rises to 154%. The meaning of the numbers is plain: the more candidates and the noisier the measurement, the more water there is in whatever gets picked, and all of this happens with nobody lying.

The mechanism in one sentence: taking the maximum is an operation that specifically favours candidates with large positive noise. The more luck there is in an estimate, the more easily it rises to the front and gets selected, so "selecting the best" is itself systematically harvesting good luck. The interview example is the most personal: the top scorer hired on interview performance generally turns out "worse than expected" on the job, not because the interview lied to you but because you only ever saw them on their luckiest day. This is another name for regressional Goodhart, and it is a close relative of the winner's curse in auctions: the winner's curse involves many bidders, where the highest bidder is usually the one who most overestimated the item; the optimizer's curse involves one person selecting the best of several options. Both share the same order statistics. The phenomenon was noticed earlier, recorded in finance by Brown in 1974 and in organization studies by Harrison and March in 1984 (who named it postdecision surprise), and Smith and Winkler's contribution was to name it, formalize it, and stress that it touches every decision that selects the best from noisy estimates, not only bidding. Their prescription is called disciplined skepticism: treat estimates as noisy signals, mix them with a prior, and discount the winner's score, more heavily the more candidates and the more noise there is. The technical name for this operation is Bayesian shrinkage, and its spirit is plain: you are not punishing the champion, you are asking it to return the luck it borrowed.

The optimizer's curse holds a special position: it is a second channel of eval degradation independent of reactivity. Reactivity requires the measured party to respond, selection bias does not. Even if the measured party is entirely honest and does not budge, the act of a judge selecting on score by itself manufactures inflation. A methodological corollary follows: last year's leader falling this year needs no reactivity to explain it, plain regression is enough. This is exactly the misreading Kahneman kept hammering on: performance dropping after praise and rising after criticism gets read as the causal effect of reward and punishment when it is regression to the mean. Harrison moved the same misreading into organizational settings in 1995, where the star poached on the strength of a high current score duly falls back next period, and "the star declined the moment they arrived" gets misdiagnosed as a management failure. Before claiming "the leader's fall is Goodhart," you have to subtract the regression part.

The most familiar old friend in machine learning is also an incarnation of it. Overfitting is exactly an instance of regressional Goodhart: training loss is only a noisy proxy for generalization error, and in the vast parameter space, every model configuration is a candidate. Pushing training loss to its extreme means taking the maximum of that proxy metric across these candidates. This is exactly a restatement of "pick the one with the highest estimate," and the same mechanism holds: taking the maximum favors the candidates with the most noise. The more the selected model consists of "happening to fit the noise," the worse its real performance. The three things you use daily (early stopping, regularization, a held-out validation set) are all essentially the same sentence: do not run unconstrained strong optimization on a noisy proxy. In the same way, when reading any leaderboard, repeat under your breath: the true capability of the top model is, in expectation, below its leaderboard score, because part of the credit for reaching the top belongs to "the noise of this evaluation happened to help it," and the more the ranking was squeezed out of a large field of candidates, the larger that part is.

Other trades already have formal antidotes to this statistics, and you can copy them directly. The post-selection inference line in statistics proved that if you use the data to pick the best model and then compute its confidence interval on the same data, the interval is necessarily too narrow (Berk et al. 2013). Gelman and Loken's "garden of forking paths" points out that even if a researcher does not actively hunt for significance, significance is already inflated as long as the analysis path was chosen while looking at the data. Quantitative finance's deflated Sharpe ratio (Bailey and López de Prado 2014) exists specifically to correct the exaggeration of "backtested N strategies and reported only the best," subtracting the luck component of "how many times you tried." Backtesting a hundred strategies and publishing only the champion, and training a hundred checkpoints and publishing only the top score, are the same coin.

One open question: in the fact that "the top model's true capability is below its score," how much comes from reactivity (the measured party responding to the measurement) and how much from selection bias (the judge selecting on score)? The two channels stack, but no framework currently separates them and corrects for both jointly. The D6 report considers this the most-missing next step in research on eval degradation.

The one-line takeaway: a champion's score always has luck mixed in, and the more candidates and the more noise, the more of it; the smart move is not to stop selecting but to discount after you select.