Medicine: Report Cards & Cherry-Picking

To make heart surgery more transparent, the government published a report card of every surgeon's mortality rate, and some surgeons started refusing the sickest patients. The intent was accountability; what landed was patient selection. This is not a story about bad doctors, it is a story about how a good institution pushes rational people into bad choices, and it puts the hardest mechanism in the K01 case library, the selection effect, onto a table where lives are at stake.

Two things to recall first. Campbell's law in B02 says that the more a quantitative indicator is used for consequential decisions, the more it corrupts the very process it was meant to monitor. K01 gave the definition of the selection effect: the measured party does not change its own real level, it changes the population being measured, picking the healthy and avoiding the severe. This case is the sharpest confluence of those two in medicine. Note the institutional background: New York State built a cardiac surgery registry in 1989 and in 1990 to 1991 first published risk-adjusted coronary artery bypass mortality rates by hospital and even by surgeon, with Pennsylvania following in 1992. Risk adjustment will come back; it is the crux of the whole dilemma.

The hardest causal evidence comes from Dranove, Kessler, McClellan and Satterthwaite's 2003 study in the Journal of Political Economy. They used nationwide Medicare cardiac surgery data to identify the net effect of these two report cards, and one sentence of the conclusion is enough to sting: the report cards led to higher resource consumption and worse health outcomes, and harmed sicker patients in particular, reducing patient welfare and social welfare at least in the short run. The mechanism is not hard to follow: mortality rates are published for all to see. If a severely ill patient dies on the table, it stains the surgeon's public report card. So the rational response is not to operate better but simply not to accept this high-risk patient, pushing them to someone else or switching to a treatment that does not enter these statistics. The metric did not get the people who most needed saving better care; it made it harder for them to find a hand willing to cut.

Doctors' own words supply the first person account. Schneider and Epstein surveyed a random sample of Pennsylvania cardiac specialists in the New England Journal of Medicine in 1996, and two numbers say a lot: 63% of cardiac surgeons said that because of the report cards they were less willing to operate on severely ill patients; 59% of cardiologists reported that it had become harder to find a surgeon willing to operate on severely ill patients who genuinely needed surgery. What do those two near-sixty-percents mean: avoiding high risk is not a few individuals with bad intentions, it is more than half the profession responding in the same direction to the same incentive. A colder trace to add: within two years of publication, over 20% of surgeons in the worst quartile of risk-adjusted mortality simply stopped performing bypass surgery, against about 5% in the control group. The pressure of bad ratings did clear out a group, and the problem is how it cleared them: partly by letting surgeons dodge the difficult patients.

The change in patient composition is not just an abstract proposition, it leaves measurable fingerprints in the data, including along racial lines. Werner, Asch and Polsky found in 2005 that after New York's first bypass report card was published, the gap in receiving bypass surgery between white patients and black and Hispanic patients widened significantly, with no such change in control states that had no report card. The mechanism: if surgeons treat minority patients as high risk by default, then the strategy of avoiding high risk incidentally worsens existing racial inequality. In interventional cardiology the evidence is sharper. Wasfy et al. gave a quantified trace in 2015: under public reporting pressure, the proportion of New York cardiogenic shock patients receiving intervention fell from 2.28% of all cases in 2003 to about 1.29%. What does this near-halving mean: the most dangerous patients, the ones most in need of an immediately opened vessel, are precisely the group most likely to be moved off the table, because they are the most likely to die there and stain the report card. Blumenthal et al. asked interventional cardiologists directly in 2018, and they too acknowledged that public reporting made them less willing to operate on the most critically ill patients.

One open question: the report card dilemma is real: without publication there is no accountability and bad surgeons hide inside the average; with publication comes patient selection and the sickest patients get sacrificed. And the switch on that dilemma rests almost entirely on how well risk adjustment is done: the more accurate the adjustment, the less a surgeon can score by selecting patients. But no amount of risk adjustment can ever adjust away the details of a condition that were never recorded, and a surgeon's private judgement about unrecorded risk will always be ahead of any formula. So how small can a permanently imperfect risk adjustment push this dilemma? We do not have a settled answer.

The one-line takeaway: published report cards push rational doctors into selecting patients and sacrifice the severe cases who most need saving; the key to the dilemma is not whether to publish but how accurate the risk adjustment is.