Academia: Citation Arms Race

Academia outsourced the evaluation of its own output to three numbers: the Journal Impact Factor (how many times the average paper in a journal is cited), citation counts, and the h-index (a person's productivity and citations compressed into a single integer). All three use "other people cite you" as a proxy for quality, and citations are something that can be produced by coordination: a few more lines in a reference list will do it. This capacity for "coordinated production" does not stay in the hands of a single author. Positions with more power can manipulate citations just as cheaply: editors can pressure authors, and journals can form alliances to cite each other. So a complete manipulation industry grew up, running from editorial pressure to transnational cartels. This case line has two features found nowhere else. First, the measured are the people in the world who best understand measurement. You saw the Matthew effect of citation systems in R10 (high citation brings visibility, visibility brings more citations, snowballing), and what manipulators do is simply connect an artificial input to that amplifier; the reflexive-capability lever (one of the seven levers in K01) is naturally at maximum, and the gaming methods are the most refined. Second, organized resistance also started here: DORA and the Leiden Manifesto are a rare instance across fields of resistance initiated by the measured community itself.

The impact factor is defined as citations in the current year to a journal's papers from the previous two years, divided per article, and both numerator and denominator leave room for arbitrage: publishing more reviews, a naturally highly cited format, and managing the denominator's definition of "citable items" both raise the score without changing the quality of any single paper. This kind of definitional arbitrage is entirely compliant, which is exactly why it is the most widespread. One step across the line is coercive citation: an editor tells an author to cite more of this journal if they want to publish. Wilhite and Fong's survey published in Science in 2012 covered about 6700 scholars, roughly one in five of whom said a journal editor had asked them to add citations to that journal's own articles, and 86% considered the practice unethical; incidence was higher at business journals, for-profit publishers and highly ranked journals. A one-in-five rate means this is not a few bad actors but a latent channel of power in the editorial role: the editor holds acceptance and the author holds the reference list, and both sides understand the exchange. Mechanism tag: metric collusion, the editorial-power version of adversarial Goodhart.

One step further is citation stacking: journals form mutual-citation cartels, you cite me and I cite you, and everyone's impact factor rises together. The measuring side patched in response: Thomson Reuters/Clarivate, which publishes the table, applies suppression to anomalous citation patterns, expelling 51 journals from the JCR at once in 2012, and of roughly 37 newly banned journals in 2015, 14 were for stacking; 140 journals once had self-citation rates over 70% in the previous two years, where the normal value is mostly below 30%. What does a 70% self-citation rate mean: seven-tenths of these journals' "influence" is votes they cast for themselves. The Brazilian cartel in 2013 is the clearest example of the motive chain: editors at several Brazilian journals stacked citations for each other, traceable to the excessive weight the national Qualis evaluation system placed on "top journals". One country's evaluation rules shaped one country's form of gaming.

ARWU (the Shanghai world university ranking) counts the number of highly cited scientists as a hard indicator, so the list itself became a commodity. In 2011 Science revealed King Abdulaziz University in Saudi Arabia's approach: recruiting over 60 scientists from the highly cited list to sign part-time agreements listing the university as a secondary affiliation, for about 72,000 dollars a year, with the only obligation being a visit of a few days each year; at one point the university "employed" over a quarter of the highly cited mathematicians on the list. A university does not have to build a world-class mathematics department, it can rent the names and own a place on the table: this is the pure form of aiming at the metric as a target. Criticism from bibliometricians pushed the Shanghai ranking to stop counting secondary affiliations from new lists starting in 2014; in 2023 Spanish media exposed an upgraded form of paid affiliation (reporting the Saudi institution directly as the primary affiliation), for which the chemist Luque was dismissed by his own university. After Clarivate tightened its review, the number of highly cited scholars at Saudi institutions fell by about 30% within a year, and over 1000 candidates were removed from that year's list. Numbers dropping 30% the moment review tightens is itself a measurement of the bubble's prior volume.

More extreme was the collective collapse in mathematics. Docampo found that highly cited mathematics papers from 2021 to 2023 were dominated by a set of institutions with no mathematical tradition, with China Medical University in Taiwan at the top with 95 papers, up from zero a decade earlier; the mechanism was low-quality papers concentrating citations on a few "top papers". From zero to first in the world in ten years, achieved not through mathematics but through a mutual-citation factory. Clarivate's response in November 2023 was to remove the entire field of mathematics from the highly cited list: a metric played so far out of shape that the measuring side could only delist a whole discipline, the most thorough patch in the citation arms race to date. The corresponding tool at the individual level is Ioannidis et al.'s public database from 2019: standardized citation metrics for 100,000 scientists, giving both a with-self-citation and a without-self-citation reading, which exposed several hundred extreme self-citers. Mechanism tag: rules arbitrage escalating into outright fraud, with the ranking body patching round after round and both sides co-evolving.

The h-index (h papers each cited at least h times) inherits every manipulable channel of citation-based metrics: self-citation, mutual-citation groups, honorary co-authorship, each of which raises h directly. The dual-reading design of the Ioannidis database is exactly the audit aimed at this channel: strip out self-citations and a portion of highly productive, highly cited scholars slide sharply down the ranking, with the gap between the two readings being the share of self-promotion. Downstream in the industry are the grey markets for mass-produced papers and mass-manufactured citations, and the moment of reckoning on the journal side is retraction: in 2023 global retractions exceeded ten thousand, a record, driven mainly by the group of journals infiltrated by paper mills (this is a course-supplementary fact not in the research working notes, source given below). Ten thousand retractions in a year means twenty or thirty "findings" declared void every day. Retraction is to papers what JCR delisting is to journals: the measurement system's after-the-fact reckoning, always arriving after the inflation has been cashed in.

Organized resistance began in San Francisco in 2012. DORA (the Declaration on Research Assessment) makes one core claim: journal-level metrics, and especially the impact factor, must not be used to evaluate individual papers or individual researchers, on the plain grounds that a journal average says nothing about the quality of your particular paper. In 2015 the Leiden Manifesto published ten principles in Nature, the first being that quantitative evaluation should support and not replace qualitative expert judgement; the same year the Metric Tide review in the UK proposed five dimensions of responsible metrics: robustness, humility, transparency, diversity, reflexivity. The shared position of the three documents can be compressed to one line: metrics assist judgement, and judgement is not outsourced to metrics. This is not anti-measurement, it puts measurement back in the position of a tool. Mechanism tag: institutional-layer defence, the same family as G06; it accepts what these cases jointly prove, that pure metric governance in a highly reflexive population is inevitably turned back on itself by the governed.

One open question: was Clarivate removing the whole of mathematics a patch or a surrender? If colluding groups simply migrate to the next discipline, the end point of delisting discipline by discipline is having no list left to publish. Avoiding that end point means asking a more fundamental question: can a citation metric be both public and collusion-resistant? That question bears directly on the impossibility boundary in mechanism design discussed in G05.

The one-line takeaway: for a metric whose signal the measured produce themselves, gaming is only a question of price; the answer academia bought with thirty years is that scores assist judgement and judgement is not outsourced to scores.