Talk costs nothing. Anyone can open their mouth and say it, true or false. So promises are naturally worth nothing. "I guarantee it is fine" costs nothing to say, so how much it can be trusted does not depend on how well it is phrased but on how far apart the speaker's interests are from yours. The core is a theorem: how expensive a proof you need to get the truth out of someone is proportional to the conflict of interest between you. When conflict is zero, bare words suffice; the greater the conflict, the more you need expensive evidence to hold the lies down. The principal-agent lesson covered wanting a good agent while the optimizer only wants the eval to go green (B12). One layer deeper: when the two sides' interests diverge, how much of what the agent says should you believe?
Economics has a term, cheap talk, meaning statements that cost nothing to make. Crawford and Sobel worked it out thoroughly in 1982, and the conclusion is unexpectedly clean: in costless communication, how much truth can be transmitted is determined monotonically by how aligned the two parties' interests are. In their own terms, the closer the two sides' preferences, the more informative the signals in equilibrium. Look at the two extremes. If speaker and listener have perfectly aligned interests, he has no reason to lie and bare words are fully credible. If their interests are perfectly opposed, you should believe nothing he says, and in theory that case degenerates into what economists call a babbling equilibrium, where saying something equals saying nothing because any statement might be the reverse of the truth. The middle is the interesting part: with partially overlapping interests, the speaker will not consider your interests word by word but will not deceive you wholesale either. He will tell you his private information in coarse bands, and the smaller the conflict, the finer and more useful the bands. In one line, the credibility of cheap talk is a function of alignment of interest.
And when cost does have to be added, how much? The answer is the same sentence: only enough to cover the extra gain a cheat could take from deceiving you, which is the size of the conflict, and no more. Lachmann and collaborators pushed this into biology and human language in 2001, and their article title says it outright, waste is not required: sustaining honesty does not require raising costs to the sky, only making lying not worth it. There is an even more economical finding: Kartik et al. proved in 2007 that even a small definite cost on lying, nothing like a crushing barrier, substantially improves information transmission and approaches truth-telling. Around you this looks like: you ask a teammate whether this piece of code has a bug, and since he is on your team and a bug lands on his own work too, your interests are basically aligned, so an offhand answer is credible and needs no evidence. But if he is an outside contractor who can get paid and leave sooner by reporting "no problem," interests conflict, so you have to add acceptance testing and accountability, that is, add cost, and how much to add depends entirely on how far apart your interests are.
From entirely free talk to having to produce expensive evidence, there is no cliff in between but a smooth slope. Zollman et al. characterized that slope in 2013: as conflict of interest grows a notch at a time, equilibrium slides from "everything can be communicated" to "only some things can" and on to "only costly signals can barely distinguish," with a large range of "semi-honest" states in between. That warns you not to think of the problem in black and white: it is not a choice between unconditional trust and a full audit, it is first measuring the size of the conflict and then finding the just-sufficient point on that slope. Every extra unit of cost is a unit of pure friction on this machine, giving no benefit to the honest and no more information to you.
Translated into a design principle, the theorem gives an order that runs against intuition: do not start by pushing the proof threshold as high as it will go. Step one is always to reduce conflict, aligning the interests of speaker and listener as far as possible, because with small conflict a lot of statements are self-evident without verification. Step two, for whatever conflict cannot be pressed out, is to add a just-sufficient definite cost: a probability of being spot-checked, accountability after the fact, a false report leaving a stain. Only after those two steps do you need the harder kind of proof that rests on structure rather than cost, for example a signal a low quality party physically cannot produce. That is the subject of the next lesson, G01, which continues with what makes a signal particularly expensive for liars and nearly free for the honest, and why the truly good criterion is not the most expensive one.
One open question: the theorem gives a lower bound on cost, saying you must add at least enough to cover the conflict, but gives no upper bound. Over-signalling has side effects, which economics calls the welfare loss of excessive signalling: to keep the frauds out, you may drag the honest party into expensive proof as well, so everyone burns effort proving their innocence. So in an artificial eval, how high should the cost be set to suppress lying without harming the honest? There is no clean answer at present.
The one-line takeaway: first ask whether the other party has a reason to deceive you, then decide whether to demand evidence; the smaller the conflict, the more statements are self-evident.
Sources / further reading
- Crawford, V. P. & Sobel, J. (1982). "Strategic Information Transmission." Econometrica 50(6):1431–1451 (the closer the preferences, the more informative the signals; the babbling equilibrium).
- Farrell, J. & Rabin, M. (1996). "Cheap Talk." Journal of Economic Perspectives 10(3):103–118 (the most readable introductory survey).
- Lachmann, M., Számadó, S. & Bergstrom, C. T. (2001). "Cost and conflict in animal signals and human language." PNAS 98(23):13189–13194 (waste is not required, cost as needed).
- Bergstrom, C. T. & Lachmann, M. (1998). "Signaling among relatives. III. Talk is cheap." PNAS 95(9):5100–5105 (bare talk beats costly signals when interests align).
- Kartik, N., Ottaviani, M. & Squintani (2007). "Credulity, lies, and costly talk." JET 134(1):93–116 (a small lying cost substantially improves matters); Yang & Harstad (2017) (welfare loss from excessive signalling).
- The cheap talk bridge and the cost-proportional-to-conflict theorem in
research/deep/D7§D7-2; the link to Spence separating equilibria inresearch/10§2; the limits of the formal theory inresearch/06.