1This is a video about the most famous problem in game theory.
2Problems of this sort pop up everywhere,
3from nations locked in conflict to roommates doing the dishes.
4Even game shows have been based around this concept.
5Figuring out the best strategy can mean the difference between life and death,
6war and peace,
7flourishing and the destruction of the planet.
8And in the mechanics of this game,
9we may find the very source of one of the most unexpected phenomena in nature:
10cooperation.
11On the 3rd of September, 1949,
12an American weather monitoring plane collected air samples over Japan.
13In those samples, they found traces of radioactive material.
14The Navy quickly collected and tested rainwater samples from their ships and bases all over the world.
15They also detected small amounts of Cerium-141 and Yttrium-91.
16But these isotopes have half lives of one or two months,
17so they must have been produced recently
18and the only place they could have come from was a nuclear explosion.
19But the US hadn't performed any tests that year,
20so the only possible conclusion
21was that the Soviet Union had figured out how to make a nuclear bomb.
22This was the news the Americans had been dreading.
23Their military supremacy achieved through the Manhattan Project was quickly fading.
24This makes the problem of Western Europe and the United States
25far more serious than it was before and perhaps makes the imminence of war greater.
26Some thought their best course of action was to launch an unprovoked nuclear strike against the Soviets
27while they were still ahead.
28In the words of Navy Secretary Matthews to become "aggressors for peace".
29John von Neuman, the founder of game theory, said,
30"If you say why not bomb them tomorrow, I say, why not bomb them today?"
31"If you say today at five o'clock, I say why not at one o'clock?"
32Something needed to be done about nuclear weapons and fast.
33But what?
34In 1950, the RAND Corporation, a US-based think tank was studying this question.
35And as part of this research, they turned to game theory.
36That same year, two mathematicians at RAND had invented a new game,
37one which unbeknownst to them at the time, closely resembled the US-Soviet conflict.
38This game is now known as the prisoner's dilemma.
39So let's play a game.
40A banker with a chest full of gold coins invites you and another player to play against each other.
41You each get two choices.
42You can cooperate or you can defect.
43If you both cooperate, you each get three coins.
44If one of you cooperates, but the other defects,
45then the one who defected gets five coins and the other gets nothing.
46And if you both defect, then you each get a coin.
47The goal of the game is simple: to get as many coins as you can.
48So what would you do?
49Suppose your opponent cooperates,
50then you could also cooperate and get three coins
51or you could defect and get five coins, instead.
52So you are better off defecting,
53but what if your opponent defects, instead?
54Well, you could cooperate and get no coins
55or you could defect and at least get one coin.
56So no matter what your opponent does,
57your best option is always to defect.
58Now, if your opponent is also rational,
59they will reach the same conclusion and therefore also defect.
60As a result, when you both act rationally,
61you both end up in the suboptimal situation
62getting one coin each when you could have gotten three, instead.
63In the case of the US and Soviet Union,
64this led both countries to develop huge nuclear arsenals of tens of thousands of nuclear weapons each,
65more than enough to destroy each other many times over.
66But since both countries had nukes, neither could use them.
67And both countries spent around $10 trillion developing these weapons.
68Both would've been better off if they had cooperated and agreed not to develop this technology further.
69But since they both acted in their own best interest,
70they ended up in a situation where everyone was worse off.
71The prisoner's dilemma is one of the most famous games in game theory.
72Thousands and thousands of papers have been published on versions of this game.
73In part, that's because it pops up everywhere.
74Impalas living in between African woodlands and Savannahs are prone to catching ticks,
75which can lead to infectious diseases, paralysis, even death.
76So it's important for impalas to remove ticks and they do this by grooming,
77but they can't reach all the spots on their bodies and therefore they need another impala to groom them.
78Now, grooming someone else comes at a cost.
79It costs saliva, electrolytes, time and attention,
80all vital resources under the hot African sun
81where a predator could strike at any moment.
82So for the other impala, it would be best not to pay this cost,
83but then again, it too will need help grooming.
84So all impalas face a choice:
85should they groom each other or not?
86In other words, should they cooperate or defect?
87Well, if they only interact once, then the rational solution is always to defect.
88That other impala is never gonna help you, so why bother?
89But the thing about a lot of problems is that they're not a single prisoner's dilemma.
90Impalas see each other day after day and the same situation keeps happening over and over again.
91So that changes the problem
92because instead of playing the prisoner's dilemma just once,
93you're now playing it many, many times.
94And if I defect now, then my opponent will know that I'd defected
95and they can use that against me in the future.
96So what is the best strategy in this repeated game?
97That is what Robert Axelrod, a political scientist wanted to find out.
98So in 1980, he decided to hold a computer tournament.
99He invited some of the world's leading game theorists for many different subjects
100to submit computer programs that would play each other.
101Axelrod called these programs strategies.
102Each strategy would face off against every other strategy and against a copy of itself
103and each matchup would go for 200 rounds.
104That's important and we'll come back to it.
105Now, Axelrod used points instead of coins, but the payoffs were the same.
106The goal of the tournament was to win as many points as possible
107and in the end, the whole tournament was repeated five times over
108to ensure the success was robust and not just a fluke.
109Axelrod gave an example of a simple strategy.
110It would start each game by cooperating and only defect after its opponent had defected twice in a row.
111In total Axelrod received 14 strategies and he added a 15th called random,
112which just randomly cooperates or defects 50% of the time.
113All strategies were loaded onto a single computer where they faced off against each other.
114One of the strategies was called Friedman.
115It starts off by cooperating,
116but if its opponent defects just once,
117it will keep defecting for the remainder of the game.
118Another strategy was called Joss.
119It also starts by cooperating,
120but then it just copies what the other player did on the last move.
121Then around 10% of the time, Joss gets sneaky and defects.
122There was also a rather elaborate strategy called Graaskamp.
123This strategy works the same as Joss, but instead of defecting probabilistically,
124Graaskamp defects in the 50th round to try and probe the strategy of its opponent
125and see if it can take advantage of any weaknesses.
126The most elaborate strategy was Name Withheld with 77 lines of code.
127After all the games were played, the results were tallied up and the leaderboard established.
128The crazy thing was that the simplest program ended up winning,
129a program that came to be called Tit for Tat.
130Tit for Tat starts by cooperating
131and then it copies exactly what its opponent did in the last move.
132So it would follow cooperation with cooperation and defection with defection,
133but only once if it's opponent goes back to cooperating,
134so does Tit for Tat.
135When Tit for Tat played against Friedman,
136both started by cooperating and they kept cooperating
137both ending up with perfect scores for complete cooperation.
138When Tit for Tat played against Joss, they too started by cooperating
139but then on the sixth move, Joss defected.
140This sparked a series of back and forth defections, a sort of echo effect.
141Okay, so now you've got this alternating thing
142which will remind you of some of the politics of the world today
143where we have to do something to you because of what you did to us.
144And then when this... this weird program throws in a second unprovoked defection,
145now it's really bad
146because now both programs are gonna defect on each other for the rest of the game.
147And that's also like some of the things that we're seeing in politics today and in international relations.
148As a result of these mutual retaliations,
149both Tit for Tat and Joss did poorly.
150But because Tit for Tat managed to cooperate with enough other strategies,
151it still won the tournament.
152I imagine initially it'd be sort of like computer chess
153where you need a pretty complicated program to play a sophisticated game.
154But in fact it was not like that at all. It was the simplest strategy that did the best.
155So I analyzed how that happened.
156Axelrod found that all the best performing strategies, including Tit for Tat, shared four qualities.
157First, they were all nice,
158which just means they are not the first to defect.
159So Tit for Tat is a nice strategy,
160it can defect but only in retaliation.
161The opposite of nice is nasty.
162That's a strategy that defects first.
163So Joss is nasty.
164Outta the 15 strategies in the tournament,
165eight were nice and seven nasty.
166The top eight strategies were all nice
167and even the worst performing nice strategy still far outscored the best performing nasty one.
168The second important quality was being forgiving.
169A forgiving strategy is one that can retaliate but it doesn't hold a grudge.
170So Tit for Tat is a forgiving strategy.
171It retaliates when its opponent defects
172but it doesn't let affections from before the last round influence its current decisions.
173Friedman on the other hand, is maximally unforgiving.
174After the first defection just from the opponent would defect for the rest of the game.
175Okay, that's it.
176No mercy and that might feel good to do
177but it doesn't end up working out well in the long run.
178This conclusion that it pays to be nice and forgiving came as a shock to the experts.
179Many had tried to be tricky and create subtle nasty strategies to beat their opponent
180and eke out an advantage,
181but they all failed.
182Instead, in this tournament, nice guys finished first.
183Now, Tit for Tat is quite forgiving but it's possible to be even more forgiving.
184Axelrod's sample strategy only defects after its opponent defected twice in a row.
185It was Tit for Two Tats.
186Now that might sound overly generous,
187but when Axelrod ran the numbers,
188he found that if anyone had submitted the Sample strategy,
189they would've won the tournament.
190I mean it's so clever, there's so many layers to this story.
191After Axelrod published his analysis of what happened
192or circulated it among these game theorists,
193he said, now that we all know what worked well,
194let's try again.
195So he announced a second tournament
196where everything would be the same except for one change:
197the number of rounds per game.
198See, in the first tournament, each game lasted precisely 200 rounds.
199And that is important
200because if you know when the last round is,
201then there's no reason to cooperate in that round.
202So you're better off defecting.
203Of course your opponent should reason the same
204and so they should also defect in the last round.
205But if you both anticipate defection in the last round,
206then there's no reason for you to cooperate in the second to last round
207or the round before that, or before that
208and so on all the way to the very first round.
209And so in Axelrod's tournament,
210it was a very important thing that the players didn't know exactly how long they were gonna be playing.
211They knew on average it would be 200 rounds,
212but there was a random number generator
213that prevented them from knowing with certainty.
214Yeah, if you're not sure when it ends,
215then you have to kind of keep cooperating 'cause it might keep going and you need might need them on your side.
216For this second tournament,
217Axelrod received 62 entries and again, added random.
218The contestants had gotten the results and analysis from the first tournament
219and could use this information to their advantage.
220This created two camps.
221Some thought that clearly being nice and forgiving were winning qualities.
222So they submitted nice and forgiving strategies.
223One even submitted Tit for Two Tats.
224The second camp anticipated that others would be nice and extra forgiving
225and therefore they submitted nasty strategies to try to take advantage of those that were extra forgiving.
226One such strategy was called Tester.
227It would defect on the first move to see how its opponent reacted.
228If it retaliated, tester would apologize and play Tit for Tat for the remainder of the game.
229If it didn't retaliate, tester would defect every other move after that.
230But again, being nasty didn't pay.
231And once again, Tit for Tat was the most effective.
232Nice strategies again did much better.
233In the top 15, only one was not nice.
234Similarly, in the bottom 15,
235only one was not nasty.
236After the second tournament,
237Axelrod identified the other qualities that distinguished the better performing strategies.
238The third is being retaliatory,
239which means if your opponent defects, strike back immediately,
240don't be a pushover.
241Always cooperate is a total pushover.
242And so it's very easy to take advantage of.
243Tit for Tat, on the other hand, is very hard to take advantage of.
244The last quality that Axelrod identified is being clear.
245Programs that were too opaque,
246that were too similar to a random program,
247you couldn't figure them out because they were so complicated.
248It was very hard to establish any pattern of trust with a program like that
249because you couldn't figure out what it was doing.
250I mean the other programs it was playing couldn't figure them out
251and so they would end up more or less defaulting to thinking:
252every turn is like the last time I'm gonna see you. So I might as well defect.
253What to me is mind blowing about this
254is that these four principles being nice, forgiving,
255provokable and clear
256is a lot like the morality that has evolved around the world
257that is often summarized as an eye for an eye.
258It's not Christianity, by the way.
259It's not to not turn the other cheek philosophy,
260it's some older philosophy.
261What's interesting is that while Tit for Two Tats would've won the first tournament,
262it only came 24th in the second tournament.
263This highlights an important fact:
264in the repeated prisoner's dilemma, there is no single best strategy.
265The strategy that performs best always depends on the other strategies it's interacting with.
266For example, if you put Tit for Tat in an environment with only the ultimate bullies of always defect,
267then Tit for Tat comes in last.
268I wanted to see whether, for example, the Tit for Tat did well because it did well with really stupid rules
269that didn't do well with it all themselves that basically it took advantage of people.
270So he ran a simulation where successful strategies in one generation
271would see their numbers grow and unsuccessful ones would see their numbers drop.
272In this simulation, the worst performing strategies quickly shrink and go extinct,
273while the top performing strategies become more common.
274Harrington, the only nasty strategy in the top 15, first grew quickly,
275but then as the strategies it was preying on went extinct,
276Harrington's numbers also quickly dropped.
277This shows a main benefit of this simulation
278because it tests how well a strategy does with other successful strategies.
279After a thousand generations, the proportions are mostly stable
280and only nice strategies survive.
281Again, Tit for Tat comes out on top,
282representing 14.5% of the total population.
283Now this process may sound similar to evolution,
284but there is a subtle difference,
285which is that in this case there are no mutations.
286So it's actually an ecological simulation.
287But what if the world you started in was different?
288Imagine a world that is a really nasty place to live,
289more or less populated with players that always defect,
290except there's a little cluster of tit-for-tat players that live in some kind of nucleus
291and they get to play with each other a lot because they're geographically sequestered.
292They will start building up a lot of points,
293and also because that translates into offspring,
294they'll start to take over the population.
295So in fact, Axelrod showed that a little island of cooperation can emerge
296and spread and eventually will take over the world,
297which is fantastic.
298How can cooperation emerge
299in a population of players who are self-interested?
300Who are not trying to be good because they're good-hearted.
301You don't have to be altruistic.
302You could be looking out for number one for yourself and your own interests.
303And yet cooperation can still emerge.
304Some argue that this could explain how we went from a world full of completely selfish organisms
305where every organism only cared about themselves
306to one where cooperation emerged and flourished.
307From impalas grooming each other to fish cleaning sharks.
308Many life forms experience conflicts similar to the prisoner's dilemma,
309but because they don't interact just once,
310both can be better off by cooperating.
311And this doesn't require trust or conscious thought either
312because the strategy could be encoded in DNA,
313as long as it performs better than the other strategies,
314it can take over a population.
315Axelrod's insights were applied to areas like evolutionary biology and international conflicts,
316but there was one aspect that his original tournaments didn't cover.
317What happens if there is a little bit of random error in the game?
318Some noise in the system.
319For example, one player tries to cooperate, but it comes across as a defection.
320Little errors like this happen in the real world all the time.
321Like in 1983,
322the Soviet satellite-based early warning system detected the launch of an intercontinental ballistic missile from the US
323but the US hadn't launched anything.
324The Soviet system had confused sunlight reflecting off high altitude clouds with a ballistic missile.
325Thankfully, Stanislav Petrov, the Soviet officer on duty, dismissed the alarm.
326But this example shows the potential costs of a signal error
327and the importance of studying the effects of noise on those strategies.
328The word game sounds like it's a children's game or,
329you know, there's some something, a misnomer maybe in calling it game theory
330because this is, these are life and death matters obviously.
331And as you mentioned that this came up in the Cold War.
332I mean it could actually be life and death of the whole planet,
333the whole we could annihilate human civilization.
334So these are not games in any kind of trivial sense,
335it's just the term that is used by mathematicians and theorists.
336When Tit for Tat plays against itself in a noisy environment,
337both start off by cooperating,
338but if a single cooperation is perceived as a defection,
339then the other Tit for Tat retaliates and it sets off a chain of alternating retaliations.
340And if another cooperation is perceived as a defection,
341then the rest of the game is constant mutual defection.
342Therefore, in the long run,
343both would only get a third of the points they would get in a perfect environment.
344Tit for Tat goes from performing very well to performing poorly.
345So how do you solve this?
346Well, you need a reliable way to break out of these echo effects.
347And one way to do this is by playing Tit for Tat, but with around 10% more forgiveness.
348So instead of retaliating after every defection,
349you only retaliate around nine out of every 10 times.
350This helps you break out of those echoes
351while still being retaliatory enough to not be taken advantage of.
352I mean, think about it, by design, all they can do is lose or draw.
353And yet when the results of all interactions are tallied up,
354they come out ahead of all other strategies.
355Similarly, always defect can never lose a game.
356It can only draw or win,
357but overall, it performs extremely poorly.
358This highlights a common misconception
359because for many people when they think about winning,
360they think they need to beat the other person.
361In games like chess or poker, this is true
362since one person's gain is necessarily another person's loss,
363so these games are zero sum.
364But most of life is not zero sum.
365To win, you don't need to get your reward from the other player.
366Instead, you can get it from the banker.
367Only in real life, the banker is the world.
368It is literally everything around you.
369It is just up to us to find those win-win situations, and then work together to unlock those rewards.
370Cooperation pays even among rivals.
371From 1950 to 1986,
372the US and Soviet Union had trouble cooperating
373and both kept developing nukes.
374But then from the late '80s onwards,
375they started reducing their nuclear stockpiles.
376They too had learned how to resolve conflict.
377Rather than making an agreement to abolish all nuclear arms at once
378and essentially turning it into a single prisoner's dilemma,
379they would disarm slowly, a small number of nukes each year
380and then they'd check each other to see that they had both cooperated
381and then repeat the year after, and the year after that.
382All along, checking to ensure mutual cooperation.
383In the more than 40 years since Axelrod's tournaments,
384researchers have continued to study which strategies perform best
385in a variety of environments.
386In doing so, they varied everything from payoff structures to strategies to errors and more.
387Some even allowed the strategies to mutate
388while Tit for Tat or generous Tit for Tat doesn't always come out on top,
389Axelrod's main takeaways still hold:
390be nice, forgiving, but don't be a pushover.
391What I find fascinating is that one of the main things that sets life apart from non-living things
392that life gets to make decisions.
393We get to make choices.
394Choices that don't only change our future, but also the future of those we interact with.
395You see, in the short term, it is often the environment that shapes the player
396that determines who does well.
397But in the long run,
398it is the players that shape the environment.
399So let's play a game,
400the game of life,
401and make your choices wisely because their impact may reach further than you think.