[Excerpt] The Tragedy of Commonsense Morality

Part Of: Demystifying Ethics sequence
Content Summary: 1500 words, 15 min read.

Excerpts are not my writing! This comes from Joshua Greene’s excellent book:

Moral Tribes: Emotion, Reason, and the Gap between Us and Them

The book goes on to present an interesting solution to the below problem. Check it out!

The Tragedy of the Commons

The following parable – entitled tragedy of the commons – originates from Garrett Hardin’s 1968 paper:

A single group of herders shares a common pasture. The commons is large enough to support many animals, but not infinitely many. From time to time, each herder must decide whether to add another animal to her flock. What’s a rational herder to do? By adding an animal to her herd, she receives a substantial benefit when she sells the animal at market. However, the cost of supporting that animal is shared by all who use the commons. Thus, the herder gains a lot, but pays only a little, by adding an additional animal to her herd. Therefore, she is best served by increasing the size of her herd indefinitely, so long as the commons remains available. Of course, every other herder has the same set of incentives. If each herder acts according to her self-interest, the commons will be completely eroded, and there will be nothing left for anyone.

You may recognize the economic structure of this game from the Prisoner’s Dilemma. To win such a game, you must find the magic corner; that is, to accomplish cooperative outcomes despite the temptation of selfishness.

The problem of cooperation is the central problem of social existence. Fortunately, our brains come equipped with the following mechanisms, all of which foster cooperation.

  1. Concern for others. Two prisoners can find the magic corner if they place some value on each other’s payoffs in addition to their own.
    • Faculties: empathy, violence aversion.
  2. Direct reciprocity. Two prisoners can find the magic corner if they know that being uncooperative now will deny the benefits of future cooperation.
    • Faculties: punitive motivation, forgiveness, gratitude
  3. Commitments. Two prisoners can find the magic corner if they are committed to punishing each other’s uncooperative behavior.
    • Faculties: shame, guilt, loyalty.
  4. Reputation. Two prisoners can find the magic corner if they know that being uncooperative now will deny us the benefits of future cooperation with others.
    • Faculties: gossip, embarrassment.
  5. Assortment. Two prisoners can find the magic corner by belonging to a cooperative group, provided that group members can reliably identify one another.
    • Faculties: identity markers, tribalism

We have cooperative brains, it seems, because cooperation provides material benefits, biological resources that enable our genes to make more copies of ourselves. Out of evolutionary dirt grows the flower of human goodness.

The Tragedy of Common Sense Morality

To the east of a deep, dark forest, a tribe of herder raise sheep on a common pasture. Here the rule is simple: each family gets the same number of sheep. Families send representatives to a council of elders, which governs the commons. Over the years, the council has made difficult decisions. One family, for example, took to breeding exceptionally large sheep, thus appropriating more of the commons for itself. After some heated debate, the council put a stop to this. Another family was caught poisoning its neighbors’ sheep. For this the family was severely punished. Some said too severely. Others said not enough. Despite these challenges, the Eastern tribe has survived, and its families have prospered, some more than others.

To the west of the forest is another tribe whose herders also share a common pasture. There, however, the size of a family’s flock is determined by the family’s size. Here, too, there is a council of elders, which has made difficult decisions. One particularly fertile family had twelve children, far more than the rest. Some complained that they were taking  up too much of the commons. A different family fell ill, losing five of their six children in one year. Some thought it was unfair to compound their tragedy by reducing their wealth by more than half. Despite these challenges, the Western tribe has survived, and its families have prospered, some more than others.

To the north of the forest is yet another tribe. Here there is no common pasture. Each family has its own plot of land, surrounded by a fence. These plots vary greatly in size and fertility. This is partly because some Northern herders are wiser and more industrious than others. Many such herders have expanded their lands, using their surpluses to buy land from their less prosperous neighbors. Some Northern herders are less prosperous than others simply because they are unlucky, having lost their flock or their children to disease. Still other herders are exceptionally lucky, possessing large fertile plots of land, not because they are especially industrious but because they inherited them. Here in the North, the council of elders doesn’t do much. They simply ensure that herders keep their promises and respect one another’s property. The vast differences in wealth among Northern families has been the source of much strife. Each year, some Northerners die in winter for want of food and warmth. Despite these challenges, the Northern tribe has survived, and its families have prospered, some more than others.

To the south of the forest is a fourth tribe. They share not only their pasture but their animals, too. Their council of elders is very busy. The elders manage the tribe’s herd, assign people to jobs, and monitor their work. The fruits of this tribe’s labor are shared equally among all its members. This is a source of much strife, as some tribe members are wiser and more industrious than others. The council hears many complaints about lazy workers. Most members, however, work hard. Some are moved to work by community spirit, others by fear of their neighbor’s reproach. Despite these challenges, the Southern tribe has survived. Its families are not, on average, as prosperous as those in the North, but they do well enough, and in the South no one has ever died in winter for want of food or warmth.  

One summer, a great fire burned through the forest, reducing it to ash. Then came heavy rains, and before long the land, once thick with trees, was transformed into an expanse of gently rolling grassy hills, perfect for grazing animals. The nearby tribes rushed in to claim the land. This was a source of much strife. The Southern tribe proclaimed that the new pastures belonged to all people and must be worked in common. They formed a new council to manage the new pastures and invited the other tribes to send representatives. The Northern herders scoffed at this suggestion. While the Southerners were making their big plans, Northern families built houses and stone walls and set their animals to graze. Many Easterners and Westerners did the same, though with less vigor. Some families sent representatives to the new council.

The four tribes fought bitterly, and many lives, both human and animal were lost. Small quarrels turned into bloody feuds, which turned into deadly battles. A Southern sheep slipped into a Northerner’s field. The Northerner demanded a fee to return it. The Southerners refused to pay. The Northerner slaughtered the sheep. The Southerners took three of the Northerner’s sheep and slaughtered them. The Northerners took ten of the Southerner’s sheep and slaughtered them. The Southerners burned down the Northerners farmhouse, killing a child. Ten Northern families marched on the Southerner’s meeting house and set it ablaze, killing dozens of Southerners, including many children. Back and forth they went with violence and vengeance, soaking the green hills with blood.

The tribes of the new pastures are engaged in bitter, often bloody conflict, even though they are all, in their different ways, moral peoples. They fight not because they are fundamentally selfish but because they have incompatible visions of what a moral society should be. These are not mere scholarly disagreements, although their scholars have those, too. Rather, each tribe’s philosophy is woven into its daily life. Each tribe has its own version of moral common sense. The tribes of the new pastures fight not because they are immoral but because they view life on the new pastures from very different moral perspectives. I call this the Tragedy of Commonsense Morality.

Five psychological tendencies tend to exacerbate intertribal conflict:

  1. Naked group selfishness. Human tribes are tribalistic, favoring Us over Them.
  2. Moral disagreement. Tribes have genuine disagreements about how societies should be organized, with different emphases on e.g., the rights of individuals versus the greater good.
  3. Authority question begging. Tribes have distinctive moral commitments, whereby moral authority is vested in local individuals, texts, traditions and deities that other groups don’t recognize as authoritative.
  4. Asymmetry capitalization. Tribes are prone to biased fairness, allowing group-level self-interest to distort their sense of justice
  5. Punitive escalation. The way we process information about social events can cause us to underestimate the harm we cause others, leading to the escalation of conflict.

Morality is nature’s solution to the Tragedy of the Commons, enabling us to put Us ahead of Me. But nature has no ready-made solution to the Tragedy of Commonsense Morality, the problem of enabling Us to get along with Them. And therein lies our problem. If we are to avert the Tragedy of Commonsense Morality, we’re going to have to find our own, unnatural solution: what I’ve called a metamorality, a higher-level moral system that adjudicates among competing tribal moralities, just as a tribe’s morality adjudicates among competing individuals.


Evolutionary Game Theory

Part Of: Game Theory sequence
Content Summary: 1300 words, 13 min read

Prisoner’s Dilemma Review

The classical Prisoner’s Dilemma has following setup:

Two prisoners A and B are interrogated, and separated asked about one another.

  • If both prisoners betray the other, each of them serves 2 years in prison
  • If A betrays B but B remains silent, A will be set free and B will serve 3 years (and vice versa)
  • If both prisoners remain silent, they will only serve 1 year in prison.

We can express the decision structure graphically:

IPD- Prisoner's Dilemma Overview

We can also represent the penalty structure. In what follows, arrows represent preference. CC → DC is true because, given that B cooperates, A would prefer the DC outcome (0 years in prison) more than CC (1 year).

IPD- Prisoner's Dilemma Regret

Our takeaways from our exploration of the Prisoner’s Dilemma:

  • An outcome is strategic dominance happens when one choice outperforms other choices, irrespective of competitor behavior. Here, DD is strategically dominant.
  • Pareto improvement is a way to improving at least one person’s outcome without harming any other player. Here, DD → CC represents such an improvement.
  • Pareto optimal outcomes are those outcomes which cannot be Pareto-improved.

The Prisoner’s Dilemma shows us that strategically-dominant outcomes need not be Pareto optimal.  Although each arrow points towards the origin for that color, the sum of all arrows points away from the origin.

It packages together the tragedy of the commons, a profound and uncomfortable fact of social living. A person can be incentivized towards an outcome that she, and everybody else, dislikes.

Towards Iterated Prisoner’s Dilemma (IPD)

In the one-off game, mutual defection is the only (economically) rational move. If a person chooses to defect, they will likely receive a bad result.

But consider morwhat happens in a more social setting, where players compete for resources multiple times. An Iterated Prisoner’s Dilemma (IPD) has the following structure:


What strategy is best? Let’s consider two kinds of strategies we might adopt. We can imagine some vindictive prisoners always defecting (AD). Other prisoner’s might be more generous, adopting a Tit-for-Tat (TfT) strategy. This has them initially cooperating, and mirroring their opponent’s previous move.

Let’s imagine that there are 200 “prisoners” playing this game, with each strategy adopted by half of the population. Which strategy should you adopt, in such a scenario?

The games look as follows:

  • AD vs AD: { DD, DD, DD,  … }. After 10 rounds: A has 20 years, B has 20 years.
  • AD vs TfT: { CD, DD, DD,  … }. After 10 rounds: A has 18 years, B has 21 years.
  • TfT vs TfT: { CC, CC, CC, … }. After 10 rounds: A has 10 years, B has 10 years.

These computations can be generalized to n rounds:

IPD- Always Defect vs TfT

The tit-for-tat (TfT) strategy wins because TfT-TfT games are collaborative, but these players also aren’t effectively exploited by players who Always Defect (AD).

Which Kinds of Strategies Are Best?

There is an very large number of possible IPD strategies. Strategy design might include considerations such as:

  • Deterministic vs Mixed. Should we follow logical rules, or employ randomness?
  • Impersonal vs Personal. Do we remember the behavior of each opponent? Do we change strategies given what we know of other players?
  • Fixed vs Adaptive. Should we use our experiences to change the above on-the-fly?

Given this behavioral diversity, which kinds of strategy are most successful?

To answer this question, in 1980 Robert Axelrod conducted a famous experiment. He invited hundreds of scholars to enter an IPD tournament, submitting their agent’s decision algorithm digitally. In a computer simulation, every agent played every other agent 200 times. The agent with highest cumulative utility was declared the winner.

Many agent strategies employed quite complex, using hundreds of lines of code. The surprising result was that simple strategies, including Tit-for-Tat, often proved to be superior. Axelrod described three properties shared among successful strategies:

IPD- Characteristics of Winning Strategy

We can call such strategies instances of reciprocal altruism.

Moral and Emotional Implications

The theory of evolution has shown us that biological systems are the product of an optimization process known as natural selection. Only genes that improve reproductive success win over evolutionary time.

From this context, it has long seemed unclear how human beings (and other animals) came to express altruistic behavior.  W.D Hamilton’s notion of inclusive fitness explains why we behave generously to relatives. As J.B.S Haldane famously joked,

I would willingly die for two brothers or eight cousins.

Game theory explains our behavior towards non-relatives. Specifically,

IPD provides insight into moral cognition. It shows how our selfish genes might, purely for selfish reasons, come to promote behaviors that are (reciprocally) altruistic.

IPD similarly explains certain emotional processes. For example, I have posited elsewhere the existence of social intuition generators like Fairness. We can now explain why natural selection generated such “socially intelligent” mental modules.

Application: Vampire Bats

Instead of jail time, we can modify our outcome structure to be more relevant to biology.

IPD- Ecological Prisoner's Dilemma (1)

Thus, we can use game theory to interpret animals competing for resources. Consider, for example, behavior of the vampire bats.

Vampire bats feed on the blood of other mammals. Their energy budget is such that they can tolerate 2 days of food deprivation before starving to death.

On a given night, 8% of adult vampire bats will fail to find food on a given night. But when they do find food, it is often more than they need.

Of course, these animals have a genetic incentive to share blood within family. But you can also observe bats sharing their food with strangers.

How can selfish genes reward altruistic behavior? Because vampire bats are playing IPD:

  • CC (Reward). I get blood on my unlucky nights. I have to give blood on my lucky nights, which doesn’t cost me too much.
  • DC (Temptation). You save my life on my poor night. But I also don’t have to feed you on my good night.
  • CD (Sucker): I pay the cost of saving your life on my good night. But on my bad night I still may starve.
  • DD (Punishment) I don’t have to feed you on my good nights. But I run a real risk of starving on my poor nights.

Towards Evolutionary Game Theory

To show why altruistic bats are more successful? Yes; we need only invent evolutionary game theory (EGT). Recall how natural selection works:

Individuals with more biological fitness tend to leave more copies of their genes.

EGT simply adds this replicator logic to the Iterated Prisoner’s Dilemma (IPD). Players with higher final scores (most resources) leave more descendants in subsequent populations (image credit):


We saw previously that Tit-For-Tat players outperform those who Always Defect. In EGT, this fact establishes how a gene that promotes altruism successfully invaded the vampire bat gene pool:

IPD- EGT Stable Strategies (2)

Of course, iterated games don’t always have one winner. Consider the following food web (structurally similar to Rock-Paper-Scissors, of course).

Snake beats Fox. Fox beats Hawk. Hawk beats snake.

What if the size of the snake population starts out quite small? In that case, hawks and foxes predominate. Since hawks are prey to foxes, the size of the hawk population decreases. But this means the snakes have fewer natural predators.

The above traces the implications of one possible starting point. However, we can use EGT maths to model the entire dynamical system, as follows (image credit):

IPD- Food Web Rock Paper Scissors (1)

With this image, we can see that any starting point will eventually (after many generations), lead to a (⅓, ⅓, ⅓) divide of snakes, foxes, and hawks. This point is the locus of the “whirlpool”, it is also known as an attractor, or an evolutionarily stable state (ESS).


  • The Iterated Prisoner’s Dilemma (IPD) makes game theory more social, where many players compete for resources multiple times.
  • While one-off PD games favor selfish behavior, IPD can favor strategies that feature reciprocal altruism, such as Tit-for-Tat.
  • More generally, IPD strategies do best if they are nice, retaliating, and forgiving. This in turn explains how certain facets of our social and moral intuitions evolved.
  • Evolutionary Game Theory (EGT) extends IPD by adding replicator logic (more successful strategies are preferentially represented in future generations).
  • Evolutionary Stable States (ESS) encode dynamical attractors, which populations asymptotically approach.

Until next time.

An Introduction To Prisoner’s Dilemma

Part Of: Algorithmic Game Theory sequence
Content Summary: 600 words, 6 min read

Setting The Stage

The Prisoner’s Dilemma is a thought experiment central to game theory. It goes like this:

Two members of a criminal gang are arrested and imprisoned. Each prisoner is in solitary confinement with no means of speaking to or exchanging messages with the other. The police admit they don’t have enough evidence to convict the pair on the principal charge. They plan to sentence both to a year in prison on a lesser charge.

Simultaneously, the police offer each prisoner a Faustian bargain. Each prisoner is given the opportunity either to betray the other, by testifying that the other committed the crime, or to cooperate with the other by remaining silent:

  • If A and B both betray the other, each of them serves 2 years in prison
  • If A betrays B but B remains silent, A will be set free and B will serve 3 years in prison (and vice versa)
  • If A and B both remain silent, both of them will only serve 1 year in prison (on the lesser charge)

Do you “get” the dilemma? Both prisoners do better if they each cooperate with one another. But they are taken to separate rooms, where the decisions of the other are no longer visible. The question evolving towards one of trust…

This parable can be drawn in strategy-space:

Prisoner's Dilemma- Overview

Strategic Dominance

Consider Person A’s perspective:

Prisoner's Dilemma- Dominance Player A

One line of analysis might run as follows:

  • If the other person cooperates (top rectangle), Player A would do better defecting (rightmost cell).
  • If the other person defects (bottom rectangle), Player A would do better defecting (rightmost cell).

Thus, no matter what B’s choice, defection leads to a superior result. Let us call this strategic dominance.

Person B’s perspective, in the below figure, is analogous:

Prisoner's Dilemma- Dominance Person B

  • If the other person cooperates (left rectangle), Player A would do better defecting (bottom cell).
  • If the other person defects (right rectangle), Player A would do better defecting (bottom cell).

Thus, the strategically dominant outcome is Defect-Defect, or (D, D).

Pareto Optimality

If the prisoners could coordinate their responses, would they select (D,D)? Surely not.

How might we express our distaste for mutual defection rigorously? One option would be to notice that (C, C) is preferred by both players. Is there anything better than mutual cooperation, in this sense? No.

Let us call the movement from (D,D) to (C,C) a Pareto improvement, and the outcome (C,C) Pareto optimal (that is, no one player’s utility can be improved without harming that of another).

It turns out that (C,D) and (D, C) are also Pareto optimal. If we map all outcomes in utility-space, we notice that Pareto optimal outcomes comprise a “fence” (also called a Pareto frontier).

Prisoner's Dilemma- Pareto Optimality

The crisis of the Prisoner’s Dilemma can be put as follows: (D, D) doesn’t reside on the Pareto frontier. More generally: strategically-dominant outcomes need not reside on the Pareto frontier.

While there are other, more granular, ways to express “good outcomes”, the Pareto frontier is a useful way to think about the utility landscape.

Let me close with an observation I wish I had encountered sooner: utility-space does not exist a priori. It is an artifact of causal processes. One might even question the ethics of artificially inducing “Prisoner’s Dilemma” landscapes, given their penchant for provoking antisocial behaviors.


  • A strategy is called dominant when it always outperforms alternatives, irrespective of competitor behavior.
  • Pareto optimal outcomes are those for which there is no “pain-free” way to improve the outcome of any participant. All such outcomes comprise the Pareto frontier.
  • The Prisoner’s Dilemma illustrates that strategically-dominant outcomes need not reside on the Pareto frontier, or more informally, that acting in one’s self-interest can lead to situations where everyone loses.