Agent Detection: Life Recognizing Itself

Part Of: Demystifying Sociality sequence
Content Summary: 1400 words, 14 min read

David Hume once observed:

There is an universal tendency among mankind to conceive all beings like themselves, and to transfer to every object those qualities of which they are intimately conscious. We find faces in the moon, armies in the clouds; and, by a natural propensity, if not corrected by experience and reflection, ascribe malice or goodwill to every thing that hurts or pleases us … trees, mountains and streams are personified, and the inanimate parts of nature acquire sentiment and passion.

Today, we will learn how we came on the ability to discover other animals in the world.

Response Patterns To Predation

Today, we discuss behaviors induced by predation. Did you know that even bacteria can do predation?

  • On detecting energy-laden chemicals, it will swim towards it via a process known as positive chemotaxis.
  • On detecting noxious chemicals, it will instead swim away via negative chemotaxis.

Chemotaxis doesn’t require a nervous system, which is nice because bacteria don’t have one. This nicely illustrates a key lesson in biology: competent behavior does not require comprehension. The only things required here are stimulus-response (SR) maps, which are just as mechanical as the button linking the entrance of your house to a doorbell.

Contra B.F Skinner and his school of radical behaviorism, mammals construct mental representations of their environment. But you can still find SR maps in reflexes (e.g., your knee recoiling from a doctor’s mallet) and fixed action patterns (e.g., the Sphex building her nest).

Of course, S-R maps are metabolically costly, and easy for social predators to outmaneuver. Mammals improved on this approach via re-purposing their endocrine system: the amygdala drives the hypothalamus into one of two modes:

  • Sympathetic nervous system, also known as the fight-or-flight response, prepares your body for action. Symptoms include heart rate increase, tunnel vision, dilated pupils, flushed skin, dry mouth, and slowed digestion.
  • Parasympathetic nervous system, also known as rest-and-digest, restores normal metabolic function (e.g.,  digestion).

The sympathetic nervous system prepares the body for intense activity (amusingly, it is used by both predator and prey).

Two Agency Detection Faculties

Animals with complex nervous systems possess a wide range of sense data. But perceptions don’t include “Lion Warning” labels; instead, sense-data is encoded in neuronal spike trains (roughly, a string of 1s and 0s):

Agency Detection- Interpreting Sense-Data (1)

Fortunately, just as machine learning algorithms feed on data to generate prediction machines, your brain ingests such sensory data to produce inferences about predators and prey. What kinds of algorithms might it use to this effect?

Consider, for a moment, the gazelle. Lion-detector algorithms would surely benefit this creature. However, the perceptual signatures of lions significantly overlaps other gazelles: both have faces, four limbs, the ability to move at great speed, etc.

A gazelle perceives the outline of some as-yet-indeterminate animal concealed in the underbrush: should it simply try to resolve the ambiguity? By no means! While computing identity remains worthwhile, it also pays to immediately invoke the sympathetic system (“prepare for the worst, hope for the best”). We thus see a need for two distinct mental modules:

  • The Agent Detector module is responsible for detecting agents generally. The Agent Detector module is informed by multiple algorithms that search for specific features of an environment. 
  • The Agent Classifier module is responsible for differentiating between agents: predator from prey, friend from foe; it answers the question “so there’s an organism over there: what is it?” 

Perceptual Fluency, Relationship Models, Affect Signature

Like all processes subject to natural selection, the Agent Classifier is not built in the service of truth. Tinkering with the existing software is only preserved when the changes maintain or promote that organism’s biological fitness. We can nevertheless see four classifications that would honor this harsh criteria:

  • Predator: noticing that an animal is a predator, enables differential activation of fight-or-flight, which improves chances of survival.
  • Prey: noticing that an animal is prey, enables differential activation of fight-or-flight, which reduces the risk of starvation.
  • Kin: populations engaged in sexual reproduction are genetically motivated to help their kin (c.f., inclusive fitness). Noticing family members underwriting this ability.
  • Conspecific: social populations often engage in tasks which require coordination. Organisms able to recognize one another in such an environment stand to benefit politically.

The above labels are in fact used reached by a wide variety of organisms. How did they arrive at these abilities? The first clue lies in the mere exposure effect: that which is familiar exudes warmth. Two examples:

  • In studies of interpersonal attraction, the more often a person is seen by someone, the more pleasing and likeable that person appears to be.
  • In another study, subjects were shown nonsense symbols that resembled Chinese characters.  Each character was shown from 0–25 times.  The subjects were then asked to rate how they felt about each character. Eleven out of twelve times, the character was liked better when it was in the high frequency category.

The more you encounter a certain perceptual signature that doesn’t attack you, the easier that signature is to decode (perceptual fluency), and the more “good vibes” you get from the experience.

Besides these foundational mechanism, mammals have additional modules underwriting their social interactions. Significant relationships are implemented with relationship models: finite databases in your brain that track your interactions with significant individuals. But Capgras syndrome complicates this picture somewhat. An example:

Mrs. D, a 74-year-old married housewife, recently discharged from a local hospital after her first psychiatric admission, presented to our facility for a second opinion. At the time of her admission earlier in the year, she believed that her husband had been replaced by another unrelated man. She refused to sleep with the impostor, locked her bedroom and door at night, asked her son for a gun, and finally fought with the police when attempts were made to hospitalise her.

It turns out that people associate signs of normal, autonomic emotional arousal on recognizing close relationships. While Mrs. D relational model produced the same memories, her affective response to her husband was found to be damaged: he felt like a stranger to her. Your emotional encoding of significant individuals, their affect signature, is so powerful that your brain will privilege its information over your memories, should they ever contradict.

Here then is the information processing view of life recognizing itself:

Agency Detection- Information Processing v1

The Ability To See Faces

In 1976, NASA’s Viking 1 was orbiting Mars, exploring the surface for possible landing sites. Here’s one of its pictures, in the Cydonia region:

Agent Detection- Face On Mars

Striking, no?

While the popular reaction involved speculation of extraterrestrial intelligence, the scientists were, of course, a bit less credulous. Presented with such examples, it would be easy for us to get lost in the spooky feelings, or in dissecting superstitious tendencies. The most fertile explananda whispers to us but quietly. Why are such false positives more common than false negatives? We will return to this question in a moment.

Richard Feynman once said:

What I cannot create, I do not understand.

Even by this exacting metric, face detection is a solved problem. The software operating your smartphone’s camera is able to detect faces using a machine learning algorithm. We even know which area of your brain operates the wetware version of this algorithm. 

In Defense Of False Positives

For any such binary classification task, four outcomes are possible:

Agency Detection- Binary Classification Outcome Matrix (1)

There are two ways to get face detection wrong. Why are false positives so much more common than false negatives?

This question can only be satisfactorily answered by the fitness landscape. In our environment of evolutionary adaptation (EEA), these two errors induce radically asymmetric costs:

Agency Detection- Binary Classification Outcome Costs (1)

The above cost asymmetry explains this predominance of false positives, tells us why Agency Detector so often sees armies in the clouds.


  • Simple lifeforms simply move away from noxious stimuli. More complex animals instead possess the fight or flight mechanism.
  • Activating fight-or-flight requires two separate abilities: the ability to detect, and the ability to classify, other animals.
  • Animals label their perceptions of one another by three mechanisms: perceptual fluency, affect infusion, and relational models
  • Many different algorithms exist in your brain for detecting agents. One particularly well-understood example is the ability to see faces.
  • False positives appear more frequently because they cost less than false negatives.
    • This explains why we find faces in the moon, and ascribe malice or goodwill to every thing that hurts or pleases us.
Agency Detection- Information Processing v2 (1)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s