Why Serialization?

Part Of: [Deserialized Cognition] sequence

Introduction

Nietzsche once said:

My time has not yet come; some men are born posthumously.

Well, this post is “born posthumously” too: its purpose will become apparent by its successor. Today, we will be taking a rather brisk stroll through computer science, to introduce serialization. We will be guided by the following concept graph:

Concept Map To Serialization

On a personal note, I’m trying to make these posts shorter, based on feedback I’ve received recently. 🙂

Let’s begin.

Object-Oriented Programming (OOP)

In the long long ago, most software was cleanly divided between data structures and the code that manipulated them. Nowadays, software tends to bundle these two computational elements into smaller packages called objects. This new practice is typically labelled object-oriented programming (OOP).

OOP- Comparison to imperative style (1)

The new style, OOP, has three basic principles:

  1. Encapsulation. Functions and data that pertain to the same logical unit should be kept together.
  2. Inheritance. Objects may be arranged hierarchically; they may inherit information in more basic objects.
  3. Polymorphism. The same inter-object interface can be satisfied by more than one object.

Of these three principles, the first is most paradigmatic: programming is now conceived as a conversation between multiple actors. The other two simply elaborate the rules of this new playground.

None of this is particularly novel to software engineers. In fact, the ability to conjure up conversational ecosystems – e.g., the taxi company OOP system above – is a skill expected in practically all software engineering interviews.

CogSci Connection: Some argue that conversational ecosystems is not an arbitrary invention, but necessary to mitigate complexity.

State Transitions

Definition: Let state represent a complete description of the current situation. If I were to give you full knowledge of the state of an object, you could (in principle) reconstitute it.

During a program’s lifecycle, the state of an object may change over time. Suppose you are submitting data to the taxi software from the above illustration. When you give your address to the billing system, that object updates its state. Object state transitions, then, look something like this:

OOP- Object State Transitions

Memory Hierarchy

Ultimately, of course, both code and data are 1s and 0s. And information has to be physically embedded somewhere. You can do this in switches, gears, vacuum tubes, DNA, and entangled quantum particles: there is nothing sacred about the medium. Computer engineers tend to favor magnetic disks and silicon chips, for economic reasons. Now, regardless of the medium, what properties do we want out of an information vehicle? Here’s a tentative list:

  • Error resistant.
  • Inexpensive.
  • Non-volatile (preserve state even if power is lost).
  • Fast.

Engineers, never with a deficit of creativity, have invented dozens of such information vehicle technologies. Let’s evaluate four separate candidates, courtesy of Tableau. 🙂

memory technology comparison

Are any of these technologies dominant (superior to all other candidates, in every dimension)?

No. We are forced to make tradeoffs. Which technology do you choose? Or, to put it more realistically, what would you predict computer manufacturers have built, guided by our collective preferences?

The universe called. It says my question is misleading. Economic pressures have caused manufacturers to choose… several different vehicles. And no, I don’t mean embedding different programs into different mediums. Rather, we embed our programs into multiple vehicles at the same time. The memory hierarchy is a case study in redundancy.

CogSci Connection: I cannot answer why economics has gravitated towards this highly counter-intuitive solution? But, it is important to realize that the brain does the same thing! It houses a hierarchy of trace memory, working memory, and long-term memory. Why is duplication required here, as well? So many unanswered questions…

Serialization

It is time to combine OOP and the memory hierarchy. We now imagine multiple programs, duplicated across several vehicles, living in your computer:

OOP- Memory Hierarchy

In the above illustration, we have two programs being duplicated in two different information vehicles (main memory and hard drive). The main memory is faster, so state transitions (changes made by the user, etc) land there first. This is represented by the mutating color within the objects of main memory. But what happens if someone trips on your power cord, unplugging your CPU before main memory can be copied to the hard drive? All changes to the objects are lost! How do we fix this?

One solution is serialization (known in some circles as marshalling). If we simply write down the entire state of an object, we would be able to re-create it later. Many serialization formats (competing techniques for how best to record state) exist. Here is an example in the JavaScript Object Notation (.json) format:

{“menu”: {
“id”: “file”,
“value”: “File”,
“popup”: {
“menuitem”: [
{“value”: “New”, “onclick”: “CreateNewDoc()”},
{“value”: “Open”, “onclick”: “OpenDoc()”},
{“value”: “Close”, “onclick”: “CloseDoc()”}
]
}
}}

Applications

So far, we’ve motivated serialization by appealing to a computer losing power. Why else would we use this technique?

Let’s return to our taxi software example. If the software becomes very popular, perhaps too many people will want to use it at the same time. In such a scenario, it is typical for engineers to load balance: distribute the same software on multiple different CPUs. How could you copy the same objects across different computers? By serialization!

CogSci Connection: Let’s pretend for a moment that computers are people, and objects are concepts. … Notice anything similar to interpersonal communication? 🙂

Conclusion

In this post, we’ve been introduced to object-oriented programming, and how it changed software to becoming more like a conversation between agents. We also learned the surprising fact about memory: that duplicate hierarchies are economically superior to single solutions. Finally, we connected these ideas in our model of serialization: how the entire state of an object can be transcribed to enable future “resurrections”.

Along the way, we noted three parallels between computer science and psychology:

  1. It is possible that object-oriented programming was first discovered by natural selection, as it invented nervous systems.
  2. For mysterious reasons, your brain also implements a duplication-heavy memory hierarchy.
  3. Inter-process serialization closely resembles inter-personal communication.

Modularity & The Argument From Design

Part Of: Cognitive Modularity sequence
See Also: Fodor: Modularity of Mind
Content Summary: 1600 words, 16min read

Introduction

This post represents an argument for a particular thesis, known as massive modularity. This thesis, particularly popular among evolutionary psychologists, states that the mind is rife with mental modules, and that the cognitive life is the interplay between them.

What is a mental module? If you don’t have a clear grasp on what that means, I recommend just glancing my summary of Fodorian modularity. Bear in mind, though, that here the term is used somewhat differently: modules here may be some subset of the listed properties.

The following argument is not my own, it is rather an interpretation of Carruther’s argument, which is presented in this text, under Section 1.3.

Motivators From Biology

Carruthers starts by surveying the biological literature for instances of modularity. And he finds it, by the truckload:

There is a great deal of evidence from across many levels in biology to the effect that complex functional systems are built up out of assemblies of sub-components. This is true for the operations of genes, of cells, of cellular assemblies, of whole organs, of whole organisms, and of multi-organism units like a bee colony. And by extension, we should expect it to be true of cognition also, provided that it is appropriate to think of cognitive systems as biological ones, which have been subject to natural selection.

Amongst other sources, he cites the following research:

  • West-Eberhard, 2003. Developmental Plasticity and Evolution.
  • Seeley, 1995. The Wisdom of the Hive: the social physiology of honey bee colonies.

We thus possess considerable biological reason to believe that:

(3) Natural selection selects for modularity at a variety of different levels.

A Role For Evolvability

It’s one thing to observe natural selection promoting modularity, it is another to understand why it is doing so. To do this, we must appeal to the concept of evolvability.

Biological populations tend to conform themselves to ecological niches. That is, a species tends to adopt a particular survival strategy that exploits a certain subset of the local biosphere. Let me here decorate a concept I like to call niche distance: two species said to be in direct competition are so in virtue of the fact of short niche distance, etc. Thus, we could say that the niche distance between two types of weeds in your backyard is small, and the niche distance between the weed and the bald eagle is large.

The fact that niches change is one of the drivers for biological evolution. For example, as the earth warms in the coming centuries, mammalian species will need to acclimate to a different climate, which entails a changed vegetative response, which entails a need for change in eating patterns, etc. Such niche fluctuations are ubiquitous.

We know that evolution is driven by the engine of mutation. But mutation is simply a stochastic, quantum mechanical phenomenon:  there is no way to “speed it up”. Species typically cannot keep pace with niche fluctuations by directly modulating the rate of mutation. Rather, the genetic infrastructure of species must be able to harness mutations to keep pace with niche fluctuations. To put this concept of evolvability very crudely: natural selection does not only select for number of muscles, but also the ability to grow new ones.

(1) Evolvability is selected to allow for fluctuations within an ecological niche.

This video is a cute exploration of how evolvability may be supported in microorganisms by direct tampering of the genetic replication engine. But for larger organisms, the loci of behavior is trans-cellular. The sheer geometry of size compelled cells to become heterozygous, to constitute interdependent systems. The question of mutation containment, then, becomes central: is it possible for evolution to improve upon one function of an organism, without simultaneously affecting other functions?

Here, finally, is where modularity comes into play. One of the most important features of modularity is encapsulation: the hiding of information within specific containers. Rather than all functions affecting all other functions, computational processes erect walls around themselves, and communicate through them in a controlled fashion. Modular encapsulation is thus seen as a prerequisite for mutation containment:

(2) Modular subsystems are a necessary ingredient for evolvability.

Taken together, premise (1) and (2) support (3) in the following way:

Massive Modularity- Argument From Design- Evolvability

Motivators From Computer Science

In the above section, we were given a nice intuition regarding Premise 2: that modularity affords for mutation containment. But perhaps this intuition can be buffered with evidence from somewhere else entirely:

The basic reason why biological systems are organized hierarchically in modular fashion is a constraint of evolvability. Evolution needs to be able to add new functions without disrupting those that already exist; and it needs to be able to tinker with the operations of a given functional sub-system – either debugging it, or altering its processing in response to changes in external circumstances – without affecting the functionality of the remainder. Human software engineers have hit upon the same problem, and the same solution.

Two of the most widely used languages nowadays are C++ and Java. Languages in this class are often described as ‘object-oriented’. Many programming languages now require a total processing system to treat some of its parts as ‘objects’ which can be queried and informed, but where the processing that takes place within those objects isn’t accessible elsewhere. This enables the code within the ‘objects’ to be altered without having to make alterations in code elsewhere, with all the attendant risks that this would bring; and it likewise allows new ‘objects’ to be added to the system without necessitating wholesale re-writings of code elsewhere. And the resulting architecture is regarded as well nigh inevitable (irrespective of the programming language used) once a certain threshold in the overall degree of complexity of the system gets passed.

Interestingly, since the need for modular organization increases with increasing complexity, we can predict that the human mind will be the most modular amongst animal minds. This is the reverse of the intuition shared by many philosophers and social scientists, who would be prepared to allow that animal minds might be organized along modular lines, while believing that with the appearance of the human mind most of that organization was somehow superseded and swept away.

We extract the following argument from the above appeal to object-oriented programming (OOP):

(4) Software engineering suggests that OOP (modularization) is necessary to manage increasing complexity.
(5) Biological systems are very complex.

These premises buffer our Premise 2.

(2) Modular subsystems are a necessary ingredient for evolvability.

Massive Modularity- Argument From Design- OOP

I particularly enjoyed the originality of this argument. Even though software engineering is notoriously bad at quantifying its practices, its trajectory surely sheds some light on other disciplines. As a computer scientist, this argument made me speculate what other trends, current or future, could be brought to bear on such questions. The interchange between computer science and cognitive neuroscience is broad… with things like neuromorphic computing flowing in one direction, and information theory flowing in the other…

Is Mind Subject To Natural Selection

This phase of the argument is the most philosophical. The question is whether mental processes are subject to the forces of natural selection.

Carruthers begins with a fairly uncontroversial premise:

(6) The central nervous system is subject to natural selection.

So much, so obvious. But the crux of the issue is how to relate mind and brain. Carruthers wants to argue that:

(7) The central nervous system underwrites the mind.

However, this premise falls squarely into a philosophy of mind morass. Carruthers suggests a way forward is to notice that most mainstream approaches (“anyone who is neither an an epiphenomenalist nor an eliminativist about the mind”) support such a premise (see this post for some definitions).

If we find ourselves sympathetic to 7, we are led by the nose to Proposition 8:

(8) Mental processes are subject to natural selection.

Massive Modularity- Argument From Design- Mental Evolution

How Many Minds

While the weight of this argument labors to support the reality of computational modules, we must also spare some words to motivate massive modularity. Carruthers, leveraging Simon, H’s 1962 paper The Architecture of Complexity, points out that the question is one of degrees. Let us try to imagine a modularity thesis that is non-massive:

Moderate Modularity

The x-axis captures number of modules, the y-axis leverages David Marr’s concept of Tri-Level Analysis.  The concave shape of the curve represents the claim that, while the number of neurological functions may be large, the number of computational processes (e.g., belief, desire, motivation) is small.

In contrast, the shape of massive modularity thesis is convex:

Massive Modularity

While Carruthers elsewhere motivates massive modularity by way of task analysis and ethological surveys, he here defends this latter thesis by appealing to the empirically-robust observation that the brain appears to process its algorithms in parallel, and this would be impossible without a relatively plentiful number of processing units. So we have stumbled upon our last premise:

(9) In the mind, massive modularity is computationally superior to moderate modularity.

Putting It All Together

All that remains is to glue together the sub-conclusions of the above arguments. Specifically, take the following propositions:

(3) Natural selection selects for modularity at a variety of different levels.
(8) Mental processes are subject to natural selection.
(9) Within the mind, massive modularity is computationally superior to moderate modularity.

From these, it is clear we have successfully motivated our thesis:

(10) Natural selection selects for massive modularity in the mind

The entire argument, then, is pictured below.

Massive Modularity- Argument From Design- Summary

Concluding Thoughts

While I happen to affirm Premise 8, I feel like Carruthers – and even more so myself – do a poor job at motivating it. This observation is particularly painful because it is arguably the central thesis of evolutionary psychology. Mental note-to-self: revisit that section of the argument.

All told, I find this argument fairly compelling, although I would like to get more clear on several of its distinctions.