The Origin of Reason

Associative Memory

For all we know, animals are only capable of memory association. Of course, we don’t know as yet what really goes on inside their minds, nor for that matter do we know much about what goes on in  human minds other than our own. But for humans we extrapolate from the little we know about our own minds. Nevertheless, there is little evidence that any animal is capable of verbal reasoning.

Notwithstanding, memory association is a powerful cognitive faculty. A very wide range of the animal kingdom - not only dolphins, parrots and higher primates but even what we might consider fairly unsophisticated animals, display capabilities such as responding appropriately to a situation, memory and learning. They are even capable of that essential element of reasoning, the conditional response (if A then B). Humans, too, can use this faculty without conscious effort, awareness of the process itself, or any verbal reasoning.

The core functionality in this faculty is the ability to associate. The sight of snow creates an expectation of cold, wet, soft etc. The sight, sound or smell of a predator creates panic and flight. A smell associated with sight, will bring the sight to mind before the sight is seen. Fruit will be sweet and will nourish. Some mushrooms produce the desire to eat them, others will not. At a specific moment in time the association exists. Over time the association may be learnt; the association is created. The storage of the association is the faculty of memory. Learning is a capability to create association. Memory and association are two facets of the same function.

Association is not an exact process. The predator need not be seen at the same angle, or need not be the same individual. A family of resemblances is sufficient and it is not binary (all or nothing) but fuzzy; one instance will create the association strongly and another weakly or bordering on not at all. 

When we open up the brain of an animal or human we find the hardware capable of producing this function. The science of Neurobiology and the Computer Science discipline of Neural Networks provide ever-growing understanding of how neurons, their activation and the synaptic connections between them can produce the range of association effects, learning capabilities and memory that are observed in living creatures.

How does human reason come into this? There is no second brain to be found only in humans that works like a symbolic serial computer. There is only (roughly) the same neural substrate (but more of it and arranged in some new configurations). Where is the additional sequential, symbolic processor we might have expected to find? What follows is one possible answer. The justification of the claim is that the answer might be provably capable of providing all the known features associated with reason using only the associative capabilities currently known to be provided for by the brain’s hardware. The phrase “might be provably capable” refers to the hope of producing simulations of the known observations starting from the premises of the theory. It may not be much to be going on, but, then again, there are widely accepted scientific theories that rely on far less.

Words

To paraphrase, the first step was the word. A human being makes a sound or a syllable. This artificial creation nevertheless comprises sound-sense data just like any feature of the world. The word can associate with an image or a sound (or a feeling, action, physical response etc.) using the same associative memory mechanism that already exists. Every dog owner knows that animals are capable of making this association too. The outstanding feature is the artificial creation of a sense-data object that can then serve in the traditional associative system.

The question now becomes what is the full range of the implications of this new associative mechanism and how this relates to the development of reason.

Associating a Word with a Word

The first innovation that the use of this artificial audio construct allows, is the association of one word with another. Remember that the original use for a word is the association between the artificial audio instance and some non-artificial production (including a human-initiated action, which for this purpose will be treated as a natural, non-artificial, phenomenon). However once there is some limited vocabulary of words, associations can be formed between them. Assume we have the words, “John”, “rock” and “hit”. “John” works with “hit” in a way that’s different from the way “hit” works with “rock”. These can be expressed in rules with an implication for both grammar and reasoning.

What is important is not that grammar and reasoning imply application of rules. That much is obvious. Rather it is that (i) adding words to the associative interplay allows for such rules and (ii) that such rules themselves require no additional capability other than the associative memory functionality of assigning associations and retrieving them.

The second point needs proving. The programming language Wittgen was created for doing just that. Wittgen consists of only two instructions. One assigns one text string to variable of any name and the other retrieves the association. Thus Wittgen incorporates on a textual basis (words only, in the terms of the current discussion) only the associative memory capabilities of the brain discussed so far.

Rules of both grammar and inference can be created using Wittgen. This proves that memory association is sufficient to create these rules with no additional brain hardware needed. Specific examples of such rules are available at the Wittgen web site.

Categorization

A central feature of reasoning is the ability to categorize. Symbolic computer processing goes to great lengths to produce such an ability. However, for memory association systems such as neural networks this ability in inherent. No two sounds are exactly identical and yet both serve as the same key in a memory association retrieval. Thus the two sounds have been classed as belonging to the same category. 

It is not sufficient to be able to name one person “John” and the other “Paul”. There is a need for the word “man” to describe them both; an act of categorization. Once words exist, there are two ways such categorizations can occur. The first is that the word “man” is directly associated with sense-data instances; the experience of seeing a man, whoever he may be. The second is that the words themselves “John”, “Paul” get categorized. This latter effect need not be associated with a generic name. It could simply arise from the fact that “John” and “Paul” can both be used in the same way, with, say, a word such as “hit”. 

Using Words to Describe Words

Once an artifact such as a word is created and included in the associative gameplay, another innovation can arise. A word can be used to describe a group of other words. Just as a word can be created and associated with a group of sense-data or a category of objects, so too a word can also be created and associated with a group of words. A paradigmatic example of this is a word such as “noun”. This allows second-order thinking, or thinking about thought. It also allows rules of logic. 

Free Speech

A word is created. It is associated with objects from the world we interact with. The objects are seen and the word comes to mind unbidden; “red”, “cold” or “home”. Any association can work both ways. Now I say the word “cold” and the memory of cold appears. The memory of cold appears but not the actual experience. The word “John” is said, the memory surges forward but John is not present. 

The existence of words in the universe of association allows disconnect between the word and its association. The word can be said but the object is not present. This leads to two important consequences, the first of which will be dealt with in this section.

The phrase “John is standing here” has a very high change cost. This means that denying it when John is indeed present, is difficult. Without words, much of the cognitive apparatus is involuntary or at least very constrained. The dissociation between words and their references allows the creation of sentences that have almost no change cost. I can say the words or say their opposite; it is a matter of indifference to me which. Now we have a form of association, saying words, which has a low change cost. 

The importance of low change cost sentences is that it sets us on the road to common sense predictions of reality and ultimately to the modern scientific and technological capability of accurate prediction. Knowledge and understanding of the world consists in the ability to create constructs (and ultimately simulations) that will predict the sense-data high cost sentences. These constructs will be procedures, pattern generators, equations etc. However, the entire structure of such predictability, relies on the dichotomy between sense-data high change cost sentences and low change cost procedures and sentences. It is the dissociation between the word and its reference which allows for one side of this dichotomy; the other side was always been there.

This point needs qualification. This reality dissociation existed before the word. After all, there is the memory of a person without the presence of a person. There are dreams. Both of these do not require verbal skills and presumably existed prior to the word. This is thus not a complete innovation. However, the quality, the range and possibilities are far greater once words can be used. A story can now be told with great detail. What is more, memory and dreams are, at best, only semi voluntary. Telling a story can be entirely voluntary.

Truth

This section presents another important consequence of the ability to dissociate the word and its reference. There now arises the concept of falsity and, from this, the concept of truth.

One can say a sentence that does not match the sense-data sentence. This state of not matching is described by a new word: “false”. The causes are many and diverse. One could have created a procedure that achieves the false result. In that case one is “wrong”. Another important possibility is that of lying. I can say any sentence I can, regardless of what I actually saw or experienced or which sentences I find in the privacy of my own mind to have high change cost. I can say any sentence out loud and I can tell another person any sentence. We call this process lying. Falsity and lying are a direct consequence of the potential dissociation between words and their reference.

Once we can label sentences and a lie or as simply false, other sentences may be labeled as “true”. This is a syntactic version of the concept of “truth”.

Simulations

Words allow the development of verbal reasoning in the form of simulations. A simulation is a procedure or calculation whose results can be compared to a sense-data high change cost sentence. However, the simulation does more than just produce a testable result. It suggests that the simulation is in some way mirroring a procedure in an “external” world that is actually producing this result. In other words the simulation is understood as a description of “what is actually going on”.

In some cases a simulation is a description of an unfolding of situations that could, in theory, be observed and, if so observed, would elicit high change-cost sentences matching those in the simulation itself. However, this is not true of all simulations.

Simulations consist of component mechanisms, procedures or rules. The component rules may take part in more than one simulation. For many reasons, these component rules are expected to work in a similar manner in any simulation in which they take part. For example, rules of logic are expected to work in the same way regardless of the simulation/calculation they are used in. This requires the creation of rules about rules or descriptions and procedures whose reference is the verbal reasoning process itself. The more simulations a rule is required for, the higher the change cost of the rule.

Writing

30k+ years ago, drawing existed. Yet 2,300 years ago advanced societies (Celts/Gauls) in the same region (Non-Mediterranean Europe) had not developed writing. Creating a written symbol for the word is an essential advance in the process of creating complex procedures of verbal reasoning. A drawing is an artifact that associates, for obvious reasons, with visual sense-data. A word, too is an artifact that shares some of the properties of a drawing in its association with the visual. However a written symbol that associates to a word is a far later development. 

The nature of memory brings about significant failures in verbal reasoning. In short term memory, holding new facts crowds out other facts. Holding a six digit number in short memory is possible, but most people will not be able to repeat a twenty digit number presented to them seconds earlier. In Artificial Neural Networks too, simulations of associative memory can be shown to limit the total number of different associations held in a given number of neurons. (It is possible to memorize long passages but perhaps one may suggest that this involves a different mental process. It seems, instead to be related to the ability to execute a long sequence of physical actions with very little conscious intervention in the stages of the sequence itself.)

This means that verbal reasoning must necessarily consist of short sequences of procedural processing. This greatly limits the possibilities for sophisticated sequences of reasoning that require more components. Writing symbols down allows for overcoming these limitations. Examples range from long multiplication to written texts and logical proofs.

Mathematical and Logical Symbols

There is another kind of word or symbol that plays an important role in symbolic reasoning but is not a simple association with visual sense data nor is it, strictly speaking a word describing other words. A integer, for example, is a form of symbol that enables processing at a far higher level than processing without it. It comes with its own rules of how to manipulate it which have become ever-more sophisticated over time. The history of mathematics and logic can be seen as the development of an ever growing library of new processing symbols and rules that go along with them.

Conclusion

I have tried to show in this post the power of using a model based on neural networks associative memory alone. All the major features of human reasoning as well as a plausible development path can be explained using this perspective.