I conjecture that all minds are real-time control systems.

In this post I will explain what that means and why it seems to be true.

Creatures and Real-Time Systems

Consider, if you will, artificial creatures that exist in either the real world or some model thereof. These [ro]bots do not know the environment beforehand, at least not all of it. Sure they may know and learn some universal traits of the environment. But there will always be changes.

And those changes can happen quickly. Meanwhile the creature itself can move very quickly. Even if it doesn't want to move, it may be moved. Or it may fall down. All of these interactions are happening in a range of speeds dictated by physics.

An artificial creature that lives just in a software simulation could have a lot of freedom as far as time spent thinking before reacting. This freedom is not necessarily available in reality.

In software land, it's quite easy to make a program that responds much slower than an animal to events. And any system, be it software or a mix of hardware and software, can get fouled up temporarily and miss a deadline. In fact, that's the normal state of affairs for a lot of the computing devices you interact with on a daily basis.

Does your desktop or laptop computer or tablet or phone respond to every one of your interactions instantly and consistently? Of course not. Cheap general-purpose consumer computing devices and operating systems (e.g. Linux, Windows, Darwin) are slower or faster depending on what else they are processing and what other networks/systems they are waiting on. They are at best soft real-time.

The consequence of a missing a deadline on a soft real-time system is merely annoying the fuck out of the user. But for other applications, such as patient heart-rate monitoring, aircraft jet engine control, car airbag deployment, offshore drilling platform positioning, etc. we have hard real-time systems.

The consequence of missing a hard real-time deadline could be fatal to either the system or to various humans involved. Or as Payton and Bihari [1] put it, "A timing constraint is hard if small violations of the constraint result in significant drops in a computation's value."

The Stubborn Real World

It has been pointed out that "the real world stubbornly refuses to operate faster than real time” [2]. Likewise, the real world stubbornly refuses to operate slower than real time.

Imagine a four-legged "intelligent" robot which does not respond reliably or quickly enough for its real-world timing constraints. You walk over to it while it is walking, touch it, and it falls over. That doesn't seem to be very intelligent or useful. Try tipping over a cat while it's walking.

Now imagine balancing on one foot. Try it. If you can't do this, then you need to practice. There are machines that can do similar things, and they all involve feedback control loops. In other words, measuring where your body is and correcting the position of your various body parts so you don't have an unfortunate interaction with the floor. But this correction loop has to run at the right speed. If it slows down, it gets out of whack and you're flailing wildly around to keep balance. Anything more out of whack and the next stop is Floor via the wonderful free transportation system known as gravity.

So, real-time control means that the system reacts in a timely fashion, where "timely" relates to a real world context, not in software land.

Or to put it another way: All systems have constraints on various resources and correct behaviors. But real-time systems cannot waste great wads of time in order to satisfy the other constraints [1].

Danger and Mystery

From the viewpoint of control theory, intelligence might be defined as a knowledgeable “helmsman of behavior”. --James S. Albus [3]
Environments are dangerous and unpredictable. For one thing there's agents, such as humans, cats, insects, etc. wandering about. There are all the inanimate objects in an environment like chairs and rocks and tumbleweeds. And the agents are always moving the inanimate objects around and changing them, taking them away, introducing new things.

Large inanimate objects, such as walls and buildings often change slowly, but sometimes relatively big objects are changing and moving such as doors and windows. Or the creature may be changing position fast relative to one of those big objects, such as a bird flying into a building.

And then there's the weird fact that the bodies of meat machines (aka biological organisms) are always changing. The most obvious phase is of course the baby growing into an adult. Of course, you might argue that's a slow change. Perhaps it's not enough to warrant a real-time control system.

But we still have all those things happening around the body like people talking, fierce creatures attacking, cars careening down the street, etc. And there's also all that internal subconscious upkeep going on. Biological bodies, as well as many electromechanical machines, are constantly keeping track of and updating their vital survival needs. In animals this is called homeostasis.

Homeostasis means that the creature keeps its internal environment stable. This is typically accomplished with various feedback control mechanisms.

All of these considerations--interacting with the real world, interacting with other creatures, internal self-maintenance--seem to imply that the mind is a real-time control system.


Argument: But surely only part of the mind is a real-time control system. Our higher mental capacities like language, social interaction, conscious reflection, planning, reasoning, arguing on blogs, etc. are seemingly uncoupled from the real time nature of the lower quick-reactive support system.

In fact, one might prefer to think in terms of a simplified layered architecture in which the lower layer is real-time (and perhaps itself split between hard real-time and soft real-time). The upper layer contains all those capacities of mind which aren't real-time.

A cognitive architecture in which the higher layer is not real-time.

However, as Albus [3] has proposed, an engineered mind may have response time constraints throughout the entire hierarchy that are due to the nature of real-time control systems. In his example, you start at the bottom servo layer with a 30 millisecond short term memory or planning horizon (sensory sample and output update intervals of 3 milliseconds). Every layer going up has an approximate order of magnitude increase in time scale. Level 2 is 300 milliseconds memory and planning with a 30 millisecond detection and command interval, Level 3 is 3 seconds with 300 milliseconds intervals, and so on until you get to Level 7 in which the time scale is for an entire day.

Real-time control layers of a cognitive architecture (adapted from [3]).

Another consideration is that the pressures of real-world time constraints could result in high frequency "dumb" reactions being better for a creature than low frequency "smart" decisions:
For example, it may be necessary to avoid a hazard before there is time to assess all possible paths for avoidance. Such actions may be suboptimal when considered in the absence of time constraints. However, when time is taken into account, these actions may be the best possible under the given circumstances. Taking the time to assess the alternatives may leave no time to execute the best alternative once it is found [1].
So...at the very least there is a real-time control layer in all minds. For the more intelligent minds like humans, the real-time reaction requirements may time-constrain the higher layers. Slower but smarter assimilation and decision-making is often trumped by reliable fast ignorant behaviors in real-world contexts.


[1] D.W. Payton&T.E. Bihari, "Intelligent Real-Time Control of Robotic Vehicles," COMMUNICATIONS OF THE ACM, August 1991, Vo1.34, No.8.

[2] S. Thrun, J. Canny, S. Russell&P. Norvig, Artificial Intelligence: A Modern Approach, 2nd Ed. Prentice Hall, 2002.

[3] J.S. Albus, "Outline for a Theory of Intelligence," IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 21, NO. 3, MAY/JUNE 1991.