The Seed And The Flower
    By Samuel Kenyon | December 30th 2011 11:11 PM | 9 comments | Print | E-mail | Track Comments
    About Samuel

    Software engineer, AI researcher, interaction designer (IxD), actor, writer, atheist transhumanist. My blog will attempt to synthesize concepts...

    View Samuel's Profile
    Right now I'm reading an architecture book from the 1970s called The Timeless Way of Building. So far it has to do with theories of how towns and buildings and other things seem more "alive" than others, and how to achieve this quality--the "quality without a name".

    This of course goes far beyond merely architecture; indeed this book was brought to my attention not by an architect but by people in the UX (user experience) design community. Anyway, this blog post only covers a couple pages out of the book.

    The author, Christopher Alexander, says that we have come to think of buildings, towns, and works of art as "creations." And that "creation" is thought of as a monumental design task, "brought to birth, suddenly, in a single act, whose inner workings cannot be explained, whose substance relies ultimately on the ego of the creator."

    I would interject that the creator might understand the inner workings, but even then, for a complicated project attempted in a process with this mindset, the end result would probably not be completely understandable by the creator. More on that in a minute...

    As Alexander writes:

    The quality without a name cannot be made like this.

    Imagine, by contrast, a system of simple rules, not complicated, patiently applied, until they gradually form a thing. The thing may be formed gradually and built all at once, or built gradually over time--but it is formed, essentially, by a process no more complicated than the process by which the Samoans shape their canoe.

    And if you're thinking that this sounds very much like how biology works, then you have predicted the next key statement on the same page:

    The same thing, exactly, is true of a living organism.

    An organism cannot be made. It cannot be conceived, by a willful act of creation, and then built, according to the blueprint of the creator. It is far too complex, far too subtle, to be born from a bolt of lightning in the creator's mind. It has a thousand billion cells, each one adapted perfectly to its conditions--and this can only happen because the organism is not "made" but generated by a process which allows the gradual adaptation of these cells to happen hour by hour....

    And Alexander claims that there is no other way. Of course, as a transhumanist and a roboticist and an occasional cognitive architect (oh, maybe there is architecture here after all!) I want to be able to create and modify life forms. I want to make artificial organisms, and interfaces between the organic and the non-organic.

    However, I have enough experience with software development and project management to know that what Alexander says is true. I.e., I believe it based on my experience and observations.

    So I have two immediate responses when I read this part of the book:

    1. Examples of systems that look complex, but evolved and/or iterated via simple rules.

    For instance, the cellular automata popularized by Stephen Wolfram in A New Kind of Science create complexity and even randomness from ridiculously simple rules. Behavioral robotics, which started in the 1980s (although there were a few examples long before then), embraced the artifical organism in the environment concept. They had simple programs and no traditional internal models.

    If you've ever made any kind of robot, or a software agent in some environment, you may have discovered how complicated and or unplanned the behavior becomes so easily.

    The adaptability and flexibility of organisms has not yet been matched by artificial creatures. I've discussed some of this before in my article "Softer, Better, Faster, Stronger: The Coming of Soft Cybernetics".

    If you've ever made a mobile robot or any other electromechanical system, and expected it to work perfectly as designed, you have probably encountered a rude awakening. It never works "out of the box" (unless you went through many cycles before putting it in the box). In fact, engineering in general is more of a building out of existing working things. And then your specific project still has to have cycles of test and/or integration, etc. And that brings me to response two: the meta.

    2. The Meta: Development Methodology

    The meta aspect is management of projects and what process the "creator" and his/her team uses. I don't particularly care how similar a process is to biology...although biological evo-devo is quite interesting. I'd love to find ways of doing things that are totally alien to biology.

    It's just that I've found that cycles of development ("iterations" or "sprints") have to happen. And it's much better if there are feedback loops between testing with contexts. And contexts are environments, other artificial equipment, users, and so on.

    Iterative development processes almost always work better than sequential types like Waterfall. In fact, I've never seen a sequential development process work out well. Others have discovered this to. It's probably part of the reason why Agile software development processes are becoming the most popular.

    Anyway, comments about why generative and iterative processes work in our version of the universe are welcome.


    Gerhard Adam
    Isn't that really the essence of chaos theory?  The idea that things can be deterministic, but have so many possible variations that predictability is effectively removed?

    If we consider possible "origin of life" scenarios [without arguing about what occurred] we can surmise that whatever original organisms may have formed would largely have been simple, probably quite deterministic structures.  However, those that could handle variation better would have had a higher rate of survival, and consequently been more successful (i.e. more stable).  At some point, these organisms would've begun the process of reproduction at which point, selection would begin to play a role in determining which competitors would be most able to enter the "next round" and so on.

    As a result, we have a biological system that has developed which is oriented around the traits necessary to survive and replicate, so that the most suitable organisms for any given circumstance will be the most likely to succeed into the future. 

    I would argue that the fundamental principle involved here is unpredictability.  There is no perfect design [nor can there be], which is why there is a perpetual series of developmental niches that are filled by increased variety.  What works well in one situation may not be so good in another.

    Similarly with anything we want to engineer that is to have broader applicability, it must be adaptable to changing circumstances, and also be able to "learn" from that experience.  One of the primary difficulties associated with artificial life ideas is that it is trying to go against what biology does.  In biology, individuals don't evolve; populations do.  More specifically, it is the offspring that will either adapt or go extinct.  In engineering, the attempt is to try and incorporate evolution at the individual level, which seems to be based on the notion that all adaptability can be "learned".

    I expect that if we could engineer a machine that was capable of engineering its own offspring making subtle improvements based on the experience of the original [more like a Lamarckian approach], we would see the same sort of divergence of machine types as we see with species.  The "down-side" is that evolution can be quite wasteful and consequently does require predictability in determining which traits may or may not be successful.  It is similar to what occurs in any kind of general engineering, where people use something in applications that the original designer may never have considered (think spreadsheets).  In other words, we can't think about utility to "evolve" utility because that automatically narrows the scope of possibilities.  Biological evolution succeeds because it only addresses the immediate needs of survival and reproduction working with what it has.    It is this drive which increases diversity.  If you can't fly, then perhaps you can jump?  If you can't run fast, then perhaps you can hide?  It creates no "super-species", but it creates a system that works without intervention.

    While I haven't actually said anything of real significance here and probably just restated the obvious, I thought it might be interesting to start the dialogue to see what else emerges.
    Mundus vult decipi
    Gerhard,  thanks for the comment.

    In engineering, the attempt is to try and incorporate evolution at the individual level, which seems to be based on the notion that all adaptability can be "learned".

    I'm not sure if that's true since most machines and programs aren't designed to handle very much change at all and are not capable of self-modification. Obviously researchers have make self-modifying, learning and/or evolving software but I don't know of much of that making it out of the lab.

    Also it seems that there are design processes which are similar to evolving populations. For example, in graphic design it's common to create many thumbnails or rough concepts, then chose the best out of those based on various kinds of feedback, and keep cycling until the best one makes it to final draft. User interface designers do a similar thing with "prototyping" and wireframes.

    However, as you say, we don't have any machines that autonomously make their own offspring. There are of course software simulations for evolution of agents, but not physical machines.

    One of the big challenges, once you've opened the door to learning and evolution of your agents/machines/whatever, is how to balance the ontogenesis vs. phylogenesis. An individual can learn a lot, but it has to have the learning mechanisms in place. And those came from evolution and were shaped as the individual developed in a context. So there's a lot of different kinds of learning/development in biology and most are not used yet in artificial systems. Now, I don't have the answers or expertise yet on best ways to do this with AI, but I have been thinking about this since around 2003.

    In 2005 I found a draft of a paper by Aaron Sloman and Jackie Chappell called "The Altricial-Precocial Spectrum for Robots" which overlapped with some of my ideas (there's a version of it here). Perhaps I should write a blog post about that. I actually talked to Sloman in person about it. I won't try to summarize the whole thing here, but one important concept to think about is how animals have a mixture of precocial capabilities and altricial capabilities. AI software / robots could be made to have altricial learning of information, e.g. during runtime not design time. And then these chunks of info when combined with combinatorial mechanisms so they can be used and combined in new ways.
    Gerhard Adam
    I think one of the problems regarding machines is that it's like trying to construct a new species without having a defined niche in which it is exist.  In that respect, all existing species are heavily dependent on their history, having obviously benefited from the millions of years in took for them to be refined.

    In my view, this is the primary difficulty in AI considerations;  constructing the necessary motivation.  For whatever reason, each animal has a "desire" to survive and ultimately strives for reproduction.  What motivates a machine?  Why should it care about outcomes?  What does it have at stake?  Without answering these questions, it becomes difficult to analyze why any particular action should be appropriate or not.  Even the evolution of the human brain can be argued to simply be a "short-cut" to avoid having to have too many specific solutions on hand, by allowing the creature to engage in problem solving for "uncertain futures".  In this way, humans can take a much longer maturation process, since we aren't dependent on the immediate knowledge of our parents to develop survival skills.  Instead we are taught general learning and are adaptable, just in case the world is now different than it was when our parents survived in it.

    I guess I'm trying to say that for every mistake in biology, there is a certain irrevocable cost to the organism in question.  As a result, those that survive invariably become 'better".  What is the cost to a machine?  I suppose, in part, I'm arguing that it is the existence of death which creates the necessary incentive which generates the motivation that life depends on.  If you can't actually die, then where does that motivation originate from?

    In that respect, I fault AI for not truly exploring machine intelligence, but rather attempting to simulate human intelligence.  For me, the question would be ... what would a machine do in these circumstances?  what would motivate a machine to take this action?  what is a machine looking to get out of this?

    Despite how they sound, these questions don't require consciousness or intelligence, since they are capable of being addressed by something as simple as bacteria.  Therefore it is something that is fundamental to the process of being alive.

    Anyway ... I suspect I'm rambling now, but that's kind of where I see this going.
    Mundus vult decipi
    In that respect, I fault AI for not truly exploring machine intelligence, but rather attempting to simulate human intelligence. 

    Indeed. I'm perhaps one of only a few who is interested in AI as making artificial creatures. Obviously the field of ALife goes into that to some degree by making simple simulated "creatures" but to my knowledge they aren't trying to achieve the old fashioned AI dream of a full system with common sense, etc. that operates in the real world.

    For me, the question would be ... what would a machine do in these circumstances?  what would motivate a machine to take this action?  what is a machine looking to get out of this?

    I also think that is very important. And we can expand it not just to motivation but to the actual information structures or symbols in a mind. I first realized this back in 2003 when I started studying emotions. Even if you make a biology-inspired emotional architecture, it's not at all like a biological system unless the emotions and feelings bottom out eventually to survival related things.

    It would seem that our brains "store" information in semantic networks with various kinds of structures in which a piece of information is actually links between other types of information and so on. And eventually that must all bottom out to innate egocentric concepts. Without symbol grounding there is no meaning.
    Samuel, Interesting thoughts. Christoper Alexandra was very late in discovering stuff discovered by biologist long time ago. He also appears to have thought that 'Samoans built boats using primitive mental processes". The world of architecture is full of popular inspirational leaders who refuse to investigate or be interested in natures design processes unless it serves to unnecessary curve a form in a way that it will harness more attention.

    The big difference between natural and human design process is the the lack of "design intent" in natural processes. Hence the difficulty in implementing Larmarckian methods. Human designers have intent and this intent reduces both solution space and search space from which come the "efficiency" of human design process - which are indeed poor in exploratory capabilities.

    Gerhad, the view that nature is inefficiency is popular among engineers whose live are spent on creating small inefficiency within well bound search spaces. Search has a computational cost and nature is very careful with costs. The stillborn in the mammalian world are a result of natures drive for efficiency - where it kills off those that less chance of survival before they are born. Nature in human terms is both cruel and miserly.

    Genetic algorithms seem to be for now the only biologically useful approach but suffers from lack of "creative structuring" capable of creating significant diversity in complex designs.

    I a working on CAD based methods to help explore greater possibilities, but allow the human designer to make the selection as it is very difficult to quantify some of the more complex criteria. In a very stage way it resonates with the idea of "Simulating Human Intelligence" . Samuel I would like the reference for this.

    My own thoughts are similar to what is expressed here in that designers have internal models or capabilities of recognizing prospective solutions or emergent solutions and that could be a good basis for a good human computer interaction for exploring design possibilities.

    Sivam, thanks for informing us of your design software.

    However, your comment indicates a severe lack of reading and writing and thinking capabilities on your part. Why even bother to type all that if you're going to basically ignore and misquote everything on this page?
    Gerhard Adam
    The stillborn in the mammalian world are a result of natures drive for efficiency - where it kills off those that less chance of survival before they are born.
    I won't quibble about the concept of efficiency as it applies to biological systems, my point was that nature can be more "wasteful" than most engineering activities are willing to be.  Some species may lay thousands of eggs in the hopes that one survives.  Even in humans we produce millions of sperm, so that one can fertilize an egg.  How many eggs are routinely lost by females?  These are not isolated events, but the hallmark of systems that will exploit an existing process to whatever degree necessary in order to obtain the slightest probability of success.

    These are principles that are well outside the normal view of engineering, but they are quite necessary when looking at evolutionary or iterative processes.
    Mundus vult decipi
    hmmmm..... Creating may eggs and sperms are hallmark of natural processes. I agree that there are "normal views of engineering". But the issue is that computation is now cheap. All optimization methods run through thousands of cycles in search of optimizing an outcome. Genetic algorithms create populations of designs. These populations are part of the exploration strategy - but we do bot see them as waste. But we see the biological version of this search appears wasteful. So should we not change our view on what waste is ? Is it not the price that we have to pay for exploration ?

    Gerhard Adam
    Optimizing an outcome is not the same as actually surviving.  The real work requires real interactions and not simply simulations.  As a result, that's where the "waste" comes into play.

    Biological systems may not be optimally "designed" nor efficient, but they are effective at what they do.  Engineering tends to try and solve efficiency problems in the design, while invariably results in limited successes, and huge failures.
    Mundus vult decipi