How To Put Art And Brain Together
    By Mark Changizi | March 4th 2010 07:26 PM | 19 comments | Print | E-mail | Track Comments
    About Mark

    Mark Changizi is Director of Human Cognition at 2AI, and the author of The Vision Revolution (Benbella 2009) and Harnessed: How...

    View Mark's Profile
    A generation ago it was only a brave eclectic minority of psychologists and neuroscientists who dared to address the arts. Things have changed considerably since then. “Art and brain” is now a legitimate and respected target of study, and is approached from a variety of viewpoints, from reductionistic neurophysiology to evolutionary approaches.

    Things have changed so quickly that late 20th century conversations about how to create stronger art-science collaborations and connections are dated only a decade later – everyone’s already doing it! And the new generation of students being trained are at home in both the arts and sciences in a way that was rare before.

    Although we are all now more culturally comfortable bathing in conversations about art and brain, are we making progress? Has looking into the brain helped us make sense of the arts? Here I will briefly explain why I believe we have made little progress. And then I will propose an alternative route to understanding art and its origins.

    Perhaps the most common modus operandus in the cognitive and brain sciences approach to art is (i) to point to some known principle of brain science, and then (ii) to provide examples of art showing conformance with that principle. As fun as it may be to read explanations of art of this kind, the approach suffers from two fundamental difficulties – one on the brain side, one on the arts side.

    Let’s start with the “brain” difficulty, which is simply this: we don’t understand the brain. Although the field is jam-packed with fantastically clever experiments giving us fascinating and often valid data, there is usually very little agreement (or ought to be little agreement) about how to distill the data into broad principles. And the broader and higher-level the supposed principle, the more controversial and difficult-to-defend it is. Consequently, most of the supposed principles in the brain sciences remotely rich enough to inform us about the arts are deeply questionable.

    If we are so ignorant of the brain, why is the modus operandus above sometimes seemingly able to explain art? There is a lot of art out there, and it comes in a wide variety. Consequently, given any supposed principle from neuroscience or psychology, one can nearly always cherry pick art pieces fitting it. What very few scientific studies do is attempt to quantitatively gauge whether the predicted feature is a general tendency across the arts. The fundamental difficulty on the “arts” side is that we often don’t have a good idea what facets of art are universal tendencies that need to be explained.

    These difficulties for the brain and arts make the common modus operandus a poor way to make progress comprehending art and brain. What initially looks like neuroscientific principles being used to explain artistic phenomena is, more commonly, suspect brain principles being used to explain artistic phenomena that may not exist. (A second common approach to linking art and the brain sciences goes in the other direction: to begin with a piece of art, and then to cherry-pick principles from the brain sciences to explain it.)

    How, then, should we move forward in our quest to understand the arts? Here I will suggest to you a path, one that addresses the brain and art difficulties above.

    The “arts” difficulty can be overcome by identifying regularities actually found in the arts, whether universals, near-universals, or statistical tendencies. One reason large-scale measurements across the arts are not commonly carried out may be that any discipline of the arts tends to be vast and tremendously diverse, and it may seem prima facie unlikely that one will find any interesting regularity. With a strong stomach, however, it is often possible to collect enough data to capture a signal through the noise.

    The “arts” difficulty, then, can be addressed by good-old-fashioned data collection, and distillation of empirical regularities. But even so, we are left with another big problem to overcome. “Good-old-fashioned data collection” involves more than simply collecting data. Which data should one collect? And which kinds of regularities should be sought after? Although it is well-known that data helps drive theory, it is not as widely appreciated that theory drives data. There’s effectively infinitely many ways of collecting data, and effectively unlimited ways of analyzing any set of data. Without theory as a guide, one is not likely to identify empirical regularities at all, much less ones that are interesting. Good-old-fashioned theory is required in good-old-fashioned data collection. We need predictions about empirical regularities, and then need to gather data in a manner designed to test the prediction.

    But this brings us back to our first difficulty, the “brain” one. If we are so ignorant of the principles of the brain, then how can we hope to use it to make predictions about regularities in art?

    We are, indeed, woefully ignorant of the brain, but we can make progress in explaining art. Here is the fundamental insight I believe we need: the arts have been culturally selected over time to be a “good fit” for our brain, and our brain has been naturally selected over time to be a good fit to nature …so, perhaps the arts have come to be shaped like nature, exactly the shape our brain came to be highly efficient at processing. For example, perhaps music has been culturally selected to be structured like some natural class of stimuli, a class of stimuli our auditory system evolved via natural selection to process. (See Figure 1.)

    natural selection and cultural selection in shaping the brain

    If the arts are as I describe just above – selected to harness our brains by mimicking nature – then we can pursue the origins of art without having to crack open the brain. We can, instead, focus our attention on the regularities found in nature, the regularities which our brains evolved to competently process. I’ll suggest in a moment that we can do exactly this, and give examples where I have been successful at doing so. But first let’s deal with a potential problem…

    Don’t brains have quirks? And if so, couldn’t the arts tap into our quirks, and then no analysis of nature would help explain the arts? What do I mean by a quirk? Brains possess mechanisms selected to work well when the inputs to the mechanisms are natural. What happens when the inputs are not natural? That is, what happens when the inputs are of a kind the mechanism was not selected to accommodate? The answer is, “Who knows?!” The mechanism never was selected to accommodate non-natural inputs, and so the mechanism may carry out some arbitrary, inane computation.

    To grasp what the mechanism does on these non-natural inputs, we may have no choice but to crack open the hardware and figure out how it actually works. If the arts tended to be culturally selected to tap into the brain’s quirks, then nature wouldn’t help us, and we’d be bound to the brain’s enigmatic details in our grasp of the arts.

    There is, however, a good reason to suspect that cultural selection won’t try to harness the brain’s quirks, and the reason is this: quirks are stupid. When your brain mechanisms are running as nature “intended,” they are exceedingly sophisticated machines. When they are run on inputs not in their design specs, however, the behavior of the brain’s mechanisms (now quirks) are typically not intelligent at all. For example, the plastic fork in front of me is well-designed for muffin eating, and although I can comb my hair with it, it is a terribly designed comb. The quirks will usually be embarrassing in their lack of sophistication for any task. …because they weren’t designed for any task. And that’s fundamentally why we expect the arts to have culturally been selected to tap into our functional brain mechanisms, running roughly as nature “intended”.

    If we can set aside the quirks, then we can side-step the brain in our attempt to grasp the origins of the arts. If I am correct about this, we can remove the most complicated object in the universe from the art equation!

    With the brain put on the shelf, the goal is, instead, to analyze nature, and use it to explain the structure of the arts. Is this really possible? And isn’t nature just as complicated as the brain, or, at any rate, sufficiently complicated that we’re headed for despair?

    No. Nature is filled with simple regularities, many of them having physics or mathematical foundations. And although it may not be trivial to discover them, our hopes should be far greater than our hopes for unraveling the brain’s mechanisms. Our presumption, then, is that our brains evolved to “know” these regularities of nature, and if we, as scientists, can unravel the regularities, we have thereby unraveled the brain’s competencies. What regularities from nature am I referring to? For the remainder of this piece, I’ll give you three brief examples from my research. Only one is explictly about the arts, but all three concern the cultural evolution of human artifacts, and how they harness our brains via mimicking nature. (See Figure 2.)

    shaping culture to look like nature in cultural selection

    The first concerns the origins of writing, and why letters are shaped as they are. Our visual systems evolved for more than a hundred million years to be highly competent at visually processing natural scenes. One of the most central features of these natural scenes was simply this: they are filled with opaque objects strewn about. And that is enough to lead to visual regularities in nature. For example, there are three junction types having two contours – L, T and X. Ls happen at many object corners, Ts when one edge goes behind an object, and these two are accordingly common in natural scenes. X, however, is rare in natural scenes.

    Matching nature, letter shapes with L and T topologies are also common across languages, but X topologies rare. More generally, the shapes found more commonly in natural scenes are those found more commonly in writing systems. [See this SB piece for more: ]

    The second concerns the origins of speech, and why speech sounds as it does. Our auditory systems evolved for tens of millions of years to be highly efficient at processing natural sounds.

    Although nature consists of lots of sounds, one of the most fundamental categories of sound is this: solid-object events. Events among solid objects, it turns out, have rich regularities that one can work out. For starters, there are primarily three kinds of sound among solid objects: hits, slides and rings, the latter occurring as periodic vibrations of objects that have been involved in a physical interaction (namely a hit or a slide). Just as hit, slides and rings are the fundamental atoms of solid-object physical events, speech is built out of hits, slides and rings – called plosives, fricatives and sonorants. For another starter example, just as solid-object events consist of a physical interaction (hit or slide) followed by the resultant ring, the most fundamental simple structure across language is the syllable, most commonly of the CV, or consonant-sonorant form. More generally, and as I describe in my upcoming book, Harnessed: How Language and Music Mimicked Nature and Transformed Ape to Man (2011), spoken languages share a wide variety of solid-object event signatures.

    Written and spoken language look and sound like fundamental aspects of nature: opaque objects strewn about and solid-objects interacting with one another, respectively. Writing thereby harnesses our visual object-recognition mechanisms, and speech harnesses our event-recognition mechanisms. Neither opaque objects nor solid objects are especially evocative sources in nature, and that’s why the look of most writing and the sound of most speech is not evocative. [See this SciAm piece for more: ]

    Music – the third cultural production I have addressed with a nature-harnessing approach – is astoundingly evocative. What kind of story could I give here? A nature-harnessing theory would have to posit a class of natural auditory stimuli that music has culturally evolved to mimic, but haven’t I already dealt with nature’s sounds in my story for speech? In addition to general event recognition systems, we probably possess auditory mechanisms specifically designed for the recognition of human behavior. Human gait, I have argued, has signature patterns found in the regularities of rhythm. Doppler shifts of movers have regularities that one can work out, and these regularities are found in music’s melodic contours. And loudness modulations due to proximity predict how loudness is used in music.

    These results are described in my upcoming book, Harnessed. For example, just as faster movers have a greater range of pitches from their directed-toward-you high pitch to their directed-away-from-you low pitch, faster tempo music tends to use a wider range of pitches for its melody. [See this SB piece for more: ]

    structure of nature harnessing arguments for speech writing and music

    Many other aspects of the arts are potentially treatable in a similar fashion. For example, color vision, I have argued is optimized for detecting subtle spectral shifts in other people’s skin, indicating modulations in their emotion, mood or state. That is, color vision is a sense designed for the emotions of other people, and it is possible to understand the meanings of colors on this basis, e.g., red is strong because oxygenated hemoglobin is required for skin to display it. The visual arts are expected to have harnessed our brain’s color mechanisms via using colors as found in nature, namely principally as found on skin. Again, the strategy is to understand art without having to unravel the brain’s mechanisms.

    One of the morals I want to convey is that you don’t have to be a neuroscientist to take a brain-based approach to art. The brain’s competencies can be ferreted out without going inside, by carving nature at its joints, just the joints the brain evolved to carve at. One can then search for signs of nature in the structure of the arts. My hope is that via the progress I have made for writing, speech and music, others will be motivated to take up the strategy for grappling with all facets of the arts, and cultural artifacts more generally.


    Given this audience I think most of these points will be all well and good (and this is brilliantly written!)  but using modus operandus instead of operandi, when operandi is not nominative plural but rather genitive singular, is going to cause a fight among our Latin purists on the site - and they are legion!
    Given this audience I think most of these points will be all well and good (and this is brilliantly written!)  but using modus operandus instead of operandi, when operandi is not nominative plural but rather genitive singular, is going to cause a fight among our Latin purists on the site - and they are legion!

    I fully endorse both of Hank's points, except for the bits I disagree with.

    1. I enjoy reading examples of good English.

    2. My modus ponendo ponens analysis of your use of Latin reveals an error: every schoolboy knows that modus operandus means "my modem works".
    modus operandus means "my modem works".
    Which explains why Skara Brae was abandoned after 600 years.  They'd heard that being so remote, they weren't going to get broadband.
    Robert H. Olley / Quondam Physics Department / University of Reading / England
    Even in Italian you get problems.  If you go to you will find

    Da quel giorno che t'ho conosciuta
    o fanciulla di questo mio cuor,
    speme e pace per te ho perduto
    perché t'amo d'un immenso amor.

    Now that made me worry.  Shouldn't it be conosciuto, rhyming with perduto?  (For one thing, it rhymes.) The Italian lady a few doors away says that it is conosciuto, and explained the grammar.

    Now why do I concern myself with this song?  Apart from the fact that it is one of my favourites to sing, I have explained this on SciBlog    HERE.
    Robert H. Olley / Quondam Physics Department / University of Reading / England
    Mark Changizi

    If that's correct, then...editor, edit err!
    I don't edit. I don't even correct for spelling so you think I am jumping into correcting Latin declination? People will even argue I should use the word declension instead. But I was curious because I thought it was operandi for the reasons above so I did a quickie search and we are not the only ones to have this debate.
    operandi is actually what is known as a gerund in Latin grammar. It's a way of converting a verb (to operate, to speak) into a noun (of operating, in speaking, etc.) Now, gerunds don't actually appear in the nominative case. The form of the verb used if you want to say, for example, "making errors is characteristic of humans" is the infinitive. Errare est humani The genitive of the gerund can limit an adjective or noun in some way: vir bonus dicendi peritus a good man skilled in speaking difficultas navigandi difficulty in sailing ... Its other cases are used in some other circumstances - Lex est recta ratio in iubendo et vetando (the law is the right method in ordering and forbidding). There's a gerundive, which is an adjective which implies that someone is able, worthy or necessary of, for example, being loved or heard. That answer was longer than I intended to write. I'll put away my language nerd hat and back away from the dictionary now.
    Enjoyed the article. I'm sure I mentioned before that I was involved in art-science collaborations about 10 years ago - from what you say, seems like the idea was a success.

    Are you aware of the work done by mathematicians in analyzing art, works by Michele Emmer, or ISAMA? Some of them are creating mathematical art but many articles are on a mathematical analysis of artistic structures and patterns as well as patterns in nature.
    Mark Changizi

    At some point I'd like to hear about the art-science collaborations.  I'm aware of some math analyses of art, but will have to check on those.
    Mark: indirectly related to your excellent article: Susan Aldworth, artist.

    Mark Changizi
    Thanks for the link. The pieces are striking...
    Though the field is jam-packed with fantastically clever experiments giving us fascinating and often valid data, there is usually very little agreement (or ought to be little agreement) about how to distill the data into broad principles. And the broader and higher-level the supposed principle, the more controversial and difficult-to-defend it is. Consequently, most of the supposed principles in the brain sciences remotely rich enough to inform us about the arts are deeply questionable.
    I see some parallels here with molecular/cell/systems biology. Finding principles is a tough problem, and yet without them it's hard to say that we understand what's going on.
    The study of the arts have been a fundamental part of psychology from cognitive, affective, and philosophical perspectives since the beginning. That does not in any way diminish the importance of recent work. But come on, let's not lose sight of our own history here. There are so many lessons to be learned and carried forward.

    Mark Changizi

    I'm not intending to suggest that new work is better than old work (if anything, I'd suggest the other way around).  Nor am I suggesting that there haven't been researchers interested in the psychology of art for a long time; I said there have been. My suggestion in the introduction is only that there has been a resurgence of interest and respect more widely for questions about the arts. But none of that is really my point -- that was just by way of introduction. 

    I'm skeptical, though, about the lessons such research -- old and new -- teaches us about the arts.
    Gerhard Adam
    Mark, from a purely speculative perspective, I think that the arts can be viewed as an aspect of the psychology of control.  As I've mentioned elsewhere, I think the belief systems provide an data organizational structure that allows people to make sense of the world around them.  From this, I would argue that one of the primary psychological objectives of humans is to exercise "control" over their environment.  Basically my point here is that it is necessary to eliminate or avoid the randomness of the world, that our beliefs and attitudes provide a means by which we can feel in control. 

    In many respects the arts provide a means by which humans can demonstrate a degree of "control" by presenting or reshaping natural phenomenon into controlled organized displays.  I don't believe it's coincidental that music, painting, photography, etc. are all oriented towards being structured variations of the natural world.  In fact, much of what we consider creativity is based on being surprised or impressed by the degree of "control" the artist can exercise over their presentation.   Even science is premised on the assumption that organizing our thought processes and testing ideas will lead to greater knowledge, which ultimately results in greater "control" of the natural world. 

    I want to be clear that "control" is not necessarily manipulative, but it does feed into our psychological need to feel that we understand the world around us and that we've reduced some of the randomness intrinsic in it.  I realize these ideas are not fully formed, but I wanted to just toss it out there.
    Mundus vult decipi
    Mark Changizi

    Interesting. I don't really have a full theory of the arts. My theory is that the arts must usually have structure "shaped like nature" so as to not simply "bounce off" our sensory system. The lower-level structure must be shaped like nature, but it need not look like nature "all the way up." (That's my story in my explanation of the origins and structure of music in my upcoming book, Harnessed.) In fact, my guess is that by having artistic stimuli not look like nature all the way up, it allows the stimuli to be non-consciously evocative, and avoids letting our cognition get into the game and notice what it mimics, which will tend to lower the evocativeness. E.g., the Valentine's heart has many lower-level features of an upturned engorged red female rump, but doesn't look completely like it, and that may be what's crucial for the arts. But I wouldn't count that as a "theory", much less one that would be necessary and sufficient.

    My point: I'm open to ideas, like yours, about what art really means.
    Gerhard Adam
    In fact, my guess is that by having artistic stimuli not look like nature all the way up, it allows the stimuli to be non-consciously evocative, and avoids letting our cognition get into the game and notice what it mimics, which will tend to lower the evocativeness.
    I understand, but consider something like a botanical gardens.  It is one of the interesting elements of gardens is to consider that there is little desire to emulate nature, but rather to reshape it into a more organized form that fits our sense of "control".  Part of the appeal of realistic paintings is their ability to emulate natural representations.  Of course, it's equally interesting that a truly artistic photograph is usually complemented as looking like a painting (while a realistic painting is often thought to resemble a photograph).

    Even when viewing nature, we tend to be more enthralled with those things that appear to be organized into patterns (i.e. canyons, rock formations, etc.).  It also seems that our appreciation of natural beauty is inversely proportional to the likelihood that we have to actually survive in it.  If you examine nature photographs, it seems that the primary effort is to demonstrate imbedded patterns and to emphasize those elements that appeal to our sense of order. 

    I think your point about it having to appeal to our sensory system is true, which is why it is most telling when art (or nature) becomes increasingly uncomfortable the more chaotic it appears to our senses.  In nature it becomes stressful, while in art it is simply unappealing or even irritating.  It is interesting to listen to several different songs simultaneously, where you can almost feel the effort of your brain to elicit patterns and to identify something coherent. 

    In any case, it seems like a curious synergy between our senses and the means by which we create stimuli for our senses.
    Mundus vult decipi
    Hi Mark,

    nice post. However, I found it quite abstract, and consequently a little hard to follow your line of thinking in places (NB: this is probably my issue much more than yours, which is why you are writing awesome books about science and I'm merely blogging about making stuff easier to understand ;o). Would this post maybe benefit from some more concrete examples near the beginning? Depending, of course, on whom you are writing for :)


    PS — I would have gone with modus operandi too, being a big fan of the genitive case, but was surprised to hear that this is contentious!

    Mark Changizi

    Thanks, Chris, for the read and the feedback. It is short on examples! (Loads of details in the upcoming book, of course.)  -Mark