An oldfield mouse.Our behavior is critically linked to our genetics but the relationship remains murky. Genes, by constructing our brain and nervous system and by regulating our hormones, influence behavior — in humans and in all animals.

Ironically, the science is the least clear where the connection between genetics and behavior should be, in theory, the most direct — in cases of “hard-wired” behavior such as the construction of beehives or the courtship rituals of some bird species or a human infant’s rooting; behaviors that are common across a species without having to be taught. Instinct.

Biologist-author Richard Dawkins put forth the highly controversial idea, in his 1982 book The Extended Phenotype, that we should stop assuming that the phenotype — the outward expression of an organism’s genes — end with, for instance, the fur on a beaver’s back. If genes also influence the beaver’s behavior, and therefore its dam construction, we should consider the length of a given beaver’s dam as valid a measurable product of the beaver’s genotype as the length of its tail. This is the extended phenotype.

Many scientists contest this view. However, in what appears to be a confirmation of some of Dawkins’s theory, a recent paper in Nature demonstrated that a handful of discrete genetic modules are responsible for the burrowing behavior in two closely-related, mice species: oldfield mice and deer mice. The researchers explicitly chose burrows as an extended phenotype; they were working in the intellectual context Dawkins proposed.

The study, by Harvard zoologist Hopi Hoekstra and her team, has already been reported on in an excellent piece at Ed Yong’s National Geographic blog, Not Exactly Rocket Science. Here’s here’s the study in brief:

Captive-bred mice, even those with no previous exposure to sand, demonstrated the same burrow construction behavior as their wild counterparts. This must be a genetically controlled behavior. So the scientists crossed oldfield mice, which build escape tunnels in their burrows, with deer mice, which do not, and end up with an entire generation that built burrows matching that of their oldfield mice parents — including the escape burrow.

Some of you may recognize in this pattern the simple dominance of Mendelian genetics, except in these crosses we’re not dealing with pea color or shape but the complex behavior of an animal. To push behavioral genetics forward, these researchers dug into the same breeding toolkit that Gregor Mendel used a century-and-a-half ago to found the field.

The researchers then back-crossed the hybrid mice with deer mice, and analyzed the genotypes of the mice they were working with to identify regions associated with the observed behavioral changes. Ultimately, they came to the conclusion that burrowing behavior is controlled by a remarkably small number of genes packaged into discrete modules. One module, for instance, influences whether or not the mice will build escape tunnels, and just because a given hybrid inherits the module for an escape tunnel (an oldfield mouse trait) does not mean it will make a burrow of the same dimensions as oldfield mice.

The team’s next step is to narrow identify precisely which genes, rather than regions of DNA, influence borrowing behavior in these mice, and to what extent.

Aside from linking classical genetics and traditional breeding experiments to modern molecular genetics and behavioral studies, Hoekstra’s work opens a door to a wide range of possibilities, not least of which is addressing the question of how genes influence our own extremely sophisticated behavior. It’s not hard to imagine this study as tugging at strands that connect mouse burrows to beaver dam to, to Gorilla nests, and eventually to whatever genetic impetus allowed mankind to become the greatest builders natural history has ever seen.

In The Extended Phenotype, Dawkins wrote, “The extended phenotype may not constitute a testable hypothesis in itself, but it so far changes the way we see animals and plants that it may cause us to think of testable hypotheses that we would otherwise never have dreamed of.” Twenty-one years later in 2004 he wrote that, as far as he knew, “nobody had done a genetic study using animal artefacts [sic] as the phenotype.”

Thirty years later, it was not beavers (one of which sits on the cover of my 1999 edition of The Extended Phenotype) but another rodent that provided the first empirical work suggesting the utility of Dawkins’s idea.

Reprinted from Genetic Literacy Project GeneTrends: Human newsletter.