In the final chapter of the book Complexity: A Guided Tour, Mitchell gets to the heart of the real issues that I've been griping about in this blog. She begins by citing a harsh, 1995 piece by John Horgan, “Is Complexity A Sham?”
These are still the main criticisms many have, including myself. Maybe there are common or general principles of complex systems out there, maybe there aren’t - and if there aren’t, it might not ever be obvious. The worst possibility is that people keep searching, decade after decade, for something that doesn’t exist. Maybe there are systems that just are really, really complex, in their own unique way, extremely heterogeneous systems whose properties determined primarily by a non-reproducible history.
Take cellular automata for example. There has been some fruitful work done on relatively simple cellular automata, and the impressive result is that systems of simple elements that obey local rules can produce very rich, complex, but understandable behavior.
So what about systems comprised not of relatively simple elements, but complex ones? Instead of cellular automata made up of cells with two or three possible states and a handful of rules, what of ones with a dozen different types of cells, arranged in some complex spatial pattern, each of which can be in one of a dozen different states, following local rules that are specific to the type and state of the cell? Contemplating a system like that, it’s hard not to consider the possibility that there are no general principles governing most or all complex systems.
The awkwardness of some attempts to apply universal principles to very different systems is particularly clear in network science. Applying a network analysis to understand the vulnerabilities of the power grid or the Internet (where the meaning of edges and nodes in the network is fairly consistent) seems natural. It’s not so natural to apply network ideas to protein-protein interaction networks (with a few limited exceptions), or genetic networks, when the relationship between nodes isn’t so consistent across the entire network. Perhaps molecular biological ‘networks’ are more like the parts of your car: is it helpful, really, to know whether the network of physical part interactions inside my Mazda 626 is scale free? Even if general principles can be found, they may not be fruitful - which is something that would certainly prevent complexity from ever being a mature science.
Second, there is a lot of fact-free science in the world of the complexity sciences. Perhaps because many complex systems researchers are so excited about focusing in on commonalities, they not only ignore system-specific details they deem irrelevant, but fail to actually learn what they’re ignoring, leaving them unable to judge how well their theories succeed, because they don’t know much about evolution or ecology or economics or whatever. There is a lot of biology tourism by some computational people who really are doing fact-free science. They are happy if they can successfully use their model to number-crunch some data set (typically the other half of the single dataset they used to train their model); they declare victory and say that their success means that some grand idea (typically untested, if not untestable) that motivated their model has been vindicated.
In this fashion, the yeast cell cycle transcription network has been ‘shown’ to be robust 50 times over. As far as I can tell, this computational finding has not inspired a single new experiment. I once asked a speaker who had built a model that ‘explained’ known gene expression patterns in a developing embryo, how he planned to test his model. His answer was, literally, that he had no idea; in his mind, the fact that he could computationally reproduce known experimental facts was enough. That’s just bad science.
And so, I finish Complexity: A Guided Tour having read in it many claims about how new (allegedly complexity-inspired) ideas in biology are overthrowing long-held doctrines about genes and evolution, and yet I didn’t find more than two examples of any application of non-linear dynamics, mathematical network theory, cellular automata, information theory, Turing machines, Gödel’s incompleteness theorem, genetic algorithms, game theory, or fractals actually applied successfully to a genuine biological question. And in both cases rigorous experimental testing is lacking. One is Robert Axelrod’s game theoretic exploration of cooperation in biology. The other is metabolic scaling theory (which looks at the relationship between metabolic rate and organism size), which has so far failed to be widely convincing, and, worse, is associated with grandiose claims about its potential to be the unifying theory of biology, on par with Newton’s Laws in physics. When it comes to biology, complexity sciences have produced more hype than fruitful results. This stands in stark contrast to the successes of genuinely hot fields in biology, like genome biology and human genetics.
I won’t end on a completely negative note. The mediocre treatment of biology in this book has got my hackles raised, but the book’s author, Melanie Mitchell (a computer science professor) is honest throughout the book about the criticisms that have been raised.
I can understand why this subject is so seductive. (If you don't believe me, check out the name of my blog.) There is such rich theory in physics and mathematics about computation, about information processing, about emergent properties, theory that has been quite successful when applied to the typical systems physicists study. The link between thermodynamics and information is stunning. How could there not be a fruitful way to apply these ideas to information processing inside of the cell? How does a physical-chemical system of metabolites, proteins, nucleic acids and lipids sense the environment, make a decision, and execute that decision? Narrative models (of the type typically found in Figure 7 or 8 of your average Cell paper) are unsatisfying Rube Goldberg contraptions.
And yet, as Mitchell relates, as seductive as these ideas are, they’ve failed repeatedly to live up to their promise over the decades (especially in biology) - from Cybernetics to General Systems Theory to the new sciences of complexity. They have been largely “intriguing analogies among different systems without producing a coherent and rigorous mathematical theory that explains or predicts their behavior.” Maybe most complex systems really are unique Rube Goldberg machines.
Previous failures obviously don’t mean there won’t be success in the future, but one sure path to failure is for complex systems scientists to fool themselves with their own hype. Wishful thinking can’t replace the rigor that has characterized the best science of the past 400 years.
Read the feed:
The article contained two main criticisms. First, in Horgan’s view, it was unlikely that the field of complex systems would uncover any useful general principles, and second, he believed that the predominance of computer modeling made complexity a “fact-free science.”
These are still the main criticisms many have, including myself. Maybe there are common or general principles of complex systems out there, maybe there aren’t - and if there aren’t, it might not ever be obvious. The worst possibility is that people keep searching, decade after decade, for something that doesn’t exist. Maybe there are systems that just are really, really complex, in their own unique way, extremely heterogeneous systems whose properties determined primarily by a non-reproducible history.
Take cellular automata for example. There has been some fruitful work done on relatively simple cellular automata, and the impressive result is that systems of simple elements that obey local rules can produce very rich, complex, but understandable behavior.
So what about systems comprised not of relatively simple elements, but complex ones? Instead of cellular automata made up of cells with two or three possible states and a handful of rules, what of ones with a dozen different types of cells, arranged in some complex spatial pattern, each of which can be in one of a dozen different states, following local rules that are specific to the type and state of the cell? Contemplating a system like that, it’s hard not to consider the possibility that there are no general principles governing most or all complex systems.
The awkwardness of some attempts to apply universal principles to very different systems is particularly clear in network science. Applying a network analysis to understand the vulnerabilities of the power grid or the Internet (where the meaning of edges and nodes in the network is fairly consistent) seems natural. It’s not so natural to apply network ideas to protein-protein interaction networks (with a few limited exceptions), or genetic networks, when the relationship between nodes isn’t so consistent across the entire network. Perhaps molecular biological ‘networks’ are more like the parts of your car: is it helpful, really, to know whether the network of physical part interactions inside my Mazda 626 is scale free? Even if general principles can be found, they may not be fruitful - which is something that would certainly prevent complexity from ever being a mature science.
Second, there is a lot of fact-free science in the world of the complexity sciences. Perhaps because many complex systems researchers are so excited about focusing in on commonalities, they not only ignore system-specific details they deem irrelevant, but fail to actually learn what they’re ignoring, leaving them unable to judge how well their theories succeed, because they don’t know much about evolution or ecology or economics or whatever. There is a lot of biology tourism by some computational people who really are doing fact-free science. They are happy if they can successfully use their model to number-crunch some data set (typically the other half of the single dataset they used to train their model); they declare victory and say that their success means that some grand idea (typically untested, if not untestable) that motivated their model has been vindicated.
In this fashion, the yeast cell cycle transcription network has been ‘shown’ to be robust 50 times over. As far as I can tell, this computational finding has not inspired a single new experiment. I once asked a speaker who had built a model that ‘explained’ known gene expression patterns in a developing embryo, how he planned to test his model. His answer was, literally, that he had no idea; in his mind, the fact that he could computationally reproduce known experimental facts was enough. That’s just bad science.
And so, I finish Complexity: A Guided Tour having read in it many claims about how new (allegedly complexity-inspired) ideas in biology are overthrowing long-held doctrines about genes and evolution, and yet I didn’t find more than two examples of any application of non-linear dynamics, mathematical network theory, cellular automata, information theory, Turing machines, Gödel’s incompleteness theorem, genetic algorithms, game theory, or fractals actually applied successfully to a genuine biological question. And in both cases rigorous experimental testing is lacking. One is Robert Axelrod’s game theoretic exploration of cooperation in biology. The other is metabolic scaling theory (which looks at the relationship between metabolic rate and organism size), which has so far failed to be widely convincing, and, worse, is associated with grandiose claims about its potential to be the unifying theory of biology, on par with Newton’s Laws in physics. When it comes to biology, complexity sciences have produced more hype than fruitful results. This stands in stark contrast to the successes of genuinely hot fields in biology, like genome biology and human genetics.
I won’t end on a completely negative note. The mediocre treatment of biology in this book has got my hackles raised, but the book’s author, Melanie Mitchell (a computer science professor) is honest throughout the book about the criticisms that have been raised.
I can understand why this subject is so seductive. (If you don't believe me, check out the name of my blog.) There is such rich theory in physics and mathematics about computation, about information processing, about emergent properties, theory that has been quite successful when applied to the typical systems physicists study. The link between thermodynamics and information is stunning. How could there not be a fruitful way to apply these ideas to information processing inside of the cell? How does a physical-chemical system of metabolites, proteins, nucleic acids and lipids sense the environment, make a decision, and execute that decision? Narrative models (of the type typically found in Figure 7 or 8 of your average Cell paper) are unsatisfying Rube Goldberg contraptions.
And yet, as Mitchell relates, as seductive as these ideas are, they’ve failed repeatedly to live up to their promise over the decades (especially in biology) - from Cybernetics to General Systems Theory to the new sciences of complexity. They have been largely “intriguing analogies among different systems without producing a coherent and rigorous mathematical theory that explains or predicts their behavior.” Maybe most complex systems really are unique Rube Goldberg machines.
Previous failures obviously don’t mean there won’t be success in the future, but one sure path to failure is for complex systems scientists to fool themselves with their own hype. Wishful thinking can’t replace the rigor that has characterized the best science of the past 400 years.
Read the feed:
