Emergent Modularity

Last week I attended a talk at MIT by Michael Deem, a professor at Rice University who has done some very interesting work on the emergence of modularity in evolution. This is a topic that I have long thought about, as it seems that modularity is intrinsic to many biological phenomena, and it also seems that modular systems would by construction have certain internal structure that can be exploited in computational modeling. My thinking on the topic has been crude and qualitative, and so it was with some delight that I discovered Michael’s work in this area, as his group has placed this problem on very firm quantitative footing. The talk was thought provoking, and left me with genuinely new insights, something that happens with only a small, small fraction of the talks I attend.

His presentation focused mostly on the results and implications of their methodology, as opposed to its mathematical underpinnings (field theory). Suffice it to say that they have developed a theoretical framework for studying evolution, and have, to a first order approximation, shown that modularity can emerge spontaneously in evolved systems. In particular they have identified three properties that, if present, would result in the emergence of modularity. First, the environment must undergo dramatic changes on a regular basis. Second, there must be a mechanism for exchanging large chunks of genetic information between organisms, with “genetic information” here referring abstractly to the material that encodes the evolving unit. In biology this mechanism is horizontal gene transfer, and perhaps gene duplication (he didn’t say anything about gene duplication, and obviously while gene duplication enables large chunks of genetic content to be introduced, it is not the same as having that information directly injected into another organism). Third, the evolutionary landscape must be rugged.

It appears that the first factor is the most important, and it was the main topic of the talk so I will focus on it. Before I get there however I will first describe what he means by modularity. The basic notion is that, given a system of interacting parts, modular systems are structured in a way such that there are a lot of interactions among parts within a group and few interactions between parts in different groups. This is the familiar concept of clusters in unsupervised machine learning. If we represent individual parts by rows and columns in a matrix, and the presence of an interaction by a red entry in the matrix, then this is what modularity would look like:

Modularity

It is a continuous quantity, with some systems more modular than others. The main result of the talk was that evolved systems tend to a more modular architecture if the environment in which they operate undergoes substantial changes periodically. The intuition behind this result is as follows. When a system is undergoing evolution, it is exploring the space of possible configurations that optimize its fitness in the environment. One can think of the system as exploring the space of all possible matrices pictured above. If there are structural constraints on the space of matrices that it can explore, for example if the system must be modular and thus the matrix roughly block-diagonal, then the system will find the (local) optimum more quickly than if it were exploring the entire space of all possible matrices, because it is searching a constrained and thus smaller space. On the other hand, if the system were unconstrained in its search, then it will find the same or better optimum found by the constrained search, but in a longer time. This is visualized below:

Fitness vs Time

The key feature of the above plot is the time gap during which the fitness of the modular system is higher than that of the non-modular one. If the periodic environmental changes occur on timescales shorter than the gap, then modularity will be preferred, because modular systems could find a better solution in time before a major environmental change occurs and resets the system. Although he didn’t show this, I assume that in the above there is actually a third axis representing modularity, with more modular systems having sharper curves like this:

Fitness vs Time Multi Fitness, Time, Modularity Surface

Then on the resulting two-dimensional surface evolution is optimizing for the level of modularity that yields the best solutions given the expected timescale of environmental changes. In environments that are highly stable, the evolved systems would in fact be less modular, and arguably more fragile and less resistant to change.

A related concept here is the notion of hierarchical modularity. Although he didn’t show the formalism behind this, the idea is that modularity can be defined on multiple levels, presumably by coarse-graining the matrix of interactions and then looking at block-level interactions. Their results suggest that after a system’s modularity at one level has saturated/been optimized, higher-order modularity starts emerging, and it does so right around the time a system’s modularity at one level starts tapering off, as schematized below:

Hierarchical Modularity

This is of course consistent with our current understanding of biological phenomena and the multiple levels of organization that we observe.

I will finish up with a quick sojourn into a non-biological example that Michael presented. His group also studies other systems, including global trade. He made three interesting observations about what they see there. First, the world as a whole, as a result of globalization and better telecommunication, is getting increasingly less modular. Second, historically speaking the world tends to become more modular in the aftermath of a major financial crisis. Presumably this is because modular systems are able to better insulate disruptions from affecting the entire system, and so the world as a whole tends to tighten up post-crises. Third, since the overall trend has been toward decreasing modularity, the average recovery time from crises has been increasing. This is interesting for many reasons, not least of which is the implication that as we become increasingly more interconnected, our global financial systems are becoming more fragile and slower to recover. On one level this is obvious, as can be evidenced by the fact that floods in Thailand can now have a major impact on supply chains in the US. The drive by some US companies to bring back manufacturing to the US is also driven by similar considerations. But in spite of these measures, the long-term trajectory is clear. Our interconnectedness is potentially making us a more fragile species.


5 comments

  1. Hi Mohammed,

    Your post on emergent modularity got me thinking about what modular systems being ‘meta-optimized’ for evolving optimal solutions quickly might imply in terms of their structure. I think a very crude argument implies such systems should be scale free networks:

    Say you have a modular system ‘M’ with N components, divided into K modules of N/K components each. The system can evolve by exploring the space of its possible configurations, and the time taken to reach the optima is an increasing function of the number of configurations. Each module can interact with another module in a binary way (~ 2^K^2 configurations), and each component within a module can interact with a component in the same module only (~ 2^(N/K)^2 configurations).

    The log of the number of configurations of the system is S(M) = ((N/K)^2 + K^2) log 2.

    By symmetry (or direct differentiation), at the minimum K must equal N/K. That is, the system can most easily evolve an optimal solution if the numbers of modules is close to the number of components within each module. This generalizes to any number of layers of modules and implies a ‘scale free’ organization where ‘zooming out’ from a single module to all modules reveals roughly the same level of complexity at each level of granularity. Such systems are ‘meta-optimized’ to find optima quickly since they minimize the size of their configuration space for a given number of components and modular layers.

    • Hi Tom! It’s nice to hear from you!

      Your argument is interesting. Are you assuming that the way components interact within a module must be the same from module to module? If not, wouldn’t that imply that S(M) = (K*(N/K)^2 + K^2) log 2? I.e. the total configuration space for just the submodule interactions is (2^(N/K)^2)^K. Or am I missing something?

  2. Hi, thanks for considering this. The intent here is to get some idea on how long it would take for such a model to reach an optimum, and I was assuming each module’s evolution could take place at the same time as each other module’s evolution, so long as they don’t interact. Say the organism is a bird, and within the bird’s genes, one module controls beak shape, and another one controls the color of its plumage. Evolving the optimal beak and plumage could proceed in parallel as long as they weren’t interacting (say the color of its plumage affected the habitat the bird could live in, which could affect its beak shape… lots more possibilities to consider). In other words, we only need to consider the time for finding the optimum in the configuration space of a single module (assuming they are all the same size) plus inter-module interactions, which is S(M) as I’ve shown. I hope this makes sense… 🙂

    • Yes I think it’s reasonable to assume that the modules can evolve independently and thus do so in a parallel fashion. In effect each module can expose an interface, not unlike a software API, and as long as the interface is held fixed then the internal workings and evolution of a module can proceed independently from all the others.

      If that’s the case though then I think the original expression is not quite right, because you would have to decouple the intermodule configuration from intramodule configuration. The way you have it right now, you have a term for intermodule configuration that’s being multiplied by a (single) term for intramodule configuration, which doesn’t make sense if they’re evolving independently (if the intermodule configuration does depend on the intramodule configuration, then we’re back to square one and they all have to be coupled). It’s either one big system, in which you have to take the product of the intermodule configuration with all the intramodule configurations, or they’re all independent and are evolving in parallel, in which case it’s max(2^K^2, 2^(N/K)^2) and not (2^K^2)(2^(N/K)^2). Right?

  3. 私はよ すぐ グラブ |私はとしてRSSフィードRSSあなたすることはできません 見つけることにサブスクリプションハイパーリンクまたは Eニュースサービス} {あなたのメールを検索する|見つけます。行うあなたが持っているいずれかを? 親切 許可 ように 理解私は私がちょうどすることができる購読してください。ありがとうございます。
    特別sale価格 国内即発 http://mceurope.bg


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s