Find the most recent information on EU Funding activities in the field of Information and Communication Technologies (ICT) by visiting our ICT in FP7 website, which covers ICT in the 7th Framework Programme (FP7) 2007 - 2013.
Find the most recent information on Future and Emerging Technologies activities in
the 7th Framework Programme 2007-2013 on our FET Proactive - FET Open Web sites.
Complex Systems (CO)
Focus document
Ralph Dum, Dec. 2001
Rationale: Broadly speaking, complex systems are structures with many interacting, often heterogeneous, components whose elaborate collective behaviour cannot be inferred from studying the behaviour of individual components. Such an intuitive characterisation holds for a multitude of systems ranging from interacting particles in physics to cellular living organisms in biology and from societies of 'agents', embodied or simulated, in IT to social groups in the social sciences, examples include:
- Interacting spin systems with phase transitions to collective macroscopic states,
- 'networks' for transport of information, energy,....(internet, power grid, food web),
- non-linear dynamical systems characterised by chaotic behaviour,
- Traders in stock markets adapting to the market and influencing each other,
- Animal populations in evolutionary biology co-evolving in a 'race' for fitness,
- Program modules in Open software evolving via copying and (competing) 'bug'-fixes,
- Neurones and their synaptic connections in the brain,
- 'Agents' in IT systems with rich collective behaviour.
Such systems exhibits a variety of characteristics that often are not predictable from knowledge at the component level only. These systems often expose the limits of top-down thinking as their behaviour can neither be designed nor controlled in a hierarchical approach. Very often, rather than relying on pre-engineered shapes and functionality, the system components evolve and develop novel characteristics in an attempt to adapt to their environment. These systems scale-up easily because of a very effective management of the information flow between their components.
In studies of complex systems largely context-independent general concepts have emerged. Such concepts touch on fundamental issues of information theory as applied to computing, neurosciences, physics, biology, economy and social sciences. We therefore hope to develop ultimately a systematic scientific approach based on a conceptual framework of complex systems. This will enable a paradigm shift in the conceptual approach to system design: the goal will no longer be global system optimisation and top-down design, but to develop mechanisms and techniques that can guide/regulate the evolution of systems. A first example of a man-made system that is not consciously designed is the internet, which grew much like a living organism strongly driven by economic interests. The Internet has proven that some extremely large and powerful systems can grow quickly without human guidance.
In a spirit similar to system theory, 'complexity', as a discipline, is based on the belief that, despite the apparent diversity of instantiations of complex systems, a common vocabulary and a common set of characteristics and theories exist and will have a considerable impact on our understanding and ultimately control of systems ranging from the natural to the social sciences. In contrast to standard system theory, 'complexity' acknowledges the need to weigh description on a system level against description on a component level. In particular the global system behaviour cannot be the result of a purely top-down construction plan. Ecology, the neuro-sciences, as well as economics provide examples of systems where knowledge on a component level has to be reconciled with knowledge on a system level. Each of these disciplines studies a subset of reality interconnected with studies on different levels in another discipline. Consequently, researchers with diverse backgrounds start to engage in a multidisciplinary attempt to explore novel ways of understanding system behaviour drawing from knowledge of the system on all its (sub-)levels. Such an understanding can result in novel approaches to system design and system control and is therefore at the heart of problems in the information society.
Framework: The currently developing common framework to describe such systems is that of Complex Adaptive Systems (CAS). A CAS is an abstraction from real complex systems and tries to grasp those characteristics of the components and the interactions between them necessary to model the resulting system level behaviour. The characteristics of the components will evolve (adapt) in time 1.
The bottom-up versus top-down approach reflected in the system versus component perspective is accompanied by various rather general characteristics:
- Diversity (heterogeneity) of CAS components.
- Interactions are adaptive: the system evolves (adapts) local connections until the desired global behaviour emerges.
- Non-equilibrium situations are dominant.
- Processes are taking place on many temporal and spatial scales.
- Agents have limited (local) information of the system as a whole.
Methods and concepts: Empirical studies of real-world systems are an essential first step to understand the system and component characteristics which are of interest. In many complex systems such empirical studies can not be obtained via the known methods from the natural science: the holy grail of science - the repeatable experiment- is absent. Here, 'experiments in silicon', that is simulations based on (empirically grounded) models of the real-world, substitute for standard experiments.
The ultimate aim is to use simulations and empirical data on real-world systems in order to develop a set of concepts addressing system behaviour. Very often these concepts will serve as a framework to describe various phenomena in a common language; ultimately, however, we would like qualitative understanding and quantitative predictions of system behaviour.
Some general concepts shared by diverse complex systems are:
- Self-organisation of systems into critical states,
- Emergence of rich collective behaviour (related to symmetry breaking in physics) ,
- Existence of system conditions resulting in chaotic behaviour,
- Robustness: adaptation of responses to changing external influence,
- Self-similarity: existence of power laws,
- Co-evolution of system components.
A static (equilibrium) approach to system behaviour is in general flawed. Already in physics, complex systems are normally open exchanging energy with their surroundings. The long-established equilibrium approach to economics has given way to an approach which understands economy as a ongoing sequence of processes (market mechanisms) which continuously drive the system out of equilibrium. Different formulations of evolutionary principles are at the core of many operational approaches to complexity. In biology it takes the form of Darwinian or Lamarckian evolution in physics it evokes the principle of self-organisation, complexity is an evolutionary unfolding rather than an a priori imposed characteristic.
Where do complex systems occur?
Physics:
Many concepts in complexity were first developed in physics. Phase transitions, spontaneous symmetry breaking, spontaneous pattern formation, and self-organisation were first identified in interacting many-particle systems. Such systems are characterised by non-linear dynamics, in open systems, that is non-equilibrium systems. In the theory of critical phenomena the occurrence of power laws, related to the loss of typical length scales, lead to a successful use of scaling theories which now is applied in other disciplines.
Computer science:
In computer science largely based on Shannon's 'thermodynamic' formulation no general theory including characteristics of complex systems was attempted up to now. Even Von Neumann's cellular automata overall fit in the standard classical automata theory. Novel approach include the physical, spatial constraints of the system components ('physics is computational' - 'information is physical' ), e.g. quantum information theory. The most successful applications so far have been to problems in pattern recognition, where e.g. Kohonen's self-organising feature maps and other neural network models have provided surprisingly efficient and generally applicable methods for the early stages of visual and auditory processing, time series forecasting, adaptive control, etc. Another promising area is combinatorial optimization, where good results have been claimed using such general-purpose complex systems methods as simulated annealing and genetic algorithms. Also cellular automata -based based techniques have been attracting increasing attention lately in areas such as image processing, computer graphics, and even cryptography.
Dynamical systems:
Non-linear dynamics, idea of order despite 2nd law of thermodynamics (broken ergodicity); chaos theory provides important concepts, although chaos applies strictly speaking only to low-dimensional systems.
Biology:
Biology deals with real-world 'complex' systems: metabolic cell networks in organisms, self-regula-tory gene networks, the immune system...; in turn ideas in complexity are applied to immunology.
Material sciences: complex adaptive matter
The search for the organising principles that govern emergent, collective behaviour in soft, hard, and biological matter.
Evolutionary biology:
Darwinian and Lamarckian evolution show concepts like variety/diversity and selection for sustaining complex systems. Evolutionary computation, evolutionary robotics, artificial life and genetic algorithms are bio-inspired methods of artificially creating highly interconnected systems.
Zoology:
Animal societies, ant colonies, fish schools, and wasp nests, provide striking examples of the concepts like swarm intelligence. Analogies with animal societies might lead to new organisational paradigms. 'Ant-algorithms' are already used to improve routing in telecom networks or supply chains.
Neuro-sciences:
Work on the brain as the primary complex system touches on the 'emergence of the conscious'. Ideas to mimic results from studies of brain functioning in new computing models necessarily touches upon issues like emergence of characteristics or self-organisation.
Evolutionary linguistics models language as an 'emergent' phenomenon.
Networking and distributed computing:
The Web exemplifies the concept of emergence of order in a priori random networks; software agents are a pertinent example of a complex system where a desired collective behaviour must relate to programs (strategies) of individual agents. Methods to secure computer networks against virus attacks are based on ideas from immunology. Internet Ecologies research focuses on the relation between the local actions and the global behaviour of large distributed systems.
Information theory: Measures of complexity
Information theoretical aspects like flow of information in complex systems needs to be taken into account. How does the distribution of information in a biological, social ... system influence its characteristics? How is information incorporated in complex systems?
Economics:
Finance takes a biological perspective: markets, institutions and investors evolve according to the "law" of economic selection; financial agents compete and (most importantly) adapt. The resulting picture is very different (and much more realistic) from the one resulting from the 'efficient market hypothesis'.
Axelrod's approach to emergence of collaboration is another example.
Sociology:
People and their interacting strategies that shape human societies and cultures are examples for concepts like co-evolution . Dynamical simulation has become a key conceptual tool in understanding how the existing diversity of complex systems has evolved in the absence of rational planning or central direction. Implications for contemporary political and economic issues include benefits of de-centralised evolutionary processes (tacit/local versus formal/expert knowledge)
What are approaches to study complex systems?
Statistical and structural analysis of complex systems: Studies of Form
Despite their diversity complex systems follow similar organisational principles. They often show high degree of order, somewhere between strictly regular 'lattices' ('map of Manhattan'), which might suggest conscious design, and random networks that grow haphazardly. How does one reconcile the notion of emergence of global order with the known effectiveness of structures based on imposed control and design? Empirical studies of statistical and structural properties try to identify generic 'forms', that is spatial shapes, hierarchical (multilevel) structures, and connectivity patterns. Can we, for example, discern universal scaling laws relating certain system properties to, say, physical size? Modelling efforts subsequently try to relate global ('macroscopic', 'collective') forms to interaction characteristics of the ('microscopic') system components. As seen above these components can be atoms, cells, internet sites, software agents, stock brokers...Universal characteristics do not show up in all systems all the time, but some are common to very diverse systems.
Often we are seeking general properties that apply to (topologically) equivalent connected systems. In consequence we can use similar methods of analysis for seemingly very different systems. E.g. sequence analysis can be used to characterise molecular sequences, such as DNA, but also literary text, music, or computer programs. Furthermore, any physical system with topologies of connections similar to the studied system can be used for modelling: work has been done using Cellular Automata and Boolean Networks, but also general concepts derived from studies of physical systems like spin glasses.
Example: Connections in networks follow a power law distribution.
Empirical studies of the Web or food webs in ecosystems, reveal the existence of (scale-free) power law distributions of connection in network structures: the number of nodes n with a given number of connections c decreases as n = const/ c^a (a @ 1). Work on allometric scaling of plant energetics showed scaling laws characteristic of many living organisms. Power laws in social systems are not new: Herbert Simon gave already 1955 a convincing explanation for their occurrence: "them that has, gets". The extension to other networks is new, however, and it is surprising that certain structural features are found in very diverse biological and technological networks, which display high homeostasis against random removals of nodes, a first example of function deriving from form.
Example: Non-Gaussian distributions in stock prices and internet load
Empirical studies of the statistics of stock market prices and time distributions of data packets on the Internet reveal self-similar stochastic distribution patterns over many time scales. Such systems are characterised by 'fat-tail' distributions where rare, but significant events (stock market crash), determine the behaviour of the system. The standard hypothesis of Gaussian distributions is therefore in stark contrast with the empirical findings. New models, e.g. the price dynamics induced by competing financial agents (strategies), show how these interacting agents amplify noise (volatility). Like in other complex systems it is impossible to deduce collective behaviour from individual strategies. Consequences are striking for finance where pricing of options (Black-Scholes) was based on Gaussian distributions of stock prices ; consequences to management of the Internet are studied.
Example: Physical concept of phase transitions in theory of traffic jams.
Concepts like phase transitions and self-organisation turn out useful in characterising certain phenomena. For example cellular automata models of Internet and car traffic show phase transitions at the border from low-traffic phase to a congestion phase. In car traffic a fluid phase with rather homogeneous distribution goes over to a jammed phase with car jams with fractal like distributions. In traffic planning methods based on these concepts are now considered pertinent.
Example: Percolation theory of cohesion: small world networks.
Cohesion is a concept inspired by sociology. The cohesion between two parts of a network is the number of nodes that must be removed to completely disconnect them. So-called small-world network structures found in many real and artificially grown networks can be classified according to their level of cohesion. Percolation theory helps to better understand how 'tightly' connected networks are. For example simple models of disease transmission are studied, in which either the probability of infection by a disease or the probability of its transmission is varied: epidemic behaviour results when the infection or transmission probability rises above the threshold for site percolation on the network.
Functional characteristics of complex systems: 'From Form to Function'
What can we learn from studies of form? What can we conclude from network structure about how the network functions? Identifying general principles relating form to function could give guidelines to understand the preferences in nature for certain network forms. This understanding, in turn, could help in the 'conscious' construction of artificial complex systems.
Example: Form-adapted searching
A group at Cornell has devised a new search-engine that finds sites that are "authoritative" on a given topic. They infer authority, not by trying to analyse the content of the sites, but by analysing the pattern of links that turn up in a key-word search for the query terms. In this example, cohesion defined according to links responding to keyword searches is a structural property used to improve the search.
Example: complexity catastrophe
How does the amount of 'connectivity' influence functioning of the system? E.g. NK models study the influence of an increase in interactions in a system; N stands for the number of components each with a 'fitness' associated to its possible states and interacting with K other. This is reminiscent of spin glass models (where fitness corresponds to energy). If the number of interactions in systems is too big (K approaches N-1). the result might be a 'complexity catastrophe': too many constraints make increases in 'fitness' for individual agents impossible without perturbing the fitness of others. Relate ideas are raised by the internet concerning scalability, 'what is the optimal size of a system?' and decision theory where too many constraints make decisions increasingly difficult.
Example: circulatory systems: required function determines form
Recent studies and models of circulatory systems in trees, frogs and humans show that they share the same structure, since they are driven by the same considerations of scale, hydraulics, and the requirements to distribute nutrients.
Mechanisms to create and sustain complexity: 'Growth of Form'
The emergence of new qualities at a macroscopic level goes along with changes of the properties and interaction between the microscopic components (agents), therefore processes of adaptation and evolution play a dominant role in forming complex systems. But also physical systems where the interactions are fixed can sustain complexity (spontaneous pattern generation via symmetry breaking) in open systems far-from equilibrium ' Growth' via evolution or adaptation creates complexity via e.g. increases in species diversity (speciation) or an increase in structural sophistication that is accumulation of more and more subsystems (structural deepening, hierarchical, multi-level structures), 'growth' of complexity in non-equilibrium systems is due to constant conversion of energy in information content. Are there other guiding principles for 'adaptation' than increasing some fitness function?
Example: Modelling of evolution: artificial life
Studies of the relation between evolution and complexity are hampered by the fact that evolution progresses slowly in smooth and small changes which make it difficult to gather statistically relevant data on real-world systems. This is why modelling based on methods like Genetic Algorithms is pertinent. Work on artificial life helps deciphering general characteristics. One has to avoid, though, that this area reduces to 'video games production' where results of artificial life simulations have no relevance for real-world systems and are not grounded in empirical studies .
Example: Equilibrium approaches versus dynamics in non-equilibrium systems
Simulations track over time the microscopic characteristics of local interactions of members (agents) of a population. This stands in contrast to modelling techniques where changes in time of averaged, macroscopic quantities characteristics of the population are simulated (corresponding to a mean-field approach in physics where fluctuations are neglected). Many disciplines, like economy and population biology, still address systemic issues mainly with, very often only linear, macroscopic or equilibrium approaches which neglect the dynamics of the system towards the equilibrium state. Such approaches ignore systems working 'far from equilibrium' studied in the context of chemical systems. Fluctuations are a crucial to explore new possible states and appearance of new forms, the result of a process, which selects the attractor of the dynamics among the huge number of possible (accessible) configurations.
Some generalities of evolution and adaptation gleaned from biology and physics can be identified. Features common to all studied systems include:
- Variation: diversity of system types; mutation and innovation.
- Selection: success criteria, notion of fitness.
- Increase in complexity: structural deepening, increase in diversity
- Interaction/Connectivity: non-linearity results in multiple equilibria
- Co-evolution: the adaptive moves of one 'agent' influences others.
1 Systems are not necessarily adaptive (e.g. in physics). Studying non-adaptive complex systems is often a first step to study the adaptation process in real-world biological and sociological systems.