Final Report Summary - CHAOPT (Chaotic Cellular Neural/Nonlinear Networks for Solving Constraint Satisfaction)
SUMMARY DESCRIPTION OF THE PROJECT OBJECTIVES
Solving constraint satisfaction problems has long been one of the most challenging tasks in computer science. These lie at the basis of many decision, scheduling, error-correction and bio-computational applications. Developing efficient algorithms is extremely important for science and technology. The industry of digital computers has gone through enormous progress in just a few decades: computer power has been increasing exponentially (as expressed by Moore’s law). This process is strongly connected to the characteristic size of the elements of the processors, which has been constantly decreasing. Nowadays this process has saturated, and instead the number of processors is increasing. This leads to new cellular-type architectures. With the emergence of quantitative neuroscience and new bio-computational applications, unconventional analog computing methods, such as the Cellular Neural/Nonlinear Network (CNN) computers appeared.
The CHAOPT project has continued a previous study of Ercsey-Ravasz, which revealed a novel connection between constraint satisfaction problems and chaotic dynamical systems. It presented an exact mapping of Boolean satisfiability (k-SAT) into a deterministic continuous-time dynamical system with a unique correspondence between its set of stable attractors and the k-SAT solutions. It was shown that optimization hardness is fundamentally equivalent to the phenomenon of chaos and turbulence. Using the same principles the goal of this project has been to develop a deterministic continuous-time CNN model for solving constraint satisfaction. CNN models were already implemented in analog computers so this could result in a possible physical implementation of the model in the future, leading to numerous applications. The main objectives of the project were: 1) Developing the CNN model and analyzing its mathematical properties. 2) Studying the efficiency of the CNN solver and properties of the transient chaotic behavior in hard problems. 3) Testing the robustness of the model to noise effects.
SIGNIFICANT RESULTS
For realising the first objective, the CNN model has been successfully developed to solve constraint satisfaction problems. Three mathematical theorems have been proved showing the equivalence of stable fixed points and SAT solutions.
For achieving the second objective, simulations were performed on thousands of random 3-SAT, 4-SAT and 5-SAT problems with different sizes and constraint densities. The efficiency and chaotic behaviour of the SAT-solver CNN model has been tested as function of the two important parameters of the model. The optimal regions of these parameters have been determined. Simulations have shown that there are three significantly different regions of the parameter map and these are fairly independent of the number of variables, constraint density and even the parameter k of k-SAT problems.
Implementing the third objective it was shown that the dynamics is robust to relatively high levels of white noise, colored noise described by the Langevin equation and connection weight errors. It was shown that the system tolerates much larger noise levels than those found on real hardware, such as the Q-Eye CNN chip.
Besides the agreed milestones the fellow has participated in an international collaboration with a group of neuroscientists from France, from the Stem Cell and Brain Research Institute (INSERM) in Lyon, lead by Dr. Henry Kennedy and Prof. Zoltán Toroczkai from the University of Notre Dame, IN, USA. The study concentrates on the physical structure of the brain. The French group performs retrograde tracing experiments, which provide data about the inter-areal cortical network and how the functional areas of the brain are connected. Ercsey-Ravasz has studied the properties of the inter-areal cortical network from the network science perspective. She discovered an interesting property of the cortex: the frequency distribution of axons of a given length connecting different areas of the cortex decreases exponentially with distance. This is called the Exponential Distance Rule (EDR) and based on this rule a one-parameter random graph model (EDR model) has been developed, which was shown to reproduce many global and local network properties of the inter-areal cortical network showing the importance of this physical rule. Based on data characterizing the feed-forward and feed-back nature of each inter-areal cortical connection a bow-tie structure of the cortical network was discovered with a highly dense core (with 92% density) distributed over the whole cortex.
DISSEMINATION
The project yielded 5 articles in peer-reviewed journals (Science, Neuron, PNAS, PLoS One, EPL), 4 conference proceedings papers (CNNA2012, two in CNNA2014, IPSEN2014) and 1 book chapter. Results were presented in 5 conference talks and two invited seminars.
The fellow also put great emphasis on transfer of knowledge activities by involving students and postdocs in the research project and maintaining international collaborations with INSERM, Lyon, University of Notre Dame, IN, USA and the Peter Pazmany Catholic University from Hungary.
POTENTIAL IMPACT
By solving constraint satisfaction problems with CNN models, CNN computing (and analog computing in general) could move to the centre of attention in the research field. Any previous analog computation model received criticism for requiring high precision and being vulnerable to noise. This model avoids these problems. Using “chaotic algorithms” on real hardware is a revolutionary idea. Implementations of this type of models in the future can lead to serious applications in computer science and technology.
Understanding the physical structure of the brain is one of the great challenges of modern neuroscience. The discovery of the exponential distance rule opened the possibility of modelling the structure of the inter-areal cortical network. Comparing experimental data to these simple models reveals some basic structural principles of the brain and also opens new ways of identifying important specificities, which are not described by these physical rules.
The results can also offer an interesting continuation to the present study related to CNN dynamics. The discovered physical, geometrical properties of the cortex can be used also in neural network models as a wire-cost minimisation principle.
Solving constraint satisfaction problems has long been one of the most challenging tasks in computer science. These lie at the basis of many decision, scheduling, error-correction and bio-computational applications. Developing efficient algorithms is extremely important for science and technology. The industry of digital computers has gone through enormous progress in just a few decades: computer power has been increasing exponentially (as expressed by Moore’s law). This process is strongly connected to the characteristic size of the elements of the processors, which has been constantly decreasing. Nowadays this process has saturated, and instead the number of processors is increasing. This leads to new cellular-type architectures. With the emergence of quantitative neuroscience and new bio-computational applications, unconventional analog computing methods, such as the Cellular Neural/Nonlinear Network (CNN) computers appeared.
The CHAOPT project has continued a previous study of Ercsey-Ravasz, which revealed a novel connection between constraint satisfaction problems and chaotic dynamical systems. It presented an exact mapping of Boolean satisfiability (k-SAT) into a deterministic continuous-time dynamical system with a unique correspondence between its set of stable attractors and the k-SAT solutions. It was shown that optimization hardness is fundamentally equivalent to the phenomenon of chaos and turbulence. Using the same principles the goal of this project has been to develop a deterministic continuous-time CNN model for solving constraint satisfaction. CNN models were already implemented in analog computers so this could result in a possible physical implementation of the model in the future, leading to numerous applications. The main objectives of the project were: 1) Developing the CNN model and analyzing its mathematical properties. 2) Studying the efficiency of the CNN solver and properties of the transient chaotic behavior in hard problems. 3) Testing the robustness of the model to noise effects.
SIGNIFICANT RESULTS
For realising the first objective, the CNN model has been successfully developed to solve constraint satisfaction problems. Three mathematical theorems have been proved showing the equivalence of stable fixed points and SAT solutions.
For achieving the second objective, simulations were performed on thousands of random 3-SAT, 4-SAT and 5-SAT problems with different sizes and constraint densities. The efficiency and chaotic behaviour of the SAT-solver CNN model has been tested as function of the two important parameters of the model. The optimal regions of these parameters have been determined. Simulations have shown that there are three significantly different regions of the parameter map and these are fairly independent of the number of variables, constraint density and even the parameter k of k-SAT problems.
Implementing the third objective it was shown that the dynamics is robust to relatively high levels of white noise, colored noise described by the Langevin equation and connection weight errors. It was shown that the system tolerates much larger noise levels than those found on real hardware, such as the Q-Eye CNN chip.
Besides the agreed milestones the fellow has participated in an international collaboration with a group of neuroscientists from France, from the Stem Cell and Brain Research Institute (INSERM) in Lyon, lead by Dr. Henry Kennedy and Prof. Zoltán Toroczkai from the University of Notre Dame, IN, USA. The study concentrates on the physical structure of the brain. The French group performs retrograde tracing experiments, which provide data about the inter-areal cortical network and how the functional areas of the brain are connected. Ercsey-Ravasz has studied the properties of the inter-areal cortical network from the network science perspective. She discovered an interesting property of the cortex: the frequency distribution of axons of a given length connecting different areas of the cortex decreases exponentially with distance. This is called the Exponential Distance Rule (EDR) and based on this rule a one-parameter random graph model (EDR model) has been developed, which was shown to reproduce many global and local network properties of the inter-areal cortical network showing the importance of this physical rule. Based on data characterizing the feed-forward and feed-back nature of each inter-areal cortical connection a bow-tie structure of the cortical network was discovered with a highly dense core (with 92% density) distributed over the whole cortex.
DISSEMINATION
The project yielded 5 articles in peer-reviewed journals (Science, Neuron, PNAS, PLoS One, EPL), 4 conference proceedings papers (CNNA2012, two in CNNA2014, IPSEN2014) and 1 book chapter. Results were presented in 5 conference talks and two invited seminars.
The fellow also put great emphasis on transfer of knowledge activities by involving students and postdocs in the research project and maintaining international collaborations with INSERM, Lyon, University of Notre Dame, IN, USA and the Peter Pazmany Catholic University from Hungary.
POTENTIAL IMPACT
By solving constraint satisfaction problems with CNN models, CNN computing (and analog computing in general) could move to the centre of attention in the research field. Any previous analog computation model received criticism for requiring high precision and being vulnerable to noise. This model avoids these problems. Using “chaotic algorithms” on real hardware is a revolutionary idea. Implementations of this type of models in the future can lead to serious applications in computer science and technology.
Understanding the physical structure of the brain is one of the great challenges of modern neuroscience. The discovery of the exponential distance rule opened the possibility of modelling the structure of the inter-areal cortical network. Comparing experimental data to these simple models reveals some basic structural principles of the brain and also opens new ways of identifying important specificities, which are not described by these physical rules.
The results can also offer an interesting continuation to the present study related to CNN dynamics. The discovered physical, geometrical properties of the cortex can be used also in neural network models as a wire-cost minimisation principle.
