CORDIS

As a paradigm of computation, reservoir computing has gained an enormous momentum. In principle, any sufficiently complex dynamical system can be used for any computation if it can be equipped by an adjustable readout layer (e.g. a linear readout is sufficient to achieve universal computation). Owing to this flexibility, new applications of reservoir computing are being reported at a constant rate. However, relatively few studies focus on sensing, and in the ones that do, the reservoir is often exploited in a somewhat passive manner, to merely post-process the signal from sensing elements (that are placed separately). Such approaches will be referred to as “reservoir computing and sensing” where the use of reservoir computing occurs in parallel with sensing, and the reservoir could be replaced by other information processing system without loss of functionality of the sensor.

The overall aim of the project is to demonstrate an entirely different novel class of sensing approaches, to be referred to as “reservoir computing for sensing”, where the reservoir plays a central role, and cannot be replaced without rendering the sensor non-functional. As a realization of this principle, we suggested a generic sensing framework with the related rigorous mathematical formalization, to be referred to as the State Weaving Environment Echo Tracker (SWEET) sensing framework (procedure, paradigm, setup), and the SWEET algorithm respectively.

The key idea behind our approach to implementing the reservoir computing for sensing principle is to exploit the reservoir as the sensing element. This is illustrated in the attached figure. The concept of time plays a crucial role. We exploit some distinct characteristics of reservoir computing, in particular the separability and the echo state properties, to achieve sensing functionality. The sensing element and the environment are both treated conceptually as one dynamical system, a super-reservoir in which the states of the reservoir and the environment are entangled. The reservoir “weaves” the states together (over time), and will be also referred to as the state weaver. Owing to the echo state property, the state of the super-reservoir at a particular time instance embodies everything that both the weaver and the environment have experienced in the past, as a unit. However, ultimately, it is the states of the weaver one is interested in: the information about the environment will be embedded (encoded) into the state of the weaver. We argue that there is a way to analyze the reservoir/weaver states to infer indirectly about the environment, a principle we wish to investigate theoretically and demonstrate experimentally.

In the Reservoir Computing with Real-time Data for future IT (RECORD-IT) project the consortium implements and further develops the SWEET idea in the context of ion sensing. The aim of the project is to develop a biocompatible sensing device to detect and classify complex behavioral changes in ion concentrations on the biological scales at the cell level or higher. Accordingly, the terms SWEET/RECORD-IT can be used interchangeably, depending on the algorithmic/application oriented discussion context addressed.

The two major objectives of the RECORD-IT project are: (O1) Build a reservoir computation device prototype and (O2) Explore the capabilities of the device. O1 includes: Build all the sensing components; Couple the components to ensure reservoir computing properties; Test against known ionic concentration profiles. O2 includes: Identify which concentration profiles can be recognized (classified); Optimize device parameters; Validate whether the key concept of using reservoir computing for sensing/measurement can be used.

These objectives will be implemented by realizing the suggested super-reservoir concept: We found a very generic way of extracting the information stored in the weaver, and suggested a rigorous mathematical description of it: the SWEET algorithm

During the first year of the project we have taken the first steps to validate the SWEET/RECORD-IT sensing concept and to strengthen our theoretical and practical understanding of how to realize it. We have progressed towards building a prototype: Since we target the classification of complex behavioral changes in ion concentrations, we have investigated detection possibilities of several biologically relevant ions, and the related sensor components that are meant to interact with these ions. There were several challenges that needed to be overcome. For example, the integration issue has been targeted very aggressively, and as a result we have a clear vision of the prototype.
We are maintaining a careful balance between the theoretical and the experimental work to achieve the objectives.
We tested the concept theoretically (O2.d) and advance our theoretical understanding of the sensing paradigm forward (the provisional patent and the open access foundational publication where two mathematical theorems are stated regarding the expressive power of the sensing model).

We also did a considerable amount of work aimed at coupling these components (O1.b). The successful establishment of heterointegration of the individual sensors, which differ significantly with respect to sensing concept, materials, and input/output parameters, is a challenge we had to overcome. We used the first year to

=> establish the physical sensing concept – motherboard-daughterboard strategy (O1)
=> establish the individual sensor fabrication strategies and heterointegrate the sensors (O1),
=> provide a user-friendly, multifunctional platform for addressing all individual sensors physically (O2), which significantly aids post-project use of the technology.

These steps were critical for the project, as they are the necessary prerequisites for delivering reliable sensor data to the theoretical groups, and in turn are necessary for leveraging the theoretical findings and model development which provide useful information regarding which drive and the feedback mechanism should be used for a sensing task. In doing so, we not only developed a feasible integration concept, we even developed a strategy for unifying two physically very different sensors on the same silicon wafer.

We already made a progress in the direction of achieving (O2). For example we tested some components against a constant concentrations of ions (O2.a)

In the traditional sensing setup that employs reservoir computing the reservoir if often used passively. In such approaches, we refer to them as “reservoir computing and sensing” approaches the reservoir is used to post-process the information that is obtained from sensors that are placed separately from the reservoir. In such a setup the reservoir can be safely replaced by some other equivalent module, i.e. an artificial neural network, without any impact on the functionality of the sensor. However, in the suggested approach, which we refer to as “reservoir computing for sensing”, the reservoir is an integral part of the sensor, and it cannot be replaced without worsening the performance of the sensor. Thus we are providing an entirely novel solution to a very hard computational problem.

In the traditional sensing setup, the sensor-environment interaction has to be carefully engineered and it should be strong in some sense (e.g. to avoid problems with noise). In our setup, the sensor-environments interaction can be weak, not fully known, it does not be carefully engineered, and it can be possibly random. This flexibility releases many engineering constraints, which might be extremely important in embedded biotech sensing applications.

The key impact is in providing the society with the new concept of operating a group of sensors using reservoir computing. The application possibilities and the socio-economic impact is enormous since sensors are ubiquitous in nearly all technological applications. At the moment, we are still developing the concept, and so far the impact has been only within academia, and the impact on the society has been limited.

We anticipate considerable future impact on the sensing industry, possibly within the project duration. The novel way of sensing being developed in the project can be used to both design novel sensing applications, but also to re-use the existing ones in unexpected ways.

It is too early to tell, but we anticipate further impact on other industries, since the algorithm is universally applicable to several societal challenges where advanced neuromorphic/embedded/unconventional information processing techniques could be used with a great advantage (energy consumption control, traffic jam prediction, transport logistic planning, security, medical applications, etc.). We are already exploiting several application possibilities in these areas.