Neural network applications are mainly devoted to classification. Unfortunately, most of the usual neural algorithms are unusable for practical applications: for instance, the architecture cannot easily grow, and it is difficult to learn new classes. The project's aim is to study and develop neural algorithms with evolutionary architectures and to finally propose a consistent set of efficient preprocessing and classification algorithms.
The research addresses learning in neural networks, both by adding or removing neurons in the networks and by synaptic adaption, for classification tasks. The research includes theoretical work on these algorithms, simulations and benchmarks, especially on realistic industrial data, and very large scale integration (VLSI) hardware implementations.
Asymptotic learning theorems have been proved, that show that any classifier minimizing a quadratic error over a learning set can ultimately reach the Bayes bound only under certain conditions. If the learning set is finite, asymptotic results hold when noisy replicates are recycled infinitely many times. Probabilities of errors are chosen as performance criteria, and calculated by Jackknife or resampling techniques. The ultimate bounds on performance are supposed to be reached by the best Bayesian classifier, based on densities of examples in each class. These are calculated with the help of Parzen type kernel estimators whose width varies with location. It has been demonstrated that Gaussian kernels are inadequate in dimensions larger than 5, because of the empty space phenomenon. In this context, an efficient kernel has been proposed, that takes into account the dimension, the finite number of samples, and their density. The expression of the asymptotic optimal variable width is used in this kernel estimator. For databases that are recognized to be small, it is proposed to perform an independent component analysis preprocessing in order to allow a reliable approximate density estimate.
To test new results and algorithms and to ensure consistent benchmarking, a smart software environment has been developed. The package, Packlib, is an open programming toolset including a communication library and several graphic tools. All algorithms studied and developed in the project are implemented as Packlib modules.
APPROACH AND METHODS
Theoretical work will be carried out in order to predict the efficiency of algorithms and to choose optimal parameters. Various strategies for changing the neural architecture will be evaluated, and problems of memory assignments addressed. Comparisons with classical methods are included in the theoretical approach.
Experimental work will be performed with the most efficient algorithms known. These tests and benchmarks will be carried out on industrial data in a standard graphic environment.
The design of hardware implementations of this class of algorithms will be studied in order to achieve high computation speed. The robustness of the algorithms, and the influences of limited precision, noise, drifts, etc, will all be studied in an initial phase. Performances must also be evaluated on various architectures, from classic computers to dedicated VLSI. It will then be necessary to define basic blocks and to design complete architectures. Finally, a few VLSI prototypes will be designed, using both digital and analogue technologies.
Theoretical results, as well as the hardware and software tools developed in this project, will provide amenable methods for solving realistic problems in classification. Furthermore, the test and benchmark results should constitute convincing demonstrations of the power of neural networks.