Community Research and Development Information Service - CORDIS


Weight decay and pruning are techniques frequently used to control the generalisation ability of neural network models. The Bayesian approach to parameter estimation and model comparison gives a probabilistic interpretation of weight decay in terms of some prior overweights and defines a framework for choosing one model from a set. The choice of prior distributions is usually left to some heuristic explanation, but an entropy optimisation principle, such as maximum entropy (MaxEnt), should be preferred. This paper reviews the first part of MacKay's framework for backpropagation, and reassesses his a priori hypothesis using MaxEnt.

Additional information

Authors: PERROTTA D, JRC Ispra (IT)
Bibliographic Reference: Paper presented: Neural Networks and their Applications - Neuronimes '94, Marseille (FR), December 15-16, 1994
Availability: Available from (1) as Paper EN 38816 ORA
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top