Artificial intelligence and, particularly, neural networks are successfully conquering more and more application domains. They help to improve our quality of life and to rid us from repetitive and tedious duties. Their applications range from noise-canceling hearing aids over machine translation to powerful image processing algorithms detecting and classifying objects in real-time video. Albeit amazingly effective, the deployment of neural networks poses an enormous computational challenge. The acceleration by power-hungry GPU farms is the norm rather than the exception. However, neural networks have been shown to be extremely resilient against the quantization of the backing computation to numerical values of harshly constraint ranges. Researchers working with programmable hardware, including the hosting research team of Michaela Blott at Xilinx Ireland, have demonstrated that even binary quantization, leaving only two possible numerical options for each operand, can yield capable neural network implementations in some application domains.
The successful quantization of neural network inference is highly relevant as it allows to simplify the backing arithmetic. The platforms that are able to extract benefit from every single saved bit are programmable hardware devices as made by Xilinx. These reprogrammable physical electrical circuits are able to translate simpler operations directly into greater operational density and concurrency. Thus, quantization allows small, power-efficient devices to deploy capable neural networks. Their use becomes a green option and is enabled in more difficult application environments as in embedded or remote contexts or in cyberphysical systems.
The goal of TPANN was the rigorous optimization of the neural network inference on programmable hardware devices. Particularly in the ubiquitous convolutional networks, it is the computation of a vast number of dot products that poses a critical challenge. His strong background in digital design and specialization in computer arithmetic of the fellow, Thomas Preußer, was key in this effort. One illustrative result of the work was the development of an object detection demo working on a live video stream running on a small embedded heterogeneous all-programmable device. The work also yielded two invention disclosures that are currently undergoing internal patent review.
