Periodic Reporting for period 2 - REDIAL (Re-thinking Efficiency in Deep Learning under Accelerators and commodity and processors)
Période du rapport: 2022-03-01 au 2023-08-31
By inventing a series of core algorithms with theoretical underpinning and applying these to micro-controllers, we increased the capabilities of the micro-controller in terms of machine learning. The methods include a novel neural architecture method called uNAS and the first ever differentiable pruning method able to work with the extreme constraints encountered my micro-controllers.
As a result, our methods allow a micro-controller to execute the inference of machine learning models at a low cost and privacy preserving manner that has not been seen before. I would add we also performed a first-of-its-kind measurement study to understand deeply the capabilities of micro-controllers under common deep learning operations.
The second is that of advancing the ability of individuals to collaborate when designing machine learning methods. We have done this by advancing a method known as federated learning. We invented a framework called Flower that has become globally popular and is now the most popular way to develop federated learning workloads in the community. A key innovation to support this is the secure aggregation strategy we developed within this framework, which works alongside our design of the framework more generally.
Our closing work performed is more general that the first element mentioned, although is related to it. That is a meta-learning informed pruning strategy. We demonstrate that this technique can outperform all known methods of this type currently known. The core idea is to consider second order effects during the pruning process, in a way that makes such consideration tractable.
In relation to results expected in the remainder of REDIAL I anticipate the following:
= Development of co-design accelerators that correspond to the algorithmic contributions we have made thus far
= Improved data-movement methodologies
= Increased outreach and dissemination.