Periodic Reporting for period 1 - HomE (Enabling Homomorphic Encryption of Deep Neural Network Models and Datasets in Production Environments)
Reporting period: 2022-09-01 to 2025-02-28
Deep learning (DL) is widely used to solve classification problems previously unchallenged, such as face recognition, and presents clear use cases for privacy requirements. Homomorphic encryption (HE) enables operations upon encrypted data, at the expense of vast data size increase. RAM sizes currently limit the use of HE on DL to severely reduced use cases. Recently emerged heterogeneous memory systems offer larger-than-ever RAM spaces, but their performance is highly dependent on data access patterns. This project aims at sparking a new class of system architectures for encrypted DL workloads, by eliminating or dramatically reducing data movements across memory/storage hierarchies and network, supported by heterogeneous memory technology, overcoming its current severe performance limitations. HomE intends to be a first-time enabler for the encrypted execution of large models that do not fit in DRAM footprints to execute local to accelerators, hundreds of DL models to run simultaneously, and large datasets to be run at high resolution and accuracy. Targeting these ground-breaking goals, HomE enters unexplored field resulting from the innovative convergence of several disciplines, where wide-ranging research is required to assess current and future feasibility. Its main challenge is to develop methodology capable of breaking through the existing software and hardware limitations. HomE proposes a holistic approach yielding highly impactful outcomes that include novel comprehensive performance characterisation, innovative optimisations upon current technology, and pioneering hardware proposals. HomE can spawn a paradigm shift that will revolutionise the convergence of the machine learning and cryptography disciplines, filling a gap of knowledge and opening new horizons such as DL training on HE, currently too demanding even for DRAM. HomE, based on solid evidence, will unveil the great unknown of whether large memory pools formed of heterogeneous memory systems are a practical enabler for encrypted DL workloads.
During this first part of the project, we have been working mainly on three main points. We first had to select the most appropriate encryption scheme for our purpose. The two main schemes, namely CKKS and TFHE, feature pros and cons that make this choice far from trivial. While several reviews comparing both counterparts may be found in the literature, unsurprisingly none covers the specifics of an ERC project. Hence, we have been working toward a fair comparison considering our focus. After careful evaluation, both theoretical by our cryptographer and empirical by our research engineers, and not without many hours of discussion, we finally decided that CKKS is the choice of the project, since it shall potentially expose further parallelism and performance at batched workloads at the expense of memory consumption, which is inherently designed to be addressed during the project. We are currently finalising a review article that will discuss the CKKS versus TFHE schemes from HomE’s point of view.
Unfortunately, we found no open-source inference engine equipped with state-of-the-art techniques. This led us to engage in developing our own. HomE’s inference engine, to be released and open sourced very soon, will not only equip state-of-the-art techniques, but also novel methodology that we are cooking. With this, we expect our software to become a reference in the world of homomorphically-encrypted deep learning inference, attracting not only researchers and users, but also contributors and collaborators.
On a third major action, we have been developing, jointly with ETH Zurich, novel methodology to implement NTT -the core operation in the homomorphically-encrypted cyphertext- in processing-in-memory (PIM) devices, providing highly efficient operations in the memory device, hence saving data movements and consequently energy consumption. We plan to integrate this feature, along with that on top of other devices, in the next stage of development of HomE’s inference engine.
Unfortunately, we found no open-source inference engine equipped with state-of-the-art techniques. This led us to engage in developing our own. HomE’s inference engine, to be released and open sourced very soon, will not only equip state-of-the-art techniques, but also novel methodology that we are cooking. With this, we expect our software to become a reference in the world of homomorphically-encrypted deep learning inference, attracting not only researchers and users, but also contributors and collaborators.
On a third major action, we have been developing, jointly with ETH Zurich, novel methodology to implement NTT -the core operation in the homomorphically-encrypted cyphertext- in processing-in-memory (PIM) devices, providing highly efficient operations in the memory device, hence saving data movements and consequently energy consumption. We plan to integrate this feature, along with that on top of other devices, in the next stage of development of HomE’s inference engine.
While we have released no tangible outcomes yet, our most immediate results include an open-source inference engine implementing the CKKS encryption scheme that will incorporate beyond-state-of-the-art methodology. This will be one of its kind with no direct competitors, and hence we expect it to become a reference in the field. This action has clear potential to become the first breakthrough of the project. This requires further research and development efforts and credible demonstration, apart from wide communication activities.
On the other hand, the NTT techniques that we have developed for PIM shall be of direct use not only for upcoming devices of this nature, but also for other energy-efficient (and hence constrained) accelerators, such as GPUs, FPGAs, SmartNICs, or even our planned domain-specific accelerator based on RISC-V technology. Again, further R&D efforts are required to consolidate this line of research.
On the other hand, the NTT techniques that we have developed for PIM shall be of direct use not only for upcoming devices of this nature, but also for other energy-efficient (and hence constrained) accelerators, such as GPUs, FPGAs, SmartNICs, or even our planned domain-specific accelerator based on RISC-V technology. Again, further R&D efforts are required to consolidate this line of research.