Periodic Reporting for period 1 - FLASH (FLASH - Federated Learning Supporting Efficient and Reliable Inference over Vehicular Networks)
Reporting period: 2022-05-01 to 2024-04-30
Summary of the context and overall objectives of the project
The FLASH project aims to establish the theoretical foundations of machine learning and wireless communications to enable the vision of assisted and self-driving systems. Unfortunately, current systems cannot provide safe and reliable driving because they lack distributed and real-time learning algorithms meeting the critical latency and reliability requirements in highly dynamic and fast-varying wireless channels. Although the fifth generation of cellular systems supports the communication demands for assisted and self-driving, and machine learning proposes federated learning (FL) for distributed scenarios, the wireless communications and machine learning domains are not sufficiently integrated for real-time critical applications. Yet, it is only by their integration that the vision of assisted and self-driving systems will become real. To this end, we will establish a theoretical and algorithmic integration of FL and cellular networks that serve vehicles, which we refer to as FL supporting efficient and reliable inference over vehicular networks (FLASH). FLASH builds on the co-design of a fundamentally new ecosystem in which FL algorithms address critical constraints from vehicular applications, while resource allocation algorithms adapt wireless communication resources to the inference tasks.
The goal of this project is to enable assisted and self-driving systems by providing novel theoretical methods on FL and vehicular communications that integrate both paradigms. To reach our goal, we investigate two intertwined research objectives:
• Objective 1 [Inception of FLASH for assisted driving systems]: To establish novel theoretical and algorithmic methods for the efficient and reliable integration of FL and vehicular communications for assisted driving systems. Specifically, we will pursue a novel direction by joining theoretical models in the fields of machine learning (federated learning and differential privacy) and wireless communication (finite blocklength theory and physical layer security). The project will proceed in three steps to fulfil this objective. The first step will establish novel theoretical foundations integrating FL and vehicular communications using the finite blocklength theory. The second step will establish strong privacy preserving and/or security guarantees to the theoretical foundations and algorithms established in the first step. These guarantees can be investigated from two perspectives: differential privacy and physical layer security. The third step will consist of the numerical validation of the algorithms from the previous steps using the publicly available datasets for assisted driving systems.
• Objective 2 [Extend FLASH to self-driving systems]: To extend the proposed novel theoretical and algorithmic methods of FL and vehicular communications to self-driving systems. Specifically, we intend to integrate transfer learning and model compression to the novel algorithms in Objective 1 to fulfil the more critical requirements of self-driving systems. The project will proceed in two steps to fulfil this objective. The first step will consist of generalizing the novel FLASH theoretical foundations and algorithms established in Objective 1 to autonomous driving systems, with critical latency, reliability, and data rate requirements. The fulfilment of these requirements will use the theories of transfer learning and model compression. The second step will consist of the numerical validation of the first step using public datasets for self-driving systems. Such datasets include the Lyft Level 5 with tasks on traffic detection, motion prediction, and trajectory planning.
The goal of this project is to enable assisted and self-driving systems by providing novel theoretical methods on FL and vehicular communications that integrate both paradigms. To reach our goal, we investigate two intertwined research objectives:
• Objective 1 [Inception of FLASH for assisted driving systems]: To establish novel theoretical and algorithmic methods for the efficient and reliable integration of FL and vehicular communications for assisted driving systems. Specifically, we will pursue a novel direction by joining theoretical models in the fields of machine learning (federated learning and differential privacy) and wireless communication (finite blocklength theory and physical layer security). The project will proceed in three steps to fulfil this objective. The first step will establish novel theoretical foundations integrating FL and vehicular communications using the finite blocklength theory. The second step will establish strong privacy preserving and/or security guarantees to the theoretical foundations and algorithms established in the first step. These guarantees can be investigated from two perspectives: differential privacy and physical layer security. The third step will consist of the numerical validation of the algorithms from the previous steps using the publicly available datasets for assisted driving systems.
• Objective 2 [Extend FLASH to self-driving systems]: To extend the proposed novel theoretical and algorithmic methods of FL and vehicular communications to self-driving systems. Specifically, we intend to integrate transfer learning and model compression to the novel algorithms in Objective 1 to fulfil the more critical requirements of self-driving systems. The project will proceed in two steps to fulfil this objective. The first step will consist of generalizing the novel FLASH theoretical foundations and algorithms established in Objective 1 to autonomous driving systems, with critical latency, reliability, and data rate requirements. The fulfilment of these requirements will use the theories of transfer learning and model compression. The second step will consist of the numerical validation of the first step using public datasets for self-driving systems. Such datasets include the Lyft Level 5 with tasks on traffic detection, motion prediction, and trajectory planning.
Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far
The project ended on March 31, 2023, because the researcher obtained an Assistant Professor position at Uppsala University, Sweden. Due to this reason, some objectives of the project could not be developed. Specifically, the tasks related to Objective 2 were not developed. Nevertheless, the project investigated several interesting topics from Objective 1, which are the activities discussed in the following.
The main research activities developed in the project were related to the research on FL and wireless communications, specifically focused on vehicular communications for assisted driving systems. For Objective 1, the researcher investigated the first, and parts of the second and third steps. For the first step, the researcher investigated several aspects of FL and wireless communications, including the proposal of novel FL methods and wireless resource allocation methods suitable to vehicular communications. Specifically, the research proposed novel communication-efficient FL methods, and investigated the convergence of FL methods in wireless communication scenarios with latency, reliability, data rate, and maximum transmitted power requirements.
For the second step, the researcher investigated the integration of differential privacy aspects into FL. Specifically, the research analysed the convergence of FL methods through the interplay between learning aspects, such as the convergence of the FL method, privacy aspects, such as the privacy leakage of the FL method, and wireless communication aspects, such as the transmitted power and device scheduling.
For the third step, the researcher investigated the previous steps in a learning task closely related to assisted driving systems, and in a wireless communication scenario with dynamic vehicles. Specifically, the researcher analysed the FL task of traffic sign classification and used a recent open-access dataset with traffic signs from six European countries, including Belgium, Croatia, France, Germany, The Netherlands, and Sweden. Overall, these investigations across the three steps were conducted by the researcher together with the group of Prof. Carlo Fischione, at KTH Royal Institute of Technology, and the group of Prof. H. Vincent Poor, at Princeton University. In the next section, the researcher's results obtained from the activities within each group are described with more details.
Beyond the research activities, the researcher participated in training, transfer of knowledge, supervision activities. In the training activities, the researcher participated in a mentoring course from the McGraw Center for Teaching and Learning from Princeton University. This course consisted of 8 virtual sessions to develop mentoring expertise, and aimed at postdoctoral researchers and new faculty. Moreover, the researcher had training at the PDC Center for High-Performance Computing at KTH Royal Institute of Technology and the Princeton Institute for Computational Science and Engineering (PICSciE) at Princeton University. The researcher attended seminars, lasting 2-4 hours, on the utilization of high-performance computing resources, particularly GPUs, containers, and libraries designed to enhance the efficiency of simulations across various programming languages.
In the transfer of knowledge activities, the researcher participated in bi-weekly group meetings at KTH Royal Institute of Technology and Princeton University. The researcher exchanged his knowledge with both groups through research collaborations with students and postdoctoral researchers. Moreover, the researcher participated in the organization of a workshop at IEEE Global Communications Conference (GLOBECOM) 2022, titled Wireless Communications for Distributed Intelligence (WCDI). This workshop was co-organized by Prof. Fischione, Prof. Poor, the researcher, and three other professors that were introduced to the researcher: Prof. Zheng Chen and Prof. Erik G. Larsson, from Linköping University in Sweden, and Prof. Osvaldo Simeone, King’s College London in the UK.
For the supervision activities, the researcher was the co-supervisor of two doctoral students in the group of Prof. Fischione at KTH Royal Institute of Technology.
The main research activities developed in the project were related to the research on FL and wireless communications, specifically focused on vehicular communications for assisted driving systems. For Objective 1, the researcher investigated the first, and parts of the second and third steps. For the first step, the researcher investigated several aspects of FL and wireless communications, including the proposal of novel FL methods and wireless resource allocation methods suitable to vehicular communications. Specifically, the research proposed novel communication-efficient FL methods, and investigated the convergence of FL methods in wireless communication scenarios with latency, reliability, data rate, and maximum transmitted power requirements.
For the second step, the researcher investigated the integration of differential privacy aspects into FL. Specifically, the research analysed the convergence of FL methods through the interplay between learning aspects, such as the convergence of the FL method, privacy aspects, such as the privacy leakage of the FL method, and wireless communication aspects, such as the transmitted power and device scheduling.
For the third step, the researcher investigated the previous steps in a learning task closely related to assisted driving systems, and in a wireless communication scenario with dynamic vehicles. Specifically, the researcher analysed the FL task of traffic sign classification and used a recent open-access dataset with traffic signs from six European countries, including Belgium, Croatia, France, Germany, The Netherlands, and Sweden. Overall, these investigations across the three steps were conducted by the researcher together with the group of Prof. Carlo Fischione, at KTH Royal Institute of Technology, and the group of Prof. H. Vincent Poor, at Princeton University. In the next section, the researcher's results obtained from the activities within each group are described with more details.
Beyond the research activities, the researcher participated in training, transfer of knowledge, supervision activities. In the training activities, the researcher participated in a mentoring course from the McGraw Center for Teaching and Learning from Princeton University. This course consisted of 8 virtual sessions to develop mentoring expertise, and aimed at postdoctoral researchers and new faculty. Moreover, the researcher had training at the PDC Center for High-Performance Computing at KTH Royal Institute of Technology and the Princeton Institute for Computational Science and Engineering (PICSciE) at Princeton University. The researcher attended seminars, lasting 2-4 hours, on the utilization of high-performance computing resources, particularly GPUs, containers, and libraries designed to enhance the efficiency of simulations across various programming languages.
In the transfer of knowledge activities, the researcher participated in bi-weekly group meetings at KTH Royal Institute of Technology and Princeton University. The researcher exchanged his knowledge with both groups through research collaborations with students and postdoctoral researchers. Moreover, the researcher participated in the organization of a workshop at IEEE Global Communications Conference (GLOBECOM) 2022, titled Wireless Communications for Distributed Intelligence (WCDI). This workshop was co-organized by Prof. Fischione, Prof. Poor, the researcher, and three other professors that were introduced to the researcher: Prof. Zheng Chen and Prof. Erik G. Larsson, from Linköping University in Sweden, and Prof. Osvaldo Simeone, King’s College London in the UK.
For the supervision activities, the researcher was the co-supervisor of two doctoral students in the group of Prof. Fischione at KTH Royal Institute of Technology.
Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)
Although the project terminated more than two years before its intended date, there were strong results obtained in the fields of FL and vehicular communications.
The results obtained by the project include novel FL methods that are more suitable to wireless and vehicular communications, and novel wireless resource allocation methods that are suitable to the FL training. Specifically, the researcher proposed novel FL methods that achieve faster convergence, thus improving the communication efficiency of learning task. The researcher investigated the convergence of FL methods when used over wireless and vehicular communication, including analysis the convergence speed when considering privacy, latency, and reliability requirements. Moreover, the researcher proposed novel wireless resource allocation methods that consider the FL task performed, thus adapting the device selection, transmitted power, and data rate to the requirements of different FL tasks.
The results in the project were obtained jointly with the group of Prof. Fischione and Prof. Poor, and led to several publications in high-impact journals and conference venues. Specifically, the results in the project led to four journal publications, one conference publication, the submission of three journal publications, and two journal manuscripts currently undergoing internal review and expected to be submitted in late March or early April 2024. In the following, the details of each activity conducted with Prof. Fischione and Prof. Poor are explained.
Together with Prof. Fischione, the researcher was involved in the co-supervision of two doctoral students and collaboration with a third doctoral student. In the first co-supervision, the doctoral student and the researcher proposed a novel communication-efficient federated learning methods that considered the impact of the learning methods in the energy consumption and latency on the wireless communication. This work has been recently accepted in a high-impact factor journal at IEEE, the IEEE Transactions on Wireless Communications. The results demonstrate the importance of proactively designing optimal cost-efficient stopping criteria to avoid unnecessary communication-computation costs to achieve a marginal FL training improvement.
In the second co-supervision, the doctoral student and the researcher proposed novel wireless over-the-air computation and communication methods compatible with current digital communication systems, which are also beneficial to FL and able to reduce the latency and energy consumption. This research indicated that it is possible to reduce the latency of the computation and communication of FL methods while maintaining high learning accuracy of the FL task. This research direction led to the publication of one journal at IEEE Transactions on Communications, and one conference at IEEE International Conference on Communication. Moreover, two journal publications were submitted to high-impact factor journals at IEEE and are currently under review.
In the collaboration with the third doctoral student, the student and the researcher proposed a novel FL method that learned a model using data at the user and server’s side, thus improving the communication efficiency and convergence speed of the FL method. This collaboration led to the publication of one journal at the IEEE Journal of Selected Topics in Signal Processing - Special Issue on Distributed Signal Processing for Edge Learning in Beyond 5G IoT Networks. Moreover, this work was selected by the IEEE Sweden VT-COM-IT Chapter among the 5 top student journal papers published in Sweden in 2023. Overall, the research conducted with Prof. Fischione and his group was closely related to the first step of Objective 1. The researcher further extended his knowledge of FL methods and their impact on wireless communications.
Together with Prof. Poor, the researcher conducted his own research direction in the development of FLASH, collaborated with two postdoctoral researchers in the group, and with a visiting professor, Prof. Piet Van Mieghem, from Delft University of Technology, The Netherlands.
In the development of FLASH, the researcher investigated the integration of the finite blocklength theory with FL methods. The researcher analysed the impact of latency and reliability requirements from vehicular communications in the convergence of FL methods, using the finite blocklength theory to model these requirements. The wireless communication scenario included a single base station with multiple-antennas and multiple devices with high reliability and low latency requirements. The FL task was related to traffic sign recognition and used the open-access dataset previously mentioned. The results indicated that fulfilling the high reliability and low latency requirements with high learning accuracy in traffic sign identification are possible, provided that appropriate device selection in the training process, transmit power allocation, and blocklength allocation are conducted. This research is currently undergoing internal review between the researcher and supervisors, and expected to be submitted to a high-impact factor journal at IEEE in late March or early April 2024. This research direction consisted of the first and third step towards fulfilling Objective 1.
In the collaboration with the postdoctoral researchers, the researcher investigated differential privacy aspects for FL methods in a wireless system with multiple base stations. Specifically, this research proposed novel differentially-private FL methods suitable to wireless communication scenarios with multiple base stations, considering the impact of privacy loss in FL and adapting the privacy levels to the conditions of the transmitted power of the devices and latency requirements. This work led to a journal publication at IET Communications – Special Issue on Advanced Technologies in Physical Layer for 6G Wireless Communication Networks. This research direction helped the research to understand the theory of differential privacy, which is closely related to the second step of Objective 1.
In the collaboration with Prof. Mieghem, the researcher collaborated with a doctoral student from Prof. Mieghem’s group on device selection for FL methods over vehicular communications. Specifically, this research direction proposed novel device selection methods considering aspects related to FL metrics, such as the loss function in the learning task, and wireless communications, such as the energy consumption, latency, and rate requirements. This research investigated vehicles driving at 60 km/h in a wireless communication scenario with multiple-antennas at the base station and vehicles. The FL task considered was traffic sign classification, and used the same open-access dataset of traffic sign images mentioned before. This research led to the submission of one journal publication and to the preparation of a second journal to be submitted to a high-impact factor journals at IEEE. Note that the journal under preparation is on the final stages, and is currently undergoing internal review with the researcher and the co-authors of the manuscript. The goal is to submit this second journal to a high impact factor journal at IEEE in late March or early April 2024. Overall, this collaboration contributed to the first and third step towards fulfilling Objective 1.
Therefore, the results of the project not only led to publications in prestigious journals and conferences, but also open several directions for further research. The results indicate that integrating FL and vehicular communications is possible, and that more research is needed in the subject. Specifically, the second step of Objective 1 could be further extended to include security FL aspects together with differential privacy. Moreover, the first step of Objective 2 could be extended to consider asynchronous communications between vehicles.
The results obtained by the project include novel FL methods that are more suitable to wireless and vehicular communications, and novel wireless resource allocation methods that are suitable to the FL training. Specifically, the researcher proposed novel FL methods that achieve faster convergence, thus improving the communication efficiency of learning task. The researcher investigated the convergence of FL methods when used over wireless and vehicular communication, including analysis the convergence speed when considering privacy, latency, and reliability requirements. Moreover, the researcher proposed novel wireless resource allocation methods that consider the FL task performed, thus adapting the device selection, transmitted power, and data rate to the requirements of different FL tasks.
The results in the project were obtained jointly with the group of Prof. Fischione and Prof. Poor, and led to several publications in high-impact journals and conference venues. Specifically, the results in the project led to four journal publications, one conference publication, the submission of three journal publications, and two journal manuscripts currently undergoing internal review and expected to be submitted in late March or early April 2024. In the following, the details of each activity conducted with Prof. Fischione and Prof. Poor are explained.
Together with Prof. Fischione, the researcher was involved in the co-supervision of two doctoral students and collaboration with a third doctoral student. In the first co-supervision, the doctoral student and the researcher proposed a novel communication-efficient federated learning methods that considered the impact of the learning methods in the energy consumption and latency on the wireless communication. This work has been recently accepted in a high-impact factor journal at IEEE, the IEEE Transactions on Wireless Communications. The results demonstrate the importance of proactively designing optimal cost-efficient stopping criteria to avoid unnecessary communication-computation costs to achieve a marginal FL training improvement.
In the second co-supervision, the doctoral student and the researcher proposed novel wireless over-the-air computation and communication methods compatible with current digital communication systems, which are also beneficial to FL and able to reduce the latency and energy consumption. This research indicated that it is possible to reduce the latency of the computation and communication of FL methods while maintaining high learning accuracy of the FL task. This research direction led to the publication of one journal at IEEE Transactions on Communications, and one conference at IEEE International Conference on Communication. Moreover, two journal publications were submitted to high-impact factor journals at IEEE and are currently under review.
In the collaboration with the third doctoral student, the student and the researcher proposed a novel FL method that learned a model using data at the user and server’s side, thus improving the communication efficiency and convergence speed of the FL method. This collaboration led to the publication of one journal at the IEEE Journal of Selected Topics in Signal Processing - Special Issue on Distributed Signal Processing for Edge Learning in Beyond 5G IoT Networks. Moreover, this work was selected by the IEEE Sweden VT-COM-IT Chapter among the 5 top student journal papers published in Sweden in 2023. Overall, the research conducted with Prof. Fischione and his group was closely related to the first step of Objective 1. The researcher further extended his knowledge of FL methods and their impact on wireless communications.
Together with Prof. Poor, the researcher conducted his own research direction in the development of FLASH, collaborated with two postdoctoral researchers in the group, and with a visiting professor, Prof. Piet Van Mieghem, from Delft University of Technology, The Netherlands.
In the development of FLASH, the researcher investigated the integration of the finite blocklength theory with FL methods. The researcher analysed the impact of latency and reliability requirements from vehicular communications in the convergence of FL methods, using the finite blocklength theory to model these requirements. The wireless communication scenario included a single base station with multiple-antennas and multiple devices with high reliability and low latency requirements. The FL task was related to traffic sign recognition and used the open-access dataset previously mentioned. The results indicated that fulfilling the high reliability and low latency requirements with high learning accuracy in traffic sign identification are possible, provided that appropriate device selection in the training process, transmit power allocation, and blocklength allocation are conducted. This research is currently undergoing internal review between the researcher and supervisors, and expected to be submitted to a high-impact factor journal at IEEE in late March or early April 2024. This research direction consisted of the first and third step towards fulfilling Objective 1.
In the collaboration with the postdoctoral researchers, the researcher investigated differential privacy aspects for FL methods in a wireless system with multiple base stations. Specifically, this research proposed novel differentially-private FL methods suitable to wireless communication scenarios with multiple base stations, considering the impact of privacy loss in FL and adapting the privacy levels to the conditions of the transmitted power of the devices and latency requirements. This work led to a journal publication at IET Communications – Special Issue on Advanced Technologies in Physical Layer for 6G Wireless Communication Networks. This research direction helped the research to understand the theory of differential privacy, which is closely related to the second step of Objective 1.
In the collaboration with Prof. Mieghem, the researcher collaborated with a doctoral student from Prof. Mieghem’s group on device selection for FL methods over vehicular communications. Specifically, this research direction proposed novel device selection methods considering aspects related to FL metrics, such as the loss function in the learning task, and wireless communications, such as the energy consumption, latency, and rate requirements. This research investigated vehicles driving at 60 km/h in a wireless communication scenario with multiple-antennas at the base station and vehicles. The FL task considered was traffic sign classification, and used the same open-access dataset of traffic sign images mentioned before. This research led to the submission of one journal publication and to the preparation of a second journal to be submitted to a high-impact factor journals at IEEE. Note that the journal under preparation is on the final stages, and is currently undergoing internal review with the researcher and the co-authors of the manuscript. The goal is to submit this second journal to a high impact factor journal at IEEE in late March or early April 2024. Overall, this collaboration contributed to the first and third step towards fulfilling Objective 1.
Therefore, the results of the project not only led to publications in prestigious journals and conferences, but also open several directions for further research. The results indicate that integrating FL and vehicular communications is possible, and that more research is needed in the subject. Specifically, the second step of Objective 1 could be further extended to include security FL aspects together with differential privacy. Moreover, the first step of Objective 2 could be extended to consider asynchronous communications between vehicles.