Periodic Reporting for period 2 - REAL-NET (REAL-time monitoring and mitigation of nonlinear effects in optical NETworks)
Berichtszeitraum: 2021-01-01 bis 2023-06-30
The REAL-NET project addressed the "nonlinear capacity crunch" in optical transmission systems, which affected reliable high-speed internet vital for a digital society. It aimed to devise AI-based solutions for sturdier optical systems, train ESRs in machine learning and optical engineering, and pivot the telecom industry from hardware to software using innovative optical tools. The project reached all its research and training objectives, enhancing the telecom sector and the wider digital economy. It equipped six ESRs with skills that enhanced their industry employability. As the Optical Communication Systems & Components sector is expected to grow by 8% annually, REAL-NET's achievements positioned Europe at the forefront of telecom advancements. The pandemic highlighted the crucial role of robust internet in sectors like education and healthcare, emphasizing the project's societal importance.
REAL-NET has produced a number of research results at the edge of modern communication technology.
ESR1, Mohammad Hosseini, spearheaded a study on optimizing metro-aggregation optical networks. He employed innovative P2MP transceivers through digital subcarrier multiplexing (DSCM) and used Integer Linear Programming (ILP) for network challenges. Findings indicated DSCM-based P2MP could cut transceiver costs by 44% compared to conventional setups. Comprehensive strategies reduced these costs further by 32%. Although ILP had its constraints, genetic algorithms outperformed other solutions. His research emphasised the importance of custom solutions for network challenges.
ESR2, Pedro Freire, conducted an extensive study on the application of neural network (NN)-based equalizers in high-speed optical transmission systems, addressing the critical trade-off between performance and computational complexity. Utilizing machine learning techniques, especially deep artificial neural networks, Freire aimed to surpass the limitations of traditional deterministic equalizers. He employed a combination of multidimensional regression and sophisticated model compression methods, including pruning, weight clustering, and quantization, to design a more efficient yet less complex optical channel equalizer. Through a series of experimental setups, Freire validated that his NN-based approach yielded significant Q-factor gains at manageable computational costs. His groundbreaking research has not only been published in multiple high-impact journals but also moves the industry closer to commercially viable, resource-efficient optical transmission systems.
Industrial supervisor, Antonio Napoli: Pedro was with Infinera for 6 months and worked on the real-time implementation of ML algorithms – based on neural networks - on a FPGA to compensate for fiber nonlinearities in dispersion uncompensated links. His work has been very successful, as it can be evinced by his numerous publications.”
ESR3, Mohannad Abu-Romoh and ESR4, Jamal Darweesh, developed post-training and training-aware quantization algorithms for quantizing the NNs equalizers in optical fiber communication. Their results showed that the computational complexity and memory requirements of the NN can be significantly reduced, typically by over 90%, with negligible drop in the Q-factor performance. Furthermore, they developed an LDBP for mitigating the nonlinear effects in DM transmission systems. Their Q-factor and complexity analysis showed that LDBP provides significant improvements in performance or complexity over DBP.
Industrial supervisor, Antonio Napoli: “Mohannad was with Infinera for 6 months and worked on the optimization of neural networks for compensating fiber nonlinearities in dispersion-managed links. His work has been successful, and the results were later published in the OPTICA journal OSA Continuum.”
ESR5, Masab Iqbal, focused on enhancing the efficiency of optical networks, especially in the realms of data traffic patterns and security. He conducted a comparative study of point-to-point (P2P) and point-to-multipoint (P2MP) technologies, specifically Digital Subcarrier Multiplexing (DSCM) and a novel technology, Optical Constellation Slicing (OCS). His findings indicate that P2MP technologies can offer cost savings and greater efficiency in handling dynamic and heterogeneous data flows. On the security front, Masab has developed Light Path SECurity (LPsec), a cryptographic solution for optical connections that significantly speeds up encryption while maintaining robust security. He also proposed protocols to make quantum communication more reliable. Quantum communication is the future for overcoming the security threats posed by quantum computers.
ESR6, Diogo Sequeira, focused on real-time performance monitoring of optical networks using machine learning. He has developed OCATA, a deep-learning-based digital twin for optical time domains, to model lightpath features such as linear and nonlinear noise. OCATA is designed to offer high accuracy in representing actual network conditions, making it suitable for applications like network automation and fault management. His work also extends to the estimation of bit error rate degradation and failure detection, leveraging monitored data to train, test, and validate the digital twin.
The six ESRs showcased their work at top photonics conferences with 18 conference talks and papers, and published 18 peer-reviewed journal papers. REAL-NET exceeded expectations by hosting 22 events instead of the planned 14, including 3 skills training, 5 annual workshops, and 14 other training events, reaching over 1,150 external researchers. They also engaged in 14 outreach activities touching various societal sectors. REAL-NET's impact is evident and is expected to grow in the coming years.
ESR1, Mohammad Hosseini, spearheaded a study on optimizing metro-aggregation optical networks. He employed innovative P2MP transceivers through digital subcarrier multiplexing (DSCM) and used Integer Linear Programming (ILP) for network challenges. Findings indicated DSCM-based P2MP could cut transceiver costs by 44% compared to conventional setups. Comprehensive strategies reduced these costs further by 32%. Although ILP had its constraints, genetic algorithms outperformed other solutions. His research emphasised the importance of custom solutions for network challenges.
ESR2, Pedro Freire, conducted an extensive study on the application of neural network (NN)-based equalizers in high-speed optical transmission systems, addressing the critical trade-off between performance and computational complexity. Utilizing machine learning techniques, especially deep artificial neural networks, Freire aimed to surpass the limitations of traditional deterministic equalizers. He employed a combination of multidimensional regression and sophisticated model compression methods, including pruning, weight clustering, and quantization, to design a more efficient yet less complex optical channel equalizer. Through a series of experimental setups, Freire validated that his NN-based approach yielded significant Q-factor gains at manageable computational costs. His groundbreaking research has not only been published in multiple high-impact journals but also moves the industry closer to commercially viable, resource-efficient optical transmission systems.
Industrial supervisor, Antonio Napoli: Pedro was with Infinera for 6 months and worked on the real-time implementation of ML algorithms – based on neural networks - on a FPGA to compensate for fiber nonlinearities in dispersion uncompensated links. His work has been very successful, as it can be evinced by his numerous publications.”
ESR3, Mohannad Abu-Romoh and ESR4, Jamal Darweesh, developed post-training and training-aware quantization algorithms for quantizing the NNs equalizers in optical fiber communication. Their results showed that the computational complexity and memory requirements of the NN can be significantly reduced, typically by over 90%, with negligible drop in the Q-factor performance. Furthermore, they developed an LDBP for mitigating the nonlinear effects in DM transmission systems. Their Q-factor and complexity analysis showed that LDBP provides significant improvements in performance or complexity over DBP.
Industrial supervisor, Antonio Napoli: “Mohannad was with Infinera for 6 months and worked on the optimization of neural networks for compensating fiber nonlinearities in dispersion-managed links. His work has been successful, and the results were later published in the OPTICA journal OSA Continuum.”
ESR5, Masab Iqbal, focused on enhancing the efficiency of optical networks, especially in the realms of data traffic patterns and security. He conducted a comparative study of point-to-point (P2P) and point-to-multipoint (P2MP) technologies, specifically Digital Subcarrier Multiplexing (DSCM) and a novel technology, Optical Constellation Slicing (OCS). His findings indicate that P2MP technologies can offer cost savings and greater efficiency in handling dynamic and heterogeneous data flows. On the security front, Masab has developed Light Path SECurity (LPsec), a cryptographic solution for optical connections that significantly speeds up encryption while maintaining robust security. He also proposed protocols to make quantum communication more reliable. Quantum communication is the future for overcoming the security threats posed by quantum computers.
ESR6, Diogo Sequeira, focused on real-time performance monitoring of optical networks using machine learning. He has developed OCATA, a deep-learning-based digital twin for optical time domains, to model lightpath features such as linear and nonlinear noise. OCATA is designed to offer high accuracy in representing actual network conditions, making it suitable for applications like network automation and fault management. His work also extends to the estimation of bit error rate degradation and failure detection, leveraging monitored data to train, test, and validate the digital twin.
The six ESRs showcased their work at top photonics conferences with 18 conference talks and papers, and published 18 peer-reviewed journal papers. REAL-NET exceeded expectations by hosting 22 events instead of the planned 14, including 3 skills training, 5 annual workshops, and 14 other training events, reaching over 1,150 external researchers. They also engaged in 14 outreach activities touching various societal sectors. REAL-NET's impact is evident and is expected to grow in the coming years.
In the REAL-NET project, we revolutionised optical networks by introducing three major advancements. Firstly, we integrated machine learning algorithms, transforming the traditionally rigid networks into adaptable systems that could predict and rectify issues on their own. Secondly, our innovative data transmission techniques not only boosted efficiency but also consumed less energy and utilized fewer resources, optimizing both performance and cost. Lastly, we fortified the networks with cutting-edge security protocols, specifically designed for high-speed data flow, ensuring both speed and safety for users.
Our machine learning algorithms matured to autonomously identify, diagnose, and address most network challenges, dramatically reducing the need for human oversight. Moreover, we honed our data transmission methods, yielding networks that were not only faster but also environmentally friendlier, thereby supporting worldwide sustainability aspirations. Additionally, our advanced security measures have established a new gold standard in the industry, fortifying networks against potential unauthorized access and breaches.
Our machine learning algorithms matured to autonomously identify, diagnose, and address most network challenges, dramatically reducing the need for human oversight. Moreover, we honed our data transmission methods, yielding networks that were not only faster but also environmentally friendlier, thereby supporting worldwide sustainability aspirations. Additionally, our advanced security measures have established a new gold standard in the industry, fortifying networks against potential unauthorized access and breaches.