Periodic Reporting for period 2 - ECOLE (Experience-based Computation: Learning to Optimise)
Période du rapport: 2020-04-01 au 2022-03-31
Europe is facing challenges for growth of technological companies as the increasing complexity of products, development and production processes requires advanced integration skills and innovative minds. There is an acute shortage of human experts with sufficient skills in tackling current industrial challenges in a holistic manner, and meanwhile there is a need for solving ever more complex problems to tackle such challenges.
The aim of the ECOLE ITN is to contribute towards shortening product cycles, reducing resource consumption during complete engineering processes and creating more balanced and innovative products. This will involve not only the development of novel approaches to solving complex engineering optimisation problems, but also the equipment of early career researchers with the skills needed to advance European technological companies in the years to come, enhancing their employability and strengthening European innovation capacity. In particular, the ECOLE project will train early career researchers with strong skills in artificial intelligence and optimisation, which are in high demand by European industries.
ECOLE’s aim has been achieved through meeting the following core objectives:
(1) The development of a novel “Learning to Optimise” framework. ECOLE has taken a bold step forward in solving complex engineering optimisation problems. Instead of just developing technologies to solve a given optimisation problem instance, it has proposed novel techniques to optimise automatically across problem instances. Through knowledge, skills, and practices derived from problem solving processes in time, the experience of optimising one product or process can be learned and transferred automatically to better solve other complex optimisation problems.
(2) The development of early-stage researchers’ (ESRs’) transferrable skills and industrial experiences. Among others, this has included technical knowledge that enables them to understand the interplay between learning and optimisation, practical skills in integrating various disciplines into innovative solutions to complex industrial problems, competences related to intellectual property, management and entrepreneurship, and communication skills.
The aim of the ECOLE ITN is to contribute towards shortening product cycles, reducing resource consumption during complete engineering processes and creating more balanced and innovative products. This will involve not only the development of novel approaches to solving complex engineering optimisation problems, but also the equipment of early career researchers with the skills needed to advance European technological companies in the years to come, enhancing their employability and strengthening European innovation capacity. In particular, the ECOLE project will train early career researchers with strong skills in artificial intelligence and optimisation, which are in high demand by European industries.
ECOLE’s aim has been achieved through meeting the following core objectives:
(1) The development of a novel “Learning to Optimise” framework. ECOLE has taken a bold step forward in solving complex engineering optimisation problems. Instead of just developing technologies to solve a given optimisation problem instance, it has proposed novel techniques to optimise automatically across problem instances. Through knowledge, skills, and practices derived from problem solving processes in time, the experience of optimising one product or process can be learned and transferred automatically to better solve other complex optimisation problems.
(2) The development of early-stage researchers’ (ESRs’) transferrable skills and industrial experiences. Among others, this has included technical knowledge that enables them to understand the interplay between learning and optimisation, practical skills in integrating various disciplines into innovative solutions to complex industrial problems, competences related to intellectual property, management and entrepreneurship, and communication skills.
Eight ESRs have been recruited. All of them stayed full-time until the last day of the project. They have completed all planned training activities, including training on the core algorithms and theories required to conduct research on Learning to Optimise, reading skills, presentation skills and networking. First, they attended a number of academic modules offered by the Universities, broadening their knowledge. Second, they were embedded into research training programmes at the Universities of Birmingham and Leiden and at the Honda Research Institute Europe and NEC Lab Europe, gaining first-hand research and industrial skills and experiences. Third, they attended a number of transferable skill training activities. Fourth, they have practiced their skills through conference presentations, posters, public talks, social media posts, etc.
The Learning to Optimise framework for solving complex engineering optimisation problems requires (a) advancing learning, (b) advancing optimisation, and (c) developing novel approaches to integrate learning with optimisation. The following progress has been achieved by the ECOLE project:
a) Learning:
• Learning from imbalanced data: Real world problems frequently suffer from data imbalances, where data of a given type is available in much more limited quantity than data of other types. Machine learning algorithms struggle to learn well from such data. We investigated the adequacy of different class imbalance techniques to deal with this problem and applied them to classify mesh quality in aerodynamics simulation of a benchmark vehicle body, which allows to realize a fast and resource efficient design process. Algorithms to automatically tune parameters of class imbalance techniques were proposed.
• Text representation: Many real-world problems involve making predictions based on text data. For that, existing machine learning algorithms use text representations that are not human-interpretable, leading to lack of human trust in the results. We proposed a novel human-interpretable representation for text and applied it to interpretable user profiling. We validated the method on user generated product review data and showed promising explanation about users’ preference for feature optimization of product design.
• Prediction approaches: We developed a novel prediction approach with demonstrated improvements in medical data analysis for patient status prediction. We integrated patient meta data, such as disease and demographic information, to model correlations of patient time series, by which prediction depends on not only historic patient information but also information of related patients.
b) Optimisation:
• Supporting automobile design: We collected data from various steps of vehicle development process to create a challenging test framework for new optimisation and learning methods. We improved a novel geometric representation based on cutting-edge deep learning technology for practical application on 3D (car) shapes. Such representation has shown to enable efficient optimisation of the design of a benchmark vehicle body.
• Optimisation under uncertainty: Real-world optimisation frequently has to deal with problems whose features are not entirely and accurately known beforehand. We developed a novel optimisation algorithm able to improve optimisation of such challenging problems.
c) Integration:
• Learning to optimise for dynamic problems: Real-world optimisation problems frequently suffer changes over time. We proposed a new algorithm to learn from the optimisation results obtained before a change to more efficiently optimise after the change.
• We also studied using multi-objective evolutionary optimisation algorithm for learning, where we can consider multiple objectives simultaneously without assigning weights to objectives.
The Learning to Optimise framework for solving complex engineering optimisation problems requires (a) advancing learning, (b) advancing optimisation, and (c) developing novel approaches to integrate learning with optimisation. The following progress has been achieved by the ECOLE project:
a) Learning:
• Learning from imbalanced data: Real world problems frequently suffer from data imbalances, where data of a given type is available in much more limited quantity than data of other types. Machine learning algorithms struggle to learn well from such data. We investigated the adequacy of different class imbalance techniques to deal with this problem and applied them to classify mesh quality in aerodynamics simulation of a benchmark vehicle body, which allows to realize a fast and resource efficient design process. Algorithms to automatically tune parameters of class imbalance techniques were proposed.
• Text representation: Many real-world problems involve making predictions based on text data. For that, existing machine learning algorithms use text representations that are not human-interpretable, leading to lack of human trust in the results. We proposed a novel human-interpretable representation for text and applied it to interpretable user profiling. We validated the method on user generated product review data and showed promising explanation about users’ preference for feature optimization of product design.
• Prediction approaches: We developed a novel prediction approach with demonstrated improvements in medical data analysis for patient status prediction. We integrated patient meta data, such as disease and demographic information, to model correlations of patient time series, by which prediction depends on not only historic patient information but also information of related patients.
b) Optimisation:
• Supporting automobile design: We collected data from various steps of vehicle development process to create a challenging test framework for new optimisation and learning methods. We improved a novel geometric representation based on cutting-edge deep learning technology for practical application on 3D (car) shapes. Such representation has shown to enable efficient optimisation of the design of a benchmark vehicle body.
• Optimisation under uncertainty: Real-world optimisation frequently has to deal with problems whose features are not entirely and accurately known beforehand. We developed a novel optimisation algorithm able to improve optimisation of such challenging problems.
c) Integration:
• Learning to optimise for dynamic problems: Real-world optimisation problems frequently suffer changes over time. We proposed a new algorithm to learn from the optimisation results obtained before a change to more efficiently optimise after the change.
• We also studied using multi-objective evolutionary optimisation algorithm for learning, where we can consider multiple objectives simultaneously without assigning weights to objectives.
Progress beyond state-of-the-art has been summarised in the previous section and contributes towards different aspects of the Learning to Optimise framework. More details can be found in the 36 fully refereed publications by our ESRs. Our proposed Learning to Optimise framework demonstrated improvements on efficiency and effectiveness of solving complex real-world optimisation problems, and in particular automobile design optimisation. Long-term impact of the ECOLE project includes shortening product cycle, reducing resource consumption during the automobile engineering process, and an integrative and seamless man-machine design approach towards the creation of automobile innovations. The framework is transferrable to other domains, contributing towards European technology industry in general. Most importantly, the ECOLE project has trained 8 ESRs. All of them stayed until the last day of the project. The ECOLE training model has contributed towards structuring ESR training that involves contributions of the non-academic sector at the European level.