Periodic Reporting for period 2 - AssAssiNN (Development of a multifunctional system for complex aerostructures ASSembly, ASSisted by Neural Network)
Okres sprawozdawczy: 2021-07-01 do 2022-12-31
The future of aircraft manufacturing factories is oriented to more flexible and adaptable manufacturing systems, with new processes that enable shorter manufacturing cycles and higher environmental friendliness, energy efficiency and integration. New challenges such as a highly customized production (aimed at manufacturing new aircrafts under request) and new materials (aimed at making aircrafts lighter and, therefore, more environmentally friendly) are driving further changes in how new aircrafts are manufactured.
The main objective of the ASSASSINN project is to develop and validate a robust multifunctional assembly cell able to assist manual activities as the installation of typical fuselage systems and equipment, including cabling through the cabin structures or the application of sealant. This cell will guide the worker using Mixed and Augmented Reality during the assembly and inspection processes. The worker will be assisted by a Co-Robot while Artificial Intelligence algorithm based on Neural Networks will check the quality of the results.
This cell will be applied to three use cases, which represent different manufacturing activities: Structure installation assisted, wire assembling and sealant application.
The overall objective of the cell is to improve the quality of these activities. The cell proposed in ASSASSINN will help achieving this goal in three main ways:
• By providing instructions to workers to guide them through the assembly process. A mixed reality application and an augmented reality application have been developed to assist the operator in assembly processes. In one case by means of holograms integrated into the environment and in the other case by showing text, images and diagrams of the operations. In both applications, algorithms have been integrated to monitor the assembly process and identify errors.
• By using a collaborative robot to help the operator performing a task or completing it autonomously. The sealant application process has been automated.
• By automatically checking the quality of the processes in all the use cases using artificial intelligence.
Different approaches have also been analysed to develop a methodology to identify FOD using artificial intelligence techniques
These use cases have served to demonstrate, test and validate the developed technologies and has been carried out at Topic Manager Plant for REG IADP Fuselage/Cabin full scale demonstrator assembly.
The main objective of the ASSASSINN project is to develop and validate a robust multifunctional assembly cell able to assist manual activities as the installation of typical fuselage systems and equipment, including cabling through the cabin structures or the application of sealant. This cell will guide the worker using Mixed and Augmented Reality during the assembly and inspection processes. The worker will be assisted by a Co-Robot while Artificial Intelligence algorithm based on Neural Networks will check the quality of the results.
This cell will be applied to three use cases, which represent different manufacturing activities: Structure installation assisted, wire assembling and sealant application.
The overall objective of the cell is to improve the quality of these activities. The cell proposed in ASSASSINN will help achieving this goal in three main ways:
• By providing instructions to workers to guide them through the assembly process. A mixed reality application and an augmented reality application have been developed to assist the operator in assembly processes. In one case by means of holograms integrated into the environment and in the other case by showing text, images and diagrams of the operations. In both applications, algorithms have been integrated to monitor the assembly process and identify errors.
• By using a collaborative robot to help the operator performing a task or completing it autonomously. The sealant application process has been automated.
• By automatically checking the quality of the processes in all the use cases using artificial intelligence.
Different approaches have also been analysed to develop a methodology to identify FOD using artificial intelligence techniques
These use cases have served to demonstrate, test and validate the developed technologies and has been carried out at Topic Manager Plant for REG IADP Fuselage/Cabin full scale demonstrator assembly.
The aim of the ASSASSINN project was to develop a multifunctional cell that would optimise both the quality and time of assembly processes. In particular, assembly processes that occur during the construction of an aircraft fuselage. Technology such as artificial intelligence, augmented reality, mixed reality and collaborative robotics have been integrated into the cell.
The methodology for developing the cell consisted of defining use cases that represent assembly processes in the construction of a fuselage. The use cases have been sequentially defined and implemented.
The first case involved the assembly of intercostal parts. First, the operator must uninstall the intercostal part, then apply sealant to the part and, once the intercostal part is ready, reinstall it on the fuselage. First, a module called activity manager was developed, which controls the different components of the cell and the status of the assembly process. The following modules were then developed:
- A mixed reality application to assist the operator. The application shows the operator the assembly instructions and, using holograms, where to install the intercostal part. The operator communicates with the cell via this application.
- An automatic application of sealant. When the operator places the intercostal part on a table, the cell identifies its position using an area scanner and applies the sealant using a robotic arm.
- A quality inspection system. Using a linear scanner guided by the robotic arm, the quality of the sealant that has been applied is analysed and in case of errors, rework is carried out.
In the second use case, an augmented reality system was developed to assist the operator during cable assembly. The application provides the operator with all the information needed to perform each assembly step. In some cases, after the step has been performed, artificial intelligence algorithms are run to identify whether the step has been performed correctly. Mainly, to identify if the correct cable has been inserted in the correct hole of the connector.
The next use case involved the installation of structural parts as brackets on the fuselage. The operator is assisted with a mixed reality application similar to the one developed in the first use case. Once all the parts have been assembled, the operator using a collaborative robot and an area scanner reconstructs all the assembly area. An algorithm checks if all parts have been assembled correctly, if not, it informs the operator of the errors for repair.
Also, a feasibility evaluation to study the use of AI for automatic FOD recognition was carried. Two approaches, NERF (Neural Radiance Fields) and OLN(Object Localizer Network), have been considered to address the issue. Each has a different approach: the first one tries to find the differences between 2 images and the second one tries to learn what the foreign objects are.
After developing each use case, they have been validated at Topic Manager Plant.
Various dissemination tasks have been carried out during the project, mainly its dissemination in social networks and press. As a result of the work carried out, four articles have been published in open access journals and it is expected to publish other results the next months.
The methodology for developing the cell consisted of defining use cases that represent assembly processes in the construction of a fuselage. The use cases have been sequentially defined and implemented.
The first case involved the assembly of intercostal parts. First, the operator must uninstall the intercostal part, then apply sealant to the part and, once the intercostal part is ready, reinstall it on the fuselage. First, a module called activity manager was developed, which controls the different components of the cell and the status of the assembly process. The following modules were then developed:
- A mixed reality application to assist the operator. The application shows the operator the assembly instructions and, using holograms, where to install the intercostal part. The operator communicates with the cell via this application.
- An automatic application of sealant. When the operator places the intercostal part on a table, the cell identifies its position using an area scanner and applies the sealant using a robotic arm.
- A quality inspection system. Using a linear scanner guided by the robotic arm, the quality of the sealant that has been applied is analysed and in case of errors, rework is carried out.
In the second use case, an augmented reality system was developed to assist the operator during cable assembly. The application provides the operator with all the information needed to perform each assembly step. In some cases, after the step has been performed, artificial intelligence algorithms are run to identify whether the step has been performed correctly. Mainly, to identify if the correct cable has been inserted in the correct hole of the connector.
The next use case involved the installation of structural parts as brackets on the fuselage. The operator is assisted with a mixed reality application similar to the one developed in the first use case. Once all the parts have been assembled, the operator using a collaborative robot and an area scanner reconstructs all the assembly area. An algorithm checks if all parts have been assembled correctly, if not, it informs the operator of the errors for repair.
Also, a feasibility evaluation to study the use of AI for automatic FOD recognition was carried. Two approaches, NERF (Neural Radiance Fields) and OLN(Object Localizer Network), have been considered to address the issue. Each has a different approach: the first one tries to find the differences between 2 images and the second one tries to learn what the foreign objects are.
After developing each use case, they have been validated at Topic Manager Plant.
Various dissemination tasks have been carried out during the project, mainly its dissemination in social networks and press. As a result of the work carried out, four articles have been published in open access journals and it is expected to publish other results the next months.
ASSASSINN project will have several results beyond the state of the art:
• A mixed/augmented reality to assist operators in assembly process. The AI cell monitor the assembly process and provide the worker with up-to-date information.
• Artificial intelligence for inspection, quality control and assembly errors identification.
• Collaborative robotics to assist the worker and complete some tasks autonomously.
Concerning to the socio-economic and societal potential impacts: It is estimated a reduction of assembly production time of around 20%. Additionally, it is expected up to 80% reduction in time needed for the inspection for complex aerostructures.
ASSASSINN outputs are expected to have essential societal impact.
1. The assistance tools based on MR/AR and neural networks will allow the future definition of a more robust assembly process (same result with less effort / less hazard).
2. Human Robot Collaboration (HRC) reduces the physical workload to the human, and improve the Job Quality Index, the workers life-quality and enables to keep older people or “not physically strong” profiles at work, making the profession available to any gender o demography.
3. Public Administration (health related) cost improvement and all process related to workers’ injuries, consequent sickness absence, and similar societal impact will decrease.
4. Technologies that will result from this proposal increase the European market from the current level in the next 10 years, and will provide opportunities for the employment of highly skilled professionals.
5. The new technology has broad potential applications in many other industries (automotive and general transportation, etc.) creating opportunities for further employment.
• A mixed/augmented reality to assist operators in assembly process. The AI cell monitor the assembly process and provide the worker with up-to-date information.
• Artificial intelligence for inspection, quality control and assembly errors identification.
• Collaborative robotics to assist the worker and complete some tasks autonomously.
Concerning to the socio-economic and societal potential impacts: It is estimated a reduction of assembly production time of around 20%. Additionally, it is expected up to 80% reduction in time needed for the inspection for complex aerostructures.
ASSASSINN outputs are expected to have essential societal impact.
1. The assistance tools based on MR/AR and neural networks will allow the future definition of a more robust assembly process (same result with less effort / less hazard).
2. Human Robot Collaboration (HRC) reduces the physical workload to the human, and improve the Job Quality Index, the workers life-quality and enables to keep older people or “not physically strong” profiles at work, making the profession available to any gender o demography.
3. Public Administration (health related) cost improvement and all process related to workers’ injuries, consequent sickness absence, and similar societal impact will decrease.
4. Technologies that will result from this proposal increase the European market from the current level in the next 10 years, and will provide opportunities for the employment of highly skilled professionals.
5. The new technology has broad potential applications in many other industries (automotive and general transportation, etc.) creating opportunities for further employment.