Periodic Reporting for period 3 - RobDREAM (Optimising Robot Performance while Dreaming)
Periodo di rendicontazione: 2017-02-01 al 2018-01-31
On the way towards flexible co-workers in industrial manufacturing environments researchers and robot manufacturers have enhanced the capabilities of mobile manipulators both on hardware and software level. Although much effort has been spent to find intuitive programming methods for such robots as well as general manipulation skills, the overall performance of flexible mobile manipulators still falls short of the expectations of potential end users.
Based on the observation of the short-comings in mobile manipulation systems outlined above, RobDREAM aims at the enhancement of mobile manipulation systems in two ways:
-Acceleration of the setup process for new automation tasks through more structured and intuitive programming of the robot at task level. Allowing the system to exchange parts of the program with automatically generated and optimized routines.
-Speed up of planning and execution for general methods to solve navigation, manipulation and grasping tasks as well as general approaches for object recognition and object localization using context information and automated algorithm configuration techniques.
Over the duration of the RobDREAM action the consortium has made significant achievements in the research fields navigation, perception, manipulation and grasping as well as in parameter optimization. The developments have been successfully integrated on a mobile manipulation system and the performance has been evaluated in numerous tests in a realistic environment. It has been shown, that automatically optimizing parameters of algorithms with real-world data is a valid approach for increasing the performance of mobile manipulation systems and that the complexity of the application can still be handled at the same time.
Automatically optimizing complete mobile manipulation applications is yet a task, which still should be tackled by future work in research and technology development. The RobDREAM project has paved the way towards this goal by providing algorithms and frameworks for optimization and derived requirements, which must be met by advanced simulation systems required for the optimization over the complete application.
Based on the observation of the short-comings in mobile manipulation systems outlined above, RobDREAM aims at the enhancement of mobile manipulation systems in two ways:
-Acceleration of the setup process for new automation tasks through more structured and intuitive programming of the robot at task level. Allowing the system to exchange parts of the program with automatically generated and optimized routines.
-Speed up of planning and execution for general methods to solve navigation, manipulation and grasping tasks as well as general approaches for object recognition and object localization using context information and automated algorithm configuration techniques.
Over the duration of the RobDREAM action the consortium has made significant achievements in the research fields navigation, perception, manipulation and grasping as well as in parameter optimization. The developments have been successfully integrated on a mobile manipulation system and the performance has been evaluated in numerous tests in a realistic environment. It has been shown, that automatically optimizing parameters of algorithms with real-world data is a valid approach for increasing the performance of mobile manipulation systems and that the complexity of the application can still be handled at the same time.
Automatically optimizing complete mobile manipulation applications is yet a task, which still should be tackled by future work in research and technology development. The RobDREAM project has paved the way towards this goal by providing algorithms and frameworks for optimization and derived requirements, which must be met by advanced simulation systems required for the optimization over the complete application.
Work in RobDREAM has been performed in six objectives:
1.Requirements Assessment: at the start of the development work a requirements analysis has revealed the most common requirements for mobile manipulation and the specification of the realistic manufacturing environment (RME) has been derived.
2.Simplification of Setup: all technological and integrated developments on the robotic mobile manipulation system have been made accessible for the operator on a graphical programming interface for setting up and operating applications.
3.DREAM Architecture: a common framework and optimization methods for optimizing (DREAMing) the parameters of the key technologies for mobile manipulation systems has been designed and implemented with taking into account the specific requirements of each technology field.
4.Optimization: the key technologies (navigation, perception, manipulation and grasping) have been equipped with interfaces to the DREAMing framework and data from the evaluation environment has been used to optimize all individual technology fields.
5.Portfolio of algorithms: state-of-the-art approaches have been taken up by the consortium partners and further research and development has been performed. Interface structures for integration have been implemented, which can also be used for the integration of future developments.
6.Testing and Validation : the consortium has designed and implemented a joint integrated evaluation mobile robot system which has been used for development and evaluation constantly over the duration, especially in three evaluation rounds.
The consortium has published more than 31 scientific publications in major conferences and peer-reviewed journals. Dedicated workshops have been organized and the consortium has also been disseminated the developments at trade fairs and industrial meetings. Within the project scope five patents have been prepared. A most notable achievement is the preparation of three realistic business plans for commercial exploitation of the project results, which will be further pursued by the partners. Finally, the project has already planned for a publication of the final results and all public deliverables on the project’s website.
1.Requirements Assessment: at the start of the development work a requirements analysis has revealed the most common requirements for mobile manipulation and the specification of the realistic manufacturing environment (RME) has been derived.
2.Simplification of Setup: all technological and integrated developments on the robotic mobile manipulation system have been made accessible for the operator on a graphical programming interface for setting up and operating applications.
3.DREAM Architecture: a common framework and optimization methods for optimizing (DREAMing) the parameters of the key technologies for mobile manipulation systems has been designed and implemented with taking into account the specific requirements of each technology field.
4.Optimization: the key technologies (navigation, perception, manipulation and grasping) have been equipped with interfaces to the DREAMing framework and data from the evaluation environment has been used to optimize all individual technology fields.
5.Portfolio of algorithms: state-of-the-art approaches have been taken up by the consortium partners and further research and development has been performed. Interface structures for integration have been implemented, which can also be used for the integration of future developments.
6.Testing and Validation : the consortium has designed and implemented a joint integrated evaluation mobile robot system which has been used for development and evaluation constantly over the duration, especially in three evaluation rounds.
The consortium has published more than 31 scientific publications in major conferences and peer-reviewed journals. Dedicated workshops have been organized and the consortium has also been disseminated the developments at trade fairs and industrial meetings. Within the project scope five patents have been prepared. A most notable achievement is the preparation of three realistic business plans for commercial exploitation of the project results, which will be further pursued by the partners. Finally, the project has already planned for a publication of the final results and all public deliverables on the project’s website.
In the architectural field a „DREAM architecture enabling to intuitively tune operational parameters of mobile manipulation applications“ (TRL2->5) has been developed. The developed architecture allows the employment of algorithms in a running application in similar way as during simulation (DREAM-Phase) and it also supports the collection of data.
For optimization, existing optimization schemes (Bayesian Optimization) have been made available to the project with the required interfaces (pySMAC). Further investigations have been carried out to deal with large amounts of sparse data sets, these led to methods for „Reducing dimensions of a data vector using background knowledge“ from TRL1->5.
Autonomous Navigation for mobile robots has been one of the key technologies interfaced and further developed in RobDREAM. Significant technological achievements have been made in self-localization „ Localization based on CAD floor plans“ (TRL0->2), which supports also the setup of applications and in self-positioning („Positioning using laser-scan matching“ TRL3->4) of the robots. Further highly successful developments were contributing to a better matching of user intentions and later robot behavior: „ Adaptive Teach-and-Replay” (TRL3->4) and „ Experience-based Navigation according to User Preferences“ (TRL3->5).
For manipulation and grasping numerous devlopments were made for grasp and motion planning. These developments led to significant advances, such as in „ Hierarchical Fingertip Space Grasp Planner“ (TRL3->4) and „ Integrated Grasp Planner“ (TRL1->4) allowing the robot to grasp objects with high flexibility. The motion planning efforts could even be advanced into a state, where commercial exploitation will be possible: „ Reactive planner for bin-picking and visual pick and place“ (TRL4->7) and „ Less programming and planning-Planning with Re-Use“ (TRL2->7).
As perception of the environment is the key for interaction with objects in the surroundings of the robot, also a whole set of technologies has been developed in this field: „ Combined 2D/3D object detection and localisation“ (TRL2->6) allows reliable identification and localization of parts, while the methods for „ Autonomous and assisted object modelling“ (TRL4->7) provide a ground truth in perception optimization („Automatic optimization of pose estimation“, TRL3->6).
Development and advancement of the technology readiness levels of the technologies have been made possible by integrating the developed systems on state of the art mobile manipulation hardware and their evaluation in the realistic manufacturing environment (RME), which re-creates a real use case scenario.
With the achievements made in RobDREAM, the European Robotics sector receives a significant contribution to an increase in market-share. Especially SMEs can benefit and the deployment of robotics to new areas of application is supported. Finally RobDREAM has an impact on the cross-fertilisation between industry and academia as the frameworks for integration and evaluation can be further used in future joint efforts.
For optimization, existing optimization schemes (Bayesian Optimization) have been made available to the project with the required interfaces (pySMAC). Further investigations have been carried out to deal with large amounts of sparse data sets, these led to methods for „Reducing dimensions of a data vector using background knowledge“ from TRL1->5.
Autonomous Navigation for mobile robots has been one of the key technologies interfaced and further developed in RobDREAM. Significant technological achievements have been made in self-localization „ Localization based on CAD floor plans“ (TRL0->2), which supports also the setup of applications and in self-positioning („Positioning using laser-scan matching“ TRL3->4) of the robots. Further highly successful developments were contributing to a better matching of user intentions and later robot behavior: „ Adaptive Teach-and-Replay” (TRL3->4) and „ Experience-based Navigation according to User Preferences“ (TRL3->5).
For manipulation and grasping numerous devlopments were made for grasp and motion planning. These developments led to significant advances, such as in „ Hierarchical Fingertip Space Grasp Planner“ (TRL3->4) and „ Integrated Grasp Planner“ (TRL1->4) allowing the robot to grasp objects with high flexibility. The motion planning efforts could even be advanced into a state, where commercial exploitation will be possible: „ Reactive planner for bin-picking and visual pick and place“ (TRL4->7) and „ Less programming and planning-Planning with Re-Use“ (TRL2->7).
As perception of the environment is the key for interaction with objects in the surroundings of the robot, also a whole set of technologies has been developed in this field: „ Combined 2D/3D object detection and localisation“ (TRL2->6) allows reliable identification and localization of parts, while the methods for „ Autonomous and assisted object modelling“ (TRL4->7) provide a ground truth in perception optimization („Automatic optimization of pose estimation“, TRL3->6).
Development and advancement of the technology readiness levels of the technologies have been made possible by integrating the developed systems on state of the art mobile manipulation hardware and their evaluation in the realistic manufacturing environment (RME), which re-creates a real use case scenario.
With the achievements made in RobDREAM, the European Robotics sector receives a significant contribution to an increase in market-share. Especially SMEs can benefit and the deployment of robotics to new areas of application is supported. Finally RobDREAM has an impact on the cross-fertilisation between industry and academia as the frameworks for integration and evaluation can be further used in future joint efforts.