Functionalised Soft robotic gripper for delicate produce harvesting powered by imitation learning-based control

Project Information

SoftGrip

Grant agreement ID: 101017054

DOI

10.3030/101017054

Project closed

EC signature date 4 December 2020

Start date 1 January 2021

End date 30 June 2024

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 2 998 955,00

EU contribution

€ 2 998 955,00

Coordinated by

SCUOLA SUPERIORE DI STUDI UNIVERSITARI E DI PERFEZIONAMENTO S ANNA
Italy

Periodic Reporting for period 2 - SoftGrip (Functionalised Soft robotic gripper for delicate produce harvesting powered by imitation learning-based control)

Reporting period: 2022-07-01 to 2024-06-30

Summary of the context and overall objectives of the project

The fresh food industry is under growing pressure to reduce production costs due to high levels of competition. Moreover, in the specific case of mushroom industry, it is highly labour-intensive, with much of the manual work requiring rapid, repetitive, and monotonous movement that lead to poor quality control and a high incidence of industrial accidents.

Therefore, the food industry is looking increasingly towards automation and robotics to help lower production costs and improve job quality and safety. However, the capabilities of the human hand in terms of dexterity, sensitivity and precision are still unmet. So, the main objective of the SoftGrip project was to develop a self-actuating soft gripper for the autonomous picking of delicate white button mushrooms.

We studied and developed low-cost, soft robotic grippers having built-in actuation, sensing and embodied intelligence that enable safe-grasping, adaptability to object shape and size, and grasping versatility for reliable and efficient picking of mushrooms. We engineered blending of novel materials that offer precise tuning of fundamental material properties, that interact safely with the food, and with minimum impact on the environment (through recyclability). We used a set of accelerated continuum mechanics modelling algorithms that facilitate sophisticated model-based control schemes. Moreover, we investigated advanced cognition capabilities in the soft gripper through a learning by demonstration framework.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During the first period, the consortium work was mainly focused on knowledge exchange, definition of requirements and fundamental research on stand-alone subsystems and simple devices. Despite in-person hands-on meetings were limited by the pandemic situation, the consortium also developed preliminary prototypes for the gripper and the robotic platform moving first steps towards an early integration of the full platform on a realistic scenario.

In the second period, the design underwent several iterations to comply with requirements but also technical constraints. In-person workshops have been held to finalize the integration in a simplified scenario and finally to test the system in a realistic environment.

The main results include:

• Three versions of fingered grippers which embed sensors for proprioception (pose estimation) and exteroception (fingertip contact force) and an in-hand camera for alignment adjusting
• A robot base fully compliant with Dutch shelves
• A low-cost vision system that showed good reliability on the identification of the correct mushroom to pick
• A simulation framework that includes the mushroom root failure mode
• Model-based control schemes for bending angle control and gentle grasping realization
• An imitation learning-based adaptive motion planner that learns to react to tactile feedback
• A complete robotic system able to perform a harvesting routine in a realistic environment (tested at Teagasc facilities)
• Several low TRL ideas which are shared through scientific publications to allow future initiatives to explore them further.

Along with these main results, the project achieved unplanned results:

- A general colorimetric method to grade the mushroom quality, to be used to assess the damage generated at picking.
- The development of artificial mushrooms as physical twins for assessing harvesting main parameters, usable for training purposes too.

The results have been presented in a major dissemination event (the Dutch Mushroom Days) and in a stakeholder event organized by one of the partners. Furthermore, the results have been also disseminated through scientific publications (in indexed journals and esteemed conferences), events, website, social networks, and press releases.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Manipulating fragile and delicate objects is a difficult task for a robotic device, especially because it involves multiple interconnected features and abilities: delicate yet firm grasp, motion/deformation capabilities, effective and adaptable motion execution, advanced sensing, durability, contact safety, just to name the most important. In this context, mushroom harvesting stands as a very challenging scenario and as a paradigmatic example to explore new approaches and technological routes. There exist some ongoing studies on the field, but with limited success so far, especially regarding the achievable produce quality. The main innovation brought by the project consists of an advanced robotic platform that exploits a multidisciplinary approach combining material science, soft mechatronics, advanced vision algorithms, and cutting-edge learning techniques.

The SoftGrip project was set to be a feasibility study with high ambitions to apply the latest technologies in soft-robotics, vision systems and imitation learning algorithms to the harvesting of soft delicate produce, such as mushrooms. Consequently, it had a low technology readiness level (TRL) of 4 assigned to it, which is defined as “technology validated in lab”. However, we pushed the technology well beyond to TRL 5 that is defined as “technology validated in an industrially relevant environment”. The SoftGrip technologies were demonstrated in the industrially relevant environment of Teagasc’s mushroom growing rooms with beds of mushrooms prepared and grown at a semi commercial scale.

SoftGrip approach