Periodic Reporting for period 2 - GREENPATROL (Galileo Enhanced Solution for Pest Detection and Control in Greenhouse Fields with Autonomous Service Robots)
Reporting period: 2019-05-01 to 2020-09-30
GreenPatrol solution is focused on tackling the EU need of producing more with less, as the European population grows while the area available for agriculture is gradually declining due to increased forestry and urbanisation. Greenhouse crop production emerges (22% area increase since 2011) as a way to protect crops from adverse weather conditions, allowing year-round production. However, it also creates favourable conditions for many devastating pests and diseases that can cause losses of about 15%. This has significantly increased the need for pesticides. At the same time, legislative measures and requirements regarding the quality and safety of vegetables have become increasingly demanding. Consumers awareness has risen and thus the demand for pesticide-free products.
GreenPatrol aims at developing an innovative and efficient robotic solution for Integrated Pest Management in crops, which has the ability to navigate inside greenhouses while performing localized early pest detection and treatment application in an autonomous way. The GreenPatrol navigation capability is enabled by Galileo’s new signals and the implementation of sensor fusion techniques. These capabilities redound in an increase of the production quality, yield and revenues of the farmer, while reducing the pesticide usage and its environmental impact.
The main overall objectives include: 1) Robot precise positioning solution able to operate in the greenhouse, providing accurate and detailed pest maps for decision making about precise case-specific treatment, 2) Integration of Galileo better capabilities in light indoor environment, 3) Perception with visual sensing for on-line pest detection and reasoning mechanisms for efficient action selection and 4) Control strategies for manipulation and motion planning based on pest monitoring feedback.
GreenPatrol aims at developing an innovative and efficient robotic solution for Integrated Pest Management in crops, which has the ability to navigate inside greenhouses while performing localized early pest detection and treatment application in an autonomous way. The GreenPatrol navigation capability is enabled by Galileo’s new signals and the implementation of sensor fusion techniques. These capabilities redound in an increase of the production quality, yield and revenues of the farmer, while reducing the pesticide usage and its environmental impact.
The main overall objectives include: 1) Robot precise positioning solution able to operate in the greenhouse, providing accurate and detailed pest maps for decision making about precise case-specific treatment, 2) Integration of Galileo better capabilities in light indoor environment, 3) Perception with visual sensing for on-line pest detection and reasoning mechanisms for efficient action selection and 4) Control strategies for manipulation and motion planning based on pest monitoring feedback.
The main outcome of GreenPatrol is the implementation of a complete robotic solution prototype that implements the concept of Integrated Pest Management in crops, with the ability to navigate inside greenhouses while performing early pest detection and treatment in an autonomous way.
GreenPatrol specifications were determined based on interviews to potential users, stakeholders and analysis of greenhouses. The system architecture was designed from a functional and modular point of view. Tests were defined in order to verify the fulfilment of each requirement at subsystem level, and for the whole system. Safety mechanisms were also considered both at hardware and software levels.
The GreenPatrol robotic prototype is the actual hardware-software system, including all the developments made within the GreenPatrol project. The GreenPatrol prototype is consisting of the following hardware components:
- Global Localization Unit
- Robotic platform
- Robotic arm manipulator
- Sensors (encoders, inertial sensors, 3D Lasers, cameras and lighting)
- Actuators (sprayer)
- Processing unit and communications
Apart from the GreenPatrol prototype itself, the project has also delivered a Global Localisation prototype Unit, GNSS-based and Galileo-enabled, aided by inertial sensors and odometry.
The levels of accuracy of the absolute localization solution combined with the relative localization one inside of the greenhouse, together with the autonomous navigation capabilities, allow the robot to accurately navigate and reach the target positions. This feature makes it possible for the system to carry out the inspection and treatment missions according to the IPM strategy within the required periodicity, guaranteeing that the GreenPatrol solution is a valid and efficient alternative to the traditional pest inspection and control management.
GreenPatrol concept allows achieving a remarkable reduction of production losses and chemicals usage, as well as an increase in quality and competitiveness, leading to more sustainable farming and enhanced food safety and soil and water protection.
The project included a validation stage where the prototype was tested both in simulated and real environment reaching a TRL7. GreenPatrol will be commercially exploited under a business plan that shows profits starting in year 3.
The pest detection and identification images database is a public available result of the project. By filling in a form on GreenPatrol´s website, the access to the database and labels containing images of Tuta absoluta, Bemisia tabaci and Trialeurodes vaporarium on leaves is granted.
Dissemination and communication activities included website updates, scientific papers and web press releases, appearance in radio and TV, and other channels such as Linkedin, Facebook and Twitter, participation in events, workshops and congresses.
Three components of the GreenPatrol can be exploited separately: the navigation solution, the IPM robotic strategy and the pests and diseases identification system. Patents are started for the pest detection and identification module in combination with the manipulation capabilities, and for the IPM strategy.
GreenPatrol specifications were determined based on interviews to potential users, stakeholders and analysis of greenhouses. The system architecture was designed from a functional and modular point of view. Tests were defined in order to verify the fulfilment of each requirement at subsystem level, and for the whole system. Safety mechanisms were also considered both at hardware and software levels.
The GreenPatrol robotic prototype is the actual hardware-software system, including all the developments made within the GreenPatrol project. The GreenPatrol prototype is consisting of the following hardware components:
- Global Localization Unit
- Robotic platform
- Robotic arm manipulator
- Sensors (encoders, inertial sensors, 3D Lasers, cameras and lighting)
- Actuators (sprayer)
- Processing unit and communications
Apart from the GreenPatrol prototype itself, the project has also delivered a Global Localisation prototype Unit, GNSS-based and Galileo-enabled, aided by inertial sensors and odometry.
The levels of accuracy of the absolute localization solution combined with the relative localization one inside of the greenhouse, together with the autonomous navigation capabilities, allow the robot to accurately navigate and reach the target positions. This feature makes it possible for the system to carry out the inspection and treatment missions according to the IPM strategy within the required periodicity, guaranteeing that the GreenPatrol solution is a valid and efficient alternative to the traditional pest inspection and control management.
GreenPatrol concept allows achieving a remarkable reduction of production losses and chemicals usage, as well as an increase in quality and competitiveness, leading to more sustainable farming and enhanced food safety and soil and water protection.
The project included a validation stage where the prototype was tested both in simulated and real environment reaching a TRL7. GreenPatrol will be commercially exploited under a business plan that shows profits starting in year 3.
The pest detection and identification images database is a public available result of the project. By filling in a form on GreenPatrol´s website, the access to the database and labels containing images of Tuta absoluta, Bemisia tabaci and Trialeurodes vaporarium on leaves is granted.
Dissemination and communication activities included website updates, scientific papers and web press releases, appearance in radio and TV, and other channels such as Linkedin, Facebook and Twitter, participation in events, workshops and congresses.
Three components of the GreenPatrol can be exploited separately: the navigation solution, the IPM robotic strategy and the pests and diseases identification system. Patents are started for the pest detection and identification module in combination with the manipulation capabilities, and for the IPM strategy.
The major asset of GreenPatrol project is the introduction of a robotic platform for autonomous and effective scouting and pest control based on an Integrated Pest Management (IPM) decision support system. Unlike most of the robotic solutions for pest detection and treatment (mobile bases including inspection systems), GreenPatrol has a manipulator arm on a mobile base, that provides a higher mobility to the inspection system and is able to analyse plants in different situations and plant growth stages.
Precise smart agriculture entails considerable demands in the development and implementation of new processes, addressed by this project. Artificial intelligence and an autonomous robot are used to early detect and evaluate potential crop pests. A new IPM strategy was developed especially for robotized agriculture, which is another goal. A huge number of images is needed to generate Artificial Intelligence detectors, that was not available before the project.
Currently, there is no automatic method used for directly monitoring plants for pest detection in greenhouses using a robotic platform. The proposed automated identification will provide an objective measure to evaluate the pest status and possible extension. All data generated during the pest detection task is stored using a data management system that will be very useful to develop prediction models based on environmental conditions. Sharing the generated datasets will be very beneficial for the scientific and agriculture communities.
Most robots devoted to greenhouses are specialized to particular crop and environment. For navigation, it is common to rely on rails or heating pipes or to require heavy environment instrumentation, such as beacons, tags or landmarks. In the case of autonomous robots, the precise topology of the greenhouse (number of rows, length, separation...) is often required. GreenPatrol robot provides a fully autonomous solution able to navigate without the aforementioned restrictions. It uses a combination of global and local information to generate maps of the environment to locate itself and the detected pests. The global localisation uses Precise Point Positioning techniques to get high accuracy in any location, and the Galileo E5 AltBOC signal to help improve tracking and reduce multipath in the greenhouse environment.
Precise smart agriculture entails considerable demands in the development and implementation of new processes, addressed by this project. Artificial intelligence and an autonomous robot are used to early detect and evaluate potential crop pests. A new IPM strategy was developed especially for robotized agriculture, which is another goal. A huge number of images is needed to generate Artificial Intelligence detectors, that was not available before the project.
Currently, there is no automatic method used for directly monitoring plants for pest detection in greenhouses using a robotic platform. The proposed automated identification will provide an objective measure to evaluate the pest status and possible extension. All data generated during the pest detection task is stored using a data management system that will be very useful to develop prediction models based on environmental conditions. Sharing the generated datasets will be very beneficial for the scientific and agriculture communities.
Most robots devoted to greenhouses are specialized to particular crop and environment. For navigation, it is common to rely on rails or heating pipes or to require heavy environment instrumentation, such as beacons, tags or landmarks. In the case of autonomous robots, the precise topology of the greenhouse (number of rows, length, separation...) is often required. GreenPatrol robot provides a fully autonomous solution able to navigate without the aforementioned restrictions. It uses a combination of global and local information to generate maps of the environment to locate itself and the detected pests. The global localisation uses Precise Point Positioning techniques to get high accuracy in any location, and the Galileo E5 AltBOC signal to help improve tracking and reduce multipath in the greenhouse environment.