Final Report Summary - CROPS (Intelligent sensing and manipulation for sustainable production and harvesting of high value crops, clever robots for crops)
Executive Summary:
The following results were obtained in the CROPS project.
For the robotic middleware ROS was chosen as software framework. The supervisory control system as well as the high-level software architecture have been developed and tested.
The design and implementation of the sensory systems for CROPS have been completed and tested , including the sensory system for detection and localisation of fruits in orchards and greenhouses, the sensory system for detection and classification of objects, the sensory system for fruit ripeness evaluation, the sensory system for diseases detection in crops and the sensory system for ground bearing in forestry.
A nine degree of freedom manipulator was manufactured. End-effectors were tested. A prototype of a canopy sprayer was manufactured. The first manipulator prototype, the grippers, and the precision spray end-effector were tested in laboratory and field experiments. Bases on these tests a final manipulator was designed and tested in a commercial greenhouse for harvesting sweet pepper.
For sensor fusion the system architectures for sensing, grasping and fusion and algorithms for sensor fusion, learning in sensing and grasping were developed. The adaptive sensor fusion algorithm was implemented and tested for apples and sweet pepper and for disease and ripeness detection .A method for construction of a discrete fuzzy grasp affordance manifold based on learning from human demonstration was developed and tested.
For sweet pepper harvesting the requirements for a harvesting robot were obtained. The manipulator, a platform to transport the manipulator through the greenhouse, a gripper and a sensing system were integrated into a complete system. This system was successfully tested and demonstrated to growers in both a laboratory setting as in a commercial greenhouse.
For harvesting of grapes and apples requirements have been defined based on discussions with the growers. To maximize the visibility and reachability of the fruits, the so-called ‘walls of fruit trees’ growing system has been chosen. The CROPS manipulator, grippers, sensors and software architecture have been tested both in laboratory as well as in apple orchards and as in vineyards. All the modules have been integrated into one system, which was successfully tested in the laboratory and in an apple orchard.
For canopy optimised spraying and close range precision spraying requirements were selected in discussion with spraying specialists and growers. A canopy optimised sprayer was designed as a trailed sprayer with centrifugal blower. An eight DOF hydraulic driven manipulator with three arms was used. Orchard experiments were performed with the canopy optimised sprayer during 2013. A good spraying quality was achieved with significant reduction of pesticide use. The close range precision spraying was focused on testing disease detection with various sensing principles. The manipulator with waterproof protecting case, sensors precision spraying end-effector were integrated in a precision spraying robot for viniculture. The robot was successfully tested in a greenhouse environment and the attained pesticide reduction was 84%.
For the forestry application the requirements for the detection of bushes, rocks, and trees and for the estimation of ground bearing capacity (for propulsion of forest machines) have been specified. Two sensory system for detection and classification of trees and humans respectively have been evaluated in a forestry environment.
As part of the training work package 44 graduate students were recruited in the project. 33 graduate courses were proposed at the participating universities for the students. The students took 17 advanced courses, participated in conferences and visited various universities and industries. Eight different workshops took place during the CROPS meetings.
For dissemination tha website www.crops-robots.eu has been made. 25 articles in technical journals, 10 articles in farmer’s magazines, 69 papers in conference proceedings and 12 papers on scientific journals were published. 76 presentations on conferences were given. The final CROPS workshop was held during the AgEng2014 conference in Zurich, July 2014.
Project Context and Objectives:
The main objective of the EU-FP7 project CROPS (GA 246252) is to develop a highly configurable, modular and clever carrier platform comprising a carrier plus modular parallel manipulators and “intelligent tools” (sensors, algorithms, sprayers, grippers) that can easily be installed onto the carrier and that are capable of adapting to new tasks and conditions. Both the scientific know-how and a number of technological demonstrators will be developed for the agro management of high value crops like greenhouse vegetables, orchard fruits, and grapes for premium wines. The CROPS robotic platform will be capable of site-specific spraying (targeted spraying only on foliage and selected targets) and selective harvesting of fruit (i.e. it will detect the fruit, determine its ripeness, move towards the fruit and grasp it and softly detach it). Another objective of CROPS is to develop techniques for reliable detection and classification of obstacles and other objects to enable successful autonomous navigation and operation of the platform in plantations and forests.
The CROPS project context and the CROPS objectives are described per work package:
WP1 System Engineering and Architecture:
Work package 1 is devoted to the research and development of an open systems architecture that provides a modular configurable platform for all software and hardware, sensors and controllers in CROPS. It aims at specifying and validating the major system components, some of which are developed as parts of other work packages. Other important components, the hardware and software system containing interfaces to sensors, controllers, actuators and human operators, will be developed as part of WP1.
The major objectives of WP1 are:
1. Definition of the overall system specifications
2. Design and construction of a computer hardware platform, including interfaces
3. Design and implementation of a software framework and supervisory control system
4. Validation of system components
5. Integration of system components
WP2 Sensing:
Work package 2 is devoted to the research and development of a low cost, efficient, adaptable, robust, secure and intelligent sensing system for the CROPS robotic platform. With this sensing technology, the CROPS robotic platform will be able to: navigate autonomously in greenhouses and outdoor
environments, detect fruits during different stages of their life cycle, provide sensory input for agromechanical operations, and perform sophisticated tasks, such as ripeness evaluation and disease
detection on fruits and crop canopy. Forestry tasks will also benefit from these WP2 developments. The scope of WP2 includes the following major objectives:
1. Localization and characterization of fruits in a controlled environment
2. Localization and characterization of fruits in natural environments
3. Detection and classification of objects in plantations and forests
4. Obstacle detection for autonomous navigation
5. Sensing of ripeness
6. Detection of diseases
7. Formulation of evaluation methodologies for sensing results
WP3 Manipulators and End-Effectors:
The objectives of work package 3 are the design, modelling, manufacture, start-up and optimization of low cost, lightweight, modular and compliant manipulators with exchangeable end-effectors and their motion control.
The major objectives of this work package are to design and to develop:
1. Prototype of manipulator for large motion
2. Prototype of manipulator for accurate positioning
3. Prototype of end-effector for gripping or fruit removal
4. Prototype of end-effector for spraying
5. Hierarchical simulation model of the manipulator system
6. Design of a low-level controller interface
7. Motion and grasping control.
WP4 Intelligent Sensor Fusion and Learning Algorithms:
Work package 4 is devoted to the research and development of intelligent algorithms for sensing and grasping in the CROPS robotic platform. The algorithms will enable the CROPS robotic platform to: (i) perform robustly and efficiently in a variety of operating and sensing conditions by fusing the information from multiple and different types of sensors, and (ii) easily adapt to new varieties of crops and to crops other than those of the CROPS project through continuous learning.
Specifically, WP4 focuses on the development and implementation of:
1. Adaptive sensor fusion framework and algorithms
2. Learning algorithms for the different sensing tasks
3. Learning algorithms for grasping.
WP5 Sweet pepper – protected cultivation:
The objectives of work package 5 are:
1. To develop and demonstrate a robotic system, focussed on harvesting, for sweet pepper production in protected cultivation systems.
2. To develop and evaluate generic concepts that can be added to this platform to perform other functions like planting, attaching plants to supporting wires, pruning, monitoring yield and quality, and local crop protection.
WP6 Harvesting systems in orchards: grapes and apples:
The objectives of work package 6 are:
1. To develop and demonstrate a robotic system, focused on harvesting, for apples and grapes.
2. To develop and evaluate generic concepts that can be added to this platform to perform other functions like planting, pruning, monitoring yield and quality, and local crop protection as well.
WP7 Precision spraying:
The objectives of work package 7 are:
1. To achieve canopy optimised spraying where the spraying operating parameters are continuously adapted to the canopy characteristics, such as volume, shape, density, etc., varying in space and time with the aim to reduce pesticide application by 30%.
2. To achieve high-precision close range target spraying to selectively and precisely apply chemicals solely on targets susceptible to specific diseases/pests with a 90% success rate and a 90% reduction in pesticide use for selected varieties.
WP8 Forestry:
Work package 8 is devoted to sensory and perception solutions for forestry automation applications. The proposed tasks are object detection/classification (bushes, rocks and trees) and estimation of ground bearing capacity (for propulsion of both manual and autonomous forest machines). The developed solutions will also be applied to robots in vineyard environments (WP7) and greenhouse environments (WP5).
The specific objectives of WP8 are to specify, adapt, implement, test and evaluate sensing solutions in close cooperation with WP2 and WP4. As a result of WP8, parts of the CROPS project will be
of immediate application for forestry automation.
WP9 Training:
Work package 9 includes several activities to provide multidisciplinary training in a joint industrial and academic
environment, aiming to create a new generation of agro-forestry robotic researchers and engineers with a broad understanding of the needs and opportunities for intelligent robots that function in the agro-forestry environment. To succeed in the complex agro-forestry environment, all partners must be exposed to in-depth knowledge in the different domain areas. The main objectives of WP9 are to:
1. Train the staff of robotics/mechanical/sensing industries involved in CROPS developments in such a way that they will be able to adapt their developments to the specific characteristics of the agricultural environment that influence design and performance.
2. Train the staff of agricultural industries involved in CROPS developments in such a way that they will be able to adapt their developments to the use of advanced robotic systems.
3. Train Young Researchers (YRs) and engineers to give them in-depth knowledge from the many disciplines involved in CROPS.
4. Train YRs to work in an industrial-academic environment and endow them with complementary skills (project management, team work, presentations).
WP10 Dissemination:
The objectives of work package 10 are:
1. To inform the scientific community, especially in the areas of robotics and agricultural engineering, about the relevant results arising from the project and to disseminate experimental data and source code.
2. To ensure that all research results are disseminated to European industry.
3. To inform the farmers, through their local and national associations, about those results of the project that are ready for use and about new products/techniques possibly arising in the near future from the application of project outcomes.
WP11 Final demonstration:
The objectives of Work package 11 are:
Demonstrate final CROPS platforms under a variety of operating conditions for the following tasks:
1. Selective harvesting of sweet peppers in a closed cultivation system
2. Selective harvesting of apples
3. Selective harvesting of grapes
4. Canopy optimized sprayers for vineyards and orchards
5. Targeted sprayers for vineyards and orchards
6. Forestry machine capable of classifying obstacles.
WP12 Economics, Social Aspects, Sustainability and Exploitation:
The objectives of work package 12 are:
1. Analyse the parameters and factors influencing economic viability and social aspects and determine other requirements for sustainability with the aim to enable correct design requirements that will ensure successful market penetration
2. Evaluate and validate the performance of the developed technologies from a business point of view with the aim to ensure practical applications of the products developed and easy and fast market penetration and adoption of the technologies by farmers
3. Protect intellectual property rights and ensure open-source availability
4. Ensure exploitation of the project results.
WP13 Coordination:
The objectives of work package 13 are:
1. Manage the project and fulfil of all its goals.
2. Draft the Consortium Agreement and submit it to the partners for approval.
3. Organize meetings (including minutes) for the governing and management bodies and the Advisory
Group.
4. Deliver periodical reports for the Commission.
5. Monitor deliverables and milestones.
6. Create and manage project web-portal for internal and external communication.
Project Results:
Main S&T results
CROPS
Intelligent sensing and manipulation for sustainable production and harvesting of high value crops, clever robots for crops
Grant Agreement number 246252
Consortium:
1. Stichting Dienst Landbouwkundig Onderzoek (WUR), The Netherlands
2. Katholieke Universiteit Leuven (KU Leuven), Belgium
3. Ben-Gurion University of the Negev (BGU), Israel
4. Univerza V Ljubljani (UL), Slovenia
5. Umea Universitet (UMU), Sweden
6. Università degli Studi di Milano (UniMI), Italy
7. Agencia Estatal Consejo Superior De Investigaciones Cientificas (CSIC), Spain
8. Technische Universität München (TUM), Germany
9. CNH Industrial Belgium NV (CNHi), Belgium
10. Instituto De Investigaciones Agreopecuarias (INIA), Chile
11. FORCE-A SA (Force_A), France
12. Festo AG & Co (Festo), Germany
13. Sveriges Lantbruksunivesitet (SLU), Sweden.
Work packages:
The CROPS project consists of the following 13 work packages, with between brackets the lead beneficiary.
1. Systems engineering and architecture (TUM)
2. Sensing (CSIC)
3. Manipulators and end-effectors (TUM)
4. Intelligent sensor fusion and learning algorithms (BGU)
5. Sweep pepper – protected cultivation (WUR)
6. Harvesting systems in orchards: grapes and apples (KULeuven)
7. Precision spraying (UL)
8. Forestry (UMU)
9. Training (BGU)
10. Dissemination (UniMI)
11. Final demonstration (UniMI)
12. Economics, social aspects, sustainability and exploitation (WUR)
13. Coordination (WUR)
Introduction:
The main objective of CROPS (www.crops-robots.eu) is to develop a highly configurable, modular and clever carrier platform comprising a carrier plus modular parallel manipulators and intelligent tools” (sensors, algorithms, sprayers, grippers) that can easily be installed onto the carrier and that are capable of adapting to new tasks and conditions. Both the scientific know-how and a number of technological demonstrators will be developed for the agro management of high value crops like greenhouse vegetables, orchard fruits, and grapes for premium wines. The CROPS robotic platform will be capable of site-specific spraying (targeted spraying only on foliage and selected targets) and selective harvesting of fruit (i.e. it will detect the fruit, determine its ripeness, move towards the fruit and grasp it and softly detach it). Another objective of CROPS is to develop techniques for reliable detection and classification of obstacles and other objects to enable successful autonomous navigation and operation of the platform in plantations and forests. The rationale for this aspect of the project is that agricultural and forestry applications share many common research areas, primarily regarding sensing and learning capabilities.
The results of the CROPS project will be given by work packages.
Results:
1. Systems engineering and architecture (work package 1)
Work package 1 aims at formulating the system architecture and specifying and validating the system components, also with regard to the interfaces in between.
Therefore, high-level requirements for the components have been derived from different harvesting (sweet pepper, grapes, and apples) and spraying scenarios based on the input of all project partners in a product requirements document. Functional specifications for all subsystems have been defined during an iterative process in collaboration with all partners according to the VDI2519 guideline, which resulted in a functional specification document. Based on this document, the development and the subsequent testing and evaluation process has been performed.
During the decision process, a meeting about the robotic middleware was held at the Technical University Munich, where the hardware framework has been specified, and the, at that time very new, Robot Operating System (ROS) was chosen as software framework. To become familiar with the framework, a ROS school has been organized by the Technical University of Munich (TUM) for all interested partners.
The supervisory control system and the high-level software architecture has been developed. It consists of multiple modules which communicate via defined interfaces. With the handover of the first manipulator prototype, including the manipulator for large motion and for accurate positioning as well as the gripper, from the Technical University of Munich and Festo to WUR, the basis for the integration process was provided.
Furthermore, a web application (wiki) has been set up in order to support the testing and integration process. This wiki is a common internet-based platform for the project CROPS. It was accessible and could be modified by all partners and contains general information about the hardware and software modules, definitions of the communication as well as electrical and mechanical interfaces between the subsystems. A first manipulator prototype system has been circulated among the application work packages resulting into an extensive testing in the field of the integrated robot systems. The computer hardware platform, the supervisory control system as well as testing, evaluation and integration of the subsystems for achieving a fully integrated harvesting or spraying robot system were finalized.
2. Sensing (work package 2)
Work package 2 is devoted to the research on and the development of a low cost, efficient adaptable, robust, secure and intelligent sensing system for the CROPS robotic platform. With this sensing technology the CROPS robotic platform should be able to navigate autonomously in greenhouses and outdoor environments, detect fruits during different stages of their life cycle, provide sensory input for agromechanical operations, and perform sophisticated tasks, such as ripeness evaluation and disease detection on fruits and crop canopy.
Firstly the requirements and specifications of the sensing systems for CROPS were completed, resulting in the decisions for each sub-system: sensory system for fruit detection, sensory system for obstacle detection, sensory system for ripeness evaluation, disease detection system and system for estimation of ground bearing capacity. A first report on active perception algorithms for CROPS describes the role that will play the active perception algorithms on dealing with common complexities that will be found in the working scenarios, such as the occlusions, the variations in the apparent colour and the changeable illumination conditions. The sensory system for fruit detection and the sensory system for obstacle detection in the sweet pepper application was completed. It was decided to incorporate a mini-camera in the end-effector of the manipulator. This was intended to improve the detection in conditions in which the occlusion of the fruit is high, and also to improve the detection of the peduncles (and cutting position) of the sweet-peppers.
Furthermore the design and implementation of the sensory system for detection and localisation of fruits in orchards, the sensory system for detection and classification of objects, the sensory system for fruit ripeness evaluation and the sensory system for diseases detection in crops was completed.
Numerous experimental campaigns have been carried out during this period in different scenarios and with varied environmental conditions not only for accomplishing successful designs and implementations of the mentioned sensory systems, but also for their validation.
These modules embrace from basic pixel classification based on multispectral signature, through analysis of the 2D image for regions of interest, to the detection and localisation of targets in 3D space.
For the classification of objects in forest, a number of existing algorithms for detection of trees and estimation of tree diameter were evaluated and three new and more accurate algorithms for the stated purpose were suggested. Finally, the sensor system for fruit ripeness evaluation, as well as the sensor system for disease detection have been designed and tested successfully in field conditions.
A sensory system for ground bearing estimation has been designed and tested, this sensing could be both applied in forestry as in orchards.
3. Manipulators and end-effectors (work package 3)
Work package 3 is focussed on the design, modelling, manufacture, start-up and optimization of low cost, lightweight, modular and compliant manipulators with exchangeable end-effectors and their motion control.
First the high level system requirements for the manipulator and the end-effectors were derived on the basis of the application work packages 5 (Sweet pepper harvesting), 6 (apple and grape harvesting) and 7 (precision spraying).
A preliminary study on pneumatic actuator concepts for the manipulator resulted in the rejection of this concept and electrical actuators were applied for the complete manipulator system. Several kinematic manipulator designs were investigated and evaluated for the different applications. A nine degree of freedom, redundant and modular, manipulator turned out to be the most promising concept and a kinematic model was derived as well as a multi body simulation was performed for the mechatronic design of the system. The first manipulator was manufactured, assembled and started-up at the Technische Universität München (TUM).
End-effector prototypes for gripping and fruit removal for the described application scenarios were designed, manufactured, started up, and tested by partner Festo. The requirements for the spraying prototype were derived and based on numerical simulation a first prototype end-effector for spraying was manufactured. At TUM the integration of the first manipulator and the end-effector for gripping was done and the complete system was delivered to the Stichting Dienst Landbouwkundig Onderzoek (WUR) for the installation of the manipulator on a robotic platform and first field tests in sweet-pepper greenhouses were carried out.
The first manipulator prototype, the grippers, and the precision canopy sprayer were tested in laboratory and field experiments. Additionally, a simulation model with direct and inverse kinematics algorithms for serial manipulators, as well as equations to describe the manipulator dynamics was implemented.
Especially the fruit removal for sweet-pepper harvesting turned out to be very challenging for the gripper and sensor system design. Two end-effector concepts for the sweet-pepper harvesting application were designed and built by respectively WUR and one by FESTO. The final gripper prototypes include a Time of Flight (ToF) and RGB camera.
The first manipulator prototype has been adapted for the precision spraying application and was integrated with the spraying platform in an experimental greenhouse of the University of Milano, Italy (UniMi).
Based on the experiments with the first manipulator prototypes and the feedback of the project partners TUM has designed the final manipulator prototype. This prototype is assembled out of custom made robot drive modules.
The final manipulator prototype, designed by TUM, has been manufactured and assembled. After the start-up of the prototype in Munich in the beginning of 2014 and a complete functional test, the robot has been integrated in Wageningen for the sweet pepper harvesting application in February 2014 and was evaluated in field experiments by WUR mid-2014. Major improvements of the final manipulator prototype (compared to the first manipulator) are:
• Increased robustness of the mechatronic design required for agricultural applications
• Custom designed drive modules allowing for a high modularity
• Higher Power allowing for faster motions and more payload.
Further field experiments with the first manipulator prototype were carried out by the Katholieke Universiteit Leuven (KULeuven). A motion and grasping strategy for apple harvesting, based on dynamic motion primitives has been developed by the Ben-Gurion University of the Negev (BGU).
4. Intelligent sensor fusion and learning algorithms (work package 4)
Work package 4 is devoted to the research and development of intelligent algorithms for sensing and grasping in the CROPS robotic platform. The algorithms will enable the CROPS robotic platform to: (i) perform robustly and efficiently in a variety of operating and sensing conditions by fusing the information from multiple and different types of sensors, and (ii) easily adapt to new varieties of crops and to crops other than those of the CROPS project through continuous learning.
The system architectures for sensing, grasping and fusion (inputs, outputs, connection with other system modules) and basic algorithms for sensor fusion, learning in sensing and learning in grasping were characterized and developed. The adaptive sensor fusion architecture was designed and implemented in Matlab and in ROS. It includes a module for threshold optimization and an object unification module. The algorithms developed were tested on the first two acquired databases with measurement from field tests. Preliminary results using a limited database from a sweet pepper greenhouse indicates that by fusing several algorithms detection of peppers is increased resulting in robustness for changes in lighting conditions.
A generic learning framework consisting of a Sensor Layer, a Feature Layer and a Classification Layer was developed. To demonstrate the framework a depth sensor and an RGB camera were used to generate foreground images to learn features for classification of forestry objects. A database of images was collected and labelled. The objects were classified as bush, tree, stone and human. Adding depth based features improved classification accuracy considerably.
A method for construction of a discrete fuzzy grasp affordance manifold based on learning from human demonstration has been developed. It includes quality grade determination, manifold structure determination, and a method for adapting the human grasp manifold to different manipulators and grippers. Field experiments were performed for apples and sweet peppers. The affordance manifolds were tested in the laboratory for two different grippers proving feasibility.
An adaptive sensor fusion algorithm and an algorithm for determining thresholds adaptively were implemented and analyzed for two databases. The databases include apple databases (acquired in Chile 2012) and a pepper database acquired in the Netherlands in 2012. The results were analyzed with respect to several parameters.
A database with forest data was acquired in Swedish forests. Data comprises RGB images and laser data for scenes with trees, humans, rocks, and bushes. A classifier system was constructed and algorithms for image segmentation and semi-automatic labeling were developed.
Theoretically grounded grasp quality measures that can be computed directly from a point cloud were developed along with efficient graspability map computation methods that facilitate map computation during run-time. The computed maps were tested in a physical environment. Dynamic motion primitives (DMP) were applied to apple harvesting in a laboratory setup. The parameters of each DMP were learned based on demonstration.
A new methodology for evaluating the connection between gripper design and sensing was developed in addition to formulating measures for sensor fusion evaluation.
The adaptive sensor fusion algorithm with an algorithm for determining thresholds adaptively was adjusted to the new large database of peppers (acquired in the Netherlands, November 2013). New sensor fusion algorithms were developed for ripeness detection in grapes and apples. A new sensor fusion algorithm was developed for disease detection in grapes. ROS modules were developed and implemented for final prototypes for apples and peppers.
The database with forest data previously acquired has been extended with more data from Swedish forests. Data comprises RGB images, thermal images, and laser data for scenes with trees, humans, rocks, and bushes. Algorithms for detecting trees and humans in forest environments were developed.
A gripper collision detection with the environment and its integration with graspability map-based grasp pose selection was developed, tested in simulation and implemented in ROS as part of the grasp node. The grasp node was enhanced to include pepper, apple, and grape grasp pose selection. The grasp node was successfully integrated with the complete CROPS ROS implementation. An innovative algorithm for reach-to-grasp motion planning which integrates graspability maps with the rapid random trees (RRT) algorithm was developed and successfully tested in simulation.
5. Sweep pepper – protected cultivation (work package 5)
Work package 5 is mainly dealing with the development and demonstration of a robotic system, focussed on harvesting, for sweet pepper production in protected cultivation systems.
The design objectives and requirements for a sweet pepper harvesting robot were determined. The current situation in greenhouse sweet pepper production was illustrated. The terminology and the crop system were explained as well as detailed information about fruit geometry and plant dimensions was provided. Also data about the cropping cycle and the climate conditions in a greenhouse were given. The information which was assembled was based on discussions and interviews held with a number of Dutch sweet pepper growers and the knowledge base of Wageningen University & Research Centre, complemented with available literature and reports.
A number of design sessions with experts (growers, trained workers and project partners) were organized to find and to define functions to fulfill the earlier defined requirements. These sessions yielded insight on how humans harvest sweet peppers. From this insight came the important notion that solving occlusion and finding the abscission zone, where the sweet pepper should be cut, are two important aspects which are difficult to solve but need to be solved in order to successfully harvest a fruit. The functions were translated to so called morphological charts. A morphological chart consists of a list of functions, in the left column and in the rows behind each function a working principle is shown. Also a description of requirements, functions and working principles for a number of cultivation tasks other than harvesting was produced. Crop monitoring, planting, pruning and tying up the plant were worked out in more detail.
The first prototype of the manipulator from WP3 and a carrier platform to transport the manipulator through the greenhouse and a gripper were integrated for the very first test in the greenhouse environment.
The conceptual design including a design evaluation of prototype of harvester for sweet pepper was finished. An analysis of the different sub-systems for cropping systems and crop manipulation; concepts for fruit and obstacle detection, ripeness determination and localization and concepts to reach, grasp and detach fruit was given. The chosen design is based on the current good practice cropping system of sweet-peppers in the Netherlands. The concepts for a number of cultivation related tasks for sweet-pepper production other than harvesting have been described. Four tasks were identified to be promising to automate which are: monitoring, planting, pruning and tying up the plant. A mobile platform that can move between the plant rows, as under development in this project offers opportunities also for plant monitoring. The other tasks are most likely not economically feasible to automate. Different cameras were tested for their suitability to detect and localize sweet-pepper fruits and obstacles and to determine fruit ripeness. After evaluation a sensor rig with a Time of Flight 3D camera mounted close to a high-resolution colour camera was assembled. Two different end-effectors to detach fruits from the plant are developed and analysed: a gripper with an integrated cutting tool with fingers that are based on the Finray principle (built by partner Festo) and a guidance and cutting tool approaching the fruit from below: the Lip-type end-effector, designed and built by WUR. For this last end-effector a patent has been filed. On both types of end-effectors two mini cameras are mounted: one Time of Flight camera and one colour camera. The first prototype of the manipulator was tested. It was concluded that the considered prototype is suited to harvest pepper fruits. A carrier platform for the robotic arm and the sensor systems was built. The sensor system on the platform is mounted on a linear motorized slide and can be horizontally moved out of the workspace of the robot while the robot is moving. The software of the robot was implemented for the Linux operating system and the middleware ROS. The main software is based on a finite state machine and includes diagnostic tools and performance measures of the harvesting operation. The demonstrator was tested under laboratory conditions. During these experiments data on the performance and accuracy of the different subsystems was gained. During these early tests 189 out of 194 fruit could be detected (97%), 167 fruits could be reached (86% of all fruits) and 154 picked (79% of all fruits). The integrated system was successfully demonstrated to the Dutch pepper growers which are part of the established growers advisory board.
In a number of greenhouse experiments the performance of the two sweet-pepper end-effectors (Fin Ray and lip-type) was determined. In these single module experiments the Fin ray type end-effector harvested a maximum of 80% of the fruits on the plant, the lip-type end-effector a maximum of 76% of the fruits. After final integration (using e.g. the final manipulator prototype) the robot was tested in the laboratory with an artificial plant canopy first. After that, the robot was transferred to a greenhouse. Between April and July 2014 experiments with the integrated pepper harvester were carried out in a red colored sweet-pepper crop grown in a Dutch commercial greenhouse. During these experiments the feasibility of harvesting sweet peppers autonomously has been proven. However, there were several difficulties noted which caused limited performance. Harvest success in an unmodified crop was only between 2% and 6%. After simplifying the crop by removing fruit clusters and occluding leaves harvest success improved up to 33%. The average cycle time to pick a fruit was 94 seconds. Additionally, a novel stem-dependent determination of the grasp pose method was evaluated. This indicated that by using this feature the plant stem damage decreased from 7% to 4% in an unmodified crop and from 19% to 13% in the simplified crop. For the Lip-type end-effector the grasp success increased using that feature. Several video clips of the sweet-pepper robot operating in the greenhouse were produced and diss¬eminated and are available for download in the video section of the Crops public website: http://www.crops-robots.eu/. It is expected that using the new insights gained during the experiments and a successful integration of the latest modules for sensing, motion planning and artificial intelligence developed in the project could significantly increase the performance of the system, which will be tested in the follow up project SWEEPER (www.sweeper-robot.eu).
6. Harvesting systems in orchards: grapes and apples (work package 6)
Work package 6 is dealing with testing and demonstrating of a robotic system, focused on harvesting of apples and grapes in an orchard and vineyard respectively.
The design objectives and system requirements for the apple and grape harvesting robots have been defined based on measurements in the orchards and vineyards in Chile and discussions with the stakeholders (end-users, system integrators, researchers, growers and crop specialists). As many of the CROPS partners were not familiar with commercial apple and grapes growth, an overview of the current situation in apple and grapes growth and harvesting, both manual and mechanized, has been communicated in the project. A design objectives tree was presented for the orchard harvesting robot which was complemented with separate lists of requirements for the apple and grape harvesting robots. In the brief of requirements, the specifications and design objectives were stipulated in quantitative terms, such as the required harvesting speed (in seconds per apple) and the percentage of fruit to be successfully harvested. From these requirements the required functions have been derived and possible working principles have been listed and critically evaluated. The most common crop growing practices and the most economically relevant varieties in Belgium and Chile have been critically reviewed with respect to their potential for automated cultivation and harvesting. Inputs from experts in pomology and viticulture and experienced growers have been exploited for this. To maximize the visibility and reachability of the apples and fruit bunches, high density planting of flat planar canopies supported by wires have been selected both for apple and grape growth. Although these planar growing systems are quite labour intensive to establish and train, these ‘walls of fruit trees’ have high potential for mechanized pruning and harvesting thanks to the limited depth. Besides improving the visibility and reachability, the pruning strategies in the apple orchard are also targeted at keeping the fruits close to the main branches, as this not only maximizes fruit quality, but also increases the detachment success. A next step will be to investigate thinning strategies reducing the number of apples in each bunch to one while maintaining the yield, as this would have a positive impact on the robotic harvesting success.
The conceptual designs and design evaluation of the demonstrator of a harvester for apples and grapes have been obtained. Also the description and design evaluation of generic concepts other than harvesting which could be performed by the robot have been summarized.
The manipulator, grippers, sensors and software architecture for the apples and grapes harvesting robotic system elaborated in work packages 1, 2 and 3 have been tested under laboratory as well as practical conditions in the apple orchard and grapes vineyard.
The different modules have been integrated into a working prototype. The functionality of this full prototype with all modules integrated has been demonstrated in the laboratory by fully automated picking of some plastic apple dummies attached to a metal frame by magnets.
The manipulator, grippers, software architecture and part of the sensors for the apples and grapes harvesting robotic system elaborated in work packages 1, 2 and 3 have been tested further under laboratory as well as practical conditions in the apple orchard and grape vineyard. The sensing in this prototype was performed with an alternative system elaborated by KU Leuven. This system uses an RGB-D camera (ASUS Xtion) in combination with an algorithm for apple detection and size estimation written using the Point Cloud Library in openCV.
The functional prototype of the apple harvesting robot has been tested in a ‘mur fruitier’ apple orchard of PCFruit, a Belgian fruit research station, in October-November 2013. The canopies had been pruned specifically to promote robotic harvesting. During these tests this prototype detected all apples inside the picking area, grasped 90% of these and successfully detached 80% of the successfully grasped. So, in these tests 72% of the fruits were successfully harvested in outdoor conditions. The average time needed to pick one apple was 15s when the apple was dropped in a tube after detachment and 30s when the apple was transported to a box. The causes for failure were carefully studied and suggestions for improvement were made.
During tests in August 2014 in a Belgian vineyard detection of the stalk of the grape bunches was achieved by manually attaching a label to this stalk and detecting this with the sensor platform.
The field tests in August in the vineyard with the grapes harvesting robot were performed by attaching fake grapes to the vines and picking these with the robot. In the first half of September, picking of real grapes bunches was then attempted by moving the grapes gripper to the labelled stalk. However, as this method only provides information on the position (x, y, z) of the stalk and not of the orientation, grasping was most of the time not successful. During these tests it was also observed that the grapes gripper prototype even at maximum pressure did not have sufficient force to cut the thick stalks of these grape bunches. As a result, no successful detachment could be achieved in this orchard.
Around half of September 2014 the harvesting robot was moved to the apple orchard to test the effect of the improvements made compared to the tests performed during the 2013 harvesting season. The fruit localization success rate was found to be as high as 98 %. The reason for this high success rate is that no occlusions were present because leaves were pruned before measuring. The only occlusion that could happen was due to apples hanging in front of each other. This is the reason for not reaching 100 % detection accuracy. False-positive detections were not observed (false-positive detection rate = 0 %). When apples were detected and localized in the picking window of the manipulator, they could be approached in all cases (approach success rate = 100 %).
The grasp success rate was 86 %. The main reasons for an unsuccessful grasp were a too small width between the two gripping fingers and a non-optimized positioning of the end-effector with respect to the detected fruit. The removal rate was only 60 %, with a non-optimized picking motion and a non-optimized positioning of the third finger on the stem as the main reasons for unsuccessful removal. Combination of the grasp success rate with the removal rate results in a detachment rate of 47%. When this detachment rate is combined with the fruit localization rate, a harvest rate of 46% is obtained. It is clear that within the reachable window (which was much smaller in 2014) the grasp success rate was comparable, while the detachment rate was drastically reduced. There is no clear explanation for this reduction in the detachment rate, but it might be related to the low third finger success rate (21%). The fruit characteristics could also be a possible explanation as in 2013 the fruits were more ripe and it is known that ripe fruit can be detached more easily. Finally, the lower number of tested apple fruits might also have resulted in a less accurate quantification, and therefore a bad comparison, between the two test years.
7. Precision spraying (work package 7)
Work package 7 work is has two applications, namely canopy optimised spraying and close range precision spraying and the aim is developing automated systems in order to reduce considerably the amount of pesticides
Both canopy optimised spraying and close range precision spraying requirements and specifications were selected with discussion with spraying specialists and growers.
A database of sensed variables (laser scanner, industrial RGB camera, photographic camera, thermal camera, ultrasonic sensor) was recorded and was made available for use in other work packages, mainly in WP2 and WP4.
The canopy optimised sprayer was designed and modified from existing design as a trailed sprayer with centrifugal blower. It was equipped with sensing equipment: laser scanner, ultrasonic sensors and RGB camera. Processing algorithm was derived such that multidimensional surface plane of selected degree is fitted to the canopy contour. An eight DOF hydraulic driven manipulator with three arms was used and equipped with position sensors. An inverse kinematics algorithm was derived, such that spraying arms are perpendicular to spraying surface and located at a selected distance from the centre of sprayed surface. Actuation of spraying arms is performed hydraulically and hydraulic cylinders are driven using fast relays from digital I/O module. Cycle time of spraying manipulator control is less than 10 ms, while response times of sensing elements are longer. Processing of all sensed values are delayed based on recorded sprayer velocity and stored in a buffer.
Work has been done on building of a new manipulator section, CAD design, CFD modelling and testing of subcomponents.
For the close range precision spraying initially the focus was on testing disease detection with various sensing principles. Several configurations of close range precision sprayer were tested, open, with background shielding which avoids multiple rows viewing and diffusing roof that avoids specular spots reflections. In all configurations, sensors were used in different positions, selected was a combination of sensors mounted on prime mover and on an end effector. Close range precision spraying end effector was constructed in accordance with requirements and specifications for efficient close range precision spraying. Post processing algorithms included normalisation and segmentation principles, among them noise reduction, illumination balancing among channels correlations, background and external material removing, foreground segmentation (infection symptoms identification by spectral ratios + texture indexes) and experiments on data fusion with hyperspectral images. ROI centroids distribution in spectral dimensions were also evaluated.
For canopy optimised spraying several modification and tasks related to the first prototype sprayer were performed. These included reduction of number of sensors, implementation of new single electric box for sensing, processing and arm control, improvement of sensing, processing and control software, development of new algorithm for spraying nozzles positioning, performance optimisation by laboratory experiments for accurate and smooth operation of spraying arms and implementation of orchard experiment and substantial reduction of pesticide use. The canopy optimised sprayer performed during 2013 orchard experiments reasonably well and good spraying quality with significant reduction of pesticide use was achieved. The further modified canopy optimised sprayer performed well during orchard experiments in 2014 and a good spraying quality with significant reduction of pesticide use was achieved. Fan inducer was built for regulation of the airflow rate and mounted on the sprayer. Aerodynamic measurements of fan inducer were performed. Various small modifications of the software were performed, resulting in better performance of the canopy optimised sprayer. Among them are most important arms positioning algorithm modifications, power management, installation of fast valves for continuous flow control, and selection of algorithms for continuous flow control. Also, several dissemination and demonstration activities were performed related to canopy optimised sprayer.
The close range precision spraying achieved a significant milestone under the CROPS project: for the first time small foci of diseases were robotically detected and sprayed without human intervention. System integration of close range precision sprayer included integration of prime mover with encoder, R-G-NIR multispectral camera, RGB high resolution camera, diffuse illumination panels, personal computer for analysis of sensing data for real time disease detection, manipulator control PC and power case, CROPS manipulator with waterproof protecting case, pesticide control circuit with a pump, and precision spraying end effector.
In the greenhouse experiment, a test plot was selected with 10 locations that required spraying. By a fully autonomous operation, all were actually sprayed with 25 spot sprayings. The actual reduction in pesticide consumption was 84%, although there is possibility for further 10% reduction. The data of the already performed experiments have been analysed and refined algorithms for disease detection were developed. Further activity on disease detection included sensor fusion for different disease diagnostic and advanced classification algorithms or boosting techniques for decreasing the false positive incidence. Results and analysis performed have shown that for field use shield and light-diffusing covering layer for sensing equipment is required.
8. Forestry (work package 8)
Work package 8 is devoted to the tasks object detection and classification (bushes, rocks, and trees) and estimation of ground bearing capacity (for propulsion of both manual and autonomous forest machines). Technical requirements for the two tasks were derived. For object detection and classification this gave the following result: localize, detect and classify trees, bushes, rocks, and humans with the following constraints: 95 % detection rate; 10 cm localization accuracy; less than 10 meters in front of the forestry machine; varying light conditions (sun, darkness); varying weather conditions (snowing, raining) and varying ground conditions (gravel, snow, forest vegetation). For ground bearing capacity this gave the following requirement. Estimate bearing capacity with the following constraints: 5-10 meters in front of the forest machine; only in forest terrain; only in summer.
A sensory system for detection and classification of trees has been evaluated in a forest environment. Two sensory systems able to detect and classify trees and humans respectively in a forest environment have been evaluated. The human detection works very well and can detect humans that are not visible for the naked eye (or a normal RGB camera). The tree detection works well in most situations, but as with all image analysis techniques it is sensitive to lighting conditions. A system able to estimate ground bearing capacity has been developed and evaluated. Preliminary results show that the system has potential for a non-destructive, real-time estimation of ground bearing capacity.
9. Training (work package 9)
Work package 9 includes several activities to provide multidisciplinary training in a joint industrial and academic environment, aiming to create a new generation of agro-forestry robotic researchers and engineers with a broad understanding of the needs and opportunities for intelligent robots that function in the agro-forestry environment. To succeed in the complex agro-forestry environment, all partners must be exposed to in-depth knowledge in the different domain areas.
A total of 44 graduate students were recruited to work on various aspects of the project. 33 graduate level courses were proposed at the participating universities for the students. The students took 17 advanced courses, participated in conferences and visited various universities and industries. CROPS personnel were trained in nine different workshops that took place during the half year periodical meetings.
10. Dissemination (work package 10)
The objectives of work package 10 are:
1. To inform the scientific community, especially in the areas of robotics and agricultural engineering, about the relevant results arising from the project and to disseminate experimental data and source code.
2. To ensure that all research results are disseminated to European industry.
3. To inform the farmers, through their local and national associations, about those results of the project that are ready for use and about new products/techniques possibly arising in the near future from the application of the project outcomes.
Within the website of CROPS project, a webpage specifically designated to dissemination has been launched since March 2011, with two subpages aimed to contain focused information directed toward farmers and companies, respectively. The webpages were updated on a regular basis, with popularizing, technical and scientific publications produced within CROPS.
Close contacts with three other European projects were established through dedicated meetings with the coordinators of RHEA, SPICY and RoboEarth. These are all FP7-projects focused in area of field robotics, advanced crop production and fundamental robotics. Among other topics, possibilities of reinforcing dissemination activity through joint actions was discussed and considered for implementation.
A CROPS project session was organized within the 1st RHEA Conference held in September 2012, with four presentations from CROPS partners and six CROPS presentations were given on the 2nd RHEA Conference hold in May 2014 in Madrid.
Based on the CROPS concepts and initial research activities, several publications were produced within the consortium. Specifically: 25 articles on specialized webpages or in technical journals and newspapers; 10 popularizing articles in farmers magazines; 76 presentations at international scientific conferences, with 70 extended papers included in proceedings; 12 papers in main scientific journals in the area of field robotics and vision theory.
The CROPS project was one of the subjects of a special report “Le serre a Rotterdam” of the science popularizing TV programme “SuperQuark” broadcasted by the Italian network Rai Uno on July 2012 (http://www.rai.tv/dl/RaiTV/programmi/media/ContentItem-afa68735-1a96-4d31-a5a3-5435f5c9cea9.html). CROPS project activity was reported by the science popularizing TV programme “Dobra ura” broadcasted by the Slovenian network RTVSLO on 24 April 2014 (http://4d.rtvslo.si/arhiv/dobra-ura/174272608) and the CROPS project was the main subject of a report in “Regio Nieuws” broadcasted by the Netherland TV network WOS on 4 September 2014
(www.wos.nl/televisie/uitzendinggemist/player/item/20140904-regionieuws-van-donderdag/).
Popularizing description of selected experiments and explicative videos relevant for demonstrating the main capabilities of developed subsystems and their possible applications have been published on the homepage of CROPS website.
The CROPS Final workshop was hold in Zurich on July 9th, 2014 as a special session within the AgEng2014 conference, with a world-wide academy attendance and also R&D representatives from main machinery manufacturers. The workshop lasted a whole day, with 16 extended presentations (including one overall presentation from RHEA project) and 4 summary posters discussions. Two of the extended presentations were given by industry partners, namely by company Festo (on new passive grippers for agricultural applications) and by company FORCE-A (on Fluorescence sensing for ripeness and quality evaluation of specialty crops products).
A special issue of the journal "Biosystems Engineering" is prepared on advances in agricultural robotics and which will have a strong nucleus coming from “CROPS Final workshop” papers, and for which partners of the CROPS consortium have been charged as Guest Editors.
11. Final demonstration (work package 11)
Work package 11 has the task to demonstrate the final CROPS platforms under a variety of operating conditions and for the different applications.
The demonstration activities were organised and conducted mainly in the last half year of the CROPS project. To this aim, for the different application cases the scenario conditions of demonstrations were defined by the partners responsible for them, namely: sweet-pepper harvesting (WUR), apple harvesting (KULeuven), grapes harvesting (KULeuven), canopy optimised sprayer (UL), close precision selective spraying of diseases (UNIMI) and for obstacle detection in forestry operation (UMU). For all the demonstrations, specific indicators of system’s performance were also stated in order to objectively quantifying the operational success achieved in the demonstration.
The possibility of organizing in a single site all the demonstrations of the systems applied to grapes and apples (i.e. apple harvesting, grape harvesting, canopy optimised spraying and close range precision spraying) was preliminary explored as a priority option. Due to high requirements in terms of logistic efforts and resources costs, the involved partners independently organized local arrangements for applications demonstration.
The CROPS robot system was then configured according to the specific needs of each application case, by integrating manipulator, end-effectors, sensing hardware, algorithm and control software.
The demonstration schedule had to accommodate the possibility for all the application cases to carry out of the different demonstrations with the shared hardware resources (i.e. the manipulator and the control system). To this aim, the different demonstrations were time-spread from spring 2013 to autumn 2014.
The expected capabilities of apple harvesting, canopy optimised spraying, close precision selective spraying of diseases, obstacle detection in forestry operation were successfully demonstrated. For sweet-pepper harvesting the expected capabilities were demonstrated in a simplified scenario, while only partially achieved in real operative scenario. For grape harvesting the expected capabilities have not been reached in field demonstration.
12. Economics, Social Aspects, Sustainability and Exploitation (work package 12)
Work package 12 is mainly focussed on economic viability , social aspects and sustainability with the aim to enable correct design requirements that will ensure successful market penetration
market penetration and adoption of the technologies by farmers.
A simulation of economic viability was made for each application, consisting of three simulation tools to determine the economic viability of: sweet pepper harvesting (WP 5), harvesting systems in orchards (grapes and apples WP 6) and precision spraying (WP 7).
A report on social aspects, standard requirements and other requirements for sustainability, describes the social aspects, economics and sustainable requirements for: sweet pepper harvesting (WP 5), harvesting systems in orchards (grapes and apples WP 6) and precision spraying (WP 7) has been finished. To evaluate social aspects, economics and sustainability, a short version of the method “Sustainable scan for the agro food chain” is used. The method is influenced by thinking about CSR (Corporate Social Responsibility) and the ambition is to provide an understanding of the dynamic relationships between the triple bottom line, people, planet, and profit, and to measure and weight the effects of efforts to raise performance levels in all three of these domains.
For the sweet pepper harvester the evaluation is positive in general. Specific advantages for robotising the sweet pepper harvesting task are labour development: employment, education, working conditions, absenteeism. When robots replace human labour there are no limitations to climate conditions. Plants grow very well under high CO2 levels, high humidity and high temperatures, under which people cannot work. A negative condition is that robots are expensive. In the beginning the pay-back time might be up to five years and more. For the harvesting system in orchards (grapes and apples) the opportunities are almost the same as for sweet pepper harvesting and are looking positive. But there are no advantages expected in energy saving due to closing windows in greenhouses. The energy consumption will increase a bit. There will be also initial capital needed which might be difficult to find. For robotized precision spraying, opportunities are looking positive. It will give social support by using less pesticide appointed more accurate. Labourers don’t have to work under bad/protected conditions. There will be also more initial capital needed which might be difficult to find. But in the end, using less spraying fluids, robots will save money and become profitable.
The evaluation on all the aspects within People Planer Profit leads to new system requirements which were discussed with the involved work package leaders.
At May 13, 2013 a one day Exploitation Strategy Seminar was organized in cooperation with ESIC2, a EU-project, fully dedicated to exploitation of FP7 NMP research results. All members of the General Assembly,the Steering Committee (GA/SC) and the exploitation committee of CROPS were invited and participated. The seminar was led by Thomasz Cichocki (PL). The seminar was evaluated by the GA/SC as valuable to the project and can be seen as the start of exploitation activities.
The results on sensing and canopy spraying with a robot are promising and possibly exploitable. The precision spraying application is identified to be validated first in relation to an application that could be commercialized after the project.
The project partner WUR has filed a patent for the end effector of sweet pepper harvesting (Lip-type end-effector) and the partner KULeuven has filed a patent for the end-effector for apple harvesting.
The work package analysed and validated the performance of the three applications that were built and tested: sweet pepper harvester; apple harvester; canopy optimised sprayer. For these applications exploitation plans were made. Furthermore a list of open source software and algorithms was made.
13. Coordination (work package 13)
The task of work package 13 is to manage the project and to take care that everything is running smoothly and that all goals are fulfilled.
The consortium had a project meeting every 6 months, in total eight, organized by one of the partners. During these meeting also the meeting of the Steering Committee and the Grand Assembly took place. On request more meetings could be held. As an example special meetings have been held on the decision of the robotic middleware, on the gripper design for sweet pepper, on CROPS software integration and on motion planning. In total three official review meeting with the EC were organized.
The CROPS website, www.crops-robots.eu , has been available from October 28th, 2010. The website has a public part, describing the project, the partners, the work packages, a dissemination section and a contact site. Besides the public part there is a so-called member area for the partners, including a file share facility. On every project meeting an analysis on the number of visitors of the web site were given. On average there were 200 visits per day. Most popular are the videos on the website and also the abstract booklets and presentations from the Crops workshops are downloaded frequently, in average 20 to 100 times per month per booklet.
The project was advised by a Grower’s Advisory Board, which was organized locally.
Potential Impact:
Potential impact.
The CROPS project has developed a modular and generic robot platform for harvesting sweet pepper, apples and grapes. The robot platform is also been used for canopy spraying in orchards and targeted spraying in vineyards. For forestry application obstacle detection has been developed.
The CROPS robots will reduce harvest costs. It is estimated that a cost reduction of 40% can be achieved. The critical factor for economic feasibility is the yield rate which must be at least 95% to ensure successful market entrance. However, this threshold might drop due to increased labour costs and increased difficulties in finding workers which is already a bottleneck to agriculture production expansion in some countries.
The technology evaluation that will be conducted in parallel to the scientific activities will provide more detailed evaluations of harvest cost reductions for the variety of farms and markets and will include specific calculations for each of the demonstrators.
CROPS will contribute to the reduction in the use of crop protection chemicals. A CROPS robot will enable to spray only the canopy in vineyards and orchards instead of spraying the whole area, leading to a reduction of pesticide application of 30%.
For specific treatments, selective application of pesticides on susceptible targets will allow to achieve even higher reduction levels. Selective spraying of insecticides on only the grape bunches, will result in 90% reduction of chemicals.
Due to the reduction of chemical inputs there will be less residues of chemical on the products leading to improved product quality. Using a robotic system will enable to harvest the product at optimal time thereby increasing the quality of the product. A system which handles each product autonomously also increases food safety due to elimination of the need for human operators. The ability to look within the plant level and measure locally the quality of each individual fruit can open new possibilities and markets. For example, this project makes it possible to determine the chemical composition of harvested grapes at the bunch level and thus to improve the quality and consistency of the wines considerably.
Several demonstrator systems has been developed as part of the project. Successful development of the new CROPS design for agricultural and forestry robots will lead to a new product line for several European machine industries, maintaining the world-leading position of the European forestry and agricultural machine industry.
Low valued labour on the farms and in forestry will be replaced by high skilled labour. Furthermore, the development of agriculture and forestry robots will create high technology jobs in the machine and sensing industries. In forestry, recruiting of machine operators is often a limiting factor for business expansion. Introducing semi-autonomy and autonomy into the forestry machines creates additional high skilled jobs.
CROPS will contribute to sustaining available income in rural areas. By enabling introduction of robotic systems which are efficient, cost effective and can replace manual labour, the agricultural sector, and also the supplying companies, will be kept in. The developed robots can release the farmer from early rising, long work days, back breaking demands of farm work, and around-the-year engagement to the farm. Together with the previously mentioned creation of high technology jobs, the rural economy will be strengthened and there will be a reduction of migration to urban areas.
The reduction of chemical pesticides enabled by site-specific and targeting spraying will reduce chemical pollution by reducing chemically-polluted runoff from fields that cause contaminated surface and ground waters and violate ecological balance. Additionally, the reduction of chemicals disposed will reduce the medical hazards caused by exposure of workers and increase health of all via the improved environment and improved agriculture product quality.
Dissemination
The dissemination activity has been performed on a regular basis, mainly through the dedicated webpage of the CROPS project website (www.crops-robots.eu) the publication of scientific papers on journals and to international conferences in the field of agricultural and biosystems engineering, robotics, industrial automation, artificial intelligence, as well as through specialized webpages or technical journals and newspapers or on farmers magazines. Based on the CROPS concepts and initial research activities, several publications were produced within the consortium. Specifically: 25 articles on specialized webpages or in technical journals and newspapers; 10 popularizing articles in farmers magazines and 76 presentations at international scientific conferences. The CROPS project was one of the subjects of a special report “Le serre a Rotterdam” of the science popularizing TV programme “SuperQuark” broadcasted by the Italian network Rai Uno on July 2012 (http://www.rai.tv/dl/RaiTV/programmi/media/ContentItem-afa68735-1a96-4d31-a5a3-5435f5c9cea9.html). CROPS project activity was reported by the science popularizing TV programme “Dobra ura” broadcasted by the Slovenian network RTVSLO on 24 April 2014 (http://4d.rtvslo.si/arhiv/dobra-ura/174272608) and the CROPS project was the main subject of a report in “Regio Nieuws” broadcasted by the Netherland TV network WOS on 4 September 2014 (www.wos.nl/televisie/uitzendinggemist/player/item/20140904-regionieuws-van-donderdag/).
Popularizing description of selected experiments and illustrative videos relevant for demonstrating the main capabilities of developed subsystems and their possible applications have been published on the homepage of CROPS website.
Contacts with the RHEA project, another FP7-project focused on field robotics, have been maintained and strengthend. Six research contributions from CROPS partners were presented at the 2nd RHEA Conference hold in May 2014 in Madrid, and an overall presentation of RHEA project was given at the CROPS Final workshop hold in July 2014 in Zurich.
The CROPS Final workshop was hold in Zurich on July 9th, 2014 as a special session within the AgEng2014 conference, with a world-wide academy attendance and also R&D representatives from main machinery manufacturers. The workshop lasted a whole day, with 16 extended presentations (including one overall presentation from RHEA project) and 4 summary posters discussions. Two of the extended presentations were given by industry partners, namely one by the company Festo (on new passive grippers for agricultural applications) and one by the company FORCE-A (on Fluorescence sensing for ripeness and quality evaluation of specialty crops products).
70 conference papers had been published in refereed proceedings and 12 papers were published in major scientific journals and two additional submissions are under review.
In next future, this number will significantly increase since more CROPS publications are further expected. This is especially in relation to a Special Issue of the journal "Biosystems Engineering" on Advances in agricultural robotics which will have a strong nucleus coming from “CROPS Final workshop” papers, and for which partners of CROPS consortium have been charged as Guest Editors. Nevertheless, the publication of this special issue is scheduled for late spring-summer 2015.
Papers published (or submitted) in scientific journals:
1. Bac, C.W. Henten, E.J. van, Hemming J., Edan, Y., 2014. Harvesting Robots for High-value Crops: State-of-the-art Review and Challenges Ahead. Journal of Field Robotics. DOI: 10.1002/rob.21525
2. Eizicovits D., Berman S. 2014. Efficient sensory-grounded grasp pose quality mapping for gripper design and online grasp planning Robotics and Autonomous Systems. doi.org/ 10.1016/ j.robot.2014.03.011
3. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Borghese A. N. 2014. Automatic detection of powdery mildew on grapevine leaves by image analysis: Optimal view-angle range to increase the sensitivity. Computers and Electronics in Agriculture, 104: 1- 8
4. Hemming, J.; Ruizendaal, J.; Hofstee, J.W.; Henten, E.J. van Henten, 2014. Fruit detectability analysis for different camera positions in sweet-pepper Sensors 14: 6032-6044
5. Bac, C.W. Hemming J., Henten, E.J. van 2014, Stem localization of sweet-pepper plants using the support wire as a visual cue. Computers and Electronics in Agriculture 105: 111–120
6. Hellström T., Ringdahl O. 2013. A software framework for agricultural and forestry robots. Industrial Robot: An International Journal. 40(1): 20-26, 2013
7. Fernandez R., Montes H.,Salinas C., Sarria J., Armada M. 2013. Combination of RGB and multispectral imagery for discrimination of Cabernet Sauvignon grapevine elements. Sensors. 13(6), 7838-7859; doi:10.3390/s130607838
8. Ola Ringdahl O., Hohnloser P., Hellström T., Holmgren J., Lindroos O. 2013. Enhanced algorithms for estimating tree trunk diameter using 2D laser scanner. Remote Sensing. 5: 4839-4856
9. Osterman A., Hočevar M., Godeša, T., Širok B., Stopar M. 2013. Real-time positioning algorithm for variable-geometry air-assisted orchard sprayer. Computers and Electronics in Agriculture. Vol. 98, p. 175-182
10. Bac C.W. Hemming, J., Henten, E. van 2013. Robust pixel-based classification of obstacles for robotic harvesting of sweet-pepper. Computers and Electronics in Agriculture. Vol. 96, p. 148 - 162
11. Ben-Yosef G., Ben-Shahar O. 2012. A tangent bundle theory for visual curve completion. IEEE Transactions on Pattern Analysis and Machine Intelligence. 34, 1263-1280
12. Kapach K. , Barnea E., Mairon R., Edan Y., Ben-Shahar O. 2012. Computer Vision for Fruit Harvesting Robots - State of the Art and Challenges Ahead. International journal of Computational Vision and Robotics. 3, 4-34
13. Bac, C.W. Roorda T., Reshef R., Berman S., Hemming J., Henten E.J. van. 2014. Analysis of a motion planning problem for fruit harvesting in a dense obstacle environment. Submitted to Biosystems Engineering.
14. Bac C.W. Hemming J., Tuijl B. van, Barth R., Wais E., Henten E. van. 2014. Performance evaluation of a harvesting robot for sweet-pepper. Submitted to Journal of Autonomous Robots.
Presentations and published conference proceedings:
1. Barth R., Baur J., Buschmann T., Edan Y., Hellström T., Ringdahl O., Salinas C., Vitzrabin E. 2014. Using ROS for agricultural robotics - design considerations and experiences. Proc. of 2nd International RHEA Conference
2. Cohen Y., Berman S. 2014. Integrating simulation with robotic learning from demonstration. Proceedings of the 28th EUROPEAN CONFERENCE ON MODELLING AND SIMULATION (ECMS)
3. Hellström T., Ostovar A. 2014. Detection of trees based on quality guided image segmentation. Proc. of 2nd International RHEA Conference
4. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Hočevar M., Baur J., Pfaff J., Schütz C, Ulbrich, H. 2014. Crops agricultural robot: application to selective spraying of grapevine’s diseases. Proc. of 2nd International RHEA Conference
5. Osterman A., Godeša, Hočevar, M., T., Širok, B. 2014. Monitoring of growth of fruit trees during growing season with lidar. Proc. of 2nd International RHEA Conference
6. Osterman A., Godeša, Hočevar, M., T., Širok, B. 2014. Unilateral characterization of tree canopies in orchards with lidar. Proc. of 2nd International RHEA Conference
7. Reshef R., Eizicovits D., Berman S. 2014. Path planning of grasp-aimed robotic tasks using rapid-exploring random trees. Proc. of 2nd International RHEA Conference
8. Ben-Shahar O., Barnea E. 2014. Depth Based Object Detection from Partial Pose Estimation of Symmetric Objects. Proc. of European Conference on Computer Vision (ECCV)
9. Hemming J., Tuji van B., Gauchel W. 2014. Field Test of Different End-effectors for Robotic Harvesting of Sweet-pepper. Proc. of International Horticultural congress
10. Cohen A.H. Berman S. 2014. Motor control variables and dynamic motion primitives. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop on Compliant manipulation
11. Elkoby Z., Ooster B. van, Edan Y. 2014. Simulation Analysis of Sweet Pepper Harvesting Operations. Proc. of the Advances in Production Management Systems Conference (APMS)
12. Ben-Shahar O., Mairon R. 2014. A Closer Look at Context: From Coxels to the Contextual Emergence of Object Saliency. European Conference on Computer Vision (ECCV)
13. Montes H., Fernandez R., Salinas C., Sarria J., Armada M. 2014. Sistema Multisensorial para Aplicaciones en Agricultura de Precisión . Proc. of the II International Conference of Science and Technology for Sustainable Development (CICyTDS)
14. Bontsema J., Hemming J., Pekkeriet E., Saeys W., Edan Y., Shapiro A., Hočevar M., Hellström T., Oberti R., Armada M., Ulbrich H., Baur J., Debilde B., Best S., Evain S., Münzenmaier A.,Ringdahl O., 2014. CROPS: high tech agricultural robots. Proc. of the International Conference of Agricultural Engineering AgEng 2014
15. Pekkeriet E., Saeys W., Hemming J., Bontsema J., Hočevar M., Oberti R. 2014. CROPS Evaluation of economic viability, social aspects, sustainability and exploitation of robotic systems. Proc. of the International Conference of Agricultural Engineering AgEng 2014
16. Schuetz C., Pfaff J., Baur J., Buschmann T., Ulbrich H. 2014. A modular robot system for agricultural applications. Proc. of the International Conference of Agricultural Engineering AgEng 2014
17. Baur J., Schuetz C., Pfaff J., Buschmann T., Ulbrich H. 2014. Path Planning for a Fruit Picking Manipulator. Proc. of the International Conference of Agricultural Engineering AgEng 2014
18. Hemming J., Bac W., Tuijl B. van, Barth R., Bontsema J., Pekkeriet E., Henten E. van 2014. A robot for harvesting sweet-pepper in greenhouses. Proc. of the International Conference of Agricultural Engineering AgEng 2014
19. Eizicovits D., Berman S. 2014. Grasp quality measures based on point cloud input. Proc. of the International Conference of Agricultural Engineering AgEng 2014
20. Osterman A., Godeša T., Malnersic A., Hočevar M., 2014. Moving spraying arm with internal airflow for air-assisted orchard sprayer. Proc. of the International Conference of Agricultural Engineering AgEng 2014
21. Nguyen T.Th. Vandevoorde K., Kereszte J., Kayacan E., De Baerdemaeker J, Saeys W. 2014. Apple detection algorithm for robotic harvesting using a RGB-D camera. Proc. of the International Conference of Agricultural Engineering AgEng 2014
22. Berenstein R., Edan Y., Godeša T., Hočevar M. 2014. Image registration for agriculture tasks . Proc. of the International Conference of Agricultural Engineering AgEng 2014
23. Beers van R., León L., Schenk A., Nicolaï B.,Kayacan E., Saeys W. 2014. Optical measurement techniques for the ripeness determination of Braeburn apples. Proc. of the International Conference of Agricultural Engineering AgEng 2014
24. Osterman A., Godeša T., Hočevar M., 2014. LIDAR-based control of automated orchard sprayer. Proc. of the International Conference of Agricultural Engineering AgEng 2014
25. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Hočevar M., Baur J., Pfaff J., Schütz C, Ulbrich, H. 2014. Selective precision spraying of grapevine’s diseases by CROPS robot platforms. Proc. of the International Conference of Agricultural Engineering AgEng 2014
26. Klosok M. 2014. Passive, adaptive grippers as end-effectors for agricultural applications . Final CROPS Workshop at the International Conference of Agricultural Engineering AgEng 2014
27. Laurent F., Evain L. 2014. Fluorescence sensors for CROPS. Final CROPS Workshop at the International Conference of Agricultural Engineering AgEng 2014
28. Nguyen T.Th. Kayacan E., De Baerdemaeker J, Saeys W. 2014. Motion planning algorithm and its real-time implementation in apples harvesting robot. Proc. of the International Conference of Agricultural Engineering AgEng 2014
29. Barnea E., Ben-Shahar O. 2014. Depth Based Fruit Detection from Partial Pose Estimation using Symmetry. Proc. of the International Conference of Agricultural Engineering AgEng 2014
30. Ohev-Zion A., Shapiro A. 2014. Stability of Compliant Planar Robotics Grasp with Application to Fruits Grasping.. Proc. of the International Conference of Agricultural Engineering AgEng 2014
31. Oberti R., Marchi M., Tirelli P., Vitzrabin E., Edan Y. 2014. Sensor fusion of multispectral and hyperspectral imaging: preliminary analysis of disease detection in grapevine. Proc. of the International Conference of Agricultural Engineering AgEng 2014
32. Pfaff J., Baur J., Schuetz C., Buschmann T., Ulbrich H. 2014. Design of drive units for agricultural robots. Proc. of the International Conference of Agricultural Engineering AgEng 2014
33. Baur J., Pfaff J., Schuetz C. & Ulbrich H. 2013. Dynamic modeling and realization of an agricultural manipulator. Proceedings of XV International Symposium on Dynamic Problems of Mechanics.
34. Bac, C.W. Hemming, J., Henten, E.J. van. 2013. Classification of sweet-pepper plant parts using multi-spectral imaging. Proceedings of the 2013 IFAC Biorobotics conference.
35. Cohen Y., Berman S. 2013. Tight Dynamic Movement Primitives for Complex Trajectory Generation. Proc. of 2013 IEEE International Conference on Systems, Man and Cybernetics.
36. Henten E.J. Van, Bac C.W. Hemming, J., Edan Y. 2013. Robotics in protected cultivation. Proc of IFAC Agricontrol 2013 Conference, p. 170-176
37. Nguyen T.Th. Kayacan E., De Baerdemaeker J, Saeys W. 2013. Task and Motion Planning for Apple Harvesting. Proc. of IFAC Agricontrol 2013 Conference. p. 247-252
38. Tuijl van B., Wais E., Edan Y. 2013. Methodological design of an end-effector for a horticultural robot. Proc. of 4th Israeli Conference on Robotics.
39. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Hočevar M., Baur J., Pfaff J., Schütz C. 2013. Selective spraying of grapevine’s diseases by a modular agricultural robot. Proc. of AIIA 2013 - Italian Conference of Agricultural Engineering.
40. Hershkovitz Cohen A., Berman S. 2013. Path Planning of Manipulator for Harvesting using DMP. Proc. of 4th Israeli Conference on Robotics.
41. Vitzrabin E., Edan Y. 2013. Apple detection using multi-dimensional adaptive thresholding with multi-resolution windows. Proc. of 4th Israeli Conference on Robotics.
42. Eizicovits D., Berman S. 201 . Constructing successful grasps based on Graspability maps . Proc. of 4th Israeli Conference on Robotics.
43. Rehsef R., Berman S. 2013. Path smoothing for the RRT motion planning algorithm. Proc. of 4th Israeli Conference on Robotics.
44. Fernandez R., Salinas C., Montes H., Sarria J., Armada M. 2013 . Validation of a multisensory system for fruit harvesting robots in lab conditions. Proc. of ROBOT 2013: First Iberian Robotics Conference . in press
45. Baur J., Pfaff J., Schuetz C., Ulbrich H. 2013. Dynamic modeling and realization of an agricultural manipulator. DINAME 2013 - Proceedings of the XV International Symposium on Dynamic Problems of Mechanics .
46. Gauchel, W. 2013. Flexible und adaptive Greifer zum Ernten von hochwertigen Feldfrüchten. Technikforum Roboterinnovationen 2020, 41541
47. Van Beers R., Aernouts B., De Baerdemaeker J., Saeys W. 2013. Apple ripeness detection using Hyperspectral Laser Scatter Imaging. Proc. of Sensing Technologies for Biomaterial, Food, and Agriculture (SeTBio '13). Proc. of SPIE Vol. 8881, 88810K
48. A.Ohev-Zion A.Miler A.Sintov A.Shapiro. 2012. Experimental Validation of Compliant Contact Model. 2012 Israeli Conference on Mechanical Engineering.
49. Hellström, T., Ringdahl O. 2012 . A software framework for agricultural and forestry robotics . Proceedings of the first International RHEA Conference , p. 171-176
50. Gauchel W. , Saller S. 2012. Adaptive gripper jaws for high-value crops harvesting. Proc. of 8th International Conference on Fluid Power.
51. Oberti R. , Tirelli P. , Marchi M. , Calcante A. , Iriti M., Borghese N.A. 2012. Automatic diseases detection in grapevine under field conditions. International Conference of Agricultural Engineering CIGR-AgEng.
52. Baur J., Pfaff J., Ulbrich H., Villgrattner T. 2012. Design and development of a redundant modular multipurpose agricultural manipulator. Proc. of 2012 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.
53. Pfaff J., Baur J., Ulbrich H., Villgrattner T. 2012. Development of a multipurpose agricultural manipulator. Proceedings of the first International RHEA Conference .
54. Eizicovits D., Yaacobovich M., Berman S. 2012. Discrete fuzzy grasp affordance for robotic manipulators. IFAC Symposium on Robot Control, SYROCO, Sept 5-7, 2012
55. Hemming, J.; Ruizendaal, J.L.; Hofstee, J.W.; Henten, E.J. van, 2012. Fruit detectability analysis for different camera positions in sweet-pepper. Proceedings of the International conference of agricultural engineering, CIGR-AgEng2012, Valencia, Spain, 8-12 July 2012.
56. Yaacobovich M., Eizicovits D., Berman S. 2012. Grasp affordance for robotic selective harvesting based on human demonstrations. International Conference of Agricultural Engineering CIGR-AgEng, July 8-12, 2012.
57. Mostafa Pordel, Thomas Hellström, and Ahmad Ostovar2012. Integrating Kinect Depth Data with a Stochastic Object Classification Framework for Forestry Robots.
58. Tirelli P. , Marchi M. , Calcante A. , Vitalini S., Iriti M., Borghese N.A. Oberti R. 2012. Multispectral image analysis for grapevine diseases automatic detection in field conditions. Proceedings of the first International RHEA Conference, p. 101-106.
59. Montes H., Fernandez R., Salinas C., Armada M. 2012. Robotic multisensory system for precision agriculture applications. Proc. of 15th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines - CLAWAR2012, p. 731-738
60. Hočevar, M., Godeša, T., Širok, B., Malneršič, A., Jejčič, V. 2012. Canopy optimised sprayer development within CROPS EU project. Proceedings of the first International RHEA Conference .
61. Malneršič, A, Hočevar, M., Širok, B., Marchi M., Tirelli, P., Oberti, R. 2012. Close range precision spraying airflow /plant interaction. Proceedings of the first International RHEA Conference .
62. Malneršič, A, Hočevar, M., Širok, B. 2012. Interakcija med zračnim tokom in rastlino pri ciljnem pršenju/Interaction between airflow and plant at close range precision spraying Proc. of the Kuhljevi dnevi conference, 2012.
63. Nguyen T.Th. Van Essen D., De Baedermaeker J., Saeys W. 2012. Optimum detaching movement for apples harvesting robot. International Conference of Agricultural Engineering CIGR-AgEng, July 8-12, 2012.
64. Berenstein, R., Edan, Y. 2012 . Human-robot cooperative precision spraying: collaboration levels and optimization fuction . Proc. 10th IFAC Symposium on Robot Control (SYROCO 2012) . p. 799- 803
65. Thanh Nguyen T., Van Essen, D., De Baerdemaeker J., Saeys W. 2012 . Optimum detaching movement for apples-harvesting robot . Proceedings of the International conference of agricultural engineering, CIGR-AgEng2012, Valencia, Spain, 8-12 July 2012 .
66. Oberti, R., Tirelli, P., Marchi, M., Iriti, M., Borghese, A. 2012 . Automatic diseases detection in grapevine under field conditions . Proceedings of the first International RHEA Conference .
67. Berenstein, R., Edan, Y. 2012 . Robotic precision spraying methods . Proc. ASABE Annual International Meeting - Dallas, Texas, July 29-August 1, 2012 .
68. Yaacobovich M., Bechar A., Berman S. 2011 . Environment effect on grasping: case study - peppers harvesting . 7th Computational Motor Control Workshop (CMCW) - Beer-Sheva (Israel)
69. Kravits M., Shapiro A., Berman S. 2011 . Parametric Compliant contact model for human finger . 7th Computational Motor Control Workshop (CMCW) - Beer-Sheva (Israel)
70. Pomeranz D., Shemesh M., Ben-Shahar O. 2011 . A fully automated greedy square jigsaw puzzle solver . Proceedings of the IEEE conference on Computer Vision and Pattern Recognition - CVPR 2011 . 9-16
71. Adato Y., Zickler T., Ben-Shahar O. 2011 . A Polar Representation of Motion and Implications for Optical Flow . Proceedings of the IEEE conference on Computer Vision and Pattern Recognition - CVPR 2011 . 1145-1152
72. Malneršič A., Hočevar M. 2011 . Design of aerodynamic actuator for precision target spraying in orchards and measuring of flow parameters . Kuhljevi Dnevi 2011 .
73. Salinas C., Fernandez R., Montes H., Armada M. 2011 . Sensory system evaluation methodology for automatic fruits harvesting . Proc. of 14th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines - CLAWAR2011 . 801-808
74. Shemesh M., Ben-Shahar O. 2011 . Free Boundary Conditions Active Contours with Applications for Vision . Proc. of International Symposium on Visual Computing .
75. Ohev-Zion A., Shapiro A. 2011 . Grasping of Deformable Objects Applied to Organic Produce . Proc. of TAROS 2011, LNAI 6856 . May 2011, 396–397
76. Bontsema, J. 2011. CROPS: Clever Robots for CROPS. 5th Workshop of the MANUFUTURE AET- community – Agricultural Engineering Strategies for EU FP8, Hanover (Germany)
Technical and popularizing CROPS publications:
1. I robot tra I filari e negli orti. ItaliaOggi. 2014, May, 28, p. 20
2. Darbouret, A., Evain, S. Les robots des champs. Le Journal du Dimanche . 3 mars 2013
3. Darbouret, A., Evain, S. Les robots des champs. http://www.lejdd.fr/Societe/Sciences/Actualite/Les-robots-des-champs-594543
4. Oberti R., Iriti M. Nel vigneto del futuro la difesa affidata ai robot. Corriere vinicolo . 2013, March 4
5. Hemming, J. Oogstrobot voor paprika’s in najaar de kas in. Nederlandse groententeelt. http://www.groentennieuws.nl/nieuwsbericht_detail.asp?id=94975
6. Hemming, J. Prototype of greenhouse harvesting robot almost able to operate independently. http://www.hortidaily.com/article/1455/Prototype-of-greenhouse-harvesting-robot-almost-able-to-operate-independently
7. Hemming, J. Der eiserne Gärtner, Bildanalyseaufgaben im EU-Forschungsprojekt CROPS. Inspect. May 2012
8. Tinker, D. EU funded research: CROPS and RHEA . EurAgEng Newsletter. Winter 2011/2012, p. 4-5
9. Hočevar, M., Godeša, T., Širok, B., Canopy optimised sprayer design. National Instruments case study
10. Hočevar, M., Godeša, T., Širok, B., Pršilnik za krošnjam dreves prilagojeno škropljenje/Canopy optimised sprayer design. NI Days conference, Ljubljana 2012
11. Hemming, J. Prototype van een oogstrobot getest in paprika. http://www.glastuinbouw.wur.nl/NL/nieuwsagenda/nieuws/prototypeoogstrobotpaprika_.htm .
12. Markus Keßler. Roboter-Schwärme bewirtschaften Bauernhöfe (interview with J. Bontsema) . http:/ /www.pressetext.com/news/20121102003 .
13. Agricultural robot will be able to harvest high-value crops. June 2011. www.theengineer.co.uk
14. Sikkema, A., Bontsema, J. Bell pepper picking robot in the making. www.resource.nl
15. Dodde, H. WUR coöordineert roboticaproject. November 2010, www.nieuweoogst.nu
16. Bontsema, J. Slimme robot herkent en oogst paprika's. June 2011. www.agentschapnl.nl
17. CROPS - An agricultural robot that can pick the ripest fruit. April 2011. www.smashingrobots.com
18. Bontsema, J., Jongebloed, P. Wageningen UR Glastuinbouw coordinates international robotics project. November 2010. www.hortist.com
19. Paprikabot. De Ingenieur.Vol. 123, nr. 16, Oct 14, 2011
20. The Marker. Crops. 2011, March 23
21. Bontsema J. Pekkeriiet E. Robothand gaat paprika oogsten. Kennis Online. Vol. 8, December 2011, p. 11.
22. Bontsema J. Tien miljoen euro voor onderzoek robots. De Gelderlander. November 9th, 2010
23. Two large-scale integrating FP7- EU projects: CROPS and RHEA. ICT Agri Newsletter. December 2011, p. 5
24. Tinker, D. EU funded research: CROPS and RHEA. EurAgEng Newsletter. Winter 2011/2012, p. 4-5
25. Thoenes, E., Bontsema, J. Hard en hygiëniisch werken: robots in de kas. J.Sync.nl April 18th, 2011, p. 1-4
Agricultural and popularizing publications follows:
1. I robot tra I filari e negli orti. ItaliaOggi. 2014, May, 28, p. 20
2. Staalduinen, J. van; Bontsema, J. ; Hemming, J.. Tussenbalans Europees CROPS project: Oogstrobot voor paprika krijgt stap voor stap vorm. Onder Glas 10 (2013)4. - p. 20 - 21. 10 (2013)4. - p. 20 - 21.
3. Oberti R., Iriti M.. Nel vigneto del futuro la difesa affidata ai robot. Corriere vinicolo. 2013, March 4.
4. Bontsema, J., Hemming, J.. Crops Internationale Robotica. Glastuinbouwtechniek Magazine. Vol. 6, Oct. 2011, 28-29.
5. Bontsema, J.. Nieuw internationaal roboticaproject CROPS. Onder Glas. Vol. 8, March 2011 ,p. 59.
6. Thoenes, E., Bontsema, J. Robots in de kas. Groenten en Fruit Magazine. 2011, 2, p. 20-22
7. Bontsema, J.. Robots on the march. HDC News. June 2011, p. 25.
8. Verso il robot agricoltore. ItaliaOggi. 2011, April, 16.
9. Oogstrobot paprika in de maak. Agrarisch Dagblad. 2010, Nov 5, p. 11
10. Bontsema, J.. Waarom kan een robot niet wat de mens wel kan?. Vakblad voor de Bloemisterij. Vol. 65, Nr. 46, 19 November 2010, p.6.
Exploitation of results
Two patents, one on an end-effector for sweep pepper harvesting and one on an end-effector for apple harvesting have been filed by respectively partner Stichting Dienst Landbouwkundig Onderzoek and Katholieke Universiteit Leuven.
The research on the sweet pepper harvesting robot has been continued within in a new Horizon 2020 project SWEEPER (GA 644313, www.sweeper-robot.eu). The partners from CROPS, Stichting Dienst Landbouwkundig Onderzoek, Ben-Gurion University of the Negev and Umea Universitet are involved in this new project, together with a greenhouse research station from Belgium (Proefstation voor de Groenteteelt), a system integrator from the Netherlands (IRMATO) and a commercial sweet pepper grower from the Netherlands (De Tuindershoek). The project duration is three years. Associated to the new project is a grower’s advisory board consisting of two Dutch and two Belgium sweet pepper growers, to insure that the new project has commitment of the sweet pepper grower’s community.
The CROPS industrial partner, the German company Festo, has with the experience with one of the end-effectors for the sweet pepper harvesting based on the Finray fingers, made a new commercial device for the food industry. The CROPS industrial partner, the French company Force-A, has extended their handheld ripeness sensor with an optical fibre, so that this device can now also be used on the tractor.
List of Websites:
The project is co-ordinated by Wageningen UR Greenhouse Horticulture.
For general questions please contact:
Dr. Jan Bontsema
E-mail: jan.bontsema@wur.nl
Phone: +31 317-486390
Fax: +31 317-423110
Visiting address
Droevendaalsesteeg 1
6708 PB Wageningen
The Netherlands
Postal address
P.O. Box 644
6700 AP Wageningen
The Netherlands
For other information you may also contact: webmaster@crops-robots.eu
The following results were obtained in the CROPS project.
For the robotic middleware ROS was chosen as software framework. The supervisory control system as well as the high-level software architecture have been developed and tested.
The design and implementation of the sensory systems for CROPS have been completed and tested , including the sensory system for detection and localisation of fruits in orchards and greenhouses, the sensory system for detection and classification of objects, the sensory system for fruit ripeness evaluation, the sensory system for diseases detection in crops and the sensory system for ground bearing in forestry.
A nine degree of freedom manipulator was manufactured. End-effectors were tested. A prototype of a canopy sprayer was manufactured. The first manipulator prototype, the grippers, and the precision spray end-effector were tested in laboratory and field experiments. Bases on these tests a final manipulator was designed and tested in a commercial greenhouse for harvesting sweet pepper.
For sensor fusion the system architectures for sensing, grasping and fusion and algorithms for sensor fusion, learning in sensing and grasping were developed. The adaptive sensor fusion algorithm was implemented and tested for apples and sweet pepper and for disease and ripeness detection .A method for construction of a discrete fuzzy grasp affordance manifold based on learning from human demonstration was developed and tested.
For sweet pepper harvesting the requirements for a harvesting robot were obtained. The manipulator, a platform to transport the manipulator through the greenhouse, a gripper and a sensing system were integrated into a complete system. This system was successfully tested and demonstrated to growers in both a laboratory setting as in a commercial greenhouse.
For harvesting of grapes and apples requirements have been defined based on discussions with the growers. To maximize the visibility and reachability of the fruits, the so-called ‘walls of fruit trees’ growing system has been chosen. The CROPS manipulator, grippers, sensors and software architecture have been tested both in laboratory as well as in apple orchards and as in vineyards. All the modules have been integrated into one system, which was successfully tested in the laboratory and in an apple orchard.
For canopy optimised spraying and close range precision spraying requirements were selected in discussion with spraying specialists and growers. A canopy optimised sprayer was designed as a trailed sprayer with centrifugal blower. An eight DOF hydraulic driven manipulator with three arms was used. Orchard experiments were performed with the canopy optimised sprayer during 2013. A good spraying quality was achieved with significant reduction of pesticide use. The close range precision spraying was focused on testing disease detection with various sensing principles. The manipulator with waterproof protecting case, sensors precision spraying end-effector were integrated in a precision spraying robot for viniculture. The robot was successfully tested in a greenhouse environment and the attained pesticide reduction was 84%.
For the forestry application the requirements for the detection of bushes, rocks, and trees and for the estimation of ground bearing capacity (for propulsion of forest machines) have been specified. Two sensory system for detection and classification of trees and humans respectively have been evaluated in a forestry environment.
As part of the training work package 44 graduate students were recruited in the project. 33 graduate courses were proposed at the participating universities for the students. The students took 17 advanced courses, participated in conferences and visited various universities and industries. Eight different workshops took place during the CROPS meetings.
For dissemination tha website www.crops-robots.eu has been made. 25 articles in technical journals, 10 articles in farmer’s magazines, 69 papers in conference proceedings and 12 papers on scientific journals were published. 76 presentations on conferences were given. The final CROPS workshop was held during the AgEng2014 conference in Zurich, July 2014.
Project Context and Objectives:
The main objective of the EU-FP7 project CROPS (GA 246252) is to develop a highly configurable, modular and clever carrier platform comprising a carrier plus modular parallel manipulators and “intelligent tools” (sensors, algorithms, sprayers, grippers) that can easily be installed onto the carrier and that are capable of adapting to new tasks and conditions. Both the scientific know-how and a number of technological demonstrators will be developed for the agro management of high value crops like greenhouse vegetables, orchard fruits, and grapes for premium wines. The CROPS robotic platform will be capable of site-specific spraying (targeted spraying only on foliage and selected targets) and selective harvesting of fruit (i.e. it will detect the fruit, determine its ripeness, move towards the fruit and grasp it and softly detach it). Another objective of CROPS is to develop techniques for reliable detection and classification of obstacles and other objects to enable successful autonomous navigation and operation of the platform in plantations and forests.
The CROPS project context and the CROPS objectives are described per work package:
WP1 System Engineering and Architecture:
Work package 1 is devoted to the research and development of an open systems architecture that provides a modular configurable platform for all software and hardware, sensors and controllers in CROPS. It aims at specifying and validating the major system components, some of which are developed as parts of other work packages. Other important components, the hardware and software system containing interfaces to sensors, controllers, actuators and human operators, will be developed as part of WP1.
The major objectives of WP1 are:
1. Definition of the overall system specifications
2. Design and construction of a computer hardware platform, including interfaces
3. Design and implementation of a software framework and supervisory control system
4. Validation of system components
5. Integration of system components
WP2 Sensing:
Work package 2 is devoted to the research and development of a low cost, efficient, adaptable, robust, secure and intelligent sensing system for the CROPS robotic platform. With this sensing technology, the CROPS robotic platform will be able to: navigate autonomously in greenhouses and outdoor
environments, detect fruits during different stages of their life cycle, provide sensory input for agromechanical operations, and perform sophisticated tasks, such as ripeness evaluation and disease
detection on fruits and crop canopy. Forestry tasks will also benefit from these WP2 developments. The scope of WP2 includes the following major objectives:
1. Localization and characterization of fruits in a controlled environment
2. Localization and characterization of fruits in natural environments
3. Detection and classification of objects in plantations and forests
4. Obstacle detection for autonomous navigation
5. Sensing of ripeness
6. Detection of diseases
7. Formulation of evaluation methodologies for sensing results
WP3 Manipulators and End-Effectors:
The objectives of work package 3 are the design, modelling, manufacture, start-up and optimization of low cost, lightweight, modular and compliant manipulators with exchangeable end-effectors and their motion control.
The major objectives of this work package are to design and to develop:
1. Prototype of manipulator for large motion
2. Prototype of manipulator for accurate positioning
3. Prototype of end-effector for gripping or fruit removal
4. Prototype of end-effector for spraying
5. Hierarchical simulation model of the manipulator system
6. Design of a low-level controller interface
7. Motion and grasping control.
WP4 Intelligent Sensor Fusion and Learning Algorithms:
Work package 4 is devoted to the research and development of intelligent algorithms for sensing and grasping in the CROPS robotic platform. The algorithms will enable the CROPS robotic platform to: (i) perform robustly and efficiently in a variety of operating and sensing conditions by fusing the information from multiple and different types of sensors, and (ii) easily adapt to new varieties of crops and to crops other than those of the CROPS project through continuous learning.
Specifically, WP4 focuses on the development and implementation of:
1. Adaptive sensor fusion framework and algorithms
2. Learning algorithms for the different sensing tasks
3. Learning algorithms for grasping.
WP5 Sweet pepper – protected cultivation:
The objectives of work package 5 are:
1. To develop and demonstrate a robotic system, focussed on harvesting, for sweet pepper production in protected cultivation systems.
2. To develop and evaluate generic concepts that can be added to this platform to perform other functions like planting, attaching plants to supporting wires, pruning, monitoring yield and quality, and local crop protection.
WP6 Harvesting systems in orchards: grapes and apples:
The objectives of work package 6 are:
1. To develop and demonstrate a robotic system, focused on harvesting, for apples and grapes.
2. To develop and evaluate generic concepts that can be added to this platform to perform other functions like planting, pruning, monitoring yield and quality, and local crop protection as well.
WP7 Precision spraying:
The objectives of work package 7 are:
1. To achieve canopy optimised spraying where the spraying operating parameters are continuously adapted to the canopy characteristics, such as volume, shape, density, etc., varying in space and time with the aim to reduce pesticide application by 30%.
2. To achieve high-precision close range target spraying to selectively and precisely apply chemicals solely on targets susceptible to specific diseases/pests with a 90% success rate and a 90% reduction in pesticide use for selected varieties.
WP8 Forestry:
Work package 8 is devoted to sensory and perception solutions for forestry automation applications. The proposed tasks are object detection/classification (bushes, rocks and trees) and estimation of ground bearing capacity (for propulsion of both manual and autonomous forest machines). The developed solutions will also be applied to robots in vineyard environments (WP7) and greenhouse environments (WP5).
The specific objectives of WP8 are to specify, adapt, implement, test and evaluate sensing solutions in close cooperation with WP2 and WP4. As a result of WP8, parts of the CROPS project will be
of immediate application for forestry automation.
WP9 Training:
Work package 9 includes several activities to provide multidisciplinary training in a joint industrial and academic
environment, aiming to create a new generation of agro-forestry robotic researchers and engineers with a broad understanding of the needs and opportunities for intelligent robots that function in the agro-forestry environment. To succeed in the complex agro-forestry environment, all partners must be exposed to in-depth knowledge in the different domain areas. The main objectives of WP9 are to:
1. Train the staff of robotics/mechanical/sensing industries involved in CROPS developments in such a way that they will be able to adapt their developments to the specific characteristics of the agricultural environment that influence design and performance.
2. Train the staff of agricultural industries involved in CROPS developments in such a way that they will be able to adapt their developments to the use of advanced robotic systems.
3. Train Young Researchers (YRs) and engineers to give them in-depth knowledge from the many disciplines involved in CROPS.
4. Train YRs to work in an industrial-academic environment and endow them with complementary skills (project management, team work, presentations).
WP10 Dissemination:
The objectives of work package 10 are:
1. To inform the scientific community, especially in the areas of robotics and agricultural engineering, about the relevant results arising from the project and to disseminate experimental data and source code.
2. To ensure that all research results are disseminated to European industry.
3. To inform the farmers, through their local and national associations, about those results of the project that are ready for use and about new products/techniques possibly arising in the near future from the application of project outcomes.
WP11 Final demonstration:
The objectives of Work package 11 are:
Demonstrate final CROPS platforms under a variety of operating conditions for the following tasks:
1. Selective harvesting of sweet peppers in a closed cultivation system
2. Selective harvesting of apples
3. Selective harvesting of grapes
4. Canopy optimized sprayers for vineyards and orchards
5. Targeted sprayers for vineyards and orchards
6. Forestry machine capable of classifying obstacles.
WP12 Economics, Social Aspects, Sustainability and Exploitation:
The objectives of work package 12 are:
1. Analyse the parameters and factors influencing economic viability and social aspects and determine other requirements for sustainability with the aim to enable correct design requirements that will ensure successful market penetration
2. Evaluate and validate the performance of the developed technologies from a business point of view with the aim to ensure practical applications of the products developed and easy and fast market penetration and adoption of the technologies by farmers
3. Protect intellectual property rights and ensure open-source availability
4. Ensure exploitation of the project results.
WP13 Coordination:
The objectives of work package 13 are:
1. Manage the project and fulfil of all its goals.
2. Draft the Consortium Agreement and submit it to the partners for approval.
3. Organize meetings (including minutes) for the governing and management bodies and the Advisory
Group.
4. Deliver periodical reports for the Commission.
5. Monitor deliverables and milestones.
6. Create and manage project web-portal for internal and external communication.
Project Results:
Main S&T results
CROPS
Intelligent sensing and manipulation for sustainable production and harvesting of high value crops, clever robots for crops
Grant Agreement number 246252
Consortium:
1. Stichting Dienst Landbouwkundig Onderzoek (WUR), The Netherlands
2. Katholieke Universiteit Leuven (KU Leuven), Belgium
3. Ben-Gurion University of the Negev (BGU), Israel
4. Univerza V Ljubljani (UL), Slovenia
5. Umea Universitet (UMU), Sweden
6. Università degli Studi di Milano (UniMI), Italy
7. Agencia Estatal Consejo Superior De Investigaciones Cientificas (CSIC), Spain
8. Technische Universität München (TUM), Germany
9. CNH Industrial Belgium NV (CNHi), Belgium
10. Instituto De Investigaciones Agreopecuarias (INIA), Chile
11. FORCE-A SA (Force_A), France
12. Festo AG & Co (Festo), Germany
13. Sveriges Lantbruksunivesitet (SLU), Sweden.
Work packages:
The CROPS project consists of the following 13 work packages, with between brackets the lead beneficiary.
1. Systems engineering and architecture (TUM)
2. Sensing (CSIC)
3. Manipulators and end-effectors (TUM)
4. Intelligent sensor fusion and learning algorithms (BGU)
5. Sweep pepper – protected cultivation (WUR)
6. Harvesting systems in orchards: grapes and apples (KULeuven)
7. Precision spraying (UL)
8. Forestry (UMU)
9. Training (BGU)
10. Dissemination (UniMI)
11. Final demonstration (UniMI)
12. Economics, social aspects, sustainability and exploitation (WUR)
13. Coordination (WUR)
Introduction:
The main objective of CROPS (www.crops-robots.eu) is to develop a highly configurable, modular and clever carrier platform comprising a carrier plus modular parallel manipulators and intelligent tools” (sensors, algorithms, sprayers, grippers) that can easily be installed onto the carrier and that are capable of adapting to new tasks and conditions. Both the scientific know-how and a number of technological demonstrators will be developed for the agro management of high value crops like greenhouse vegetables, orchard fruits, and grapes for premium wines. The CROPS robotic platform will be capable of site-specific spraying (targeted spraying only on foliage and selected targets) and selective harvesting of fruit (i.e. it will detect the fruit, determine its ripeness, move towards the fruit and grasp it and softly detach it). Another objective of CROPS is to develop techniques for reliable detection and classification of obstacles and other objects to enable successful autonomous navigation and operation of the platform in plantations and forests. The rationale for this aspect of the project is that agricultural and forestry applications share many common research areas, primarily regarding sensing and learning capabilities.
The results of the CROPS project will be given by work packages.
Results:
1. Systems engineering and architecture (work package 1)
Work package 1 aims at formulating the system architecture and specifying and validating the system components, also with regard to the interfaces in between.
Therefore, high-level requirements for the components have been derived from different harvesting (sweet pepper, grapes, and apples) and spraying scenarios based on the input of all project partners in a product requirements document. Functional specifications for all subsystems have been defined during an iterative process in collaboration with all partners according to the VDI2519 guideline, which resulted in a functional specification document. Based on this document, the development and the subsequent testing and evaluation process has been performed.
During the decision process, a meeting about the robotic middleware was held at the Technical University Munich, where the hardware framework has been specified, and the, at that time very new, Robot Operating System (ROS) was chosen as software framework. To become familiar with the framework, a ROS school has been organized by the Technical University of Munich (TUM) for all interested partners.
The supervisory control system and the high-level software architecture has been developed. It consists of multiple modules which communicate via defined interfaces. With the handover of the first manipulator prototype, including the manipulator for large motion and for accurate positioning as well as the gripper, from the Technical University of Munich and Festo to WUR, the basis for the integration process was provided.
Furthermore, a web application (wiki) has been set up in order to support the testing and integration process. This wiki is a common internet-based platform for the project CROPS. It was accessible and could be modified by all partners and contains general information about the hardware and software modules, definitions of the communication as well as electrical and mechanical interfaces between the subsystems. A first manipulator prototype system has been circulated among the application work packages resulting into an extensive testing in the field of the integrated robot systems. The computer hardware platform, the supervisory control system as well as testing, evaluation and integration of the subsystems for achieving a fully integrated harvesting or spraying robot system were finalized.
2. Sensing (work package 2)
Work package 2 is devoted to the research on and the development of a low cost, efficient adaptable, robust, secure and intelligent sensing system for the CROPS robotic platform. With this sensing technology the CROPS robotic platform should be able to navigate autonomously in greenhouses and outdoor environments, detect fruits during different stages of their life cycle, provide sensory input for agromechanical operations, and perform sophisticated tasks, such as ripeness evaluation and disease detection on fruits and crop canopy.
Firstly the requirements and specifications of the sensing systems for CROPS were completed, resulting in the decisions for each sub-system: sensory system for fruit detection, sensory system for obstacle detection, sensory system for ripeness evaluation, disease detection system and system for estimation of ground bearing capacity. A first report on active perception algorithms for CROPS describes the role that will play the active perception algorithms on dealing with common complexities that will be found in the working scenarios, such as the occlusions, the variations in the apparent colour and the changeable illumination conditions. The sensory system for fruit detection and the sensory system for obstacle detection in the sweet pepper application was completed. It was decided to incorporate a mini-camera in the end-effector of the manipulator. This was intended to improve the detection in conditions in which the occlusion of the fruit is high, and also to improve the detection of the peduncles (and cutting position) of the sweet-peppers.
Furthermore the design and implementation of the sensory system for detection and localisation of fruits in orchards, the sensory system for detection and classification of objects, the sensory system for fruit ripeness evaluation and the sensory system for diseases detection in crops was completed.
Numerous experimental campaigns have been carried out during this period in different scenarios and with varied environmental conditions not only for accomplishing successful designs and implementations of the mentioned sensory systems, but also for their validation.
These modules embrace from basic pixel classification based on multispectral signature, through analysis of the 2D image for regions of interest, to the detection and localisation of targets in 3D space.
For the classification of objects in forest, a number of existing algorithms for detection of trees and estimation of tree diameter were evaluated and three new and more accurate algorithms for the stated purpose were suggested. Finally, the sensor system for fruit ripeness evaluation, as well as the sensor system for disease detection have been designed and tested successfully in field conditions.
A sensory system for ground bearing estimation has been designed and tested, this sensing could be both applied in forestry as in orchards.
3. Manipulators and end-effectors (work package 3)
Work package 3 is focussed on the design, modelling, manufacture, start-up and optimization of low cost, lightweight, modular and compliant manipulators with exchangeable end-effectors and their motion control.
First the high level system requirements for the manipulator and the end-effectors were derived on the basis of the application work packages 5 (Sweet pepper harvesting), 6 (apple and grape harvesting) and 7 (precision spraying).
A preliminary study on pneumatic actuator concepts for the manipulator resulted in the rejection of this concept and electrical actuators were applied for the complete manipulator system. Several kinematic manipulator designs were investigated and evaluated for the different applications. A nine degree of freedom, redundant and modular, manipulator turned out to be the most promising concept and a kinematic model was derived as well as a multi body simulation was performed for the mechatronic design of the system. The first manipulator was manufactured, assembled and started-up at the Technische Universität München (TUM).
End-effector prototypes for gripping and fruit removal for the described application scenarios were designed, manufactured, started up, and tested by partner Festo. The requirements for the spraying prototype were derived and based on numerical simulation a first prototype end-effector for spraying was manufactured. At TUM the integration of the first manipulator and the end-effector for gripping was done and the complete system was delivered to the Stichting Dienst Landbouwkundig Onderzoek (WUR) for the installation of the manipulator on a robotic platform and first field tests in sweet-pepper greenhouses were carried out.
The first manipulator prototype, the grippers, and the precision canopy sprayer were tested in laboratory and field experiments. Additionally, a simulation model with direct and inverse kinematics algorithms for serial manipulators, as well as equations to describe the manipulator dynamics was implemented.
Especially the fruit removal for sweet-pepper harvesting turned out to be very challenging for the gripper and sensor system design. Two end-effector concepts for the sweet-pepper harvesting application were designed and built by respectively WUR and one by FESTO. The final gripper prototypes include a Time of Flight (ToF) and RGB camera.
The first manipulator prototype has been adapted for the precision spraying application and was integrated with the spraying platform in an experimental greenhouse of the University of Milano, Italy (UniMi).
Based on the experiments with the first manipulator prototypes and the feedback of the project partners TUM has designed the final manipulator prototype. This prototype is assembled out of custom made robot drive modules.
The final manipulator prototype, designed by TUM, has been manufactured and assembled. After the start-up of the prototype in Munich in the beginning of 2014 and a complete functional test, the robot has been integrated in Wageningen for the sweet pepper harvesting application in February 2014 and was evaluated in field experiments by WUR mid-2014. Major improvements of the final manipulator prototype (compared to the first manipulator) are:
• Increased robustness of the mechatronic design required for agricultural applications
• Custom designed drive modules allowing for a high modularity
• Higher Power allowing for faster motions and more payload.
Further field experiments with the first manipulator prototype were carried out by the Katholieke Universiteit Leuven (KULeuven). A motion and grasping strategy for apple harvesting, based on dynamic motion primitives has been developed by the Ben-Gurion University of the Negev (BGU).
4. Intelligent sensor fusion and learning algorithms (work package 4)
Work package 4 is devoted to the research and development of intelligent algorithms for sensing and grasping in the CROPS robotic platform. The algorithms will enable the CROPS robotic platform to: (i) perform robustly and efficiently in a variety of operating and sensing conditions by fusing the information from multiple and different types of sensors, and (ii) easily adapt to new varieties of crops and to crops other than those of the CROPS project through continuous learning.
The system architectures for sensing, grasping and fusion (inputs, outputs, connection with other system modules) and basic algorithms for sensor fusion, learning in sensing and learning in grasping were characterized and developed. The adaptive sensor fusion architecture was designed and implemented in Matlab and in ROS. It includes a module for threshold optimization and an object unification module. The algorithms developed were tested on the first two acquired databases with measurement from field tests. Preliminary results using a limited database from a sweet pepper greenhouse indicates that by fusing several algorithms detection of peppers is increased resulting in robustness for changes in lighting conditions.
A generic learning framework consisting of a Sensor Layer, a Feature Layer and a Classification Layer was developed. To demonstrate the framework a depth sensor and an RGB camera were used to generate foreground images to learn features for classification of forestry objects. A database of images was collected and labelled. The objects were classified as bush, tree, stone and human. Adding depth based features improved classification accuracy considerably.
A method for construction of a discrete fuzzy grasp affordance manifold based on learning from human demonstration has been developed. It includes quality grade determination, manifold structure determination, and a method for adapting the human grasp manifold to different manipulators and grippers. Field experiments were performed for apples and sweet peppers. The affordance manifolds were tested in the laboratory for two different grippers proving feasibility.
An adaptive sensor fusion algorithm and an algorithm for determining thresholds adaptively were implemented and analyzed for two databases. The databases include apple databases (acquired in Chile 2012) and a pepper database acquired in the Netherlands in 2012. The results were analyzed with respect to several parameters.
A database with forest data was acquired in Swedish forests. Data comprises RGB images and laser data for scenes with trees, humans, rocks, and bushes. A classifier system was constructed and algorithms for image segmentation and semi-automatic labeling were developed.
Theoretically grounded grasp quality measures that can be computed directly from a point cloud were developed along with efficient graspability map computation methods that facilitate map computation during run-time. The computed maps were tested in a physical environment. Dynamic motion primitives (DMP) were applied to apple harvesting in a laboratory setup. The parameters of each DMP were learned based on demonstration.
A new methodology for evaluating the connection between gripper design and sensing was developed in addition to formulating measures for sensor fusion evaluation.
The adaptive sensor fusion algorithm with an algorithm for determining thresholds adaptively was adjusted to the new large database of peppers (acquired in the Netherlands, November 2013). New sensor fusion algorithms were developed for ripeness detection in grapes and apples. A new sensor fusion algorithm was developed for disease detection in grapes. ROS modules were developed and implemented for final prototypes for apples and peppers.
The database with forest data previously acquired has been extended with more data from Swedish forests. Data comprises RGB images, thermal images, and laser data for scenes with trees, humans, rocks, and bushes. Algorithms for detecting trees and humans in forest environments were developed.
A gripper collision detection with the environment and its integration with graspability map-based grasp pose selection was developed, tested in simulation and implemented in ROS as part of the grasp node. The grasp node was enhanced to include pepper, apple, and grape grasp pose selection. The grasp node was successfully integrated with the complete CROPS ROS implementation. An innovative algorithm for reach-to-grasp motion planning which integrates graspability maps with the rapid random trees (RRT) algorithm was developed and successfully tested in simulation.
5. Sweep pepper – protected cultivation (work package 5)
Work package 5 is mainly dealing with the development and demonstration of a robotic system, focussed on harvesting, for sweet pepper production in protected cultivation systems.
The design objectives and requirements for a sweet pepper harvesting robot were determined. The current situation in greenhouse sweet pepper production was illustrated. The terminology and the crop system were explained as well as detailed information about fruit geometry and plant dimensions was provided. Also data about the cropping cycle and the climate conditions in a greenhouse were given. The information which was assembled was based on discussions and interviews held with a number of Dutch sweet pepper growers and the knowledge base of Wageningen University & Research Centre, complemented with available literature and reports.
A number of design sessions with experts (growers, trained workers and project partners) were organized to find and to define functions to fulfill the earlier defined requirements. These sessions yielded insight on how humans harvest sweet peppers. From this insight came the important notion that solving occlusion and finding the abscission zone, where the sweet pepper should be cut, are two important aspects which are difficult to solve but need to be solved in order to successfully harvest a fruit. The functions were translated to so called morphological charts. A morphological chart consists of a list of functions, in the left column and in the rows behind each function a working principle is shown. Also a description of requirements, functions and working principles for a number of cultivation tasks other than harvesting was produced. Crop monitoring, planting, pruning and tying up the plant were worked out in more detail.
The first prototype of the manipulator from WP3 and a carrier platform to transport the manipulator through the greenhouse and a gripper were integrated for the very first test in the greenhouse environment.
The conceptual design including a design evaluation of prototype of harvester for sweet pepper was finished. An analysis of the different sub-systems for cropping systems and crop manipulation; concepts for fruit and obstacle detection, ripeness determination and localization and concepts to reach, grasp and detach fruit was given. The chosen design is based on the current good practice cropping system of sweet-peppers in the Netherlands. The concepts for a number of cultivation related tasks for sweet-pepper production other than harvesting have been described. Four tasks were identified to be promising to automate which are: monitoring, planting, pruning and tying up the plant. A mobile platform that can move between the plant rows, as under development in this project offers opportunities also for plant monitoring. The other tasks are most likely not economically feasible to automate. Different cameras were tested for their suitability to detect and localize sweet-pepper fruits and obstacles and to determine fruit ripeness. After evaluation a sensor rig with a Time of Flight 3D camera mounted close to a high-resolution colour camera was assembled. Two different end-effectors to detach fruits from the plant are developed and analysed: a gripper with an integrated cutting tool with fingers that are based on the Finray principle (built by partner Festo) and a guidance and cutting tool approaching the fruit from below: the Lip-type end-effector, designed and built by WUR. For this last end-effector a patent has been filed. On both types of end-effectors two mini cameras are mounted: one Time of Flight camera and one colour camera. The first prototype of the manipulator was tested. It was concluded that the considered prototype is suited to harvest pepper fruits. A carrier platform for the robotic arm and the sensor systems was built. The sensor system on the platform is mounted on a linear motorized slide and can be horizontally moved out of the workspace of the robot while the robot is moving. The software of the robot was implemented for the Linux operating system and the middleware ROS. The main software is based on a finite state machine and includes diagnostic tools and performance measures of the harvesting operation. The demonstrator was tested under laboratory conditions. During these experiments data on the performance and accuracy of the different subsystems was gained. During these early tests 189 out of 194 fruit could be detected (97%), 167 fruits could be reached (86% of all fruits) and 154 picked (79% of all fruits). The integrated system was successfully demonstrated to the Dutch pepper growers which are part of the established growers advisory board.
In a number of greenhouse experiments the performance of the two sweet-pepper end-effectors (Fin Ray and lip-type) was determined. In these single module experiments the Fin ray type end-effector harvested a maximum of 80% of the fruits on the plant, the lip-type end-effector a maximum of 76% of the fruits. After final integration (using e.g. the final manipulator prototype) the robot was tested in the laboratory with an artificial plant canopy first. After that, the robot was transferred to a greenhouse. Between April and July 2014 experiments with the integrated pepper harvester were carried out in a red colored sweet-pepper crop grown in a Dutch commercial greenhouse. During these experiments the feasibility of harvesting sweet peppers autonomously has been proven. However, there were several difficulties noted which caused limited performance. Harvest success in an unmodified crop was only between 2% and 6%. After simplifying the crop by removing fruit clusters and occluding leaves harvest success improved up to 33%. The average cycle time to pick a fruit was 94 seconds. Additionally, a novel stem-dependent determination of the grasp pose method was evaluated. This indicated that by using this feature the plant stem damage decreased from 7% to 4% in an unmodified crop and from 19% to 13% in the simplified crop. For the Lip-type end-effector the grasp success increased using that feature. Several video clips of the sweet-pepper robot operating in the greenhouse were produced and diss¬eminated and are available for download in the video section of the Crops public website: http://www.crops-robots.eu/. It is expected that using the new insights gained during the experiments and a successful integration of the latest modules for sensing, motion planning and artificial intelligence developed in the project could significantly increase the performance of the system, which will be tested in the follow up project SWEEPER (www.sweeper-robot.eu).
6. Harvesting systems in orchards: grapes and apples (work package 6)
Work package 6 is dealing with testing and demonstrating of a robotic system, focused on harvesting of apples and grapes in an orchard and vineyard respectively.
The design objectives and system requirements for the apple and grape harvesting robots have been defined based on measurements in the orchards and vineyards in Chile and discussions with the stakeholders (end-users, system integrators, researchers, growers and crop specialists). As many of the CROPS partners were not familiar with commercial apple and grapes growth, an overview of the current situation in apple and grapes growth and harvesting, both manual and mechanized, has been communicated in the project. A design objectives tree was presented for the orchard harvesting robot which was complemented with separate lists of requirements for the apple and grape harvesting robots. In the brief of requirements, the specifications and design objectives were stipulated in quantitative terms, such as the required harvesting speed (in seconds per apple) and the percentage of fruit to be successfully harvested. From these requirements the required functions have been derived and possible working principles have been listed and critically evaluated. The most common crop growing practices and the most economically relevant varieties in Belgium and Chile have been critically reviewed with respect to their potential for automated cultivation and harvesting. Inputs from experts in pomology and viticulture and experienced growers have been exploited for this. To maximize the visibility and reachability of the apples and fruit bunches, high density planting of flat planar canopies supported by wires have been selected both for apple and grape growth. Although these planar growing systems are quite labour intensive to establish and train, these ‘walls of fruit trees’ have high potential for mechanized pruning and harvesting thanks to the limited depth. Besides improving the visibility and reachability, the pruning strategies in the apple orchard are also targeted at keeping the fruits close to the main branches, as this not only maximizes fruit quality, but also increases the detachment success. A next step will be to investigate thinning strategies reducing the number of apples in each bunch to one while maintaining the yield, as this would have a positive impact on the robotic harvesting success.
The conceptual designs and design evaluation of the demonstrator of a harvester for apples and grapes have been obtained. Also the description and design evaluation of generic concepts other than harvesting which could be performed by the robot have been summarized.
The manipulator, grippers, sensors and software architecture for the apples and grapes harvesting robotic system elaborated in work packages 1, 2 and 3 have been tested under laboratory as well as practical conditions in the apple orchard and grapes vineyard.
The different modules have been integrated into a working prototype. The functionality of this full prototype with all modules integrated has been demonstrated in the laboratory by fully automated picking of some plastic apple dummies attached to a metal frame by magnets.
The manipulator, grippers, software architecture and part of the sensors for the apples and grapes harvesting robotic system elaborated in work packages 1, 2 and 3 have been tested further under laboratory as well as practical conditions in the apple orchard and grape vineyard. The sensing in this prototype was performed with an alternative system elaborated by KU Leuven. This system uses an RGB-D camera (ASUS Xtion) in combination with an algorithm for apple detection and size estimation written using the Point Cloud Library in openCV.
The functional prototype of the apple harvesting robot has been tested in a ‘mur fruitier’ apple orchard of PCFruit, a Belgian fruit research station, in October-November 2013. The canopies had been pruned specifically to promote robotic harvesting. During these tests this prototype detected all apples inside the picking area, grasped 90% of these and successfully detached 80% of the successfully grasped. So, in these tests 72% of the fruits were successfully harvested in outdoor conditions. The average time needed to pick one apple was 15s when the apple was dropped in a tube after detachment and 30s when the apple was transported to a box. The causes for failure were carefully studied and suggestions for improvement were made.
During tests in August 2014 in a Belgian vineyard detection of the stalk of the grape bunches was achieved by manually attaching a label to this stalk and detecting this with the sensor platform.
The field tests in August in the vineyard with the grapes harvesting robot were performed by attaching fake grapes to the vines and picking these with the robot. In the first half of September, picking of real grapes bunches was then attempted by moving the grapes gripper to the labelled stalk. However, as this method only provides information on the position (x, y, z) of the stalk and not of the orientation, grasping was most of the time not successful. During these tests it was also observed that the grapes gripper prototype even at maximum pressure did not have sufficient force to cut the thick stalks of these grape bunches. As a result, no successful detachment could be achieved in this orchard.
Around half of September 2014 the harvesting robot was moved to the apple orchard to test the effect of the improvements made compared to the tests performed during the 2013 harvesting season. The fruit localization success rate was found to be as high as 98 %. The reason for this high success rate is that no occlusions were present because leaves were pruned before measuring. The only occlusion that could happen was due to apples hanging in front of each other. This is the reason for not reaching 100 % detection accuracy. False-positive detections were not observed (false-positive detection rate = 0 %). When apples were detected and localized in the picking window of the manipulator, they could be approached in all cases (approach success rate = 100 %).
The grasp success rate was 86 %. The main reasons for an unsuccessful grasp were a too small width between the two gripping fingers and a non-optimized positioning of the end-effector with respect to the detected fruit. The removal rate was only 60 %, with a non-optimized picking motion and a non-optimized positioning of the third finger on the stem as the main reasons for unsuccessful removal. Combination of the grasp success rate with the removal rate results in a detachment rate of 47%. When this detachment rate is combined with the fruit localization rate, a harvest rate of 46% is obtained. It is clear that within the reachable window (which was much smaller in 2014) the grasp success rate was comparable, while the detachment rate was drastically reduced. There is no clear explanation for this reduction in the detachment rate, but it might be related to the low third finger success rate (21%). The fruit characteristics could also be a possible explanation as in 2013 the fruits were more ripe and it is known that ripe fruit can be detached more easily. Finally, the lower number of tested apple fruits might also have resulted in a less accurate quantification, and therefore a bad comparison, between the two test years.
7. Precision spraying (work package 7)
Work package 7 work is has two applications, namely canopy optimised spraying and close range precision spraying and the aim is developing automated systems in order to reduce considerably the amount of pesticides
Both canopy optimised spraying and close range precision spraying requirements and specifications were selected with discussion with spraying specialists and growers.
A database of sensed variables (laser scanner, industrial RGB camera, photographic camera, thermal camera, ultrasonic sensor) was recorded and was made available for use in other work packages, mainly in WP2 and WP4.
The canopy optimised sprayer was designed and modified from existing design as a trailed sprayer with centrifugal blower. It was equipped with sensing equipment: laser scanner, ultrasonic sensors and RGB camera. Processing algorithm was derived such that multidimensional surface plane of selected degree is fitted to the canopy contour. An eight DOF hydraulic driven manipulator with three arms was used and equipped with position sensors. An inverse kinematics algorithm was derived, such that spraying arms are perpendicular to spraying surface and located at a selected distance from the centre of sprayed surface. Actuation of spraying arms is performed hydraulically and hydraulic cylinders are driven using fast relays from digital I/O module. Cycle time of spraying manipulator control is less than 10 ms, while response times of sensing elements are longer. Processing of all sensed values are delayed based on recorded sprayer velocity and stored in a buffer.
Work has been done on building of a new manipulator section, CAD design, CFD modelling and testing of subcomponents.
For the close range precision spraying initially the focus was on testing disease detection with various sensing principles. Several configurations of close range precision sprayer were tested, open, with background shielding which avoids multiple rows viewing and diffusing roof that avoids specular spots reflections. In all configurations, sensors were used in different positions, selected was a combination of sensors mounted on prime mover and on an end effector. Close range precision spraying end effector was constructed in accordance with requirements and specifications for efficient close range precision spraying. Post processing algorithms included normalisation and segmentation principles, among them noise reduction, illumination balancing among channels correlations, background and external material removing, foreground segmentation (infection symptoms identification by spectral ratios + texture indexes) and experiments on data fusion with hyperspectral images. ROI centroids distribution in spectral dimensions were also evaluated.
For canopy optimised spraying several modification and tasks related to the first prototype sprayer were performed. These included reduction of number of sensors, implementation of new single electric box for sensing, processing and arm control, improvement of sensing, processing and control software, development of new algorithm for spraying nozzles positioning, performance optimisation by laboratory experiments for accurate and smooth operation of spraying arms and implementation of orchard experiment and substantial reduction of pesticide use. The canopy optimised sprayer performed during 2013 orchard experiments reasonably well and good spraying quality with significant reduction of pesticide use was achieved. The further modified canopy optimised sprayer performed well during orchard experiments in 2014 and a good spraying quality with significant reduction of pesticide use was achieved. Fan inducer was built for regulation of the airflow rate and mounted on the sprayer. Aerodynamic measurements of fan inducer were performed. Various small modifications of the software were performed, resulting in better performance of the canopy optimised sprayer. Among them are most important arms positioning algorithm modifications, power management, installation of fast valves for continuous flow control, and selection of algorithms for continuous flow control. Also, several dissemination and demonstration activities were performed related to canopy optimised sprayer.
The close range precision spraying achieved a significant milestone under the CROPS project: for the first time small foci of diseases were robotically detected and sprayed without human intervention. System integration of close range precision sprayer included integration of prime mover with encoder, R-G-NIR multispectral camera, RGB high resolution camera, diffuse illumination panels, personal computer for analysis of sensing data for real time disease detection, manipulator control PC and power case, CROPS manipulator with waterproof protecting case, pesticide control circuit with a pump, and precision spraying end effector.
In the greenhouse experiment, a test plot was selected with 10 locations that required spraying. By a fully autonomous operation, all were actually sprayed with 25 spot sprayings. The actual reduction in pesticide consumption was 84%, although there is possibility for further 10% reduction. The data of the already performed experiments have been analysed and refined algorithms for disease detection were developed. Further activity on disease detection included sensor fusion for different disease diagnostic and advanced classification algorithms or boosting techniques for decreasing the false positive incidence. Results and analysis performed have shown that for field use shield and light-diffusing covering layer for sensing equipment is required.
8. Forestry (work package 8)
Work package 8 is devoted to the tasks object detection and classification (bushes, rocks, and trees) and estimation of ground bearing capacity (for propulsion of both manual and autonomous forest machines). Technical requirements for the two tasks were derived. For object detection and classification this gave the following result: localize, detect and classify trees, bushes, rocks, and humans with the following constraints: 95 % detection rate; 10 cm localization accuracy; less than 10 meters in front of the forestry machine; varying light conditions (sun, darkness); varying weather conditions (snowing, raining) and varying ground conditions (gravel, snow, forest vegetation). For ground bearing capacity this gave the following requirement. Estimate bearing capacity with the following constraints: 5-10 meters in front of the forest machine; only in forest terrain; only in summer.
A sensory system for detection and classification of trees has been evaluated in a forest environment. Two sensory systems able to detect and classify trees and humans respectively in a forest environment have been evaluated. The human detection works very well and can detect humans that are not visible for the naked eye (or a normal RGB camera). The tree detection works well in most situations, but as with all image analysis techniques it is sensitive to lighting conditions. A system able to estimate ground bearing capacity has been developed and evaluated. Preliminary results show that the system has potential for a non-destructive, real-time estimation of ground bearing capacity.
9. Training (work package 9)
Work package 9 includes several activities to provide multidisciplinary training in a joint industrial and academic environment, aiming to create a new generation of agro-forestry robotic researchers and engineers with a broad understanding of the needs and opportunities for intelligent robots that function in the agro-forestry environment. To succeed in the complex agro-forestry environment, all partners must be exposed to in-depth knowledge in the different domain areas.
A total of 44 graduate students were recruited to work on various aspects of the project. 33 graduate level courses were proposed at the participating universities for the students. The students took 17 advanced courses, participated in conferences and visited various universities and industries. CROPS personnel were trained in nine different workshops that took place during the half year periodical meetings.
10. Dissemination (work package 10)
The objectives of work package 10 are:
1. To inform the scientific community, especially in the areas of robotics and agricultural engineering, about the relevant results arising from the project and to disseminate experimental data and source code.
2. To ensure that all research results are disseminated to European industry.
3. To inform the farmers, through their local and national associations, about those results of the project that are ready for use and about new products/techniques possibly arising in the near future from the application of the project outcomes.
Within the website of CROPS project, a webpage specifically designated to dissemination has been launched since March 2011, with two subpages aimed to contain focused information directed toward farmers and companies, respectively. The webpages were updated on a regular basis, with popularizing, technical and scientific publications produced within CROPS.
Close contacts with three other European projects were established through dedicated meetings with the coordinators of RHEA, SPICY and RoboEarth. These are all FP7-projects focused in area of field robotics, advanced crop production and fundamental robotics. Among other topics, possibilities of reinforcing dissemination activity through joint actions was discussed and considered for implementation.
A CROPS project session was organized within the 1st RHEA Conference held in September 2012, with four presentations from CROPS partners and six CROPS presentations were given on the 2nd RHEA Conference hold in May 2014 in Madrid.
Based on the CROPS concepts and initial research activities, several publications were produced within the consortium. Specifically: 25 articles on specialized webpages or in technical journals and newspapers; 10 popularizing articles in farmers magazines; 76 presentations at international scientific conferences, with 70 extended papers included in proceedings; 12 papers in main scientific journals in the area of field robotics and vision theory.
The CROPS project was one of the subjects of a special report “Le serre a Rotterdam” of the science popularizing TV programme “SuperQuark” broadcasted by the Italian network Rai Uno on July 2012 (http://www.rai.tv/dl/RaiTV/programmi/media/ContentItem-afa68735-1a96-4d31-a5a3-5435f5c9cea9.html). CROPS project activity was reported by the science popularizing TV programme “Dobra ura” broadcasted by the Slovenian network RTVSLO on 24 April 2014 (http://4d.rtvslo.si/arhiv/dobra-ura/174272608) and the CROPS project was the main subject of a report in “Regio Nieuws” broadcasted by the Netherland TV network WOS on 4 September 2014
(www.wos.nl/televisie/uitzendinggemist/player/item/20140904-regionieuws-van-donderdag/).
Popularizing description of selected experiments and explicative videos relevant for demonstrating the main capabilities of developed subsystems and their possible applications have been published on the homepage of CROPS website.
The CROPS Final workshop was hold in Zurich on July 9th, 2014 as a special session within the AgEng2014 conference, with a world-wide academy attendance and also R&D representatives from main machinery manufacturers. The workshop lasted a whole day, with 16 extended presentations (including one overall presentation from RHEA project) and 4 summary posters discussions. Two of the extended presentations were given by industry partners, namely by company Festo (on new passive grippers for agricultural applications) and by company FORCE-A (on Fluorescence sensing for ripeness and quality evaluation of specialty crops products).
A special issue of the journal "Biosystems Engineering" is prepared on advances in agricultural robotics and which will have a strong nucleus coming from “CROPS Final workshop” papers, and for which partners of the CROPS consortium have been charged as Guest Editors.
11. Final demonstration (work package 11)
Work package 11 has the task to demonstrate the final CROPS platforms under a variety of operating conditions and for the different applications.
The demonstration activities were organised and conducted mainly in the last half year of the CROPS project. To this aim, for the different application cases the scenario conditions of demonstrations were defined by the partners responsible for them, namely: sweet-pepper harvesting (WUR), apple harvesting (KULeuven), grapes harvesting (KULeuven), canopy optimised sprayer (UL), close precision selective spraying of diseases (UNIMI) and for obstacle detection in forestry operation (UMU). For all the demonstrations, specific indicators of system’s performance were also stated in order to objectively quantifying the operational success achieved in the demonstration.
The possibility of organizing in a single site all the demonstrations of the systems applied to grapes and apples (i.e. apple harvesting, grape harvesting, canopy optimised spraying and close range precision spraying) was preliminary explored as a priority option. Due to high requirements in terms of logistic efforts and resources costs, the involved partners independently organized local arrangements for applications demonstration.
The CROPS robot system was then configured according to the specific needs of each application case, by integrating manipulator, end-effectors, sensing hardware, algorithm and control software.
The demonstration schedule had to accommodate the possibility for all the application cases to carry out of the different demonstrations with the shared hardware resources (i.e. the manipulator and the control system). To this aim, the different demonstrations were time-spread from spring 2013 to autumn 2014.
The expected capabilities of apple harvesting, canopy optimised spraying, close precision selective spraying of diseases, obstacle detection in forestry operation were successfully demonstrated. For sweet-pepper harvesting the expected capabilities were demonstrated in a simplified scenario, while only partially achieved in real operative scenario. For grape harvesting the expected capabilities have not been reached in field demonstration.
12. Economics, Social Aspects, Sustainability and Exploitation (work package 12)
Work package 12 is mainly focussed on economic viability , social aspects and sustainability with the aim to enable correct design requirements that will ensure successful market penetration
market penetration and adoption of the technologies by farmers.
A simulation of economic viability was made for each application, consisting of three simulation tools to determine the economic viability of: sweet pepper harvesting (WP 5), harvesting systems in orchards (grapes and apples WP 6) and precision spraying (WP 7).
A report on social aspects, standard requirements and other requirements for sustainability, describes the social aspects, economics and sustainable requirements for: sweet pepper harvesting (WP 5), harvesting systems in orchards (grapes and apples WP 6) and precision spraying (WP 7) has been finished. To evaluate social aspects, economics and sustainability, a short version of the method “Sustainable scan for the agro food chain” is used. The method is influenced by thinking about CSR (Corporate Social Responsibility) and the ambition is to provide an understanding of the dynamic relationships between the triple bottom line, people, planet, and profit, and to measure and weight the effects of efforts to raise performance levels in all three of these domains.
For the sweet pepper harvester the evaluation is positive in general. Specific advantages for robotising the sweet pepper harvesting task are labour development: employment, education, working conditions, absenteeism. When robots replace human labour there are no limitations to climate conditions. Plants grow very well under high CO2 levels, high humidity and high temperatures, under which people cannot work. A negative condition is that robots are expensive. In the beginning the pay-back time might be up to five years and more. For the harvesting system in orchards (grapes and apples) the opportunities are almost the same as for sweet pepper harvesting and are looking positive. But there are no advantages expected in energy saving due to closing windows in greenhouses. The energy consumption will increase a bit. There will be also initial capital needed which might be difficult to find. For robotized precision spraying, opportunities are looking positive. It will give social support by using less pesticide appointed more accurate. Labourers don’t have to work under bad/protected conditions. There will be also more initial capital needed which might be difficult to find. But in the end, using less spraying fluids, robots will save money and become profitable.
The evaluation on all the aspects within People Planer Profit leads to new system requirements which were discussed with the involved work package leaders.
At May 13, 2013 a one day Exploitation Strategy Seminar was organized in cooperation with ESIC2, a EU-project, fully dedicated to exploitation of FP7 NMP research results. All members of the General Assembly,the Steering Committee (GA/SC) and the exploitation committee of CROPS were invited and participated. The seminar was led by Thomasz Cichocki (PL). The seminar was evaluated by the GA/SC as valuable to the project and can be seen as the start of exploitation activities.
The results on sensing and canopy spraying with a robot are promising and possibly exploitable. The precision spraying application is identified to be validated first in relation to an application that could be commercialized after the project.
The project partner WUR has filed a patent for the end effector of sweet pepper harvesting (Lip-type end-effector) and the partner KULeuven has filed a patent for the end-effector for apple harvesting.
The work package analysed and validated the performance of the three applications that were built and tested: sweet pepper harvester; apple harvester; canopy optimised sprayer. For these applications exploitation plans were made. Furthermore a list of open source software and algorithms was made.
13. Coordination (work package 13)
The task of work package 13 is to manage the project and to take care that everything is running smoothly and that all goals are fulfilled.
The consortium had a project meeting every 6 months, in total eight, organized by one of the partners. During these meeting also the meeting of the Steering Committee and the Grand Assembly took place. On request more meetings could be held. As an example special meetings have been held on the decision of the robotic middleware, on the gripper design for sweet pepper, on CROPS software integration and on motion planning. In total three official review meeting with the EC were organized.
The CROPS website, www.crops-robots.eu , has been available from October 28th, 2010. The website has a public part, describing the project, the partners, the work packages, a dissemination section and a contact site. Besides the public part there is a so-called member area for the partners, including a file share facility. On every project meeting an analysis on the number of visitors of the web site were given. On average there were 200 visits per day. Most popular are the videos on the website and also the abstract booklets and presentations from the Crops workshops are downloaded frequently, in average 20 to 100 times per month per booklet.
The project was advised by a Grower’s Advisory Board, which was organized locally.
Potential Impact:
Potential impact.
The CROPS project has developed a modular and generic robot platform for harvesting sweet pepper, apples and grapes. The robot platform is also been used for canopy spraying in orchards and targeted spraying in vineyards. For forestry application obstacle detection has been developed.
The CROPS robots will reduce harvest costs. It is estimated that a cost reduction of 40% can be achieved. The critical factor for economic feasibility is the yield rate which must be at least 95% to ensure successful market entrance. However, this threshold might drop due to increased labour costs and increased difficulties in finding workers which is already a bottleneck to agriculture production expansion in some countries.
The technology evaluation that will be conducted in parallel to the scientific activities will provide more detailed evaluations of harvest cost reductions for the variety of farms and markets and will include specific calculations for each of the demonstrators.
CROPS will contribute to the reduction in the use of crop protection chemicals. A CROPS robot will enable to spray only the canopy in vineyards and orchards instead of spraying the whole area, leading to a reduction of pesticide application of 30%.
For specific treatments, selective application of pesticides on susceptible targets will allow to achieve even higher reduction levels. Selective spraying of insecticides on only the grape bunches, will result in 90% reduction of chemicals.
Due to the reduction of chemical inputs there will be less residues of chemical on the products leading to improved product quality. Using a robotic system will enable to harvest the product at optimal time thereby increasing the quality of the product. A system which handles each product autonomously also increases food safety due to elimination of the need for human operators. The ability to look within the plant level and measure locally the quality of each individual fruit can open new possibilities and markets. For example, this project makes it possible to determine the chemical composition of harvested grapes at the bunch level and thus to improve the quality and consistency of the wines considerably.
Several demonstrator systems has been developed as part of the project. Successful development of the new CROPS design for agricultural and forestry robots will lead to a new product line for several European machine industries, maintaining the world-leading position of the European forestry and agricultural machine industry.
Low valued labour on the farms and in forestry will be replaced by high skilled labour. Furthermore, the development of agriculture and forestry robots will create high technology jobs in the machine and sensing industries. In forestry, recruiting of machine operators is often a limiting factor for business expansion. Introducing semi-autonomy and autonomy into the forestry machines creates additional high skilled jobs.
CROPS will contribute to sustaining available income in rural areas. By enabling introduction of robotic systems which are efficient, cost effective and can replace manual labour, the agricultural sector, and also the supplying companies, will be kept in. The developed robots can release the farmer from early rising, long work days, back breaking demands of farm work, and around-the-year engagement to the farm. Together with the previously mentioned creation of high technology jobs, the rural economy will be strengthened and there will be a reduction of migration to urban areas.
The reduction of chemical pesticides enabled by site-specific and targeting spraying will reduce chemical pollution by reducing chemically-polluted runoff from fields that cause contaminated surface and ground waters and violate ecological balance. Additionally, the reduction of chemicals disposed will reduce the medical hazards caused by exposure of workers and increase health of all via the improved environment and improved agriculture product quality.
Dissemination
The dissemination activity has been performed on a regular basis, mainly through the dedicated webpage of the CROPS project website (www.crops-robots.eu) the publication of scientific papers on journals and to international conferences in the field of agricultural and biosystems engineering, robotics, industrial automation, artificial intelligence, as well as through specialized webpages or technical journals and newspapers or on farmers magazines. Based on the CROPS concepts and initial research activities, several publications were produced within the consortium. Specifically: 25 articles on specialized webpages or in technical journals and newspapers; 10 popularizing articles in farmers magazines and 76 presentations at international scientific conferences. The CROPS project was one of the subjects of a special report “Le serre a Rotterdam” of the science popularizing TV programme “SuperQuark” broadcasted by the Italian network Rai Uno on July 2012 (http://www.rai.tv/dl/RaiTV/programmi/media/ContentItem-afa68735-1a96-4d31-a5a3-5435f5c9cea9.html). CROPS project activity was reported by the science popularizing TV programme “Dobra ura” broadcasted by the Slovenian network RTVSLO on 24 April 2014 (http://4d.rtvslo.si/arhiv/dobra-ura/174272608) and the CROPS project was the main subject of a report in “Regio Nieuws” broadcasted by the Netherland TV network WOS on 4 September 2014 (www.wos.nl/televisie/uitzendinggemist/player/item/20140904-regionieuws-van-donderdag/).
Popularizing description of selected experiments and illustrative videos relevant for demonstrating the main capabilities of developed subsystems and their possible applications have been published on the homepage of CROPS website.
Contacts with the RHEA project, another FP7-project focused on field robotics, have been maintained and strengthend. Six research contributions from CROPS partners were presented at the 2nd RHEA Conference hold in May 2014 in Madrid, and an overall presentation of RHEA project was given at the CROPS Final workshop hold in July 2014 in Zurich.
The CROPS Final workshop was hold in Zurich on July 9th, 2014 as a special session within the AgEng2014 conference, with a world-wide academy attendance and also R&D representatives from main machinery manufacturers. The workshop lasted a whole day, with 16 extended presentations (including one overall presentation from RHEA project) and 4 summary posters discussions. Two of the extended presentations were given by industry partners, namely one by the company Festo (on new passive grippers for agricultural applications) and one by the company FORCE-A (on Fluorescence sensing for ripeness and quality evaluation of specialty crops products).
70 conference papers had been published in refereed proceedings and 12 papers were published in major scientific journals and two additional submissions are under review.
In next future, this number will significantly increase since more CROPS publications are further expected. This is especially in relation to a Special Issue of the journal "Biosystems Engineering" on Advances in agricultural robotics which will have a strong nucleus coming from “CROPS Final workshop” papers, and for which partners of CROPS consortium have been charged as Guest Editors. Nevertheless, the publication of this special issue is scheduled for late spring-summer 2015.
Papers published (or submitted) in scientific journals:
1. Bac, C.W. Henten, E.J. van, Hemming J., Edan, Y., 2014. Harvesting Robots for High-value Crops: State-of-the-art Review and Challenges Ahead. Journal of Field Robotics. DOI: 10.1002/rob.21525
2. Eizicovits D., Berman S. 2014. Efficient sensory-grounded grasp pose quality mapping for gripper design and online grasp planning Robotics and Autonomous Systems. doi.org/ 10.1016/ j.robot.2014.03.011
3. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Borghese A. N. 2014. Automatic detection of powdery mildew on grapevine leaves by image analysis: Optimal view-angle range to increase the sensitivity. Computers and Electronics in Agriculture, 104: 1- 8
4. Hemming, J.; Ruizendaal, J.; Hofstee, J.W.; Henten, E.J. van Henten, 2014. Fruit detectability analysis for different camera positions in sweet-pepper Sensors 14: 6032-6044
5. Bac, C.W. Hemming J., Henten, E.J. van 2014, Stem localization of sweet-pepper plants using the support wire as a visual cue. Computers and Electronics in Agriculture 105: 111–120
6. Hellström T., Ringdahl O. 2013. A software framework for agricultural and forestry robots. Industrial Robot: An International Journal. 40(1): 20-26, 2013
7. Fernandez R., Montes H.,Salinas C., Sarria J., Armada M. 2013. Combination of RGB and multispectral imagery for discrimination of Cabernet Sauvignon grapevine elements. Sensors. 13(6), 7838-7859; doi:10.3390/s130607838
8. Ola Ringdahl O., Hohnloser P., Hellström T., Holmgren J., Lindroos O. 2013. Enhanced algorithms for estimating tree trunk diameter using 2D laser scanner. Remote Sensing. 5: 4839-4856
9. Osterman A., Hočevar M., Godeša, T., Širok B., Stopar M. 2013. Real-time positioning algorithm for variable-geometry air-assisted orchard sprayer. Computers and Electronics in Agriculture. Vol. 98, p. 175-182
10. Bac C.W. Hemming, J., Henten, E. van 2013. Robust pixel-based classification of obstacles for robotic harvesting of sweet-pepper. Computers and Electronics in Agriculture. Vol. 96, p. 148 - 162
11. Ben-Yosef G., Ben-Shahar O. 2012. A tangent bundle theory for visual curve completion. IEEE Transactions on Pattern Analysis and Machine Intelligence. 34, 1263-1280
12. Kapach K. , Barnea E., Mairon R., Edan Y., Ben-Shahar O. 2012. Computer Vision for Fruit Harvesting Robots - State of the Art and Challenges Ahead. International journal of Computational Vision and Robotics. 3, 4-34
13. Bac, C.W. Roorda T., Reshef R., Berman S., Hemming J., Henten E.J. van. 2014. Analysis of a motion planning problem for fruit harvesting in a dense obstacle environment. Submitted to Biosystems Engineering.
14. Bac C.W. Hemming J., Tuijl B. van, Barth R., Wais E., Henten E. van. 2014. Performance evaluation of a harvesting robot for sweet-pepper. Submitted to Journal of Autonomous Robots.
Presentations and published conference proceedings:
1. Barth R., Baur J., Buschmann T., Edan Y., Hellström T., Ringdahl O., Salinas C., Vitzrabin E. 2014. Using ROS for agricultural robotics - design considerations and experiences. Proc. of 2nd International RHEA Conference
2. Cohen Y., Berman S. 2014. Integrating simulation with robotic learning from demonstration. Proceedings of the 28th EUROPEAN CONFERENCE ON MODELLING AND SIMULATION (ECMS)
3. Hellström T., Ostovar A. 2014. Detection of trees based on quality guided image segmentation. Proc. of 2nd International RHEA Conference
4. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Hočevar M., Baur J., Pfaff J., Schütz C, Ulbrich, H. 2014. Crops agricultural robot: application to selective spraying of grapevine’s diseases. Proc. of 2nd International RHEA Conference
5. Osterman A., Godeša, Hočevar, M., T., Širok, B. 2014. Monitoring of growth of fruit trees during growing season with lidar. Proc. of 2nd International RHEA Conference
6. Osterman A., Godeša, Hočevar, M., T., Širok, B. 2014. Unilateral characterization of tree canopies in orchards with lidar. Proc. of 2nd International RHEA Conference
7. Reshef R., Eizicovits D., Berman S. 2014. Path planning of grasp-aimed robotic tasks using rapid-exploring random trees. Proc. of 2nd International RHEA Conference
8. Ben-Shahar O., Barnea E. 2014. Depth Based Object Detection from Partial Pose Estimation of Symmetric Objects. Proc. of European Conference on Computer Vision (ECCV)
9. Hemming J., Tuji van B., Gauchel W. 2014. Field Test of Different End-effectors for Robotic Harvesting of Sweet-pepper. Proc. of International Horticultural congress
10. Cohen A.H. Berman S. 2014. Motor control variables and dynamic motion primitives. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop on Compliant manipulation
11. Elkoby Z., Ooster B. van, Edan Y. 2014. Simulation Analysis of Sweet Pepper Harvesting Operations. Proc. of the Advances in Production Management Systems Conference (APMS)
12. Ben-Shahar O., Mairon R. 2014. A Closer Look at Context: From Coxels to the Contextual Emergence of Object Saliency. European Conference on Computer Vision (ECCV)
13. Montes H., Fernandez R., Salinas C., Sarria J., Armada M. 2014. Sistema Multisensorial para Aplicaciones en Agricultura de Precisión . Proc. of the II International Conference of Science and Technology for Sustainable Development (CICyTDS)
14. Bontsema J., Hemming J., Pekkeriet E., Saeys W., Edan Y., Shapiro A., Hočevar M., Hellström T., Oberti R., Armada M., Ulbrich H., Baur J., Debilde B., Best S., Evain S., Münzenmaier A.,Ringdahl O., 2014. CROPS: high tech agricultural robots. Proc. of the International Conference of Agricultural Engineering AgEng 2014
15. Pekkeriet E., Saeys W., Hemming J., Bontsema J., Hočevar M., Oberti R. 2014. CROPS Evaluation of economic viability, social aspects, sustainability and exploitation of robotic systems. Proc. of the International Conference of Agricultural Engineering AgEng 2014
16. Schuetz C., Pfaff J., Baur J., Buschmann T., Ulbrich H. 2014. A modular robot system for agricultural applications. Proc. of the International Conference of Agricultural Engineering AgEng 2014
17. Baur J., Schuetz C., Pfaff J., Buschmann T., Ulbrich H. 2014. Path Planning for a Fruit Picking Manipulator. Proc. of the International Conference of Agricultural Engineering AgEng 2014
18. Hemming J., Bac W., Tuijl B. van, Barth R., Bontsema J., Pekkeriet E., Henten E. van 2014. A robot for harvesting sweet-pepper in greenhouses. Proc. of the International Conference of Agricultural Engineering AgEng 2014
19. Eizicovits D., Berman S. 2014. Grasp quality measures based on point cloud input. Proc. of the International Conference of Agricultural Engineering AgEng 2014
20. Osterman A., Godeša T., Malnersic A., Hočevar M., 2014. Moving spraying arm with internal airflow for air-assisted orchard sprayer. Proc. of the International Conference of Agricultural Engineering AgEng 2014
21. Nguyen T.Th. Vandevoorde K., Kereszte J., Kayacan E., De Baerdemaeker J, Saeys W. 2014. Apple detection algorithm for robotic harvesting using a RGB-D camera. Proc. of the International Conference of Agricultural Engineering AgEng 2014
22. Berenstein R., Edan Y., Godeša T., Hočevar M. 2014. Image registration for agriculture tasks . Proc. of the International Conference of Agricultural Engineering AgEng 2014
23. Beers van R., León L., Schenk A., Nicolaï B.,Kayacan E., Saeys W. 2014. Optical measurement techniques for the ripeness determination of Braeburn apples. Proc. of the International Conference of Agricultural Engineering AgEng 2014
24. Osterman A., Godeša T., Hočevar M., 2014. LIDAR-based control of automated orchard sprayer. Proc. of the International Conference of Agricultural Engineering AgEng 2014
25. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Hočevar M., Baur J., Pfaff J., Schütz C, Ulbrich, H. 2014. Selective precision spraying of grapevine’s diseases by CROPS robot platforms. Proc. of the International Conference of Agricultural Engineering AgEng 2014
26. Klosok M. 2014. Passive, adaptive grippers as end-effectors for agricultural applications . Final CROPS Workshop at the International Conference of Agricultural Engineering AgEng 2014
27. Laurent F., Evain L. 2014. Fluorescence sensors for CROPS. Final CROPS Workshop at the International Conference of Agricultural Engineering AgEng 2014
28. Nguyen T.Th. Kayacan E., De Baerdemaeker J, Saeys W. 2014. Motion planning algorithm and its real-time implementation in apples harvesting robot. Proc. of the International Conference of Agricultural Engineering AgEng 2014
29. Barnea E., Ben-Shahar O. 2014. Depth Based Fruit Detection from Partial Pose Estimation using Symmetry. Proc. of the International Conference of Agricultural Engineering AgEng 2014
30. Ohev-Zion A., Shapiro A. 2014. Stability of Compliant Planar Robotics Grasp with Application to Fruits Grasping.. Proc. of the International Conference of Agricultural Engineering AgEng 2014
31. Oberti R., Marchi M., Tirelli P., Vitzrabin E., Edan Y. 2014. Sensor fusion of multispectral and hyperspectral imaging: preliminary analysis of disease detection in grapevine. Proc. of the International Conference of Agricultural Engineering AgEng 2014
32. Pfaff J., Baur J., Schuetz C., Buschmann T., Ulbrich H. 2014. Design of drive units for agricultural robots. Proc. of the International Conference of Agricultural Engineering AgEng 2014
33. Baur J., Pfaff J., Schuetz C. & Ulbrich H. 2013. Dynamic modeling and realization of an agricultural manipulator. Proceedings of XV International Symposium on Dynamic Problems of Mechanics.
34. Bac, C.W. Hemming, J., Henten, E.J. van. 2013. Classification of sweet-pepper plant parts using multi-spectral imaging. Proceedings of the 2013 IFAC Biorobotics conference.
35. Cohen Y., Berman S. 2013. Tight Dynamic Movement Primitives for Complex Trajectory Generation. Proc. of 2013 IEEE International Conference on Systems, Man and Cybernetics.
36. Henten E.J. Van, Bac C.W. Hemming, J., Edan Y. 2013. Robotics in protected cultivation. Proc of IFAC Agricontrol 2013 Conference, p. 170-176
37. Nguyen T.Th. Kayacan E., De Baerdemaeker J, Saeys W. 2013. Task and Motion Planning for Apple Harvesting. Proc. of IFAC Agricontrol 2013 Conference. p. 247-252
38. Tuijl van B., Wais E., Edan Y. 2013. Methodological design of an end-effector for a horticultural robot. Proc. of 4th Israeli Conference on Robotics.
39. Oberti R., Marchi M., Tirelli P., Calcante A., Iriti M., Hočevar M., Baur J., Pfaff J., Schütz C. 2013. Selective spraying of grapevine’s diseases by a modular agricultural robot. Proc. of AIIA 2013 - Italian Conference of Agricultural Engineering.
40. Hershkovitz Cohen A., Berman S. 2013. Path Planning of Manipulator for Harvesting using DMP. Proc. of 4th Israeli Conference on Robotics.
41. Vitzrabin E., Edan Y. 2013. Apple detection using multi-dimensional adaptive thresholding with multi-resolution windows. Proc. of 4th Israeli Conference on Robotics.
42. Eizicovits D., Berman S. 201 . Constructing successful grasps based on Graspability maps . Proc. of 4th Israeli Conference on Robotics.
43. Rehsef R., Berman S. 2013. Path smoothing for the RRT motion planning algorithm. Proc. of 4th Israeli Conference on Robotics.
44. Fernandez R., Salinas C., Montes H., Sarria J., Armada M. 2013 . Validation of a multisensory system for fruit harvesting robots in lab conditions. Proc. of ROBOT 2013: First Iberian Robotics Conference . in press
45. Baur J., Pfaff J., Schuetz C., Ulbrich H. 2013. Dynamic modeling and realization of an agricultural manipulator. DINAME 2013 - Proceedings of the XV International Symposium on Dynamic Problems of Mechanics .
46. Gauchel, W. 2013. Flexible und adaptive Greifer zum Ernten von hochwertigen Feldfrüchten. Technikforum Roboterinnovationen 2020, 41541
47. Van Beers R., Aernouts B., De Baerdemaeker J., Saeys W. 2013. Apple ripeness detection using Hyperspectral Laser Scatter Imaging. Proc. of Sensing Technologies for Biomaterial, Food, and Agriculture (SeTBio '13). Proc. of SPIE Vol. 8881, 88810K
48. A.Ohev-Zion A.Miler A.Sintov A.Shapiro. 2012. Experimental Validation of Compliant Contact Model. 2012 Israeli Conference on Mechanical Engineering.
49. Hellström, T., Ringdahl O. 2012 . A software framework for agricultural and forestry robotics . Proceedings of the first International RHEA Conference , p. 171-176
50. Gauchel W. , Saller S. 2012. Adaptive gripper jaws for high-value crops harvesting. Proc. of 8th International Conference on Fluid Power.
51. Oberti R. , Tirelli P. , Marchi M. , Calcante A. , Iriti M., Borghese N.A. 2012. Automatic diseases detection in grapevine under field conditions. International Conference of Agricultural Engineering CIGR-AgEng.
52. Baur J., Pfaff J., Ulbrich H., Villgrattner T. 2012. Design and development of a redundant modular multipurpose agricultural manipulator. Proc. of 2012 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.
53. Pfaff J., Baur J., Ulbrich H., Villgrattner T. 2012. Development of a multipurpose agricultural manipulator. Proceedings of the first International RHEA Conference .
54. Eizicovits D., Yaacobovich M., Berman S. 2012. Discrete fuzzy grasp affordance for robotic manipulators. IFAC Symposium on Robot Control, SYROCO, Sept 5-7, 2012
55. Hemming, J.; Ruizendaal, J.L.; Hofstee, J.W.; Henten, E.J. van, 2012. Fruit detectability analysis for different camera positions in sweet-pepper. Proceedings of the International conference of agricultural engineering, CIGR-AgEng2012, Valencia, Spain, 8-12 July 2012.
56. Yaacobovich M., Eizicovits D., Berman S. 2012. Grasp affordance for robotic selective harvesting based on human demonstrations. International Conference of Agricultural Engineering CIGR-AgEng, July 8-12, 2012.
57. Mostafa Pordel, Thomas Hellström, and Ahmad Ostovar2012. Integrating Kinect Depth Data with a Stochastic Object Classification Framework for Forestry Robots.
58. Tirelli P. , Marchi M. , Calcante A. , Vitalini S., Iriti M., Borghese N.A. Oberti R. 2012. Multispectral image analysis for grapevine diseases automatic detection in field conditions. Proceedings of the first International RHEA Conference, p. 101-106.
59. Montes H., Fernandez R., Salinas C., Armada M. 2012. Robotic multisensory system for precision agriculture applications. Proc. of 15th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines - CLAWAR2012, p. 731-738
60. Hočevar, M., Godeša, T., Širok, B., Malneršič, A., Jejčič, V. 2012. Canopy optimised sprayer development within CROPS EU project. Proceedings of the first International RHEA Conference .
61. Malneršič, A, Hočevar, M., Širok, B., Marchi M., Tirelli, P., Oberti, R. 2012. Close range precision spraying airflow /plant interaction. Proceedings of the first International RHEA Conference .
62. Malneršič, A, Hočevar, M., Širok, B. 2012. Interakcija med zračnim tokom in rastlino pri ciljnem pršenju/Interaction between airflow and plant at close range precision spraying Proc. of the Kuhljevi dnevi conference, 2012.
63. Nguyen T.Th. Van Essen D., De Baedermaeker J., Saeys W. 2012. Optimum detaching movement for apples harvesting robot. International Conference of Agricultural Engineering CIGR-AgEng, July 8-12, 2012.
64. Berenstein, R., Edan, Y. 2012 . Human-robot cooperative precision spraying: collaboration levels and optimization fuction . Proc. 10th IFAC Symposium on Robot Control (SYROCO 2012) . p. 799- 803
65. Thanh Nguyen T., Van Essen, D., De Baerdemaeker J., Saeys W. 2012 . Optimum detaching movement for apples-harvesting robot . Proceedings of the International conference of agricultural engineering, CIGR-AgEng2012, Valencia, Spain, 8-12 July 2012 .
66. Oberti, R., Tirelli, P., Marchi, M., Iriti, M., Borghese, A. 2012 . Automatic diseases detection in grapevine under field conditions . Proceedings of the first International RHEA Conference .
67. Berenstein, R., Edan, Y. 2012 . Robotic precision spraying methods . Proc. ASABE Annual International Meeting - Dallas, Texas, July 29-August 1, 2012 .
68. Yaacobovich M., Bechar A., Berman S. 2011 . Environment effect on grasping: case study - peppers harvesting . 7th Computational Motor Control Workshop (CMCW) - Beer-Sheva (Israel)
69. Kravits M., Shapiro A., Berman S. 2011 . Parametric Compliant contact model for human finger . 7th Computational Motor Control Workshop (CMCW) - Beer-Sheva (Israel)
70. Pomeranz D., Shemesh M., Ben-Shahar O. 2011 . A fully automated greedy square jigsaw puzzle solver . Proceedings of the IEEE conference on Computer Vision and Pattern Recognition - CVPR 2011 . 9-16
71. Adato Y., Zickler T., Ben-Shahar O. 2011 . A Polar Representation of Motion and Implications for Optical Flow . Proceedings of the IEEE conference on Computer Vision and Pattern Recognition - CVPR 2011 . 1145-1152
72. Malneršič A., Hočevar M. 2011 . Design of aerodynamic actuator for precision target spraying in orchards and measuring of flow parameters . Kuhljevi Dnevi 2011 .
73. Salinas C., Fernandez R., Montes H., Armada M. 2011 . Sensory system evaluation methodology for automatic fruits harvesting . Proc. of 14th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines - CLAWAR2011 . 801-808
74. Shemesh M., Ben-Shahar O. 2011 . Free Boundary Conditions Active Contours with Applications for Vision . Proc. of International Symposium on Visual Computing .
75. Ohev-Zion A., Shapiro A. 2011 . Grasping of Deformable Objects Applied to Organic Produce . Proc. of TAROS 2011, LNAI 6856 . May 2011, 396–397
76. Bontsema, J. 2011. CROPS: Clever Robots for CROPS. 5th Workshop of the MANUFUTURE AET- community – Agricultural Engineering Strategies for EU FP8, Hanover (Germany)
Technical and popularizing CROPS publications:
1. I robot tra I filari e negli orti. ItaliaOggi. 2014, May, 28, p. 20
2. Darbouret, A., Evain, S. Les robots des champs. Le Journal du Dimanche . 3 mars 2013
3. Darbouret, A., Evain, S. Les robots des champs. http://www.lejdd.fr/Societe/Sciences/Actualite/Les-robots-des-champs-594543
4. Oberti R., Iriti M. Nel vigneto del futuro la difesa affidata ai robot. Corriere vinicolo . 2013, March 4
5. Hemming, J. Oogstrobot voor paprika’s in najaar de kas in. Nederlandse groententeelt. http://www.groentennieuws.nl/nieuwsbericht_detail.asp?id=94975
6. Hemming, J. Prototype of greenhouse harvesting robot almost able to operate independently. http://www.hortidaily.com/article/1455/Prototype-of-greenhouse-harvesting-robot-almost-able-to-operate-independently
7. Hemming, J. Der eiserne Gärtner, Bildanalyseaufgaben im EU-Forschungsprojekt CROPS. Inspect. May 2012
8. Tinker, D. EU funded research: CROPS and RHEA . EurAgEng Newsletter. Winter 2011/2012, p. 4-5
9. Hočevar, M., Godeša, T., Širok, B., Canopy optimised sprayer design. National Instruments case study
10. Hočevar, M., Godeša, T., Širok, B., Pršilnik za krošnjam dreves prilagojeno škropljenje/Canopy optimised sprayer design. NI Days conference, Ljubljana 2012
11. Hemming, J. Prototype van een oogstrobot getest in paprika. http://www.glastuinbouw.wur.nl/NL/nieuwsagenda/nieuws/prototypeoogstrobotpaprika_.htm .
12. Markus Keßler. Roboter-Schwärme bewirtschaften Bauernhöfe (interview with J. Bontsema) . http:/ /www.pressetext.com/news/20121102003 .
13. Agricultural robot will be able to harvest high-value crops. June 2011. www.theengineer.co.uk
14. Sikkema, A., Bontsema, J. Bell pepper picking robot in the making. www.resource.nl
15. Dodde, H. WUR coöordineert roboticaproject. November 2010, www.nieuweoogst.nu
16. Bontsema, J. Slimme robot herkent en oogst paprika's. June 2011. www.agentschapnl.nl
17. CROPS - An agricultural robot that can pick the ripest fruit. April 2011. www.smashingrobots.com
18. Bontsema, J., Jongebloed, P. Wageningen UR Glastuinbouw coordinates international robotics project. November 2010. www.hortist.com
19. Paprikabot. De Ingenieur.Vol. 123, nr. 16, Oct 14, 2011
20. The Marker. Crops. 2011, March 23
21. Bontsema J. Pekkeriiet E. Robothand gaat paprika oogsten. Kennis Online. Vol. 8, December 2011, p. 11.
22. Bontsema J. Tien miljoen euro voor onderzoek robots. De Gelderlander. November 9th, 2010
23. Two large-scale integrating FP7- EU projects: CROPS and RHEA. ICT Agri Newsletter. December 2011, p. 5
24. Tinker, D. EU funded research: CROPS and RHEA. EurAgEng Newsletter. Winter 2011/2012, p. 4-5
25. Thoenes, E., Bontsema, J. Hard en hygiëniisch werken: robots in de kas. J.Sync.nl April 18th, 2011, p. 1-4
Agricultural and popularizing publications follows:
1. I robot tra I filari e negli orti. ItaliaOggi. 2014, May, 28, p. 20
2. Staalduinen, J. van; Bontsema, J. ; Hemming, J.. Tussenbalans Europees CROPS project: Oogstrobot voor paprika krijgt stap voor stap vorm. Onder Glas 10 (2013)4. - p. 20 - 21. 10 (2013)4. - p. 20 - 21.
3. Oberti R., Iriti M.. Nel vigneto del futuro la difesa affidata ai robot. Corriere vinicolo. 2013, March 4.
4. Bontsema, J., Hemming, J.. Crops Internationale Robotica. Glastuinbouwtechniek Magazine. Vol. 6, Oct. 2011, 28-29.
5. Bontsema, J.. Nieuw internationaal roboticaproject CROPS. Onder Glas. Vol. 8, March 2011 ,p. 59.
6. Thoenes, E., Bontsema, J. Robots in de kas. Groenten en Fruit Magazine. 2011, 2, p. 20-22
7. Bontsema, J.. Robots on the march. HDC News. June 2011, p. 25.
8. Verso il robot agricoltore. ItaliaOggi. 2011, April, 16.
9. Oogstrobot paprika in de maak. Agrarisch Dagblad. 2010, Nov 5, p. 11
10. Bontsema, J.. Waarom kan een robot niet wat de mens wel kan?. Vakblad voor de Bloemisterij. Vol. 65, Nr. 46, 19 November 2010, p.6.
Exploitation of results
Two patents, one on an end-effector for sweep pepper harvesting and one on an end-effector for apple harvesting have been filed by respectively partner Stichting Dienst Landbouwkundig Onderzoek and Katholieke Universiteit Leuven.
The research on the sweet pepper harvesting robot has been continued within in a new Horizon 2020 project SWEEPER (GA 644313, www.sweeper-robot.eu). The partners from CROPS, Stichting Dienst Landbouwkundig Onderzoek, Ben-Gurion University of the Negev and Umea Universitet are involved in this new project, together with a greenhouse research station from Belgium (Proefstation voor de Groenteteelt), a system integrator from the Netherlands (IRMATO) and a commercial sweet pepper grower from the Netherlands (De Tuindershoek). The project duration is three years. Associated to the new project is a grower’s advisory board consisting of two Dutch and two Belgium sweet pepper growers, to insure that the new project has commitment of the sweet pepper grower’s community.
The CROPS industrial partner, the German company Festo, has with the experience with one of the end-effectors for the sweet pepper harvesting based on the Finray fingers, made a new commercial device for the food industry. The CROPS industrial partner, the French company Force-A, has extended their handheld ripeness sensor with an optical fibre, so that this device can now also be used on the tractor.
List of Websites:
The project is co-ordinated by Wageningen UR Greenhouse Horticulture.
For general questions please contact:
Dr. Jan Bontsema
E-mail: jan.bontsema@wur.nl
Phone: +31 317-486390
Fax: +31 317-423110
Visiting address
Droevendaalsesteeg 1
6708 PB Wageningen
The Netherlands
Postal address
P.O. Box 644
6700 AP Wageningen
The Netherlands
For other information you may also contact: webmaster@crops-robots.eu
