Skip to main content

Aerial Data Collection and Analysis, and Automated Ground Intervention for Precision Farming

Deliverables

Weed Detection and Predictive Tracking

This deliverable consists of a working prototype for the weed detection and tracking software, running on the Bonirob platform with live data-streams.

Data Analysis and Interpretation

"• A software module for converting the raw data gathered by the UAV sensors into tested reliable vegetation indices and plant growth and vitality indicators. • A validation of the indices calculated using the UAV sensors with ""ground truth"" from the FIP. • Correlations between vegetation indices, sensor data and farm inputs along with derivation methodology. The initial version in M15 will work with the maps delivered in D2.2 at M15. The final version will work with the updated maps delivered in M35."

Development Repository

A web-based repository for the software developments with access for all partner will be setup. It will furthermore support bug tracking and project-specific wiki pages.

Dissemination and Exploitation Report

A report about achieved dissemination activities and the exploitation activities.

Long-term Navigation Optimization Module

A module that will use the experience from previous operations in the same field, to predict likely conditions, in particular possible slippage or ground softness. If possible, and when enough data will be available, this prediction could account for the weather reports and forecast. The performance of the terrain condition prediction will be assessed by integrating it in the control and planning framework and observing if and how it makes a difference.

Hardware and Software Specification

This deliverable will consist of the software specification of the new components in Flourish and those components stemming from the existing UAV and UGV platforms that need to be adapted. A second part of the deliverable will consist of the hardware modifications of the platforms, including adaptation of the existing BoniRob platform to meet the precise ground intervention requirements, as well as sensors and hardware modification needed for the aerial survey task.

Integrated Perception and Weed Treatment

This deliverable consist of a demonstration of the integration of weed detection and tracking with both treatment modules: the mechanical weed treatment module and and the selective spraying module.

Initial Data Acquisition

This deliverable will consist of log files recorded from the initial field tests.

Use-Case Analysis and Requirements

This deliverable will consist of the use-case specification of the new to develop components in Flourish and the mission scenarios that will be carried out during the field tests.

Integrated System Ready and Components Available

A report about the conducted integration activities, problems and resolutions as well as a summary of the integration weeks and related activities.

Local Collision Avoidance

A software module that computes a local obstacle map and system-compliant motions that are safe, collision free, and closely follow the mission path delivered by the module from Task 3.1.

Coordinated UGV and UAV Operations

This deliverable will provide the framework for coordinated operation of the UGV and UAV operations, making sure they successfully achieve their individual missions while being able to rendezvous or synchronize when required.

Second Periodic Report

2nd periodic report for the project.

UGV Environment Modeling

The deliverable consists of a report about the UGV mapping system. The source code for the optimisation system will be released as open source software and will be shared among the partners.

Data Management Plan

This document will outline the project’s data management plan. It will describe the types of data generated by the project, what standards will be used when publishing the data, how the data will be made available to third parties, and how the data will be preserved during and beyond the lifetime of the project.

Integration Plan

Integration plan showing how the new to develop components will integrate among each other and with the existing modules.

Third Periodic Report

Third periodic report for the project.

End-User Evaluation Report

A report on the evaluation of the system conducted by the end user. This report should drive future system specification.

Database and User Interface Development

This deliverable will consist of the development of the database and the user interface. Database deliverables include a database backend and a visual web fronted that visualize key metrics as specified. User interface deliverables include a functional fronted that communicates with the robotics system through the specified API.

Hardware Setup and Modification

This deliverable will consist of hardware modification to the UAV and UGV platforms. The modifications include changes specified in D1.3.

UAV Localization

A UAV localization module which gives robust and accurate estimates of the UAV position and orientation on the field using a combination of the on-board sensor data and communication with the UGV.

Weed Treatment Modules

This deliverable consists of a working prototype for selective spraying module and the mechanical weed treatment module, integrated on the Bonirob platform.

UAV Environment Modeling

A software module for combining the gathered sensor data with the output of the localization module into time-stamped, geolocated maps which are dynamically updated with every UAV mission.The final version will be delivered in M25.

Cooperative Environment Modeling

This deliverable will provide the results of joint UAV/UGV environment modeling.

Navigation Interface

This deliverable will consist of the interface specification for pushing navigation commands to the UGV.

Planner Module(s)

A module that publishes a planned trajectory as a list of x,y,angle poses and a module that creates motion commands for the robot to drive along these way-points. The two modules might also be combined into one.

Adaptive Mission Planning

A software module that takes coverage history and areas of interest as inputs from WP2 and plans a sequence of UAV missions, taking into account the current battery status the current map.

Model Update After Local Treatment

This deliverable consist of a working prototype for the spraying application.

First Periodic Report

First periodic report for the project.

Database and User Interface Specification

The deliverable will consist of the software specification for the database 7.1). Key metrics and interaction requirements must be specified at this time. Additionally the key user interactions, API/topics for communication, and required features of the user interface 7.4) should be specified.

Traversability Analysis Module

A module that publishes a local map with annotations if a cell is traversable by the UGV or not, according to its specifications.

UGV-based crop and weed detection

The deliverable consists of a report about the crop and weed detection system including the evaluation. In addition to that, the source code of the system will be shared among the partners. The intermediate deliverable at M15 will have limited functionality.

Localization Module

A UGV localization module which publishes both global poses (on the field or in a GPS frame) and precise local poses regarding plants/rows on the field.

Integration Evaluation Report

A report on the conducted evaluation of the system and its performance as well as a summary of the (integration and) testing weeks and related activities.

Project web-page

The website including a public document database will be operational from month 2 on. It will be updated continuously during the project

Press Video

A press video explaining the aims of the project, the scientific results, and the impact to society in plain language.

Brochure, Newsletter, and Knowledge Management Report

A nicely laid out brochure at M30 to inform the public about the project and its main objectives and content. Public newsletters will be send bi-annually from M24 on to inform the public about the Flourish project. A knowledge management report will be provided to keep internal and external experts informed about new knowledge or patents created in the project.

Searching for OpenAIRE data...

Publications

Studying Phenotypic Variability in Crops using a Hand-held Sensor Platform.

Author(s): Raghav Khanna, Joern Rehder, Martin Möller, Enric Galceran and Roland Siegwart
Published in: Proceedings of the IROS Workshop on Agri-Food Robotics, Issue 2015, 2015

Online Informative Path Planning for Active Classification on UAVs

Author(s): Marija Popovic, Gregory Hitz, Juan Nieto, Roland Siegwart, Enric Galceran
Published in: ational Conference on Robotics and Automation (ICRA), Issue 2016, 2016

Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture

Author(s): Ciro Potena, Daniele Nardi and Alberto Pretto
Published in: Proceedings of the 14th International Conference on Intelligent Autonomous Systems, Issue 2016, 2016

Towards automatic UAV data interpretation for precision farming

Author(s): Johannes Pfeifer, Raghav Khanna, Dragos Constantin, Marija Popovic, Enric Galceran, Norbert Kirchgessner, Achim Walter, Roland Siegwart, Frank Liebisch
Published in: International Conference of Agricultural Engineering 2016, Issue 2016, 2016

Flourish – A robotic approach for automation in crop management

Author(s): Frank Liebisch, Johannes Pfeifer, Raghav Khanna, Philipp Lottes, Cyrill Stachniss, Tillmann Falck, Slawomir Sander, Roland Siegwart, Achim Walter Enric Galceran
Published in: 22. Workshop Computer-Bildanalyse und Unbemannte autonom fliegende Systeme in der Landwirtschaft, Issue 2016, 2016

Fast and effective online pose estimation and mapping for UAVs

Author(s): Johannes Schneider, Christian Eling, Lasse Klingbeil, Heiner Kuhlmann, Wolfgang Forstner, Cyrill Stachniss
Published in: 2016 IEEE International Conference on Robotics and Automation (ICRA), Issue 2016, 2016, Page(s) 4784-4791
DOI: 10.1109/ICRA.2016.7487682

An effective classification system for separating sugar beets and weeds for precision farming applications

Author(s): P. Lottes, M. Hoeferlin, S. Sander, M. Muter, P. Schulze, Lammers C. Stachniss
Published in: 2016 IEEE International Conference on Robotics and Automation (ICRA), Issue 2016, 2016, Page(s) 5157-5163
DOI: 10.1109/ICRA.2016.7487720

Joint Stem Detection and Crop-Weed Classification for Plant-specific Treatment in Precision Farming

Author(s): Lottes, Philipp; Behley, Jens; Chebrolu, Nived; Milioto, Andres; Stachniss, Cyrill
Published in: International Conference on Intelligent Robots and Systems (IROS), Issue 1, 2018

Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs

Author(s): Milioto, Andres; Lottes, Philipp; Stachniss, Cyrill
Published in: International Conference on Robotics and automation (ICRA), Issue 11, 2018

Deep Auxiliary Learning for Visual Localization and Odometry

Author(s): Valada, Abhinav; Radwan, Noha; Burgard, Wolfram
Published in: International conference on Robotics and Automation (ICRA), Issue 3, 2018

Multi-agent Time-based Decision-making for the Search and Action Problem

Author(s): Miki, Takahiro; Popovic, Marija; Gawel, Abel; Hitz, Gregory; Siegwart, Roland
Published in: International Conference on Robotics and Automation (ICRA), Issue 4, 2018

Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs

Author(s): Milioto, Andres; Stachniss, Cyrill
Published in: International Conference on Robotics and Automation (ICRA), Issue 4, 2018

Non-linear model predictive control with adaptive time-mesh refinement

Author(s): Ciro Potena, Bartolomeo Della Corte, Daniele Nardi, Giorgio Grisetti, Alberto Pretto
Published in: 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), 2018, Page(s) 74-80
DOI: 10.1109/SIMPAR.2018.8376274

Design of an Autonomous Racecar: Perception, State Estimation and System Integration

Author(s): Valls, Miguel de la Iglesia; Hendrikx, Hubertus Franciscus Cornelis; Reijgwart, Victor; Meier, Fabio Vito; Sa, Inkyu; Dubé, Renaud; Gawel, Abel Roman; Bürki, Mathias; Siegwart, Roland
Published in: International Conference on Robotics and Automation (ICRA), Issue 1, 2018

The ETH-MAV Team in the MBZ International Robotics Challenge

Author(s): Bähnemann, Rik; Pantic, Michael; Popvić, Marija; Schindler, Dominik; Tranzatto, Marco; Kamel, Mina; Grimm, Marius; Widauer, Jakob; Siegwart, Roland; Nieto, Juan
Published in: arXiv, Issue 1, 2018

Flourish-A robotic approach for automation in crop management

Author(s): Achim Walter, Raghav Khanna, Philipp Lottes, Cyrill Stachnis, Roland Siegwart, Juan Nieto, Frank Liebisch
Published in: International Conference on Precision Agriculture, Issue 06, 2018

Investigation of ground based and airborne spectral information for Nitrogen fertilizer application optimization in sugar beet

Author(s): Corinne Müller-Ruh, Frank Liebisch, Johannes Pfeifer, Achim Walter
Published in: Bornimer Agrartechnische Berichte, Issue 90, 2017, ISSN 0947-7314

La robotica autonoma al servizio dell'agricoltura di precisione: primi risultati di classificazione automatica delle infestanti nel progetto Flourish

Author(s): Ciro Potena, Marco Imperoli, Alberto Pretto, Daniele Nardi, Simona Talevi and Sandro Nardi
Published in: Giornate Fitopatologiche, Issue 2016, 2016, Page(s) 641-650

D2CO: Fast and Robust Registration of 3D Textureless Objects using the Directional Chamfer Distance

Author(s): Marco Imperoli, Alberto Pretto
Published in: D2CO: Fast and Robust Registration of 3D Textureless Objects using the Directional Chamfer Distance, 2015, Page(s) 316-328
DOI: 10.1007/978-3-319-20904-3_29

Mapping and Localization using Multispectral Imaging of the Soil.

Author(s): Stefan Glaser, Alexander Schaefer and Wolfram Burgard
Published in: International Conference on Intelligent Robots and Systems (IROS) Workshop, Unconventional Sensing and Processing for Robotic Visual Perception, 2018

Multiresolution Mapping and Informative Path Planning for UAV-based Terrain Monitoring




On field radiometric calibration for multispectral cameras

Author(s): Raghav Khanna, Inkyu Sa, Juan Nieto, Roland Siegwart
Published in: 2017 IEEE International Conference on Robotics and Automation (ICRA), Issue 2017, 2017, Page(s) 6503-6509
DOI: 10.1109/ICRA.2017.7989768

UAV-based crop and weed classification for smart farming

Author(s): Philipp Lottes, Raghav Khanna, Johannes Pfeifer, Roland Siegwart, Cyrill Stachniss
Published in: 2017 IEEE International Conference on Robotics and Automation (ICRA), Issue 2017, 2017, Page(s) 3024-3031
DOI: 10.1109/ICRA.2017.7989347

Only Look Once, Mining Distinctive Landmarks from ConvNet for Visual Place Recognition




Efficient path planning for mobile robots with adjustable wheel positions

Author(s): Freya Fleckenstein, Christian Dornhege, Wolfram Burgard
Published in: 2017 IEEE International Conference on Robotics and Automation (ICRA), Issue 2017, 2017, Page(s) 2454-2460
DOI: 10.1109/ICRA.2017.7989286

Dynamic System Identification, and Control for a cost effective open-source VTOL MAV

Author(s): Inkyu Sa, Mina Kamel, Raghav Khanna, Marija Popović, Juan Nieto, Roland Siegwart
Published in: Proceedings of the Field of Service Robotics (FSR), Issue 2017, 2017, ISSN 1610-7438

Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection




Real-time blob-wise sugar beets vs weeds classification for monitoring fields based on Convolutional Neural Networks

Author(s): Andres Milioto, Philipp Lottes, and Cyrill Stachniss
Published in: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Issue 2017, 2017

Effective Target Aware Visual Navigation for UAVs

Author(s): Ciro Potena, Daniele Nardi and Alberto Pretto
Published in: European Conference on Mobile Robots (ECMR) 2017, Issue 2017, 2017

Semi-Supervised Online Visual Crop and Weed Classification in Precision Farming Exploiting Plant Arrangement

Author(s): Philipp Lottes and Cyrill Stachniss
Published in: Issue 2017, 2017

Field Coverage and Weed Mapping by UAV Swarms

Author(s): Dario Albani, Daniele Nardi, and Trianni Vito
Published in: Issue 2017, 2017

A Low-Cost System for High-Rate, High-Accuracy Temporal Calibration for LIDARs and Cameras




Closed-Form Full Map Posteriors for Robot Localization with Lidar Sensors

Author(s): Lukas Luft, Alexander Schaefer, Tobias Schubert and Wolfram Burgard
Published in: International Conference on Intelligent Robots and Systems (IROS), 2017, Issue 2017, 2017

From Plants to Landmarks: Time-invariant Plant Localization that uses Deep Pose Regression in Agricultural Fields




On the Accuracy of Dense Fisheye Stereo

Author(s): Johannes Schneider, Cyrill Stachniss, Wolfgang Forstner
Published in: IEEE Robotics and Automation Letters, Issue 1/1, 2016, Page(s) 227-234, ISSN 2377-3766
DOI: 10.1109/LRA.2016.2516509

weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming

Author(s): Inkyu Sa, Zetao Chen, Marija Popovic, Raghav Khanna, Frank Liebisch, Juan Nieto, Roland Siegwart
Published in: IEEE Robotics and Automation Letters, Issue 3/1, 2018, Page(s) 588-595, ISSN 2377-3766
DOI: 10.1109/lra.2017.2774979

Fully Convolutional Networks With Sequential Information for Robust Crop and Weed Detection in Precision Farming

Author(s): Philipp Lottes, Jens Behley, Andres Milioto, Cyrill Stachniss
Published in: IEEE Robotics and Automation Letters, Issue 3/4, 2018, Page(s) 2870-2877, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2846289

WeedMap: A Large-Scale Semantic Weed Mapping Framework Using Aerial Multispectral Imaging and Deep Neural Network for Precision Farming

Author(s): Inkyu Sa, Marija Popović, Raghav Khanna, Zetao Chen, Philipp Lottes, Frank Liebisch, Juan Nieto, Cyrill Stachniss, Achim Walter, Roland Siegwart
Published in: Remote Sensing, Issue 10/9, 2018, Page(s) 1423, ISSN 2072-4292
DOI: 10.3390/rs10091423

An Effective Multi-Cue Positioning System for Agricultural Robotics

Author(s): Marco Imperoli, Ciro Potena, Daniele Nardi, Giorgio Grisetti, Alberto Pretto
Published in: IEEE Robotics and Automation Letters, Issue 3/4, 2018, Page(s) 3685-3692, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2855052

Build Your Own Visual-Inertial Drone: A Cost-Effective and Open-Source Autonomous Drone


Published in: ISSN 1070-9932
DOI: 10.1109/MRA.2017.2771326

Detecting Changes in the Environment Based on Full Posterior Distributions Over Real-Valued Grid Maps

Author(s): Lukas Luft, Alexander Schaefer, Tobias Schubert, Wolfram Burgard
Published in: IEEE Robotics and Automation Letters, Issue 3/2, 2018, Page(s) 1299-1305, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2797317

DCT Maps: Compact Differentiable Lidar Maps Based on the Cosine Transform

Author(s): Alexander Schaefer, Lukas Luft, Wolfram Burgard
Published in: IEEE Robotics and Automation Letters, Issue 3/2, 2018, Page(s) 1002-1009, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2794602

Crop Row Detection on Tiny Plants With the Pattern Hough Transform

Author(s): Wera Winterhalter, Freya Veronika Fleckenstein, Christian Dornhege, Wolfram Burgard
Published in: IEEE Robotics and Automation Letters, Issue 3/4, 2018, Page(s) 3394-3401, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2852841

Safe Local Exploration for Replanning in Cluttered Unknown Environments for Microaerial Vehicles

Author(s): Helen Oleynikova, Zachary Taylor, Roland Siegwart, Juan Nieto
Published in: IEEE Robotics and Automation Letters, Issue 3/3, 2018, Page(s) 1474-1481, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2800109

Learning Context Flexible Attention Model for Long-Term Visual Place Recognition

Author(s): Zetao Chen, Lingqiao Liu, Inkyu Sa, Zongyuan Ge, Margarita Chli
Published in: IEEE Robotics and Automation Letters, Issue 3/4, 2018, Page(s) 4015-4022, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2859916

Robust Long-Term Registration of UAV Images of Crop Fields for Precision Agriculture

Author(s): Nived Chebrolu, Thomas Labe, Cyrill Stachniss
Published in: IEEE Robotics and Automation Letters, Issue 3/4, 2018, Page(s) 3097-3104, ISSN 2377-3766
DOI: 10.1109/LRA.2018.2849603

Improved Tau-Guidance and Vision-aided Navigation for Robust Autonomous Landing of UAVs

Author(s): Amedeo Rodi Vetrella, Inkyu Sa, Marija Popovic, Raghav Khanna, Juan Nieto, Giancarmine Fasano, Domenico Accardo and Roland Siegwart
Published in: Field and Service Robotics, Issue 2017, 2017, ISSN 1610-7438

An Analytical Lidar Sensor Model Based on Ray Path Information

Author(s): Alexander Schaefer, Lukas Luft, Wolfram Burgard
Published in: IEEE Robotics and Automation Letters, Issue 2/3, 2017, Page(s) 1405-1412, ISSN 2377-3766
DOI: 10.1109/LRA.2017.2669376

Control of a Quadrotor With Reinforcement Learning

Author(s): Jemin Hwangbo, Inkyu Sa, Roland Siegwart, Marco Hutter
Published in: IEEE Robotics and Automation Letters, Issue 2/4, 2017, Page(s) 2096-2103, ISSN 2377-3766
DOI: 10.1109/LRA.2017.2720851

Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields

Author(s): Nived Chebrolu, Philipp Lottes, Alexander Schaefer, Wera Winterhalter, Wolfram Burgard, Cyrill Stachniss
Published in: The International Journal of Robotics Research, Issue 36/10, 2017, Page(s) 1045-1052, ISSN 0278-3649
DOI: 10.1177/0278364917720510