Periodic Reporting for period 1 - FEROX (Fostering and Enabling AI, Data and Robotics Technologies for Supporting Human Workers in Harvesting Wild Food)
Reporting period: 2022-09-01 to 2024-02-29
Finnish forests and peatlands produce huge amounts of wild berries every year. It has been estimated that during a poor crop year the biological yield of wild berries is nearly 500 million kg and during an abundant crop year it reaches over 1000 mill kg.
There are 37 edible wild berry species in Finland. The economically most significant and most popular of them are lingonberry (Vaccinium vitis-idaea), bilberry (Vaccinium myrtillus) and cloudberry (Rubus chamaemorus). In an average crop year, these three berry species constitute more than 60% of the total yield of wild berries.
Wild berries have huge cultural value in Finland. A recent study confirms that almost 60 % of Finns participate annually in berry picking. One reason for the popularity of berry picking is the right of public access to forests. Finland employs the concept of "everyone's right". This principle allows everyone, regardless of land ownership, to freely collect many natural products, such as wild berries and mushrooms. This right is deeply ingrained in Finnish culture, promoting a close relationship with nature and fostering a sense of shared responsibility and respect for the environment. It embodies the idea that nature's bounty, particularly renewable resources like berries, is a communal asset available to all. Berry picking in Finland has transitioned from being a subsistence necessity to a recreational and culinary activity. While historically vital for survival, it's now enjoyed for leisure, health benefits, and cooking. Additionally, in rural areas, berry picking remains an important source of tax-free income for many residents, contributing to local economies.
It is obvious that more intensive harvesting is possible as the annual utilization rate of wild berries is only 5-10 %. However, the problem is how to intensify berry collection, particularly commercial collection. In this consideration, all the measures – including new technical solutions – that aim at solving this problem are warmly welcomed.
The FEROX project is approaching the challenge of transforming berry collecting through the development of hi-tech solutions, that will attempt to identify and map berry patches. This will enable the optimisation of foraging routes when the berries are ripe. At the same time, the project aims to monitor wild berry populations and track changes in their distribution and abundance to ensure the long-term health of the forest. FEROX is developing AI algorithms to integrate satellite imagery, geospatial data and video feeds from under-canopy drones to identify areas with a high potential for wild berry growth. As well as this, the algorithms will be trained to observe berry flowers, unripe fruit and then ripe berries. This information will be used to create AI-powered foraging apps to provide foragers with real-time information about berry patches, ripeness, terrain conditions and weather conditions. Such information will guide foragers to the most productive berry patches, optimising their foraging routes.
AI models are also being developed that can predict berry ripeness, based on berry growth patterns, prior observations and environmental factors. This is to reduce the overhead of data gathering missions in forests, i.e. the flying of drones, gathering visual data and then analysing results.
The FEROX project represents a pivotal advancement in AgriTech, primarily focusing on the development of advanced berry detection systems through the application of deep learning algorithms. The initial phase of this project involved an extensive data collection process, executed using cameras affixed to drones. These drones conducted aerial surveys of forested areas, capturing images crucial for the foundational training of our AI models.
An integral component of data preparation for AI training was the involvement of human annotators. These experts engaged in the rigorous process of meticulously labelling each image captured by the drones. This process was vital to ensure the precision and accuracy of the data fed into the AI models. It resulted in detailed datasets encompassing diverse berry species like lingonberries, bilberries, crowberries, bog berries, and cloudberries.
Equipped with this accurately annotated data, we proceeded to train our object detection algorithms. These algorithms are intricately designed to identify and classify various types of berries from the drone-captured images. The training phase of these models necessitated a delicate balancing act between achieving high accuracy and maintaining computational efficiency.
The challenge inherent in this project lies in the differentiation of various berry types; a task that our preliminary results have shown to be feasible albeit with some limitations. Specifically, the algorithms have demonstrated a commendable degree of accuracy in recognising most types of berries. However, the distinction between certain berry varieties, such as bog berries and bilberries, remains a challenge due to their close resemblance in shape and colour.
Despite these hurdles, the project team maintains a positive outlook. The initial successes in achieving notable accuracy in berry detection bolster our confidence in the project's future trajectory. Continuous refinement of the AI models and further enhancement of their learning capabilities are expected to yield improved performance, particularly in differentiating between more complex berry classes.
Objectives:
Ob1. Development of an innovative solution based on the integration of AI, computer vision and drones to monitor harvesting activities while incorporating improvements in the working conditions of berry pickers and mitigating the risk factors and threats to their safety during berry picking activities.
Ob2. Identification of needs, hazards and challenges that workers face on a daily basis in terms of occupational safety, with the aim to reduce the severity of problems and injuries by trying to provide workers with faster and more efficient first aid.
Ob3. Definition of the baseline technical and functional requirements and identification of all components to be integrated into FEROX, enabling the design of the architecture and interface of the final solution.
Ob4. Development of the necessary technical services for the solution of problems as well as for the improvement of the working conditions of the workers, bearing in mind the idea of assisting in their daily tasks.
Ob5. Validate the design and engineering approaches in two phases, focused on (i) monitoring, navigation and locating; (ii) logistics optimization and physical assistance.
Ob6. Evaluate the impact of the FEROX use cases on workers, measuring improvements in workers' health, considering first aid and anomaly detection times.
Ob7. Dissemination and communication of project updates and results and establishing connections and interactions with DIH networks and Made in Europe Partnership, achieving a wide impact of the communication, dissemination and exploitation activities.
Ob8. Development of a thorough exploitation plan for the full-scale deployment of the project results, overcoming any possible barriers regarding the future acceptance of our new technology in Europe.
There are 37 edible wild berry species in Finland. The economically most significant and most popular of them are lingonberry (Vaccinium vitis-idaea), bilberry (Vaccinium myrtillus) and cloudberry (Rubus chamaemorus). In an average crop year, these three berry species constitute more than 60% of the total yield of wild berries.
Wild berries have huge cultural value in Finland. A recent study confirms that almost 60 % of Finns participate annually in berry picking. One reason for the popularity of berry picking is the right of public access to forests. Finland employs the concept of "everyone's right". This principle allows everyone, regardless of land ownership, to freely collect many natural products, such as wild berries and mushrooms. This right is deeply ingrained in Finnish culture, promoting a close relationship with nature and fostering a sense of shared responsibility and respect for the environment. It embodies the idea that nature's bounty, particularly renewable resources like berries, is a communal asset available to all. Berry picking in Finland has transitioned from being a subsistence necessity to a recreational and culinary activity. While historically vital for survival, it's now enjoyed for leisure, health benefits, and cooking. Additionally, in rural areas, berry picking remains an important source of tax-free income for many residents, contributing to local economies.
It is obvious that more intensive harvesting is possible as the annual utilization rate of wild berries is only 5-10 %. However, the problem is how to intensify berry collection, particularly commercial collection. In this consideration, all the measures – including new technical solutions – that aim at solving this problem are warmly welcomed.
The FEROX project is approaching the challenge of transforming berry collecting through the development of hi-tech solutions, that will attempt to identify and map berry patches. This will enable the optimisation of foraging routes when the berries are ripe. At the same time, the project aims to monitor wild berry populations and track changes in their distribution and abundance to ensure the long-term health of the forest. FEROX is developing AI algorithms to integrate satellite imagery, geospatial data and video feeds from under-canopy drones to identify areas with a high potential for wild berry growth. As well as this, the algorithms will be trained to observe berry flowers, unripe fruit and then ripe berries. This information will be used to create AI-powered foraging apps to provide foragers with real-time information about berry patches, ripeness, terrain conditions and weather conditions. Such information will guide foragers to the most productive berry patches, optimising their foraging routes.
AI models are also being developed that can predict berry ripeness, based on berry growth patterns, prior observations and environmental factors. This is to reduce the overhead of data gathering missions in forests, i.e. the flying of drones, gathering visual data and then analysing results.
The FEROX project represents a pivotal advancement in AgriTech, primarily focusing on the development of advanced berry detection systems through the application of deep learning algorithms. The initial phase of this project involved an extensive data collection process, executed using cameras affixed to drones. These drones conducted aerial surveys of forested areas, capturing images crucial for the foundational training of our AI models.
An integral component of data preparation for AI training was the involvement of human annotators. These experts engaged in the rigorous process of meticulously labelling each image captured by the drones. This process was vital to ensure the precision and accuracy of the data fed into the AI models. It resulted in detailed datasets encompassing diverse berry species like lingonberries, bilberries, crowberries, bog berries, and cloudberries.
Equipped with this accurately annotated data, we proceeded to train our object detection algorithms. These algorithms are intricately designed to identify and classify various types of berries from the drone-captured images. The training phase of these models necessitated a delicate balancing act between achieving high accuracy and maintaining computational efficiency.
The challenge inherent in this project lies in the differentiation of various berry types; a task that our preliminary results have shown to be feasible albeit with some limitations. Specifically, the algorithms have demonstrated a commendable degree of accuracy in recognising most types of berries. However, the distinction between certain berry varieties, such as bog berries and bilberries, remains a challenge due to their close resemblance in shape and colour.
Despite these hurdles, the project team maintains a positive outlook. The initial successes in achieving notable accuracy in berry detection bolster our confidence in the project's future trajectory. Continuous refinement of the AI models and further enhancement of their learning capabilities are expected to yield improved performance, particularly in differentiating between more complex berry classes.
Objectives:
Ob1. Development of an innovative solution based on the integration of AI, computer vision and drones to monitor harvesting activities while incorporating improvements in the working conditions of berry pickers and mitigating the risk factors and threats to their safety during berry picking activities.
Ob2. Identification of needs, hazards and challenges that workers face on a daily basis in terms of occupational safety, with the aim to reduce the severity of problems and injuries by trying to provide workers with faster and more efficient first aid.
Ob3. Definition of the baseline technical and functional requirements and identification of all components to be integrated into FEROX, enabling the design of the architecture and interface of the final solution.
Ob4. Development of the necessary technical services for the solution of problems as well as for the improvement of the working conditions of the workers, bearing in mind the idea of assisting in their daily tasks.
Ob5. Validate the design and engineering approaches in two phases, focused on (i) monitoring, navigation and locating; (ii) logistics optimization and physical assistance.
Ob6. Evaluate the impact of the FEROX use cases on workers, measuring improvements in workers' health, considering first aid and anomaly detection times.
Ob7. Dissemination and communication of project updates and results and establishing connections and interactions with DIH networks and Made in Europe Partnership, achieving a wide impact of the communication, dissemination and exploitation activities.
Ob8. Development of a thorough exploitation plan for the full-scale deployment of the project results, overcoming any possible barriers regarding the future acceptance of our new technology in Europe.
See technical report for details...