Periodic Reporting for period 2 - AGRICORE (Agent-based support tool for the development of agriculture policies)
Berichtszeitraum: 2021-03-01 bis 2022-08-31
Common Agricultural Policy is one of the EU's most central policies and aims to ensure that food is high quality, safe, and affordable for the consumer. The design of agricultural policies is a long process involving the assessment of various local and global parameters. Therefore, from early on, agricultural models have been used to describe and interpret key aspects of agricultural policy design, assisting in the CAP instruments implementation.
The most widely used models to date compute the equilibrium between aggregate production and aggregate demand to calculate the impact on farmers' profitability and the price paid by consumers. The problem with these models is that, when aggregating, they do not take into account the heterogeneity of European farms, so that some measures that are shown to be beneficial at the aggregate level in the model may be harmful at the particular level for some types of producers or particular regions.
The AGRICORE project aims to offer a solution to this problem through the incorporation of AI, Big Data and Cloud Computing techniques to build an Agent-based Model, by which the real individual farms are simulated in the form of individual agents, allowing the impact of different policy alternatives to be tested at a much higher level of resolution (down to types of farms at a regional level).
Rural areas make up more than 77% of the EU territory, with about half of its population. There are approximately 12 million full-time farmers in the EU and the food industry and agricultural sector account for more than 6% of the EU's total GDP, providing jobs for 46 million people. The CAP currently uses almost 40% of the common budget of the EU (more than 50 billion € per year). It is therefore almost an obligation for the EU to ensure that future CAP reforms include measures whose impact has been modelled in advance, assessing direct benefits for the environment and for farmers' wellbeing, as well as showing efficiency in the use of taxpayers' resources.
The most widely used models to date compute the equilibrium between aggregate production and aggregate demand to calculate the impact on farmers' profitability and the price paid by consumers. The problem with these models is that, when aggregating, they do not take into account the heterogeneity of European farms, so that some measures that are shown to be beneficial at the aggregate level in the model may be harmful at the particular level for some types of producers or particular regions.
The AGRICORE project aims to offer a solution to this problem through the incorporation of AI, Big Data and Cloud Computing techniques to build an Agent-based Model, by which the real individual farms are simulated in the form of individual agents, allowing the impact of different policy alternatives to be tested at a much higher level of resolution (down to types of farms at a regional level).
Rural areas make up more than 77% of the EU territory, with about half of its population. There are approximately 12 million full-time farmers in the EU and the food industry and agricultural sector account for more than 6% of the EU's total GDP, providing jobs for 46 million people. The CAP currently uses almost 40% of the common budget of the EU (more than 50 billion € per year). It is therefore almost an obligation for the EU to ensure that future CAP reforms include measures whose impact has been modelled in advance, assessing direct benefits for the environment and for farmers' wellbeing, as well as showing efficiency in the use of taxpayers' resources.
The Consortium built the AGRICORE ontology, which is an extension of the DCAT-AP 2.0 ontology, allowing a more comprehensive characterisation of agricultural data sources from the highest level (the catalogue to which they belong) to the lowest level (that of their individual variables).
The Consortium built the Agricultural Research Data Index Tool (ARDIT), which is a web-based application (with APIs to allow external calls) that allows the semantic search and indexing of useful datasets for agricultural research in general, and specifically for the building of AGRICORE synthetic populations.
Almost 300 datasets were characterised and included in ARDIT.
The format required for the storage of the data contained in the different datasets and the structures of the data warehouse (DWH) have been defined. The DWH stores all the data extracted from the datasets previously identified by ARDIT, as well as the rest of the derived datasets generated by AGRICORE, will be stored. The data extraction module (DEM) and data fusion module (DFM) have also been completed, including the algorithm that allows the construction of the Bayesian network (BN) used to assign values to the attributes of each agent during the synthetic population generation (SPG).
The architecture of the AGRICORE agents has been determined, including the skeleton of attributes of interest (classified in states, inputs, outputs and disturbances), and the optimization structures for the dynamic modelling of the agent are being finalised. These optimization structures determine the actions of each agent in the short term (agro-economic dimension) using Positive Mathematical Programming (PMP) and in the long term (financial structural dimension) using Model Predictive Control (MPC).
For each of the three use cases, those attributes of the agents that can be generated with existing data sources and those ones for which data gaps exist were identified. To fill the latter, participatory research tools were designed that should allow these gaps to be filled through survey campaigns.
The design of the AGRICORE graphical user interface (GUI) was validated by experts. The libraries needed for the visualisation of results have also been identified and packaged. The implementation of the Land Market Module (LMM) have been completed, which allows the auction-based local exchange of land between agents. The basis for obtaining sales prices for each of the commodity products produced by the agents has also been established through the Product Market Module (PMM). The Key Performance Indicators (KPIs) by which the environmental and ecosystem services impact will be measured have been selected and the equations necessary for their calculation have been implemented in the corresponding Impact Assessment Modules. Finally, through the Policy Environment Module (PEM), a UML schema and an XML template have been developed to allow the standardised definition of public agricultural policies of Pillar I and Pillar II of the CAP. The communication mechanisms that allow the exchange of information between the different modules using a customisable DAPR sidecar and the module that allows the connection of the ABM engine with external biophysical models have also been implemented.
Each of the three use cases considered in the project has established its own network of contacts with those actors involved in the life cycle of the instruments that articulate each of the three policy measures analysed.
At the general level of the Project, contacts have been established with the JRC and DG-AGRI to obtain information and advice on the re-use of pre-existing models and data sources.
Finally, the External Experts Advisory Board has been set up.
In this second reporting period, the design and planning of Ex-Post and Ex-Ante analyses for each of the three use cases continued.
Open-sourcing of the software has been enabled through the choice of the collaborative development tool (GitLab) and the creation of modular repositories corresponding to each of the modules of the future tool.
The Consortium built the Agricultural Research Data Index Tool (ARDIT), which is a web-based application (with APIs to allow external calls) that allows the semantic search and indexing of useful datasets for agricultural research in general, and specifically for the building of AGRICORE synthetic populations.
Almost 300 datasets were characterised and included in ARDIT.
The format required for the storage of the data contained in the different datasets and the structures of the data warehouse (DWH) have been defined. The DWH stores all the data extracted from the datasets previously identified by ARDIT, as well as the rest of the derived datasets generated by AGRICORE, will be stored. The data extraction module (DEM) and data fusion module (DFM) have also been completed, including the algorithm that allows the construction of the Bayesian network (BN) used to assign values to the attributes of each agent during the synthetic population generation (SPG).
The architecture of the AGRICORE agents has been determined, including the skeleton of attributes of interest (classified in states, inputs, outputs and disturbances), and the optimization structures for the dynamic modelling of the agent are being finalised. These optimization structures determine the actions of each agent in the short term (agro-economic dimension) using Positive Mathematical Programming (PMP) and in the long term (financial structural dimension) using Model Predictive Control (MPC).
For each of the three use cases, those attributes of the agents that can be generated with existing data sources and those ones for which data gaps exist were identified. To fill the latter, participatory research tools were designed that should allow these gaps to be filled through survey campaigns.
The design of the AGRICORE graphical user interface (GUI) was validated by experts. The libraries needed for the visualisation of results have also been identified and packaged. The implementation of the Land Market Module (LMM) have been completed, which allows the auction-based local exchange of land between agents. The basis for obtaining sales prices for each of the commodity products produced by the agents has also been established through the Product Market Module (PMM). The Key Performance Indicators (KPIs) by which the environmental and ecosystem services impact will be measured have been selected and the equations necessary for their calculation have been implemented in the corresponding Impact Assessment Modules. Finally, through the Policy Environment Module (PEM), a UML schema and an XML template have been developed to allow the standardised definition of public agricultural policies of Pillar I and Pillar II of the CAP. The communication mechanisms that allow the exchange of information between the different modules using a customisable DAPR sidecar and the module that allows the connection of the ABM engine with external biophysical models have also been implemented.
Each of the three use cases considered in the project has established its own network of contacts with those actors involved in the life cycle of the instruments that articulate each of the three policy measures analysed.
At the general level of the Project, contacts have been established with the JRC and DG-AGRI to obtain information and advice on the re-use of pre-existing models and data sources.
Finally, the External Experts Advisory Board has been set up.
In this second reporting period, the design and planning of Ex-Post and Ex-Ante analyses for each of the three use cases continued.
Open-sourcing of the software has been enabled through the choice of the collaborative development tool (GitLab) and the creation of modular repositories corresponding to each of the modules of the future tool.
So far, AGRICORE has already proposed some technical developments and actions that represent progress with respect to the state of the art in the following aspects: (i) characterisation of agricultural data sources down to the level of variables included and the units of analysis they refer to; (ii) micro-data warehousing for automation of data extraction and merging to generate Bayesian Networks for initialising attributes of interest of the synthetic population; (iii) agent-based models with state-space feedback; (iv) participatory research design to quantify risk aversion and innovativeness of farmers in regions of interest for the use cases.
At the end of the Project, in addition to consolidating the above advances with concrete implementations and explicit results, the following additional impacts should have been produced: (v) application of advanced Big Data visualisation tools to simulation results; (vi) modularised design allowing interoperability of the suite with different existing or subsequently developed models; (vii) improvement proposals for agricultural statistics regulations and Farm Advisory Systems; (viii) increasing the involvement of farmers in the collection of agro-accounting information by making them see how this reverts into a better-calibrated model that allows to better adjust the design of future measures.
At the end of the Project, in addition to consolidating the above advances with concrete implementations and explicit results, the following additional impacts should have been produced: (v) application of advanced Big Data visualisation tools to simulation results; (vi) modularised design allowing interoperability of the suite with different existing or subsequently developed models; (vii) improvement proposals for agricultural statistics regulations and Farm Advisory Systems; (viii) increasing the involvement of farmers in the collection of agro-accounting information by making them see how this reverts into a better-calibrated model that allows to better adjust the design of future measures.