Periodic Reporting for period 2 - BigDataGrapes (Big Data to Enable Global Disruption of the Grapevine-powered Industries)
Periodo di rendicontazione: 2019-07-01 al 2020-12-31
BigDataGrapes aims to support all European companies active in two key industries powered by grapevines: the wine industry and the natural cosmetics one but also provide innovative tools to the food safety market. It will help them respond to the significant opportunity that big data is creating in their relevant markets, by pursuing two ambitious goals:
● To develop and demonstrate powerful data processing technologies that will increase the efficiency of companies that need to take important business decisions dependent on access to vast and complex amounts of data. To this end, one key outcome aims to be a set of data processing tools and methods that are produced by EU technology partners, rigorously tested on data challenges informed by the grapevine-powered industries, but also transferable into other industries within the agriculture, food and beauty sectors.
● To catalyse the creation of a data ecosystem and economy that will increase the competitive advantage of companies that serve with IT solutions to these sectors. To this end, another key outcome aims to be a data marketplace for grapevine-related data assets that will help companies and organisations evolve methods, standards and processes to help them achieve free, interoperable and secure flow of their data.
Grapes are one of the world’s largest fruit crops, with approximately 75 million tons produced each year. They are one of the top 20 agricultural commodities produced, worldwide and considering the weight of the edible portion, grape is the first most produced fruit crop in the world. It is also one of the fruits with the highest input of technology and practices for its efficient management and production. For this reason, it is the fruit crop with the highest total value of production in the world, representing almost 70 billion of US dollars.
● To develop and demonstrate powerful data processing technologies that will increase the efficiency of companies that need to take important business decisions dependent on access to vast and complex amounts of data. To this end, one key outcome aims to be a set of data processing tools and methods that are produced by EU technology partners, rigorously tested on data challenges informed by the grapevine-powered industries, but also transferable into other industries within the agriculture, food and beauty sectors.
● To catalyse the creation of a data ecosystem and economy that will increase the competitive advantage of companies that serve with IT solutions to these sectors. To this end, another key outcome aims to be a data marketplace for grapevine-related data assets that will help companies and organisations evolve methods, standards and processes to help them achieve free, interoperable and secure flow of their data.
Grapes are one of the world’s largest fruit crops, with approximately 75 million tons produced each year. They are one of the top 20 agricultural commodities produced, worldwide and considering the weight of the edible portion, grape is the first most produced fruit crop in the world. It is also one of the fruits with the highest input of technology and practices for its efficient management and production. For this reason, it is the fruit crop with the highest total value of production in the world, representing almost 70 billion of US dollars.
Work outcome: A big data software stack comprising the components that implement the novel methodologies for each technical challenge.
Related progress in the Reporting Period:
• The technical architecture and the software stack of the Big Data Platform was finalized.
• Updated the list of the big data software components and developed APIs for accessing and using the individual components and the overall software stack.
• Developed a data API for making available the datasets of the big data platform to the data marketplace and third-party systems.
Work outcome: A data marketplace where grapevine-powered data assets will be shared and exchanged in interoperable formats and versions, by companies and organisations responsible for them.
Related progress in the Reporting Period:
The BigDataGrapes Data Marketplace is developed and accessible through the BigDataGrapes website (https://marketplace.bigdatagrapes.eu/). The development of the data marketplace provides the necessary proof in action that grapevine-powered data assets are shared and exchanged in interoperable formats and versions, by companies and organisations responsible for them.
Work outcome: The project has focused on exploring different visualisation methods to provide decision support in cases when uncertainty is high.
Related progress in the Reporting Period:
During the reporting period, we have delivered various versions of interactive visualisation components that allows end-users to explore their datasets, uncertainty-aware visual analytic components designed to highlight uncertainties in prediction outcomes, and trust-aware decision support system that uses visualisation techniques to explain the influence of input (predictor) variables on prediction outcomes.
Work outcome: Design and execute the rigorous testing of the proposed technical components to be integrated in the BigDataGrapes platform, in intensive experimental testing under stressed conditions, and comparing with current state-of-art algorithms.
Related progress in the Reporting Period:
● We updated the rigorous experimental methodology.
● For each pilot three different usage scenarios were selected, and rigorous testing experiments ran such as to measure the efficiency of the platform.
● We developed a set of synthetic data generators to be used for projecting data.
Work outcome: Design, plan and execute pilots pertaining to the defined use cases and the respective requirements.
Related progress in the Reporting Period:
● Defined and updated the experimental protocols and processes to be employed in accordance to the piloting plan
● Produced a very detailed human-centered evaluation methodology, in order to evaluate the results of the application pilots qualitatively (user satisfaction and involvement) and quantitatively against the Key Performance Indicators.
Related progress in the Reporting Period:
• The technical architecture and the software stack of the Big Data Platform was finalized.
• Updated the list of the big data software components and developed APIs for accessing and using the individual components and the overall software stack.
• Developed a data API for making available the datasets of the big data platform to the data marketplace and third-party systems.
Work outcome: A data marketplace where grapevine-powered data assets will be shared and exchanged in interoperable formats and versions, by companies and organisations responsible for them.
Related progress in the Reporting Period:
The BigDataGrapes Data Marketplace is developed and accessible through the BigDataGrapes website (https://marketplace.bigdatagrapes.eu/). The development of the data marketplace provides the necessary proof in action that grapevine-powered data assets are shared and exchanged in interoperable formats and versions, by companies and organisations responsible for them.
Work outcome: The project has focused on exploring different visualisation methods to provide decision support in cases when uncertainty is high.
Related progress in the Reporting Period:
During the reporting period, we have delivered various versions of interactive visualisation components that allows end-users to explore their datasets, uncertainty-aware visual analytic components designed to highlight uncertainties in prediction outcomes, and trust-aware decision support system that uses visualisation techniques to explain the influence of input (predictor) variables on prediction outcomes.
Work outcome: Design and execute the rigorous testing of the proposed technical components to be integrated in the BigDataGrapes platform, in intensive experimental testing under stressed conditions, and comparing with current state-of-art algorithms.
Related progress in the Reporting Period:
● We updated the rigorous experimental methodology.
● For each pilot three different usage scenarios were selected, and rigorous testing experiments ran such as to measure the efficiency of the platform.
● We developed a set of synthetic data generators to be used for projecting data.
Work outcome: Design, plan and execute pilots pertaining to the defined use cases and the respective requirements.
Related progress in the Reporting Period:
● Defined and updated the experimental protocols and processes to be employed in accordance to the piloting plan
● Produced a very detailed human-centered evaluation methodology, in order to evaluate the results of the application pilots qualitatively (user satisfaction and involvement) and quantitatively against the Key Performance Indicators.
BigDataGrapes is targeting technology challenges of the grapevine-powered data economy as its business problems and decisions requires processing, analysis and visualisation of data with rapidly increasing volume, velocity and variety. BigDataGrapes achieved the following objectives:
1. Document how grapevine-powered business problems correspond to (big) data challenges: We investigate the critical decisions at the two ends of a value chain: production & buying. We translate them into cross-sector data challenges and we finally consolidated a set of innovative components to enable efficient and scalable management and processing of high-volume data
2. Develop methods and tools that go beyond the state-of-the-art in Big Data management and processing: A software stack of 11 components was developed, consisting of a component on prediction over wine ratings, 7 machine learning analytics for supporting the needs of the pilots, 2 components performing scalable operations on geospatial raster data and an application developed to monitor the status of the BDG infrastructure.
3. Leverage data value via insights and actionable recommendations: BigDataGrapes incorporated techniques for the analysis and semantic enrichment of the examined datasets. After testing and experimentation we developed customised dashboards for individual pilot partners (a total of 9 across 5 pilots).
4. Rigorously assess the improvements on performance and resource usage: We analysed state-of-the-art solutions aimed at generating artificial data and design developing synthetic data generators. We developed an intelligent panel that facilitates the process of parameterization and testing of machine learning and deep learning solutions. Finally, we proceed with the experimental evaluation of the efficiency of the BDG platform using projected datasets.
5. Evaluate the proposed technical solution within real-world settings and against realistic requirements: The development of fully defined demonstrators for each of the grapevine-powered industry use cases allowed to showcase the BigDataGrapes platform in the context of specific end-user requirements from the different areas. Moreover, we extend two specific commercial applications: SITI4farmer & FOODAKAI.
6. Explore representative data flows within the covered industrial value chains in order to define and update appropriate data standards: The creation of an environment in which BigDataGrapes partners and other collaborating groups can discover, access and freely experiment with the data assets. This resulted to the development of the data marketplace for publishing, discovering, assembling, processing and delivering data assets.
1. Document how grapevine-powered business problems correspond to (big) data challenges: We investigate the critical decisions at the two ends of a value chain: production & buying. We translate them into cross-sector data challenges and we finally consolidated a set of innovative components to enable efficient and scalable management and processing of high-volume data
2. Develop methods and tools that go beyond the state-of-the-art in Big Data management and processing: A software stack of 11 components was developed, consisting of a component on prediction over wine ratings, 7 machine learning analytics for supporting the needs of the pilots, 2 components performing scalable operations on geospatial raster data and an application developed to monitor the status of the BDG infrastructure.
3. Leverage data value via insights and actionable recommendations: BigDataGrapes incorporated techniques for the analysis and semantic enrichment of the examined datasets. After testing and experimentation we developed customised dashboards for individual pilot partners (a total of 9 across 5 pilots).
4. Rigorously assess the improvements on performance and resource usage: We analysed state-of-the-art solutions aimed at generating artificial data and design developing synthetic data generators. We developed an intelligent panel that facilitates the process of parameterization and testing of machine learning and deep learning solutions. Finally, we proceed with the experimental evaluation of the efficiency of the BDG platform using projected datasets.
5. Evaluate the proposed technical solution within real-world settings and against realistic requirements: The development of fully defined demonstrators for each of the grapevine-powered industry use cases allowed to showcase the BigDataGrapes platform in the context of specific end-user requirements from the different areas. Moreover, we extend two specific commercial applications: SITI4farmer & FOODAKAI.
6. Explore representative data flows within the covered industrial value chains in order to define and update appropriate data standards: The creation of an environment in which BigDataGrapes partners and other collaborating groups can discover, access and freely experiment with the data assets. This resulted to the development of the data marketplace for publishing, discovering, assembling, processing and delivering data assets.