Learning and Analysing Massive / Big complex Data

Project Information

LAMBDA

Grant agreement ID: 734242

DOI

10.3030/734242

Project closed

EC signature date 6 December 2016

Start date 1 March 2017

End date 31 August 2022

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 337 500,00

EU contribution

€ 337 500,00

Coordinated by

ETHNIKO KAI KAPODISTRIAKO PANEPISTIMIO ATHINON
Greece

Periodic Reporting for period 2 - LAMBDA (Learning and Analysing Massive / Big complex Data)

Reporting period: 2019-03-01 to 2022-08-31

Summary of the context and overall objectives of the project

LAMBDA aimed at transferring game changing technologies to the European industry in critical areas of Machine learning. Based on recent algorithmic breakthroughs, we adapted sophisticated methods to targeted industries to help turn cutting edge tools into innovative software products or processes, tailored to real-world issues. LAMBDA focused on two distinct application domains: 3D shape analysis and unstructured data mining. They share challenging features such as inherent complexity in modeling the data, high dimensionality which raises the issue of curse of dimensionality, and the need to address such datasets at a massive scale. 3D shape analysis is an important current problem in medicine, biology, as well as mechanical engineering and simulation. Given the huge success of manipulating language and speech (one dimensional) as well as images (2D), it is a natural next step to develop technology for 3D data. Our second domain of application is handling unstructured (such as driving-insurance) data so as to model and monitor driving behaviour, detect dangerous road segments, as well as forecast time of arrival. For each application domain the consortium included a significant industrial stakeholder within EU.

LAMBDA was characterised by a unique blend of theoretically rigorous and geometrically inclined methods, thus supporting a strong aspect of interdisciplinarity between Theory of Algorithms and Machine Learning. This was supported by software development, ranging from prototype implementations to software ready to be integrated in respective libraries. Our software methods were validated on synthetic data but possibly on real datasets. LAMBDA strengthened existing links within Europe and across the Atlantic, while creating new synergies that support knowledge transfer beyond its lifetime.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The project has achieved significant scientific progress as shown by the already published top-level research in relation to the 3 scientific Work-packages. In particular, research breakthroughs have been achieved on Dimensionality reduction by randomized linear projections, sampling and geometric random walks, Deep learning, Road segmentation / clustering, and risk assessment by anomaly detection. Articles and posters have been published in internationally established scientific journal or conferences with peer-reviewed proceedings. Workshops have been organized by the Consortium according to the provisions of the Grant Agreement.

The impact of the project can be summarized to the following points:
• Secondments have been undertaken between all members and partners. Career development of secondees (contacts with industry, experience of both sectors) has been significant. Transfer of Knowledge has been satisfactory, both intersectoral and intercontinental, mainly by means of secondment visits but also by discussion and plans for further collaboration.
• Communication is strong, given the current rate of publications as well as further activities such as use of LAMBDA to enhance course material and propose course projects, presentations in high-schools and University-level schools, general public talks. Some dissemination has occurred via LinkedIn, as well as at events targeting the wider public (the coordinator has participated in two general scientific conferences on Open Science, and on Math education).
• One industrial participant has showcased and exploited participation in LAMBDA in a recent funding round. An NDA was signed between an industrial and an academic member. Exploitable results elaborated and selected for EU's Innovation Radar concerning prospects for further innovation activities and transfer of technology in Shape representation, Geometric learning, Road segmentation / clustering, Driving behavior, anomaly detection. Joint software is being developed between academic and industrial members, and joint participation in new projects has been achieved.

Actions and deliverables relating to the LAMBDA project continue to take into account data privacy and confidentiality concerns, in order to ensure compliance with the relevant EU legislation. To that end, anonymisation and encryption techniques continue to be used, where appropriate, and specific privacy related information is not disclosed outside the context of the Project. Access restrictions to IT systems and storage such as password protection and ‘need-to-know’ access also continue to be in place, in accordance with the applicable legislation. All stakeholders in the consortium are aware of the relevant data privacy and confidentiality concerns and requirements, notably those arising from the General Data Protection Regulation (GDPR).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

LAMBDA has created a shared culture of research and innovation in crucial areas in Machine learning and Data Mining, and has shown progress and innovate on a small number of critical applications in analysing complex data, namely 3D models, and road data aimed at the insurance business. A unique feature of LAMBDA is the mathematically rigorous approach we bring into Machine Learning and Data Mining. Specifically, to address these challenges LAMBDA implements certified approximation algorithms and big data methods with guarantees. Our methods are being validated on real-world data from our European industrial participants.

Technology transfer to and exploitation by the industrial participants as well as transfer of knowledge to the wider industrial community is the final goal. Specific results were obtained in the following innovation aspects: First, efficient and compact representation of complex objects, including shapes with attributes, in high-dimensional space. Data structures and methods for efficient search and retrieval; new methods for matching and analysing 3D shapes. Second, clustering of complex data such as road segments. Analysis of car traffic and accident analysis. Both of these application domains are included in EU's Innovation Radar.

An important aspect is to offer intersectoral training to all staff, especially young scientists, to create awareness of the role of businesses in technology, and the contribution of research organisations to innovation. We are providing international training at world leader institutions. Dissemination of LAMBDA’s results to the scientific community happens with publications (including open access) and workshops. Communication of scientific and technological advances is important to inform the general public and happens via social media, the Web and public lectures.

Nearest neighbor of road segments

LAMBDA Project Logo