Periodic Reporting for period 2 - Bonseyes (Platform for Open Development of Systems of Artificial Intelligence)
Reporting period: 2018-06-01 to 2020-01-31
Artificial Intelligence is projected to add over €13 trillion to the global economy by 2030 but companies outside of the US and China risk being shut out by the ‘data wall’ dominated by internet giants. The goal of the Bonseyes project is to create an Artificial Intelligence Marketplace which will help European companies to get over the wall using the new power of edge computing and by leveraging Europe’s leadership in embedded systems.
According to Gartner, 91% of today’s data is processed in centralized data centers. But by 2022, about 74% of all data will need analysis and action on the edge. Bonseyes is a platform for the open development of systems of artificial intelligence which are emerging as a key growth driver in Smart CPS systems in the next decade. Opposed to monolithic system design methodologies currently used in closed end-to-end solutions, Bonseyes focuses on an open architecture and enables an eco-system of innovators, researchers, developers, and companies to collaborate in building highly complex distributed systems that are “intelligent”. Its objectives are to:
- Accelerate the design and programming of complex systems of artificial intelligence through secure multi-party collaboration.
- Reduce the complexity, time, and cost in training and deployment of deep learning models for distributed embedded systems.
- Foster “embedded intelligence” on low power and resource constrained Smart CPS through
- Build and promote a decentralized, multi-party AI ecosystem in Europe reducing fragmentation across researchers, start-ups, SMEs, and industry with the capacity to set the course of AI for the benefit of society.
According to Gartner, 91% of today’s data is processed in centralized data centers. But by 2022, about 74% of all data will need analysis and action on the edge. Bonseyes is a platform for the open development of systems of artificial intelligence which are emerging as a key growth driver in Smart CPS systems in the next decade. Opposed to monolithic system design methodologies currently used in closed end-to-end solutions, Bonseyes focuses on an open architecture and enables an eco-system of innovators, researchers, developers, and companies to collaborate in building highly complex distributed systems that are “intelligent”. Its objectives are to:
- Accelerate the design and programming of complex systems of artificial intelligence through secure multi-party collaboration.
- Reduce the complexity, time, and cost in training and deployment of deep learning models for distributed embedded systems.
- Foster “embedded intelligence” on low power and resource constrained Smart CPS through
- Build and promote a decentralized, multi-party AI ecosystem in Europe reducing fragmentation across researchers, start-ups, SMEs, and industry with the capacity to set the course of AI for the benefit of society.
Major milestones were reached with the development and the ongoing evaluation of proof-of-concepts of the following key-components:
- Architecture: Definition of main concepts of AI systems of systems development with multi-party challenge driven collaboration. Industry driven “Procurement” collaboration model using Secure Virtual Premise for enforcement of AI artifact licenses.
- Marketplace: Marketplace development including and integration of algorithms, toolboxes, including secure virtual premise and licensing framework. Repository allowing sharing and evaluation of deep learning-based AI Assets and AI Artifacts (data, models, benchmarks) and the buying / selling of AI Apps for various Developer Platforms.
- Deep learning toolbox: a containerized, easy-to-use end-to-end training pipeline, including deployment functionality for the (embedded) target hardware platforms. Architecture sensitive deep learning methods including pruning and hardware sensitive learning. On-device adaptation and robustness via meta-learning and demonstration of algorithms and tools on real-world use-cases. Platform-deployment methods and tools such quantization and pruning available in open source deep learning toolbox.
- Universal developer reference platforms: Deployment of AI Apps on heterogeneous embedded developer platforms from Intel, ARM, NVIDIA, and Renesas. Universal LPDNN Inference engine (LNE) and modular creation of AI-Apps with user-friendly APIs. Integration of 3rd party inference engines and interoperability with ONNX. Algorithm optimization using Winograd convolutions, optimized gemm kernel calls, depthwise convolutions, and code generation techniques.
- Demonstrators: Identifying, building and preliminary demonstration of four challenging real-world automotive, healthcare, and consumer applications targeting resource constrained devices on NVIDIA Xavier, Raspberry PI3B+, and Renesas V3M Automotive platforms.
- Sustainability and AI-on-the-Demand Platform Integration: Developed a large ecosystem of academic and industry partners and actively engaged with AI4EU to explore integration and strategic alternatives. Created a sustainability entity (Bonseyes Community Association) and set up an ambitious exploitation plan with potential commercial scalability.
- Architecture: Definition of main concepts of AI systems of systems development with multi-party challenge driven collaboration. Industry driven “Procurement” collaboration model using Secure Virtual Premise for enforcement of AI artifact licenses.
- Marketplace: Marketplace development including and integration of algorithms, toolboxes, including secure virtual premise and licensing framework. Repository allowing sharing and evaluation of deep learning-based AI Assets and AI Artifacts (data, models, benchmarks) and the buying / selling of AI Apps for various Developer Platforms.
- Deep learning toolbox: a containerized, easy-to-use end-to-end training pipeline, including deployment functionality for the (embedded) target hardware platforms. Architecture sensitive deep learning methods including pruning and hardware sensitive learning. On-device adaptation and robustness via meta-learning and demonstration of algorithms and tools on real-world use-cases. Platform-deployment methods and tools such quantization and pruning available in open source deep learning toolbox.
- Universal developer reference platforms: Deployment of AI Apps on heterogeneous embedded developer platforms from Intel, ARM, NVIDIA, and Renesas. Universal LPDNN Inference engine (LNE) and modular creation of AI-Apps with user-friendly APIs. Integration of 3rd party inference engines and interoperability with ONNX. Algorithm optimization using Winograd convolutions, optimized gemm kernel calls, depthwise convolutions, and code generation techniques.
- Demonstrators: Identifying, building and preliminary demonstration of four challenging real-world automotive, healthcare, and consumer applications targeting resource constrained devices on NVIDIA Xavier, Raspberry PI3B+, and Renesas V3M Automotive platforms.
- Sustainability and AI-on-the-Demand Platform Integration: Developed a large ecosystem of academic and industry partners and actively engaged with AI4EU to explore integration and strategic alternatives. Created a sustainability entity (Bonseyes Community Association) and set up an ambitious exploitation plan with potential commercial scalability.
The Bonseyes AI Marketplace provides the reuse of data, meta data, and models among separate legal entities, to significantly reduce design and development time in building systems of artificial intelligence. It supports many actors in the AI value chain such as Challenge Providers, Data Providers, Data Scientists, Application Developers, Infrastructure Providers, and Solution Integrators. Other available platforms are either vendor-specific or do not provide entire value chain from challenges to deployment.
Bonseyes targets high impact societal applications such as healthcare, automotive, and manufacturing that demand low latency, real-time, low power AI systems of systems while preserving user privacy. It’s potential impacts are significant to:
- Boost competitiveness through top European intelligent product/service research while reducing CAPEX, experts, and time to adoption.
- Boost technology transfer through the mobilization of talent, innovation hubs, pilots with integrated and reusable assets, tools, and European industry user-driven challenges.
- Boost tech convergence through fertilizing technology leadership, AI coalitions and interworking sharing current market best practices in deployment of AI with distributed and decentralized technologies.
Overall, the project has progressed beyond the start-of-the-art in distributed and decentralized AI systems-of-systems development through:
- Reduction in development time by 50% compared to monolithic system design methods, through the reuse of data, meta data, and models among separate legal entities made possible by the Bonseyes AI Marketplace, a data marketplace that modularizes the AI systems’ development value-chain.
- Reduction in cost of ownership by a factor of 5 related to training of deep learning models compared to current training approaches designed for the cloud. Deep learning training methods for resource constrained devices that enable models with near state-of-the-art accuracy that are tailored for embedded, constrained, distributed systems operating in real environments with noisy, sometimes missing data.
- Enabling distributed deep learning, where part of the training can be achieved in embedded devices themselves, partially alleviating the need to transmit vast amounts of labelled data back to the cloud. This step is a key enabler for eventual unsupervised learning on mobile devices.
- Predictive tools that automate optimal on-device deployment of a model on the target embedded system given a specific power/space/time constraint. Enabling low power “always-on” intelligence on edge devices. Tools optimized for various low power universal reference developer platforms supporting “always-on” intelligence paradigms.
- Privacy and data isolation for deep learning methods that can be selected and adapted by the data providers or model owners. Robust data collection tools for IoT devices scalable to a very large number of devices and maintain privacy and data isolation, while enabling real-time data processing on the “edge” device to identify when data is “abnormal”.
Bonseyes targets high impact societal applications such as healthcare, automotive, and manufacturing that demand low latency, real-time, low power AI systems of systems while preserving user privacy. It’s potential impacts are significant to:
- Boost competitiveness through top European intelligent product/service research while reducing CAPEX, experts, and time to adoption.
- Boost technology transfer through the mobilization of talent, innovation hubs, pilots with integrated and reusable assets, tools, and European industry user-driven challenges.
- Boost tech convergence through fertilizing technology leadership, AI coalitions and interworking sharing current market best practices in deployment of AI with distributed and decentralized technologies.
Overall, the project has progressed beyond the start-of-the-art in distributed and decentralized AI systems-of-systems development through:
- Reduction in development time by 50% compared to monolithic system design methods, through the reuse of data, meta data, and models among separate legal entities made possible by the Bonseyes AI Marketplace, a data marketplace that modularizes the AI systems’ development value-chain.
- Reduction in cost of ownership by a factor of 5 related to training of deep learning models compared to current training approaches designed for the cloud. Deep learning training methods for resource constrained devices that enable models with near state-of-the-art accuracy that are tailored for embedded, constrained, distributed systems operating in real environments with noisy, sometimes missing data.
- Enabling distributed deep learning, where part of the training can be achieved in embedded devices themselves, partially alleviating the need to transmit vast amounts of labelled data back to the cloud. This step is a key enabler for eventual unsupervised learning on mobile devices.
- Predictive tools that automate optimal on-device deployment of a model on the target embedded system given a specific power/space/time constraint. Enabling low power “always-on” intelligence on edge devices. Tools optimized for various low power universal reference developer platforms supporting “always-on” intelligence paradigms.
- Privacy and data isolation for deep learning methods that can be selected and adapted by the data providers or model owners. Robust data collection tools for IoT devices scalable to a very large number of devices and maintain privacy and data isolation, while enabling real-time data processing on the “edge” device to identify when data is “abnormal”.