FIRST MLOPS INTEGRATING REAL-TIME RISK MANAGEMENT, COMPLIANCE AND EXPLAINABILITY WHERE THE AI MODEL RUNS

Project Information

Deeploy

Grant agreement ID: 101216662

DOI

10.3030/101216662

EC signature date 20 June 2025

Start date 1 March 2025

End date 28 February 2027

Funded under

• The European Innovation Council (EIC)

Total cost

No data

EU contribution

€ 2 490 000,00

Coordinated by

DEEPLOY B.V.
Netherlands

Periodic Reporting for period 1 - Deeploy (FIRST MLOPS INTEGRATING REAL-TIME RISK MANAGEMENT, COMPLIANCE AND EXPLAINABILITY WHERE THE AI MODEL RUNS)

Reporting period: 2025-03-01 to 2026-02-28

Summary of the context and overall objectives of the project

The project's success could establish new industry standards for AI governance that transform current ad-hoc compliance approaches into systematic, scalable practices. This standardization potential represents perhaps the most significant long-term impact: the creation of a governance infrastructure that enables responsible AI deployment at the speed and scale that modern business environments demand, while maintaining the oversight and accountability that democratic societies require.

The strategic significance extends to European technological sovereignty in AI governance, positioning EU-based solutions as alternatives to predominantly US-based AI infrastructure providers. This alignment with broader European digital strategy objectives amplifies the project's potential impact beyond individual organizational benefits to encompass regional competitive positioning in the global AI landscape.

Deeploy's AI governance platform addresses this critical gap by enabling real-time risk management, explainability, and compliance monitoring within operational AI environments. The project emerges at a pivotal moment: organisations face potential fines of up to 7% of global annual turnover under the EU AI Act, while the global AI governance market is projected to reach $5.1 billion by 2028, a critical inflection point where regulatory compliance shifts from best practice to business requirement.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

A structured control framework was developed enabling organisations to map regulatory requirements (EU AI Act, ISO, or custom frameworks) to operational controls at organisation, workspace, use-case, and deployment level, including risk assessment, approval workflows, and periodic reviews.

An automated compliance evidence system covers the full AI lifecycle, including inference log archiving, guardrail violation tracking, metadata collection, and automated compliance checks, significantly reducing manual documentation burden.

A system for automated compliance evidence collection was developed, covering the full AI lifecycle from exploration through to retirement. This includes automated event and inference log archiving, guardrail violation tracking, key-value pair metadata collection, and automated binary and calculated compliance checks. This reduces manual documentation burden significantly.
Two Explainable AI (XAI) methods were developed and validated:
(1) a black-box method based on SHAP feature attribution, and
(2) a white-box method based on attention weights and gradient saliency, with specific implementation for generative Hugging Face models. Both methods were validated against EU AI Act requirements in collaboration with external legal and academic experts. A third XAI method targeting closed/inaccessible models is in development. Additionally, a bias detection pipeline was designed covering offline analysis, offline monitoring, online monitoring, and visualisation of sensitive attribute metrics.

A framework-agnostic model onboarding capability enables governance of any AI model (ML, GenAI, Agent) across Kubernetes (KServe), AzureML, and Sagemaker. Enterprise private cloud and hybrid SaaS architectures were validated with TVM, Novo Nordisk, and Healthplus.ai with multi-tenancy, role-based access control, and SSO.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Deeploy's results address several unresolved challenges in responsible AI deployment that current tools and frameworks do not adequately cover.

1. Automated compliance evidence across the AI lifecycle. No integrated tooling previously existed to automatically capture, structure, and verify compliance evidence throughout development, validation, and production. Deeploy's evidence pipeline, combining inference logging, guardrail enforcement, lifecycle state tracking, and policy-bound metadata, constitutes a novel operational layer above existing MLOps infrastructure, removing a core bottleneck in regulated AI adoption.

2. Explainability for generative and closed models. Deeploy developed framework-agnostic token-level attribution methods (gradient saliency, attention weights) for generative models. Work is ongoing on XAI for fully closed/external models: a largely unsolved problem. The framework targets practically useful explanations for human oversight, balancing faithfulness against complexity.

3. Governance-native infrastructure. Rather than a post-hoc audit layer, Deeploy integrates control frameworks, evidence collection, explainability, and bias monitoring directly into AI deployment infrastructure — a departure from the current standard of manual, retroactive compliance

Potential impacts include reducing the cost of EU AI Act compliance in high-risk domains (healthcare, finance, pharma, public sector). The platform has already been validated with UMCU, Novo Nordisk, and Healthplus.ai.

Key needs for further uptake include:
(1) stable and specific technical guidance from the EU AI Office on high-risk AI validation requirements, which has been repeatedly delayed;
(2) standardisation of AI governance evidence formats to enable interoperability;
(3) further enterprise-scale demonstration projects in regulated sectors; and
(4) commercial partnerships across EU geographies to localise compliance support (currently active in Benelux, DACH, Nordics, and UK/Ireland).