FIRST MLOPS INTEGRATING REAL-TIME RISK MANAGEMENT, COMPLIANCE AND EXPLAINABILITY WHERE THE AI MODEL RUNS

Información del proyecto

Deeploy

Identificador del acuerdo de subvención: 101216662

DOI

10.3030/101216662

Fecha de la firma de la CE 20 Junio 2025

Fecha de inicio 1 Marzo 2025

Fecha de finalización 28 Febrero 2027

Financiado con arreglo a

• The European Innovation Council (EIC)

Coste total

Sin datos

Aportación de la UE

€ 2 490 000,00

Coordinado por

DEEPLOY B.V.
Netherlands

Periodic Reporting for period 1 - Deeploy (FIRST MLOPS INTEGRATING REAL-TIME RISK MANAGEMENT, COMPLIANCE AND EXPLAINABILITY WHERE THE AI MODEL RUNS)

Período documentado: 2025-03-01 hasta 2026-02-28

Resumen del contexto y de los objetivos generales del proyecto

The project's success could establish new industry standards for AI governance that transform current ad-hoc compliance approaches into systematic, scalable practices. This standardization potential represents perhaps the most significant long-term impact: the creation of a governance infrastructure that enables responsible AI deployment at the speed and scale that modern business environments demand, while maintaining the oversight and accountability that democratic societies require.

The strategic significance extends to European technological sovereignty in AI governance, positioning EU-based solutions as alternatives to predominantly US-based AI infrastructure providers. This alignment with broader European digital strategy objectives amplifies the project's potential impact beyond individual organizational benefits to encompass regional competitive positioning in the global AI landscape.

Deeploy's AI governance platform addresses this critical gap by enabling real-time risk management, explainability, and compliance monitoring within operational AI environments. The project emerges at a pivotal moment: organisations face potential fines of up to 7% of global annual turnover under the EU AI Act, while the global AI governance market is projected to reach $5.1 billion by 2028, a critical inflection point where regulatory compliance shifts from best practice to business requirement.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

A structured control framework was developed enabling organisations to map regulatory requirements (EU AI Act, ISO, or custom frameworks) to operational controls at organisation, workspace, use-case, and deployment level, including risk assessment, approval workflows, and periodic reviews.

An automated compliance evidence system covers the full AI lifecycle, including inference log archiving, guardrail violation tracking, metadata collection, and automated compliance checks, significantly reducing manual documentation burden.

A system for automated compliance evidence collection was developed, covering the full AI lifecycle from exploration through to retirement. This includes automated event and inference log archiving, guardrail violation tracking, key-value pair metadata collection, and automated binary and calculated compliance checks. This reduces manual documentation burden significantly.
Two Explainable AI (XAI) methods were developed and validated:
(1) a black-box method based on SHAP feature attribution, and
(2) a white-box method based on attention weights and gradient saliency, with specific implementation for generative Hugging Face models. Both methods were validated against EU AI Act requirements in collaboration with external legal and academic experts. A third XAI method targeting closed/inaccessible models is in development. Additionally, a bias detection pipeline was designed covering offline analysis, offline monitoring, online monitoring, and visualisation of sensitive attribute metrics.

A framework-agnostic model onboarding capability enables governance of any AI model (ML, GenAI, Agent) across Kubernetes (KServe), AzureML, and Sagemaker. Enterprise private cloud and hybrid SaaS architectures were validated with TVM, Novo Nordisk, and Healthplus.ai with multi-tenancy, role-based access control, and SSO.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

Deeploy's results address several unresolved challenges in responsible AI deployment that current tools and frameworks do not adequately cover.

1. Automated compliance evidence across the AI lifecycle. No integrated tooling previously existed to automatically capture, structure, and verify compliance evidence throughout development, validation, and production. Deeploy's evidence pipeline, combining inference logging, guardrail enforcement, lifecycle state tracking, and policy-bound metadata, constitutes a novel operational layer above existing MLOps infrastructure, removing a core bottleneck in regulated AI adoption.

2. Explainability for generative and closed models. Deeploy developed framework-agnostic token-level attribution methods (gradient saliency, attention weights) for generative models. Work is ongoing on XAI for fully closed/external models: a largely unsolved problem. The framework targets practically useful explanations for human oversight, balancing faithfulness against complexity.

3. Governance-native infrastructure. Rather than a post-hoc audit layer, Deeploy integrates control frameworks, evidence collection, explainability, and bias monitoring directly into AI deployment infrastructure — a departure from the current standard of manual, retroactive compliance

Potential impacts include reducing the cost of EU AI Act compliance in high-risk domains (healthcare, finance, pharma, public sector). The platform has already been validated with UMCU, Novo Nordisk, and Healthplus.ai.

Key needs for further uptake include:
(1) stable and specific technical guidance from the EU AI Office on high-risk AI validation requirements, which has been repeatedly delayed;
(2) standardisation of AI governance evidence formats to enable interoperability;
(3) further enterprise-scale demonstration projects in regulated sectors; and
(4) commercial partnerships across EU geographies to localise compliance support (currently active in Benelux, DACH, Nordics, and UK/Ireland).