Predictive Rendering In Manufacture and Engineering

Informations projet

PRIME

N° de convention de subvention: 956585

DOI

10.3030/956585

Projet clôturé

Date de signature de la CE 17 Août 2020

Date de début 1 Octobre 2020

Date de fin 31 Août 2025

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 4 118 558,71

Contribution de l’UE

€ 4 118 558,71

Coordonné par

UNIVERZITA KARLOVA
Czechia

Periodic Reporting for period 2 - PRIME (Predictive Rendering In Manufacture and Engineering)

Période du rapport: 2022-10-01 au 2025-08-31

Résumé du contexte et des objectifs généraux du projet

The PRIME (Predictive Rendering in Industrial Manufacturing) project worked on image synthesis which delivers results that one can rely on to be visually accurate. Engineering researchers who participated in this Innovative Training Network (ITN) were trained in skills and protocols needed for industrial usage of Predictive Rendering (PR) technologies.

A variety of industries, from manufacturing to entertainment, are moving towards predictive rendering. Application areas of such systems are in product design, architecture, sensor system calibration, training of autonomous vehicle systems, manufacturing control. But even established graphics application areas like movie visual effects (VFX) can benefit from such an approach.

This is a cutting-edge area of applied computer science, in which European academia and industry are amongst the global technology leaders. Our ITN network helped with maintaining and increasing the competitive edge of Europe in this regard, and trained young researchers in a promising, future-oriented and research-driven application area.

The current industry standard in Computer Graphics in most cases only aims to convince and immerse a viewer, but not to offer an actual prediction of the appearance of a 3D scene. Genuinely predictive rendering has the potential to be a game changer for several industries, way beyond Computer Graphics proper.

A widespread industrial use of predictive rendering is starting to be technically feasible. An example is the emergence of so-called digital twins of products: CAD models which include 100% accurate appearance descriptions. These have already, within the limits of existing technology, been introduced in the car industry, and the concept is spreading to other manufacturing industries. Such digital twins become even more valuable if one can reliably visualise and manipulate them, and use them for authoritative appearance inspection. Not all such use cases have been explored yet, as the technology needed for them is still either lacking (as in the case of e.g. fluorescent materials), or not sufficiently standardised for widespread industrial use.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

PRIME was implemented from 10/2020 until 08/2025. It was extended due to impacts of force majeure, mainly the COVID-19 pandemic and global geopolitical situation.

1. The project was successfully concluded despite challenges posed by force majeure. 14 early-stage researchers (ESRs) finished their 3-year MSCA fellowship and 8 of them were awarded a PhD until October 2025.

2. The project hired 15 ESRs hired in an open, transparent, impartial and equitable recruitment process. A semi-centralized recruitment process was used to increase hiring effectiveness.

3. The training of ESRs was conducted by experts from the consortium as well as invited external experts. In the beginning of the project, the training coordinator collected training needs from the ESRs. The content of the training was consulted with the fellows during the whole project. The following network-wide trainings took place:
- 12/2020 – Kick-off meeting
- 05/2021 or 01/2022 - MSCA administrative training for all ESRs
- 07/2021 – post-EGSR PRIME training
- 12/2021 - midterm review and first network training
- 07/2022 – post-EGSR PRIME training
- 10/2022 – second network training
- 03/2023 - third network training
- 09/2023 - fourth network training
- 09/2024 - final network training

In addition to these network meetings, PRIME held several remote meetings to discuss project progress.

A reading group was organized by the ESRs from 02/2021 until 12/2023. The sessions provided our ESRs with opportunities for peer-to-peer training, offered interaction with leading experts and the acquisition of new professional expertise.

4. The research outcomes advanced the state-of-art in our area, and were published at top venues (e.g. SIGGRAPH, SIGGRAPH Asia, EGSR, CVPR…) and journals in the field (ACM Transactions on Graphics, Computer Graphics Forum). As of 10/2025, 67 publications were published within the project, 40 of them were co-authored by ESRs, mostly first-authors.

5. Partner organisations actively participated in the project. Secondments were implemented from 2022 until 2025. As a result of collaborations, 17 joint publications have been published, 12 of them are intersectoral, i.e. collaboration of an academic institution and industry.

6. PRIME’s outreach
- the project website: http://prime-itn.eu
- social media – a Twitter account ( https://twitter.com/ItnPrime ) and a PRIME YouTube channel with a video introducing the project to the general public

The project was presented through various means - press releases, local media appearances, presentations at fairs, participation at European Researcher’s Night, dedicated websites presenting the publications.

A key outreach highlight towards the expert community was PRIME’s participation in SIGGRAPH 2024, where PRIME:
- organized a Bird-of-feather session on Predictive rendering
- presented 2 technical papers authored by ESRs
- showcased 2 posters prepared by ESRs

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

Significant progress beyond the state of the art was made within the framework of PRIME, as reflected by the top tier publications supported by the PRIME project. All publications can be found at a dedicated project webpage:

https://prime-itn.eu/dissemination/publications/

For WP1 (improving capture), advances were made in practical appearance acquisition. These include a solution for the problem of SVBRDF acquisition using polarisation imaging and near-field display illumination, or an approach to acquiring spectral measurements of optical properties of translucent materials. Further results concerned the acquisition of BTFs (in particular sparse representations for BTFs, and a perceptual evaluation of BRDF models), and a method for procedural modelling of wood from photographs.

For WP2 (improved authoring), the initial focus was on translucent materials. Authoring and rendering them requires understanding human perception of appearance as well as optimisations to rendering technology, so there were multiple contributions in this direction. Contributions were also made to the area of modelling natural materials, such as wood and feathers.

WP3 (improving simulation) is at the core of the project, and there are contributions towards more efficient light transport simulation algorithms, neural methods, and denoising techniques. The contributions that were made include perceptual approaches for noise reduction, as well as denoising techniques. There were also advances in the representation and simulation of fluorescent materials.

WP4 (learning techniques) yielded a method for grid-based representation of complex signals, and in the area of generative modelling, a model that can learn view-consistent 3D scene variations from a single exemplar. Further results in this area include deep learning-based denoising methods and weathering simulations, GAN-based furniture placement in synthesized indoor scenes, and learning implicit representations for micro-geometry modelling.

All these results advance the state of the art in a transversal manner, with simultaneous contributions to different fields. Industries like entertainment (VFX, content generation) and manufacturing (computational design, creation of digital twins) are beneficiaries, as are industries that specifically develop tools for digital content creation (such as Adobe or KeyShot).

PRIME logo