Periodic Reporting for period 1 - PRIME (Predictive Rendering In Manufacture and Engineering)
Berichtszeitraum: 2020-10-01 bis 2022-09-30
Project PRIME (Predictive Rendering in Industrial Manufacturing) is working on image synthesis which delivers results that one can rely on to be visually accurate. Engineering researchers participating in this Innovative Training Network (ITN) develop skills and protocols needed for industrial usage of Predictive Rendering (PR) technologies.
A variety of industries, from manufacturing to entertainment, are now moving towards predictive rendering. Application areas of such systems are in product design, architecture, sensor system calibration, training of autonomous vehicle systems, manufacturing control. But even established graphics application areas like movie visual effects (VFX) can benefit from them.
This is a cutting-edge area of applied computer science, in which European academia and industry are amongst the global technology leaders. An ITN network in this area greatly aids in maintaining and increasing the competitive edge of Europe in this regard, and trains young researchers in a promising, future-oriented and research-driven application area.
The current industry standard in Computer Graphics in most cases only aims to convince and immerse a viewer, but not to offer an actual prediction of the appearance of a 3D scene. Genuinely predictive rendering has the potential to be a game changer for several industries, way beyond Computer Graphics proper.
And widespread industrial use of predictive rendering is starting to be technically feasible. An example is the emergence of so-called digital twins of products: CAD models which include 100% accurate appearance descriptions. These have already, within the limits of existing technology, been introduced in the car industry, and the concept is spreading to other manufacturing industries. Such digital twins become even more valuable if one can reliably visualise and manipulate them, and use them for authoritative appearance inspection. Not all such use cases have been explored yet, as the technology needed for them is still either lacking (as in the case of e.g. fluorescent materials), or not sufficiently standardised for widespread industrial use.
A variety of industries, from manufacturing to entertainment, are now moving towards predictive rendering. Application areas of such systems are in product design, architecture, sensor system calibration, training of autonomous vehicle systems, manufacturing control. But even established graphics application areas like movie visual effects (VFX) can benefit from them.
This is a cutting-edge area of applied computer science, in which European academia and industry are amongst the global technology leaders. An ITN network in this area greatly aids in maintaining and increasing the competitive edge of Europe in this regard, and trains young researchers in a promising, future-oriented and research-driven application area.
The current industry standard in Computer Graphics in most cases only aims to convince and immerse a viewer, but not to offer an actual prediction of the appearance of a 3D scene. Genuinely predictive rendering has the potential to be a game changer for several industries, way beyond Computer Graphics proper.
And widespread industrial use of predictive rendering is starting to be technically feasible. An example is the emergence of so-called digital twins of products: CAD models which include 100% accurate appearance descriptions. These have already, within the limits of existing technology, been introduced in the car industry, and the concept is spreading to other manufacturing industries. Such digital twins become even more valuable if one can reliably visualise and manipulate them, and use them for authoritative appearance inspection. Not all such use cases have been explored yet, as the technology needed for them is still either lacking (as in the case of e.g. fluorescent materials), or not sufficiently standardised for widespread industrial use.
The current report covers the first two years of PRIME’s (Predictive Rendering in Industrial Manufacturing) implementation from 10/2020 until 09/2022. During this time:
1. The consortium successfully commenced the project implementation, all parties actively participate in it.
2. All 15 ESRs were hired in an open, transparent, impartial and equitable recruitment process. In order to increase the effectiveness of the recruitment process, a semi-centralized recruitment process was used. Due to force majeure (COVID-19), PRIME experienced delays in hiring. The last two ESRs started in month 24.
3. Training of ESRs has commenced
ESRs were enrolled into PhD programmes, and primary and secondary supervisors are conducting the day-to-day training of the fellows. Each ESR formulated a Personal Career Development Plan.
Training needs were collected from ESRs by the training coordinator. The following network-wide trainings took place:
- 12/2020 – Kick-off meeting
- 02/2021 - 06/2022 PRIME reading Group self-managed by ESRs
- 05/2021 or 01/2022 - MSCA administrative training for all ESRs
- 07/2021 – post-EGSR PRIME training
- 12/2021 - midterm review and network meeting
- 07/2022 – post-EGSR PRIME training
- 10/2022 – second network training
4. ESRs commenced working on individual research projects. Five ESRs already reached an important milestone by the end of PRIME’s second implementation year – peer-reviewed scientific publications with their contributions were published.
5. Collaboration between beneficiaries and partner organisations is working well. Secondments started in early 2022, when the pandemic situation improved.
6. PRIME’s outreach commenced
The project website ( http://prime-itn.eu ) was created and is regularly updated. Social media accounts were created – a Twitter account ( https://twitter.com/ItnPrime ), a PRIME YouTube channel and a Facebook Group.
An introductory video presenting the project to the general public is available on the YouTube channel. The project was presented through various means - press releases, local media appearances, presentations at fairs, European Researcher’s Night 2022, separate websites for presentation of the publications.
Outreach towards the scientific community is the second communication pillar. Besides participation, some ESRs have already presented their peer-reviewed publications (e.g. SIGGRAPH 2022, EGSR 2022, Eurographics 2022).
1. The consortium successfully commenced the project implementation, all parties actively participate in it.
2. All 15 ESRs were hired in an open, transparent, impartial and equitable recruitment process. In order to increase the effectiveness of the recruitment process, a semi-centralized recruitment process was used. Due to force majeure (COVID-19), PRIME experienced delays in hiring. The last two ESRs started in month 24.
3. Training of ESRs has commenced
ESRs were enrolled into PhD programmes, and primary and secondary supervisors are conducting the day-to-day training of the fellows. Each ESR formulated a Personal Career Development Plan.
Training needs were collected from ESRs by the training coordinator. The following network-wide trainings took place:
- 12/2020 – Kick-off meeting
- 02/2021 - 06/2022 PRIME reading Group self-managed by ESRs
- 05/2021 or 01/2022 - MSCA administrative training for all ESRs
- 07/2021 – post-EGSR PRIME training
- 12/2021 - midterm review and network meeting
- 07/2022 – post-EGSR PRIME training
- 10/2022 – second network training
4. ESRs commenced working on individual research projects. Five ESRs already reached an important milestone by the end of PRIME’s second implementation year – peer-reviewed scientific publications with their contributions were published.
5. Collaboration between beneficiaries and partner organisations is working well. Secondments started in early 2022, when the pandemic situation improved.
6. PRIME’s outreach commenced
The project website ( http://prime-itn.eu ) was created and is regularly updated. Social media accounts were created – a Twitter account ( https://twitter.com/ItnPrime ), a PRIME YouTube channel and a Facebook Group.
An introductory video presenting the project to the general public is available on the YouTube channel. The project was presented through various means - press releases, local media appearances, presentations at fairs, European Researcher’s Night 2022, separate websites for presentation of the publications.
Outreach towards the scientific community is the second communication pillar. Besides participation, some ESRs have already presented their peer-reviewed publications (e.g. SIGGRAPH 2022, EGSR 2022, Eurographics 2022).
There has already been significant progress beyond the state of the art within the framework of PRIME, as reflected by the top tier publications resulting from research supported by the PRIME project.
For WP1 (improving capture), advances were made in practical appearance acquisition. These include a solution for the problem of SVBRDF acquisition using polarisation imaging and near-field display illumination [Nogué et al. 2022], or an approach to acquiring spectral measurements of optical properties of translucent materials [Iser et al. 2022]. Expected results in the coming months include acquisition of BTFs, and a method for procedural modelling of wood from photographs.
For WP2 (improved authoring), the focus was on translucent materials. Authoring them requires understanding human perception of appearance, so there were contributions in this direction [Lagunas et al. 2021], [Lanza et al. 2022]. Expected future results include a similarity measure for translucency, and neural network-based representations of translucent materials.
WP3 (improving simulation) is at the core of the project, and there are contributions towards more efficient light transport simulation algorithms [Rath et al. 2022, Grittmann et al. 2022], and denoising methods [Firmino et al. 2022]. More contributions are expected in the coming months, including perceptual approaches for noise reduction. Fluorescence, present in a wide range of materials, has also received attention within PRIME, with relevant advances in the representation and simulation of spectral data [Hua et al. 2021, Tódová et al. 2021, Fichet et al. 2021].
WP4 (learning techniques) has a method for grid-based representation of complex signals [Karnewar et al. 2022], and in the area of generative modelling, a model that can learn view-consistent 3D scene variations from a single exemplar [Karnewar et al. 2022]. Future results in this area include deep learning-based denoising methods or learning implicit representations for micro-geometry modelling.
All these results advance the state of the art in a transversal manner, with simultaneous contributions to different fields. Industries like entertainment (VFX, content generation) and manufacturing (computational design, creation of digital twins) are beneficiaries, as are industries that specifically develop tools for digital content creation (such as Adobe or Luxion).
For WP1 (improving capture), advances were made in practical appearance acquisition. These include a solution for the problem of SVBRDF acquisition using polarisation imaging and near-field display illumination [Nogué et al. 2022], or an approach to acquiring spectral measurements of optical properties of translucent materials [Iser et al. 2022]. Expected results in the coming months include acquisition of BTFs, and a method for procedural modelling of wood from photographs.
For WP2 (improved authoring), the focus was on translucent materials. Authoring them requires understanding human perception of appearance, so there were contributions in this direction [Lagunas et al. 2021], [Lanza et al. 2022]. Expected future results include a similarity measure for translucency, and neural network-based representations of translucent materials.
WP3 (improving simulation) is at the core of the project, and there are contributions towards more efficient light transport simulation algorithms [Rath et al. 2022, Grittmann et al. 2022], and denoising methods [Firmino et al. 2022]. More contributions are expected in the coming months, including perceptual approaches for noise reduction. Fluorescence, present in a wide range of materials, has also received attention within PRIME, with relevant advances in the representation and simulation of spectral data [Hua et al. 2021, Tódová et al. 2021, Fichet et al. 2021].
WP4 (learning techniques) has a method for grid-based representation of complex signals [Karnewar et al. 2022], and in the area of generative modelling, a model that can learn view-consistent 3D scene variations from a single exemplar [Karnewar et al. 2022]. Future results in this area include deep learning-based denoising methods or learning implicit representations for micro-geometry modelling.
All these results advance the state of the art in a transversal manner, with simultaneous contributions to different fields. Industries like entertainment (VFX, content generation) and manufacturing (computational design, creation of digital twins) are beneficiaries, as are industries that specifically develop tools for digital content creation (such as Adobe or Luxion).