Periodic Reporting for period 1 - BMC Rendering (Bayesian Monte Carlo for Global Illumination)
Periodo di rendicontazione: 2016-08-01 al 2018-07-31
Photo-realistic rendering (PBR, as it is as well called Physically Based Rendering) is the task of using computers for producing synthetic images of digital scenes that are indistinguishable from what a real photo of that same scene would look like. It is a very challenging task which requires high quality geometric models of the objects in the scene, defining and assigning realistic materials for each of those objects, and a physically-based light propagation simulation.
The realistic simulation of light propagation is a key factor for producing photo-realistic images, since it allows computing the exact amount of light that would arrive to the camera sensors after multiple bounces and interactions with the scene objects, hence realistically reproducing a virtual photo of the scene. However, such a process is very computationally demanding, and usually requires a large amount of time and resources for producing a single image (recall that, for a movie, at least 25 images per second are required, in general).
PBR has a large set of applications, ranging from movies and computer games industry to flight simulators. A typical example of the importance of this technology is architecture, where architects could interact with a realistic visualisation of a building yet to be constructed. This allows them to eventually adapt the building to their own purposes taking into account the chosen materials and illumination considerations. Other important applications can be found in the car and aeroplane industries, etc. The impact of this technology in the society is quite relevant, since it can help companies save millions of euros in expensive real prototypes by resorting to realistic digital simulations instead.
The goal of this project has been to develop innovative methods for accelerating the synthesis of photo-realistic images. To this end, we resort to machine learning-based techniques, with a particular focus on a relatively recent approach called Bayesian Monte Carlo. These techniques permit learning from the synthesis of previous images and re-use the learned information to more efficiently compute new photo-realistic images. However, due to their complexity, a direct application of these techniques to PBR is cumbersome. This project has the general objective of making such application feasible.
During the period covered by this project, solid steps were given toward the goal described above. A publication on a top-tier journal has already been made, and others should follow in the near future. In particular, a publication on how to interactively learn local features of the incident light at each particular scene point, and another on the application of deep learning techniques to high dimensional rendering problems will mark the capstone of the research conducted within this project and could open doors for industrial applications.
The realistic simulation of light propagation is a key factor for producing photo-realistic images, since it allows computing the exact amount of light that would arrive to the camera sensors after multiple bounces and interactions with the scene objects, hence realistically reproducing a virtual photo of the scene. However, such a process is very computationally demanding, and usually requires a large amount of time and resources for producing a single image (recall that, for a movie, at least 25 images per second are required, in general).
PBR has a large set of applications, ranging from movies and computer games industry to flight simulators. A typical example of the importance of this technology is architecture, where architects could interact with a realistic visualisation of a building yet to be constructed. This allows them to eventually adapt the building to their own purposes taking into account the chosen materials and illumination considerations. Other important applications can be found in the car and aeroplane industries, etc. The impact of this technology in the society is quite relevant, since it can help companies save millions of euros in expensive real prototypes by resorting to realistic digital simulations instead.
The goal of this project has been to develop innovative methods for accelerating the synthesis of photo-realistic images. To this end, we resort to machine learning-based techniques, with a particular focus on a relatively recent approach called Bayesian Monte Carlo. These techniques permit learning from the synthesis of previous images and re-use the learned information to more efficiently compute new photo-realistic images. However, due to their complexity, a direct application of these techniques to PBR is cumbersome. This project has the general objective of making such application feasible.
During the period covered by this project, solid steps were given toward the goal described above. A publication on a top-tier journal has already been made, and others should follow in the near future. In particular, a publication on how to interactively learn local features of the incident light at each particular scene point, and another on the application of deep learning techniques to high dimensional rendering problems will mark the capstone of the research conducted within this project and could open doors for industrial applications.
We have successfully developed a prototype for adaptive sampling for illumination integrals within the context of Bayesian Monte Carlo, which should be object of a publication in the near future. To reach this goal, we had to develop a set of tools which allow a better understanding and clearer comparisons of the different approaches used for estimating the value of illumination integrals. One of these tools was directly applied for deriving a new method for computing optimal sample weights which minimize the error when synthesizing photo-realistic images and resulted in a publication in the journal Computer Graphics Forum. Other tools and algorithms (not predicted in the original work-plan) will be published soon. Finally, we would also like to highlight the research conducted on the hot topic of deep learning for high dimensional rendering problems.
The work performed in this project exceeded the original expectations in terms of scientific throughput, with five potential scientific publication of high quality instead of the two initially expected. These works represent important breakthroughs for improving the efficiency of physically-based rendering methods.
We have developed a set of tools which allow getting more insights on the problem of reducing the error when synthesizing photo-realistic images. These tools per se are already of great interest for the scientific community for the knowledge they bring about such a crucial problem in PBR. We have shown that these tools can be used not only for better understanding the issues at stake, but also for improving the state of the art, as exemplified by the Optimal Sample Weights application published in the journal Computer Graphics Forum. Moreover, we have developed a prototype which is able select the most contributive sample positions in real time, hence actively learning the local features of the incident light at each point of the scene during rendering. These are important steps towards further reducing the computation time of each photo-realistic image for the same image quality, hence contributing to the trend of widening the range of application of photo-realistic image synthesis. Recall that this technology can potentially allow companies save important resources by resorting to digital simulation instead of developing real expensive prototypes. Moreover, reducing the computation time of these images, can also have an important impact in the movie and game industry.
We have developed a set of tools which allow getting more insights on the problem of reducing the error when synthesizing photo-realistic images. These tools per se are already of great interest for the scientific community for the knowledge they bring about such a crucial problem in PBR. We have shown that these tools can be used not only for better understanding the issues at stake, but also for improving the state of the art, as exemplified by the Optimal Sample Weights application published in the journal Computer Graphics Forum. Moreover, we have developed a prototype which is able select the most contributive sample positions in real time, hence actively learning the local features of the incident light at each point of the scene during rendering. These are important steps towards further reducing the computation time of each photo-realistic image for the same image quality, hence contributing to the trend of widening the range of application of photo-realistic image synthesis. Recall that this technology can potentially allow companies save important resources by resorting to digital simulation instead of developing real expensive prototypes. Moreover, reducing the computation time of these images, can also have an important impact in the movie and game industry.