Light is fundamental to how humans and machines perceive and interpret the
world. Visual data is our richest source of information, and we depend upon it
to understand the nature of our surroundings. In computer graphics, *rendering*
simulates the propagation of light to generate photo-realistic images of
virtual environments. However, in scientific applications, the goal is often
the reverse: not to create images, but to extract meaning from the ones we
already have.
The UNRAVEL project explores *Physically Based Inverse Rendering* (PBIR), a
computational approach that flips the traditional process of rendering on its
head to address this latter problem. Instead of producing images of a virtual
environment, PBIR reconstructs the environment constrained by physical laws and
visual data (e.g. photographs or other types of measurements). By simulating
how light must have interacted with the environment to produce observations,
PBIR infers a principled physical description of the world.
Take, for example, medical tomography scans or satellite-based observations of
the Earth. The underlying measurements contain valuable information about the
composition of tissue or the atmosphere, but that information is not directly
accessible from the raw sensor data. Subject experts like doctors or climate
scientists require structured, quantitative 3D models that express relevant
physical quantities like absorption or aerosol concentration. Producing such
models from the raw sensor data requires complex computational processing.
Traditional methods for solving such reconstruction problems rely on highly
simplified models. Tomographic reconstruction, for instance, normally assumes
that monochromatic X-rays pass straight along a straight line, neglecting
scattering and spectral effects. Likewise, established satellite image
reconstruction methods treat each pixel as an independent vertical column,
which breaks down in the presence of clouds or slanted viewing angles.
PBIR offers a powerful general alternative. By simulating the full physics of
light transport within a virtual environment—including scattering, absorption,
and complex geometry—PBIR can render synthetic images that mimic what a scanner
or satellite would observe. These synthetic images are then compared to
real-world data, and the virtual model is iteratively adjusted until the two
become consistent. This process offers an unprecedented level of generality and
produces physically interpretable answers. Despite this potential, PBIR is not
yet practical in most real-world applications. Current methods are
computationally expensive, lack robustness, and do not scale to large datasets.
The UNRAVEL tackles these challenges through two avenues. First, it seeks to
dramatically improve the scalability and robustness of PBIR methods, developing
algorithms that can operate efficiently on complex, real-world data. Second, it
aims to bridge the gap between PBIR and scientific applications, building
proof-of-concept systems that demonstrate the use of PBIR to solve concrete
inverse problems in several scientific fields. We selected several suitable
applications, specifically in the area of tomography, 3D printing, remote
sensing, and architecture.
