Skip to main content

Perceptual Analysis and Simulation of Real-World Materials Appearance

Final Report Summary - PASIMA (Perceptual analysis and simulation of real-world materials appearance)

The main aim of the project is to understand human perception of material appearance. As the material appearance is a complex phenomenon we focussed not only on surface material textural representation but mainly on its illumination and view-dependent effects. The ultimate goal of the project was to analyse appearance of real-materials represented by bidirectional reflectance distribution function (BRDF), bidirectional texture function (BTF) datasets and to exploit its results in various application areas, e.g. for effective material appearance representation, acquisition, editing, or synthesis.

The first part of the project was focussed on perceptual analysis of material appearance and its relation to mathematical features.

We ran series of psychophysical experiments. First, we studied visual perception of smooth material appearance degradations across different materials/shapes and compared the performance of human subjects with performance of several standard textural descriptors.

In the later experiment we analysed behaviour of human subjects when viewing interactive realistic rendering of planar material patches. The subjects could control their orientation as well as their illumination and were asked to identify those combinations of viewing and illumination angles producing the most visually appealing appearance of the material. As a result of these two experiments we have found that sensitivity of human perception is very often related to a local contrast in the data and can be simulated by features based on a local variance. Furthermore, we have found that people use different search strategies for regular and non-regular materials and they prefer to observe non-flat samples at regions that are perpendicular to their view direction.

We have also studied perceptual sensitivity of human subjects to different environment illumination of realistic stimuli represented by several BTF datasets. The results of this analysis were then applied for selection of simplified representation of individual environments. The size of minimal environment illumination representation by means of a finite set of point lights was derived from a statistics based on local mean variance of the smoothed environment map.

In the second part of the project we focussed mainly on exploiting the results achieved in the first part, for improvement of acquisition and processing techniques dealing with complex illumination and view dependent datasets (BRDF, BTF).

As the input datasets we have used public BTF datasets or our own BTF measurements. Due to insufficient accuracy of standard BTF registration approaches, we developed a novel registration method compensating for material and registration planes misalignment. This method allows more accurate alignment of measured material features and exploits the power of data compression methods more effectively.

As view and illumination dependent datasets represent massive data whose capturing is very time and resources demanding task we employed techniques of perceptual analysis of material appearance for development of intelligent sampling strategies for accurate material appearance measurement using sparse samples only.

An initial attempt of relaxation of sparse uniform samples did not bring sufficient improvement as such high contrast features (e.g. specular highlights) require relatively dense sampling. Therefore, we focussed on a simplified representation of illumination and view dependency using prior knowledge about marginal behaviour of BRDF subspace with fixed elevation angles. We represent complex apparent BRDFs using only eight predefined sparse slices using under 200 samples. Besides the fast and relatively accurate BRDF space reconstruction from such sparse data, the approach does not require any complex setup and can be realised by eight predefined continuous moves of light and camera over the material. The method allows an efficient representation of anisotropic and even non-reciprocal BRDFs as well as their fast acquisition using inexpensive consumer hardware.

To show the method's functionality for fast on-site material appearance measurement we have built a portable BRDF measurement setup, that is able to measure material's BRDF and approximate BTF in less than 10 minutes using the proposed sparse sampling. The setup consists of two arms holding light emitting diode (LED) light and consumer camera. The methods reliability was thoroughly tested on various BRDFs of anisotropic materials.

The similar principle was further extended for development of adaptive sampling strategy for highly accurate BTF measurement using the main measurement device. The proposed strategy achieves significantly better reconstruction quality using the same number of samples. However, the lack of dense accurate BRDF data makes evaluation of any adaptive sampling strategy very difficult. Therefore, the measurement of such a dense ground truth data is one of our future challenges, where we face, additionally to a long measurement times, also massive storage size of several gigabytes.

A description of the main results achieved within the project:

1. understanding perceptual differences of individual types of materials
2. identification relation between human perception of materials and corresponding mathematical features
3. applying these computational features predicting human perception for improvement of BTF data compression; effective representation of environment illumination; and development of sparse sampling strategies for more economical measurement of BRDF and BTF data
4. development of a novel BTF data registration method
5. development of fast and inexpensive portable BRDF acquisition setup
6. development of adaptive technique of dense BRDF data measurement.

The results show that by properly selected sparse subset of directions and their perceptually plausible reconstruction we are able to achieve reasonably accurate, however, very fast and affordable appearance measurement of almost any real-world material. As the proposed measurement does not require any expensive parts we believe that our research may evolve into a simplified appearance, namely BRDF, spatially varying BRDF (SVBRDF) and BTF designs. We believe that due to the project results, the capturing of accurate material appearance will not be restricted only to enterprise or research labs but more widely accessible for general professionals demanding to maintain accurate material appearance in their application areas, e.g. dermatology, architecture, interior design, culture heritage digitisation, visual safety simulations, etc.