Rendering the opaque transparent: Untangling light with bespoke optical transforms to see through scattering environments

Project Information

PhotUntangle

Grant agreement ID: 804626

DOI

10.3030/804626

Project closed

EC signature date 21 September 2018

Start date 1 November 2018

End date 31 October 2024

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 790 105,00

EU contribution

€ 1 790 105,00

Coordinated by

THE UNIVERSITY OF EXETER
United Kingdom

Periodic Reporting for period 4 - PhotUntangle (Rendering the opaque transparent: Untangling light with bespoke optical transforms to see through scattering environments)

Reporting period: 2023-05-01 to 2024-10-31

Summary of the context and overall objectives of the project

When light the emanating from an object propagates through an opaque material, such as living tissue or a hair-thin strand of optical fibre, it fragments and scatters multiple times. The emergent wavefront no longer forms an image of the object, because the spatial information it carries has been scrambled (see attached images). Reversing this scattering offers the prospect of using visible light for high-resolution imaging of structures deep inside the human body in a safe, non-ionising way. It has recently been shown that this light scattering can be characterised and inverted. Yet at the start of this project, arbitrary ‘optical inverters’ that can unscramble hundreds of light modes simultaneously, to efficiently reform an image, did not exist. The aim of this project was to understand how to design and build them.
We hope the approaches we have developed will lead to new ways to see deep inside the body and watch dynamic processes unfold at sub-cellular resolution. A driving near-term application is the development of real-time sub-cellular resolution imaging at the tip of a needle, by funnelling images through a fine strand of optical fibre. This promises a new window into living tissue, helping to guide biopsy needles and identify diseased cells.

Key aims of this project were to:
(i) Understand how to design a new class of optical elements that can efficiently unscramble scattered light on demand.
(ii) Build active transformers that can perform dynamically reconfigurable transformations at video-rates.
(iii) Apply this technology to unscramble light that has propagated through a moving multi-mode optical fibre in real-time, pushing towards ultra-thin micro-endoscopy.
(iv) Explore the wider applications of this technology to next generation imaging systems and beyond.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Objective 1: Understand
This objective was focussed on how to design of new types of optical system capable of reversing the effects of scattering media – such that light signals which have been scrambled can be unscrambled back into the state they were in before traversing the medium. We refer to these new devices as ‘optical inverters’, and they are based on an emerging technology known as ‘multi-plane light conversion’. We made good progress with this objective during the project:
Firstly, my team and I developed a new design algorithm which allows the trade-off between optical inverter fidelity, transform efficiency and channel cross-talk to be arbitrarily tuned [1].
Secondly, we designed an optical inverter capable of unscrambling up to 400 modes in parallel, that have propagated through a multimode fibre. This work represents a new approach to this very challenging problem, and our solution was made possible by taking advantage of hidden correlations within the mathematical structure of the transformation [2].
Thirdly, at the very end of the project we showed how to create ‘self-configuring’ optical inverters that automatically tune themselves to the unscrambling task at hand. We believe this is a crucial step towards real-world applications of this technology, opening up a variety of exciting new applications for high-speed adaptive control of scattered light [3,4].

Objectives 2 and 3: Build and Apply
We successfully constructed a reconfigurable optical inverter capable of handling up to 55 spatial light modes of arbitrary shape [1]. We coupled this optical inverter to a multimode optical fibre to physically reverse the fibre’s light scattering effects, and showed how this prototype could adapt to deliver a suitable new transform when the fibre was bent into different configurations [5].

Objective 4: Explore
My team and I have published a variety other papers related to the control of light through complex scattering media. Any given medium can be described by a mathematical operator known as a transmission matrix (TM), which can be experimentally measured. In [6] we showed how compressively sample the TM using a fraction of the conventional number of measurements. In [7] we also showed how the TM of a multimode fibre can be estimated with access only to the input end – key to continuously inverting a flexible optical fibre endoscope inserted into a sample. In a joint project led by the University of Glasgow (UK), we demonstrated 3D imaging through multimode fibres [8]. In [9] we discovered a new way to guide light through dynamic scattering media, by sending it through the slowest moving channels. In [10] we showed how the precise control of light scattering can be used to enhance the strength of optical tweezers that can move around particles with light. More broadly, I have also contributed to two roadmaps on the control of light within scattering media [11] and multimodal photonics [12].

References:
[1] Kupianskyi, Hlib, Simon AR Horsley, and David B. Phillips. APL Photonics 8, no. 2 (2023).
[2] Būtaitė, Unė G., Hlib Kupianskyi, Tomáš Čižmár, and David B. Phillips. Intelligent Computing (2022).
[3] Rocha, J. C., Būtaitė, U. G., Carpenter, J., & Phillips, D. B. (2025). arXiv preprint arXiv:2501.14129.
[4] Rocha, José CA, Terry Wright, Unė G. Būtaitė, Joel Carpenter, George SD Gordon, and David B. Phillips. Optics Express 32, no. 24 (2024): 43300-43314.
[5] Hlib Kupianskyi, Simon AR Horsley, & David B Phillips. Optica, 11(1), 101-112 (2024).
[6] Li, Shuhui, Charles Saunders, Daniel J. Lum, John Murray-Bruce, Vivek K. Goyal, Tomáš Čižmár, and David B. Phillips. Light: science & applications 10, no. 1 (2021): 88.
[7] Li, Shuhui, Simon AR Horsley, Tomáš Tyc, Tomáš Čižmár, and David B. Phillips. Nature communications 12, no. 1 (2021): 3751.
[8] Stellinga, Daan, David B. Phillips, Simon Peter Mekhail, Adam Selyem, Sergey Turtaev, Tomáš Čižmár, and Miles J. Padgett. Science 374, no. 6573 (2021): 1395-1399.
[9] Mididoddi, Chaitanya K., Robert J. Kilpatrick, Christina Sharp, Philipp del Hougne, Simon AR Horsley, and David B. Phillips. Nature Photonics (2025): 1-7.
[10] Būtaitė, Unė G., Christina Sharp, Michael Horodynski, Graham M. Gibson, Miles J. Padgett, Stefan Rotter, Jonathan M. Taylor, and David B. Phillips. Science Advances 10, no. 27 (2024): eadi7792.
[11] Gigan, Sylvain, Ori Katz et al. Journal of Physics: Photonics 4, no. 4 (2022): 042501.
[12] Cristiani, Ilaria et al. Journal of Optics 24, no. 8 (2022): 083001.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The papers mentioned above detail our work which goes beyond the state-of-the-art in a variety of ways. These include the design and construction of new optical devices to physically unscramble scattered light [1-3,5], new methods to measure how light has been scrambled [4,6,7], the first demonstration of time-of-flight 3D imaging through a multimode fibre [8], the discovery of a new way to guide light through dynamic complex media [9], and the development of new methods to optimise the transfer of optical momentum to trap microscopic particles in 3D [10].

Patterns depicting the intensity (brightness) and phase (colour) of light scramble by an MMF.

Experimental images recored through a pre-calibrated MMF