Resolution Enhanced Microscopy for Medical Diagnostics

Final Report Summary - REMEDI (Resolution Enhanced Microscopy for Medical Diagnostics)

Executive Summary:

Measurements on a nanometre-scale require nanometre-stability on the time-scale of the measurement. Conventional light-microscopes are not well suited for the stability requirements of super-resolution, which critically depends on the vibration susceptibility and the thermal behaviour of all opto-mechanical components, including immersion fluids connecting objective and sample.
The consortium has therefore developed a novel microscope platform concept, employing new materials with maximal inherent vibration damping capabilities, minimal or matched thermal expansion of all critical parts and, where all these measures were not sufficient, new concepts for feedback-controlled dynamic realignments. As the major enemy of nanometre stability are thermal drifts and perturbations, measures were taken to maintain a constant temperature (to 0.01°) for sample, objective and all system components involved in keeping or changing the relative position of sample and objective.
With this approach we were able to reduce short-term instabilities to levels where one could barely measure them. However, single-molecule localisation-based microscopy techniques require extended image acquisition times. We thus set out to reduce the time needed for achieving sufficient localisation accuracy in order to avoid long-term drifts, which are very hard to control in routine medical diagnostics facilities. Shortening image acquisition times was addressed by employing novel sCMOS camera technology in combination with read-out schemes tailored especially for their application in single molecule localisation imaging.
The consortium addressed resolution enhancement not only in two, but also in three dimensions. The large chip-size of the new sCMOS camera allows projecting two images obtained at different focal positions next to each other onto a single chip, and by using suitable algorithms the differential-focus recordings allow reducing the vertical position accuracy down to 20 nm levels, too. Moreover, in order to increase selectivity, each of the two images, separated by their different z-position, can be split into two emission colours and projected, side by side, on the large sCMOS chip.
Technology development was rounded up by soft- and firmware development for single molecule localisation and FPGA-based image pre-processing. Our goal was to extract super-resolution images online while thousands of localisation images are being gathered. To this end we have developed new single molecule localisation algorithms and have embedded them into the system software.
Validation was performed using two key microscopy applications: 1) diagnostic/experimental pathology using the expression of important receptors in breast cancer and 2) tumour immunology using the example of lipid raft-associated signal transduction in lymphoma cells.

Project Context and Objectives:
Valid diagnostic criteria should directly reflect functioning and malfunctioning of the patient’s organism. Since most diseases can ultimately be traced to failures of the molecular machinery underlying cellular processes, the ideal diagnosis would be based on direct inspection of the relevant molecular events, instead of analyzing organismic phenomena only. Consequently, an enormous effort has been put into the development and application of high-throughput techniques screening patients for significant molecular disease-markers. Nevertheless, this diagnostic tool can only complement, but not replace – e.g. in the field of solid tumour diagnosis – the exhaustive information provided by classical histopathological microscopic tumour typing. However, it is not merely the presence or absence of molecular components, but their interaction that determines cellular health. In addition, a protein’s functional state is dependent on its microenvironment. Thus spatial resolution constitutes an irreplaceable advantage in diagnostics.
Interactions relevant for cell-based diagnostics span a wide range of length scales: protein-protein interactions occur on a nanometre scale, protein complexes and clusters of proteins measure several tens to hundreds of nanometres and functional rearrangement of proteins to cellular substructures, can be observed at scales of hundreds of nano- to several micrometres.
Unfortunately, conventional (light-)microscopy techniques can only access 3D-volumes from 200 x 200 x 500 nm upwards, whereas electron microscopy is limited to the observation of a small number of different protein targets in fixed samples. It was thus the goal of REMEDI to adapt recently developed optical single-molecule-imaging techniques (referred to as PALM, PALMIRA, STORM, dSTORM) for the use in medical diagnostics facilities in order to address the questions of protein presence/absence, interaction, and spatial distribution in tissue samples and live cells with ultra high sensitivity.
These novel super-resolution techniques combine the advantages of optical microscopy (live-cell imaging, multi-colour imaging, 3-D imaging) with the resolution of electron microscopes and therefore can be used to determine the exact position of insular molecules within a cell with a precision of approximately 20 nanometres. Many thousands of such insular molecule localization maps are then combined into a global molecular map of the object under inspection. This will allow for studying the molecular machineries of a cell “on site”, rather than in bulk on a chip. It furthermore will extend the field of vision of the light microscope from a macroscopic level, where organismic phenomena reflect a given disease, down to the molecular level, where disease-relevant molecular events take place.
Thus, the main technological goal of the REMEDI project was the further development of super-resolution microscopy by
• increasing the resolution, which currently lies in the range of 20-50 nm, down to 10 nm or less,
• increasing the speed of image acquisition and reconstruction from minutes to seconds, or less,
• increasing the usability of the instrument for routine applications by implementing a new class of online analysis tools,
• adding 3-D capabilities, which have recently been demonstrated for the first time as well as multicolour imaging capabilities.
These technological objectives were implemented in a novel instrument based on the innovative mineral-casting microscope platform patented by TILL I.D. GmbH (patent DE 10 2006 039 896), which has been designed with such high-resolution applications in mind. The mineral-casting frame of the platform exhibits a 14-fold improvement in vibration damping properties over cast aluminium or steel, and its low thermal expansion-coefficient matches that of stainless steel. The novel frame-design acts as a 3-dimensional optical bench-system, integrating all moving parts directly into the microscope body. Together with a novel focusing concept much greater over-all stiffness is achieved ensuring the highest possible stability sufficient for super-resolution techniques. By using only tightly feedback-controlled movements of all critical parts, we achieved an unprecedented precision- and speeding advantage over competing microscope platforms. Overall, the new microscope platform combines single molecule sensitivity in 3D, multi-colour capabilities to increase selectivity, the ability to work with live cell material, an exceptionally high operating speed and last but not least, easy handling in a routine-environment.
For the biological validation and the exploration of the potential of the novel super-resolution microscope, two exemplary applications were chosen. Pilot studies in the REMEDI project were carried out in pathology, focussing on the molecular-resolution of immuno-histochemically stained solid tumour samples, and in tumour biology, where we looked at nanoscale protein and lipid structures in lymphoma and leukaemia cells. In both cases, the goal was to visualize individual protein molecules and their interaction with other proteins, all of which play a crucial role in the disease process, i.e. cancer, under study, and which hence should provide subtle, but valid, criteria for diagnosis, prognosis, and therapy monitoring.

Project Results:
Please see attached document for description of the main S&T results including tables and figures.

Work package 1: REMEDI specification refinement, development of microscopy hardware-platform

Work package leader: Stefanie Graf (TILLID)
Partners involved: TILLID, TILL, Andor, Niendorf, RUNMC

Summary
In the first work package, all beneficiaries started to specify a detailed list of required mechanical, optical and thermal stability properties of the super-resolution platform including hardware (microscope and sCMOS camera) and single molecule localisation software. The required features for two exemplary applications, pathology and immunobiology, were defined by Niendorf and RUNMC and conceptually implemented by the technology partners TILLID, TILL and Andor. Establishing a platform with maximum thermal stability turned out much more involved than anticipated, but in the end we had a microscope platform in hand, which provided the required stability with respect to stiffness, vibration-isolation and thermal stability. To achieve this goal, we had to incorporate several feedback control loops and completely re-design the volume enclosing sample and all parts needed for changing and subsequently maintaining the relative position of sample and objective. Additionally, 3D multicolour capabilities were implemented.
Task 1.1: Update requirement specification
In this first task, TILLID compiled a precise description of the required mechanical, optical and thermal stability properties of the super-resolution microscope platform. In particular the conflict between the necessity for an extremely robust and stiff sample-holder and the requirement to scan large areas of the sample in order to find regions worth imaging with super-resolution had to be solved. Beneficiary TILL concentrated on defining the requirement specifications for a novel single molecule localization software, made the final choice of the actual detection algorithm and tested the method to verify the results and clarify the requirements and specifications.
Andor specified entirely novel FPGA design and readout patterns for its scientific CMOS (sCMOS) camera that was relaunched shortly before the REMEDI. The super-resolution imaging device will feature a front-illuminated CMOS sensor and has to be conform to particularly demanding standards of imaging including fixed pattern noise correction, dual amplifier (high-low gain) optimization and the implementation of image filters suitable for spurious noise removal, sensor blemishes and other spurious events present in the image during data acquisition.
The Institute of Pathology has optimized the immunohistochemistry and molecular detection approaches (FISH, RT-PCR) of the estrogen receptor, progesterone receptor and her2 neu protein and communicated the results to TILLID for implementation into the REMEDI microscope platform.
For immunobiology applications to be performed by RUNMC, the REMEDI platform should provide a stage holding the usual coverslips and/or cell chambers of different size. Moreover, two-colour excitation and detection capabilities were requested to image multiple membrane components simultaneously.

Task 1.2: Establish microscope platform exhibiting maximal stiffness, stability and vibration isolation (passive stability)
This initial task turned out significantly more demanding than anticipated. It was found that the inherent stiffness and stability of the mineral casting microscope platform was compromised by the construction of the aluminum sample holder, which had been designed “light” in order to facilitate rapid scanning of extended areas. The different expansion coefficient of sample holder (aluminum) and microscope body material (mineral cast) led to large thermal drifts in all three dimensions. We therefore went through a sequence of designs for sample holder and focus-drive, which were all replaced by new approaches when unsatisfactory stability test resulted. This occurred repeatedly! One of the first lab prototypes of the microscope platform is shown in fig. 1.1.
In the second period, a complete redesign of all critical moving parts, i.e. sample-stage and objective focus-drive was undertaken. The concept combined the core-elements z-drive and stage into a single ensemble made from ZeroDur, a special ceramic material with zero thermal expansion. However, this new approach still did not bring the desired stability results. In a series of test measurements (using fluorescent beads and the nano-positioning software MOL-NANOPOS from TILL Photonics, see also WP3), we could identify additional factors compromising the stability performance of the instrument. The differential thermal inertia of the different materials used, i.e. the speed of their reaction to thermal changes, caused time-dependant deviations from the anticipated balance of drifts. This nonlinear behaviour necessitated reconsidering the overall platform design again.

Fig. 1.1: Lab prototype of open microscope platform

In our last and successful approach we kept the concept of moving the sample in x and y by pushing the sample chamber on a reference surface, but we completely redesigned the z-drive itself again and chose a special stainless steel instead of ZeroDur. The z-drive is now encapsulated in a “sarcophagus” carrying the sample holder and at the same time providing the reference surface for the drive itself. It also encapsulates the new objective changing mechanism, now being a carousel, not a slider. In the latest design, the chosen objective and the voice coil are clearly separated, which minimizes thermal coupling that was responsible for excursion dependent thermal expansions in previous versions. The effect of residual thermal effects is further reduced by the large heat capacitance of the stainless steel sarcophagus providing powerful thermal buffering and the fact that sarcophagus and drive-train are made from exactly the same material, thus expansions tend to compensate it other. Fig. 1.2 shows a photograph of the unit, which pervades the mineral casting and thus separates the heat source (drive) from the driven objective holder.

Fig. 1.2: Photograph of the actual focus-drive and sample holder unit (c), which pervades the mineral casting body of the microscope.
Task 1.3: Establish microscope platform exhibiting maximal speed without affecting stability, integrate feedback-control loops for all moving parts (active stability)
The stability characteristics of the microscope platform are crucial in order to achieve single molecule localization precision with nanometer resolution. Only if the overall detection system is “stable” enough, the actual software-based analysis can yield significant results. Our concept for stage movement, i.e. pushing the specimen-holder instead of carrying it using a usual stage, reduces the scan-speed in x and y. However, by implementing a novel illumination concept for scanning large areas, whereby the sample is moved (being pushed) continuously with a constant speed while the transmitted light is flashed (strobe mode), we were able to compensate for the slower scan-speed by the steady movements during a scan instead of the usual stop-and-go. In addition, the mass of the sample holder-material is no longer an issue under these conditions.
At the beginning of the second reporting period, the stability of the z-drive constituted the major remaining issue, no matter of which material (first ZeroDur, then stainless steel) it was built. Nanometre-stability comprises a vertical and a horizontal component. The vertical component was addressed by closing a feedback-loop between a position-sensor attached to a reference-surface directly connected to the holding surface for the respective objective and to the surface, on which the sample holder is being held. Thus the position sensor measured the actual distance between objective and sample and its signal could be used for the digital control of the focus-drive ensuring that the accuracy of the drive-electronics reflects the actual focus-position noise with reference to the sample.
The horizontal component of the temperature-dependent position-stability with respect to sample and objective could not be integrated into a feedback loop. Building most components of the focus-drive/sample-holder assembly from the same material minimized thermal drifts in x-y-direction, but as the bearings of the drive assembly could not be made from this material, there was a small, yet noticeable error-component left making it advisable to shield the assembly against thermal perturbations. This was addressed in task 1.4 below.

Task 1.4: Develop concept for optimal thermal management
Despite the greatly reduced susceptibility to thermal perturbations super-resolution below 100 nm still necessitates extreme thermal stability. Therefore, thermal variations had to be kept below 1/10 of a degree, which can only be achieved by maintaining a constant, yet very weak airflow that does not couple vibrations into the system. A concept was developed employing a combination of convection and radiation heating and using optimized air channels in the foam cabinet of the microscope. The first prototypes were available at the end of the first reporting period and are shown in fig. 1.3.

Fig. 1.3: Left: Partial view of the microscope with sample holder (sample removed) above the microscope’s workspace. In the foreground, the slits and filters for the airflow system are visible. Right: Microscope with dismantled and partially closed lid showing the foil heating elements.

In the second reporting period, the following four measures were taken to reduce thermal variations in the system below 0,01°:
(i) separating the voice-coil drive with its excursion dependent heat-production from the objective/sample holder assembly. The voice coil was placed below the mineral casting body, while the rest of the drive is above, and there are only two carbon-rods extending through the microscope body and connecting the two.
(ii) holding the sample in a large, mushroom-like sample-holder-top which shields the interior of the microscope against external airflows.
(iii) providing a constant, but very weak airflow from beneath the microscope lid, thermally adjusted to within 0.1°. It employs a combination of convection and radiation heating and optimised air channels in the foam housing of the platform.
(iv) surrounding stage and sample-holder by an aluminium enclosure, which opens only when a sample is loaded or withdrawn. The enclosure also holds the condenser-optics for transmitted light observation. It slides to one side during loading and is precisely repositioned afterwards so as to warrant optimal optical performance. The aluminium enclosure permits a defined air flux and protects the sample against ambient light.
Finally, these measures turned out to be sufficient for an optimal thermal management.

Task 1.5: Establish asynchronous, protocol-driven experiment control, synchronized with camera readout
TILLID has developed a real-time Linux-based electronics concept, which uses a hybrid Microcontroller/FPGA architecture and is capable of controlling all components of the system, including camera triggers, in real-time. It enables illumination schemes where a slit of laser-light is scanned over the sample in synchrony with the leading edge of the rolling shutter of the sCMOS camera used. Since the two different clock-rates of camera and digital scanner required a continuous internal recalibration and subsequent synchronisation, a real-time protocol (TMSI) was established, which allows a perfect synchronicity for extended periods of time.
Another implemented feature was a continuous monitoring of the precise focus position with feedback-controlled real-time focus readjustment.

Task 1.6: Implement multicolour capabilities
We developed an optical concept for dual-emission imaging on a single camera-chip (fig. 1.4). When testing the first opto-mechanical prototypes, the alignment precision between the two semi-images was identified as major problem. It has to be in the range of the desired resolution, i.e. 10 nm, which corresponds to an angular precision of 0.002°! This can only be achieved with piezo-elements that need to be fine-tuned for recalibration before every single experiment. As this was not very practical as well as not achievable, an alternative concept was developed using mathematical tools, i.e. algorithms for removing pixilation artifacts and shifting one image until it perfectly coincides with the position of the other one.

Fig. 1.4: Equal distortion concept for dual emission imaging on a single camera chip.

During comprehensive tests in the second period, the new prototype (fig. 1.5) proved fully functional, i.e. the diffraction-limited images of single molecule fluorescence remained diffraction limited after passage through the dual emission module.

Fig. 1.5: Photograph of the whole microscope with integrated dual emission module.

Task 1.7: Extend localization to third dimension
We developed a simplified, highly robust concept for bi-plane observation. A first version for the extension of super-resolution into the third dimension had been developed in 2011, but extensive tests revealed that it was very difficult to align and images were not as robust as needed. We therefore took a much simpler approach for splitting images in the beam-path of the dual emission module by inserting a custom-made prism-combination into the beam-path of the dual emission accessory. It can either be placed into the intermediate image plane or close to the camera chip.

Work package 2: Camera-technology development

Work package leader: Colin Coates (Andor)
Partners involved: TILLID, Andor

Summary
Throughout the duration of the project, Andor contributed multiple scientific cameras: (a) an interline CCD camera, (b) a vacuum cooled scientific CMOS (sCMOS) camera capable of sustained 30 fps (c) a water cooled sCMOS camera capable of sustained 100 fps. The sCMOS cameras in particular are seen as a highly enabling for fast single molecule based super-resolution. Andor has also untaken effort to improve the readout flexibility of the sCMOS cameras through FPGA improvements, including faster sustained ROI speeds and ‘rolling shutter’ line scan mode of imaging, specifically for this project. A splitter box solution was characterised and tested, facilitating the direct channelling of super-resolution raw data from the camera to a powerful ‘real-time’ processing unit.
In the final stages of the project, a newer variant of the sCMOS sensor design was integrated into the Andor camera platform. This variant is based on a 4-transistor (4T) pixel type, which offers notably higher QE of > 70%. This in turn presented a means to maximizing signal to noise ratio, counteracting the limited photon budget available from imaging single molecules at high speed.

Task 2.1: Optimization and delivery of front-illuminated CMOS camera
sCMOS camera delivered for the project was a 5.5 megapixel microlens front-illuminated device with 5 transistor (5T) pixel design (~60% QE max), operating at 30 frames/sec (full 5.5 Megapixel) < 2 electrons read noise. Subsequently, Andor developed a new Cameralink interface with greater bandwidth, enabling us to deliver a further sCMOS camera capable of 100 frames/sec sustained. This camera was also specially configured with water cooling to minimize fan vibration, a critical parameter for super-resolution measurements. The cameras had full external triggering functionality and offered sub-window and pixel binning selection through SDK or acquisition software.

Task 2.2: Flexible readout schemes
Work was carried out by Andor to modify the SDK to enable higher flexibility in terms of readout schemes, with the specific likely demands of this approach in mind. This block of work included provision of a row clock output that was used to synchronize the CMOS rolling shutter exposure sweep to that of the moving line scan illumination that is targeted by the project instrument.

Task 2.3: Direct data access interface
The project required the camera to offer the possibility of accessing the data direct from the capture interface. A 3rd party ‘splitter box’ from Vivid Engineering was characterised and tested, facilitating the direct channelling of data from the camera to a powerful ‘real-time’ processing entity, such as a Matrox card.

Task 2.4: Delivery of higher QE scientific CMOS camera
A newer variant of the scientific CMOS sensor design was successfully integrated into the Andor camera platform. This variant is based on a 4-transistor (4T) pixel type, which offers notably higher QE of > 70% max, at the expense of global shutter readout mode. For the current project, this is not an issue as the remaining rolling shutter mode of the 4T sensor type is well suited to the line scan illumination mechanism that is being implemented. This provision was essential, given the critical need for maximizing signal to noise ratio in the face the limited photon budget available from imaging single molecules at high speed.
Work was also undertaken to further optimize the camera to overcome some remaining drawbacks of sCMOS technology. This included: (a) FPGA minimization of column fixed pattern noise in the image, (b) implementation of a Global Multiplier, such that data is capable of filling the full 12-bit or 16-bit digital range, dependent on the amplifier mode selected, (c) further enhancement of the FPGA Dynamic baseline Clamp algorithm, required to ensure stability of the DC offset baseline (upon which data sits) across extended kinetic series, ensuring a higher degree of quantitative validity of the output data. A final version of the water-cooled sCMOS camera with this new higher QE sensor is available to the project.

Work package 3: Image acquisition and analysis

Work package leader: Frank Lison (TILL)
Partners involved: TILLID, TILL, Niendorf, RUNMC

Summary
In this work package, beneficiary TILL Photonics designed and implemented a novel single molecule localization software called MOL-NANOPOS and a graphical user interface. Furthermore, the setup was tuned for enhanced optical resolution by reducing the minimum step size for motion of the z-drive to 10 nm, optimising the timing model for focus detection and focus correction, developing real-time soft- and firmware with protocol-based and quasi continuous focus drift compensation and characterising heat dissipation of the motorised actor and timing of the heat expansion of system components. Finally, the platform has been tested intensively in terms of focal and thermal stability, analysis speed and accuracy by localization of fluorophores.

Task 3.1: Design and implementation of a robust single-molecule detection and localization algorithm
TILL designed software algorithms for single molecule localization and implemented the selected self-adjusting algorithm in a MatLab-based software tool called MOL-NANOPOS, which requires almost no input from the user for analysis. In addition, a graphical user interface was designed (fig. 3.1). The software module uses four parameters for detecting local maxima in an image and fitting the point spread function (PSF) of the extracted local sub-image to detect the position of the emitting molecule.

Fig. 3.1: Main window of the MOL-NANOPOS software for single molecule localization

Task 3.2: Choice of hardware environment (graphics card, multi-core CPU, ...) to run the analysis software based on state-of-the-art hardware available at this stage of the project
We decided to use the state-of-the-art hardware featuring a high degree of parallelism and facilitating the commercialization of the measurement system after the end of the project.
As a first step, a support code was generated and implemented, which in principle can be used for multi-core CPUs. Due to the decision to use the MatLab platform as a basis for the software package, further improvements are possible, since MatLab offers extensions, which theoretically allow parallel computations on vector-based graphics chips (GPUs) and computer clusters as well.
A parallelisation of the localisation algorithms was achieved using the OpenCL framework. It enables the execution of the resulting code on a wide range of hardware platforms, including CPUs, GPUs, FPGAs and DSPs, and even heterogeneous systems to allow for greater flexibility in the future. Since currently, the best computing performance can be found on GPUs, we decided to focus on GPUs for our tests.
Due to the results of several experiments with focus on enhanced optical resolution, we considered the effect of z drift and the precision of localization of fluorophores in the sample volume. Robust localization of fluorescent molecules with enhanced resolution requires seamless integration of focus drift detection and refocusing of the objective lens in terms of optimisation of the resulting z- positioning noise and the right timing model. Depending on the detection bandwidth, we were able to achieve a focus stability of 15-50 nm (pp).
By minimising heat dissipation from all actors and applying a closed loop thermal stabilisation of the instrument (see WP1), we had been able to eliminate thermal drift on a time-scale of several seconds. In order to allow real-time data acquisition with enhanced optical resolution using standard protocols, TILL Photonics developed an improved firmware for focus drift compensation. The firmware implementation allows protocol-based and quasi-continuous focus drift compensation with variable bandwidth of up to 1000Hz and now operates on a millisecond time scale to eliminate remaining focal-stability issues. An additional complex FPGA-based control loop allows the continuous adaption of the focus position. The combination of two complementary control loops allows not just the precise positioning of the z-drive itself, but also the precise position of the z-drive relative to the sample. As the sample position is not only measured immediately prior to image acquisition but is continuously sampled and adapted, long breaks in between measurements and consequential large steps in z-positioning are avoided and thus z-position precision increased significantly.

Task 3.3: Speed-optimized implementation of algorithm derived in Task 3.1 within parallel programming environment, based on the hardware choice made in Task 3.2
For a proof-of-principle the localization algorithm was first implemented in pure MatLab code for the test and validation phase and the complete data analysis pipeline was sub-divided into different single tasks. To further improve the speed of the analysis workflow the most demanding computational tasks were identified and the subroutines have been re-written in C to increase the speed.

Task 3.4: Test of analysis speed and accuracy using simulated and recorded datasets streamed at various sampling rates
At an early stage, several test series have been performed by TILLID and TILL with an open frame microscope and the MOL-NANOPOS software to localize fluorescent beads over time, which had their focus mainly on temperature changes and the resulting x-y drift due to the thermal issues explained in WP1 (task 1.2).
At a later stage, the focus was set on analysis speed and accuracy. First, we tested the speed for time-lapse data recording and could generate a data point with ±20nm positioning precision each 2.1ms. As a result we are able to acquire to stream images with 500 Hz.
Secondly, the speed for area screening was tested intensively by localization of fluorophore clusters in a tile experiment. Recording a tiled image of 30 x 30 single images took less than 100 ms, which corresponds to roughly 0,1 ms for the acquisition of one image file. Furthermore, the z correction due to tilted and bended cover slide surface could be compensated with the focus hold system of the microscope platform. The z position uncertainty could be compensated with the focus correction hardware, which allows to pre-scan areas of interest for data streams to collect data for images with enhanced optical resolution.

Task 3.5: Embedding of software and, if necessary, additional hardware, into the microscope. The handling of the microscope should be as similar to that of a conventional fluorescence microscope, with the analysis process running in the background
To further improve the image quality, we evaluated different approaches for improvement in terms of system stability. The performance of the chosen localisation algorithm was also checked with independent measurements, which revealed that the achievable precision is not limited by the analysis method (as expected). The main focus to further improve the resolution must be on mechanical stability of the setup. The method itself is clearly capable of achieving <10nm localisation precision.

Task 3.6: Testing of the resulting platform with recorded data streams
The stability of the microscope platform was intensively tested in series of experiments (see also WP4 and task 3.4). Some data is shown in Fig. 3.2 below, which leads to the conclusion that the stability if sufficient to reconstruct a super-resolution image after image acquisition.

Fig. 3.2: Results from stability test over a time period of 30 min – sufficient to acquire enough image data to reconstruct a super-resolution image.

Task 3.7: Testing with live-recorded data
This task was only started in 2013 during the project extension, when the novel sCMOS camera and the necessary readout schemes were ready to be used with the microscope platform.
In the meantime, most of the firmware sCMOS functionality was implemented into the image acquisition software, which allows running fast applications in live and protocol mode. Figure 3.3 shows data, which were recorded in live mode at four images per second using structured illumination microscopy (SIM) for enhancing the optical resolution. Low drift is required in order to avoid smear out effects and artefacts, which lower optical resolution due to position changes in between different raw images. The results proved the stability characteristics of the platform.

Fig. 3.3: The image shows actine from chicken ganglion (acquired with Zeiss 63x NA 1.4 oil objective, sCMOS camera EDGE). Fluorescence excitation wavelength was about 488nm. In this image structured Illumination was used in order to enhance optical resolution to 150nm in x and y, and for z section about 220nm. Thermal induced drift was in the range of 100nm with in 60 minutes in x and y. The stability in z is Δz<60nm.

Work package 4: REMEDI integration, test, validation

Work package leader: Rainer Uhl (TILLID)
Partners involved: TILLID, TILL, Andor

Summary
The integration of all hardware and software components has started late due to the delay in WP1 and WP2 and was an evolutionary tasks due to the many stages of development of the platform. Thus, the final REMEDI instrument has a completely different design than the first functional unit, but it provides maximum short- and long-term stability under conditions permitting maximum speed and a maximum degree of automation. It is a fully functional routine instrument, which excels in its transmitted light performance, its wide-field fluorescence capabilities and it provides STORM-based super-resolution!

Task 4.1: Integration of all hardware and software components
Due to the delay in delivery of the sCMOS camera and the thermal problems during the mechanical construction of the microscope platform, the integration of all hardware and software components could only be started in the second year of the project. Due to the many stages of development (see WP1), their integration into a functional unit was a truly evolutionary task. The instrument available at the end of the second reporting period bears little resemblance with the first functional unit shown in figure 4.1. Fig. 4.2 shows the interior of the instrument before it was encapsulated into a thermally and optically isolating foam cabinet.
Furthermore, while in 2011 we had obtained from Andor one of the first sCMOS camera units of the “Neo-type”, this unit was replaced by the new model Zyla in 2012. The Zyla is not cooled as extensively, but it allows novel read-out schemes required for some of the STORM-modes.

Fig. 4.1: Early microscope setup with conventional xyz-stage and Andor Neo (in the upper background) imaging.

Fig. 4.2: Microscope setup with conventional xyz-stage and Andor Neo (in the upper background) imaging.
Task 4.2: Performance test of REMEDI platform
First tests were be performed by TILLID and TILL without camera and as a next stage, the sCMOS camera Neo was integrated into the microscope setup (see fig. 4.1). As described in detail in the preceding, it was not until the end of 2012 that the REMEDI platform achieved its predicted performance figures. These, however, were tighter than those of any available instrumentation. REMEDI provides maximum short- and long-term stability under conditions permitting maximum speed and a maximum degree of automation. It is a fully functional routine instrument, which excels in its transmitted light performance, its wide-field fluorescence capabilities and it provides STORM-based super-resolution!

Task 4.3: Validation
This task was started in 2012 was only completed during the project extension. We could successfully prove the super-resolution capabilities of the REMEDI platform by measuring tubulin structures (Fig. 4.3 and 4.4). Thanks to the GPU-routines we developed and employed, the image could be reconstructed in a matter of a few seconds, only, even though several million localizations were to be computed.

Fig. 4.3: Imaging of tubulin structure in a dendritic cell. Left side: at conventional low resolution (red channel), right side: at super-resolution composed of 1.5 billion localizations (green channel). Scale bar represents 10 µm.

Fig. 4.4 Detailed view of tubulin structure. Left side: at conventional low resolution (red channel), right side: at super-resolution (green channel). Scale bar represents 2 µm (image section of fig. 4.3).

Work package 5: Application of REMEDI for diagnostic/experimental pathology

Work package leader: Axel Niendorf (Niendorf)
Partners involved: Niendorf

Summary
Our work package in this consortium was the generation of clinical and biological relevant targets to be addressed by ultra-resolution microscopy. We therefore first did work up on our case collection where we finally identified a total number of 5.000 cases. We then did a follow up in order to find out whether the patient had a relapse or died and in addition, whether they had any further therapy outside the clinic where the primary surgery had taken place.
Approximately 200 matched pairs (i. e. 400 cases) were then appropriately characterized by means of immunohistochemistry and in addition also by gene expression analysis.
We then identified interesting targets as well on the morphological as on the molecular level. In this study, clinically relevant parameters such as the estrogen receptor, progesterone receptor HER2, CD 301 and Ki67 were investigated.
In addition we investigated a collection of cases with ductal carcinoma in situ (DCIS) were the primary anatomical question, i. e. the question whether the tumour has already invaded this running tissue was in our focus.

Task 5.1: Investigation of interfaces at early invasion in breast cancer
The Institute of Pathology identified a special subgroup (estrogen receptor low and progesterone receptor high), which has been analysed morphologically as well as molecularly in order to determine the advantage over conventional histopathological information.
In addition we have entered another round of our follow-up initiative. This means that individual patients must be personally contacted in order to find out how they performed in a clinical sense. In addition to the above-mentioned experiments we have completed our marker set by the implementation of a new class of receptors i.e. glycoprotein receptors. These experiments were performed in collaboration with the group of Prof. Wagener (Labormedizin Universität Hamburg Eppendorf). This is of great interest due to the fact that it has never been performed before and allows at least hypothetically that relevant biological classification of breast cancer can be performed from theoretical background.

Task 5.2: Investigation of receptors
In 314 breast cancer samples from patients (standard therapy) protein and mRNA expression of the hormone receptors ESR1 and PGR were determined by IHC and qRT-PCR, respectively. Protein and mRNA expression of ESR1 and PGR were measured with a high concordance for the two different methods (kappa 0.87 and 0.71 respectively). Within the double-positive group, we identified a new clinically relevant subgroup that is characterized by high PGR/ESR1 expression ratios and especially favourable clinical outcomes. This observation was not limited to a certain therapy regimen. These results are validated in an external chip data set (GSE9434). The Chip Data Set GSE3494 belongs to a new Chip generation and allows reliable ratings of ESR1 and PGR. 156 cases have been selected for stratification (patients not older than 80 years, tumour size not above 5 cm).
Task 5.3: Investigation of receptor-ligand-interaction
In our initial immunohistochemical study, the expression profile of 42 different biomarkers (apoptose inhibitors, kinases, transcription factors, tumour suppressor and regulatory proteins, differentiation markers and growth binding receptors) in both tumour specimens and peritumoural normal tissue of a cohort of 179 invasive ductal breast cancer (IDC NOS) was evaluated.
For each marker the specific staining, the staining pattern and sublocalisation were evaluated and the marker-feature-combinations were correlated with the clinical outcome. The aim of the study was to evaluate the prognostic potential of these markers in a defined case series of IDC NOS cases to establish if either a single marker, a panel and/or a marker-feature combination could improve the prognostication using the Nottingham Prognostic Index (NPI). Out of 42 immunohistochemical markers and out of 232 marker-feature-combinations 15 single markers and 26 marker-feature-combinations correlated significantly (all p-values < 0.05) with the clinical outcome. Eight markers correlated with a worse and 11 markers with a good prognosis. The antiapoptotic protein survivin (BIRC5) was one of the prognostic factors in our IDC NOS patient cohort, which could identify patients with an increased risk of cancer related death.
In addition we collected our samples with ductal carcinoma in situ (DCIS) from women with breast cancer. These specimens are processed in a special way, such that the complete resection specimen is initially inked and then embedded in paraffin blocks to finally generate whole mount sections. This approach gives the most possible precise adjustment of whether an in situ carcinoma has an invasive component or not. In other words you could call this an excessive way to process a surgical specimen.
Within this collection of DCIS-cases we identified a number of regions with equivocal basal-membrane boarders. This means that one could argue whether in this regions invasion had already occurred or not. These regions will be further investigated by ultra-resolution microscopy in order to prove the clinical usefulness of this matter.

Work package 6: Application of REMEDI for monitoring of nano-scale receptor alterations in viral and tumour diseases

Work package leader: Alessandra Cambi (RUNMC)
Partners involved: RUNMC

Summary
The plasma membrane protects the cell from the external environment and allows cell signal transduction. Membrane proteins and lipids are compartmentalized into well-defined sub-micron sized clusters (nanodomains), which mediate several cellular processes ranging from cell adhesion, pathogen uptake and cell proliferation. Changes in nanodomain organization and composition appear associated with an altered physiological status of the cells, such as upon pathogen infection or malignant transformation, being responsible for deleterious prolonged signalling events. These nanodomain changes might therefore represent an unprecedented diagnostic tool to distinguish healthy from diseased cells at very early stages. In this WP6, we have analyzed a variety of membrane receptors as well as juxtamembrane cytoskeletal structures at an unprecedented high-resolution to determine their nanoscale organization in or underneath plasma membrane domains, after thorough optimization of several labelling procedures. A new generation of ultrahigh resolution microscopes will lead to a totally new type of diagnostic tool.

Task 6.1: Investigation of raft-associated signalling in lymphoma cells
To avoid unnecessary waste of precious cancer patient material as long as the microscopy platform was being assembled, tested and optimized, we have determined the optimal conditions to keep in culture several lymphoma cell lines that represent different types of the disease: Nalm-6 cell line is a pre-B cell lymphoma; JY and Raji cell lines reproduce the mature B cell lymphoma; RPMI-8226 represents a plasma-cell lymphoma. After culturing, these cells were analysed by flow cytometry using standard staining protocols available in the laboratory. In addition, we have used a human acute monocytic leukemia cell line (THP-1 cells) to test super-resolution labelling of membrane receptor. These cells were labelled with specific antibodies against a membrane receptor called CD44 and analysed by super-resolution Stochastic Optical Reconstruction Microscopy (STORM). Representative images for this receptor are shown in fig. 6.1.

Task 6.2: Investigation of virus entry via lipid rafts
In agreement with the other partners during the annual meeting in January 2011, we decided not to focus on virus-infected cells but rather exploit well-established immune cell types or the B-lymphona cell lines. This would already provide enough biological material to be tested in the REMEDI platform still needing some optimization at that early stage. However, we did perform some super-resolution imaging of CD44 and DC-SIGN, which are both membrane receptors involved in virus binding and uptake, on immature DCs (fig. 6.2).

Fig. 6.2: STORM analysis of CD44 and DC-SIGN at the plasma membrane of human antigen-presenting cells.

Task 6.3: Establishment of field-deployable medical diagnosis procedures based on molecular-resolution microscopy
This task has not been completed, because the delay in the technical WPs did not allow us to test the molecular resolution capability of the microscope. However, partners have agreed to image a number of biological samples with the REMEDI platform in the second half of 2013.

Task 6.4: Generation of protocols to fluorescently label plasma membrane proteins and lipids in healthy and diseased cells to develop a novel standardized diagnostic tool
As anticipated in the previous periodic report, we have characterised the lymphoma cell lines by labelling several plasma membrane proteins that belong to the tetraspanin family. To do this, we have optimised the protocol for Labelling of Membrane proteins of cell monolayer for use in Confocal and Epi-Fluorescence Microscopy. In addition, we have attempted to determine the structure of large cellular assemblies called podosomes that are formed underneath the ventral membrane of dendritic cells. In this way, we can exploit TIRF-based STORM imaging. Four different proteins have been stained and analysed in this way. Below, representative staining of these proteins is shown (fig. 6.3).

Fig. 6.3: TIRF-based STORM imaging of adhesive structures (podosomes) formed at the basal membrane of human dendritic cells in contact with the substrate. This type of structures is an excellent model system to test super-resolution capability of the setup under development.

Potential Impact:
Potential impact
The REMEDI project had a clear impact for the FP7 HEALTH program, because we were able to deliver a novel super-resolution instrument that is capable of identifying alterations of protein expression patterns or molecular signalling events on site i.e. directly in cells and tissue where a distinction between healthy or diseased is required.
The novel technologies, which enable extending the resolution of the conventional light microscope significantly beyond its current diffraction limits, down to single molecule resolution, has a major impact not only on basic research, but also on medical diagnostics. This “pointilist light microscopy” (PLM) that creates 3-D images by localizing individual molecules within a cell, finally unites the two completely different worlds of light- and electron microscopy. Both require completely different sample preparation schemes, and once prepared for one, a sample is no longer usable for the other. With the help of PLM the same material can be used for mass screening of cells in extended cell cultures or tissue and for intracellular localization studies with molecular resolution. Thus, it has become possible to even use patient’s data (e.g. histological slides) prepared many years ago and examine them with molecular resolution. At present we can only speculate about the immense impact this increase in sensitivity and spatial resolution will have for healthcare.
The prototype REMEDI instruments have been and will further be used to demonstrate the general capabilities of the new technology, and validation of the super-resolution microscope has taken place in two applications with major relevance for public health: a clinical/diagnostic histopathology context and a tumour immunology scenario. The work on these two applications has led to the development of improved molecular procedures (procedures and results obtained with current microscope versus new super-resolution microscope comparison) leading to the identification of close variants for the two applications.
In addition, the sCMOS camera developments that were undertaken for this project will be of benefit to several other application areas, with particular usage within live cell microscopy.
For example, the rolling shutter line scan mechanism that was developed for the scanning modality is of further use for an emerging type of microscopy called ‘Light Sheet Microscopy’, which is of particular benefit to high resolution study of relatively large living samples, such as developing embryos. The high quantum efficiency ‘4T’ camera that was developed will be of intense interest to a wide spectrum of biomedical and physical science academic end users, due to the ability to simultaneously deliver high quantum efficiency, 1 electron read noise and high resolution at speeds of up to 100 frames/sec, at a relatively accessible price.
With these new technologies, the commercialization impact of the project will be highly positive not only for the companies involved, but through anticipated licensing agreements for a much greater group of companies and institutions.
Two main scientific fields were directly targeted by the REMEDI consortium. Application 1 in diagnostic /experimental pathology in breast cancer as example for solid tumours and application 2 in tumour immunology/lymphoma, which both are gaining more and more relevance in the public health system. The consortium was able to identify clinically relevant targets in breast cancer for the study of ultra-resolution microscopy, which can serve as potential targets for drug development in breast cancer treatment.
Additionally, with respect to the application in tumour immunology, the research performed here was mainly based on the optimization of techniques and protocols for standard labelling of cells. The basic procedures already exist, but they were adapted to the experimental goals. However, we find opportune to point out that this application contributed to generate results that go far beyond the 3,5 years of this REMEDI project. In fact, the development of novel sophisticated microscopes that could be used for diagnostics of diseases at an early stage will certainly benefit the health/medicine sector. Finally, novel insights will be given in early events of invasion and transformation of malignant cells. The exploration of early signal transduction in lymphoma / leukemia diseases will contribute to find out targets for potential drug development, which is of great importance in the medical sector.

Main dissemination activities and exploitation of results
Beginning in the second period, the results of the REMEDI project were made available to the public in a variety of dissemination and demonstration activities, such as the participation in scientific and industrial conferences with oral presentations or posters of development and results, and the publication of project results in scientific journals. The dissemination activities of the second project period are listed in detail in Templates A1 and A2, but the main activities were the following:

Conferences
- Focus of Microscopy
o 2010, March 27 – April 04, Shanghai, China
o 2011, April 17-21, Konstanz, Germany
o 2013, March 24-27, Maastricht, The Netherlands
- European Cell Mechanics Meeting 2011, Amsterdam, The Netherlands
- USCAP Meeting 2012, Vancouver, Canada
- SSSMLMS 2012, Lausanne, Switzerland
- Tetraspannins Conference 2012, Nijmegen, The Netherlands
- Dijon Domains 2012, Dijon, France

Publications
- Van den Dries et al., “Interplay between myosin IIA-mediated contractility and actin network integrity orchestrates podosome composition and oscillations”, Nature Communications, 2013 (4), 1412-24.
- Meddens et al., “Automated podosome identification and characterization in fluorescence microscopy”, Microscopy and Microanalysis, 2013 (19), 180-9.

More dissemination activities are to come as soon as the final tasks of WP5 and WP6 are completed.

List of Websites:

The main achievements of the REMEDI project have been published on the Coordinator’s website and can be accessed via the following link: http://www.till-id.com/remedi.html
The website will be up to date as soon as final results are available.

Table 3 gives an overview on all beneficiaries of the consortium with their contact persons.

Institution Contact Person Contact details
TILL I.D. GmbH
Bahnhofstraße 89
82166 Gräfelfing
Germany Dr. Stefanie Graf www.till-id.com
graf@till-id.com
Andor Technology plc
Millenium Way Springvale 7
BT127AL Belfast
United Kingdom Dr. Orla Hanrahan www.andor.com
Pathologie Hamburg-West
Lornsenstraße 4
22767 Hamburg
Germany Dr. Tatjana Ballauf www.pathologie-hh-west.de/index.php?id=2
Radboud University Nijmegen Medical Center
NCMLS
Institute of Tumour Immunology
Geert Grooteplein 26
6525 GA Nijmegen
Netherlands Dr. Alessandra Cambi www.tumor-immunology.com/
FEI Munich GmbH formerly TILL Photonics GmbH
Lochhamer Schlag 21
82166 Gräfelfing
Germany Dr. Frank Lison www.till-photonics.com/

Final Report Summary - REMEDI (Resolution Enhanced Microscopy for Medical Diagnostics)

Related documents

Share this page

Download