Beyond Range: Innovating Fluorescence Microscopy

1. Converging Communities

It was the summer of 2005 when in the laboratory of Fred Wouters we started the first autonomous runs of our new in-house developed high throughput fluorescence lifetime imaging (FLIM) system. However successful that development was, one key element in allowing the biomedical community to benefit from FLIM systems, to enable the screening technologies we were pioneering [1] and to get closer to the routine application of FLIM, was still missing: replacing the expensive, comparatively fragile, and not very user-friendly, multi-channel plates used in FLIM with better technologies. The basic idea was to find a silicon-based detector capable of demodulating fluorescence signals on a bi-dimensional array of pixels. Frequent searches of primary literature returned only a few, though interesting, papers including: Nishikata et al. [2] on the development of a dedicated lock-in imager in CCD technology and, Mitchell et al. [3,4] on the adaptation of conventional CCD cameras to achieve demodulation. Unfortunately, ad hoc technologies for lock-in imaging appeared to be operating at modulation frequencies too low to be useful for FLIM [5,6] and the available CCD technologies required modifications to the camera electronics of systems which would not operate at an ideal frequency range for FLIM.

After a long literature and patent review for “lock-in imagers”, we became aware of the existence of Time-of-Flight (ToF) based technologies developed for other scopes: solid-state smart pixels were developed to replace multi channel plate- or LiDAR-based technologies to quantify the distance of objects by measuring the travelling time of a pulse of light from and to the detector/light-source. A camera that measures meter ranges with centimetre precision using the speed of light as a ruler is capable of measuring nanosecond times with tens or hundreds of picoseconds precision, exactly what we do routinely in fluorescence lifetime sensing. Soon after, we flew to Zurich to secure an agreement with Centre Suisse d’Electronique et de Microtechnique to acquire one of their prototypes. Within a year, we generated the first FLIM image using ToF technology, and the first image with an all solid state system (source and detector) within a frequency range relevant to biology [7,8].

Currently, ToF technologies are often developed also for fluorescence and fluorescence lifetime detection [9–15]. It is apparent that the engineering and biomedical communities may benefit from each other, and we are seeding new technologies and applications that could have significant impact on basic sciences and health care in the medium term.

2. Beyond Range

Time-of-Flight technologies rely on the measurement of the time it takes for light to travel from an illuminator positioned near the camera system back to the sensor after being reflected by an object. The most intuitive detection scheme is in the “time domain” [16,17]: (i) the illuminator provides a short pulse of light that triggers the electronics of the camera, (ii) light travels to and is reflected back from objects and then (iii) photons impinge on the pixels of the camera providing an image of the object and carrying information about its distance. To achieve the distance measurement, each individual pixel is equipped with electronics that sort photoelectrons according to the arrival time of photons, into different storage areas or digital counters. Statistics on the photon-arrival times will then provide information on the distance of the object. A second detection scheme is in the “frequency domain” where instead of a pulse of light, sinusoidally intensity-modulated illumination is used [16,18]. Light that is reflected by more distant objects will be detected with a larger phase-delay compared to light reflected by objects nearer to the camera. Once again, smart pixels operate on photoelectrons to generate distance maps, using fast Fourier Transform algorithms or similar approaches [19,20].

The range of applications of ToF technologies is very broad. In the automotive industry, ToF technologies are being developed to replace or assist radar in assisted breaking or parking, and also for the safe triggering of airbags. ToF technologies could also be used for the inspection of goods on production lines, for the control of automated doors, to provide depth perception for robots, to enhance object recognition, for the survey of buildings or environments, and for motion capture in biomedical applications [21–23].

Until recently, fluorescence was not within the range of applications of ToF technologies. This has changed rapidly in the past few years [7–9,11,15,24,25]. The only conceptual difference between ranging applications and fluorescence detection is that instead of detecting the light reflected by objects, a ToF sensor detects light emitted by fluorophores. The optical path between illuminator-fluorophores-camera is constant and any differences in timing properties of light depend on the photo-physics of the fluorophores. Upon excitation with light of the appropriate wavelength, a fluorescent molecule undergoes a transition to its excited state then decays back to its ground state by emitting a photon at a longer wavelength [16]. Often, the average time that a fluorophore spends in its excited state is in the nanosecond range, the same range that ToF technologies are capable of detecting.

Fluorescence is not the only process that can de-populate the excited state of a fluorophore. Excited state energy can transfer to other intra-molecular states causing non-radiative transitions, phosphorescence, and/or reversible or irreversible structural changes to the molecule [16]. For instance, some fluorophores change their fluorescence lifetime upon protonation of the chromophore or protonation of molecular regions surrounding the chromophore itself. Fluorescence lifetime sensing can thus be used to probe the acidity of samples doped with fluorescent pH-sensitive molecules. Using the same principle, biologists have exploited the environmental sensitivity of fluorophores to quantify pH, oxygen ion concentration, the metabolic status of cells, and many other properties of biological samples [26].

Furthermore, energy can migrate from a donor fluorophore to an acceptor molecule by various mechanisms [27]. The most common energy migration mechanism exploited in biology is Foerster-type resonance energy transfer (FRET). FRET causes the shortening of the fluorescence lifetime of a donor molecule with an inverse sixth power law of the donor-acceptor molecule [16,27,28]. Typically, FRET is sensitive within the 10 nm range (similar to the dimension of globular proteins), providing an invaluable tool to probe protein-protein interactions and conformational changes.

Modern technologies in the field of genetic engineering, biology and biophysics permit the design of molecules, cell lines, and animal models that encode information on specific biochemical reactions into the fluorescence of molecules that can be analyzed by FLIM.

FLIM technologies have flourished in the past two decades; however, FLIM is still a specialist tool far too often confined to the biophysical laboratory. So, what is limiting the use of FLIM in the biomedical community, medical care, and drug discovery fields? FLIM systems are comparatively expensive and they require dedicated imaging systems and specialist users. However, FLIM technologies are commercially available and they are not any more expensive than other high-end high-tech systems. Therefore, new technologies should be cost-effective, and—above all—they will have to be easy to integrate with common imaging setups, uncomplicated to operate, and simple to maintain.

Integrated silicon-based camera systems offer the opportunity to provide user-friendly and cost-effective setups that—together with the newest advances in imaging processing to simplify data analysis [29–32]—could introduce FLIM-based assays to the wider biomedical and pharma communities. In this context, ToF technologies can play a primary role.

Another feature often integrated in ranging technologies is very interesting for FLIM applications. Most ranging applications need to avoid the motion blur that occurs when multiple acquisitions of the same sample are necessary to perform the distance measurement. If imaged objects move faster than the exposure time, significant artifacts are generated in the distance maps. The best strategy to avoid motion artifacts in fluorescence lifetime imaging detection is to acquire all the needed information within a single exposure: all photon-electrons generated in the photo-sensitive layer are acquired and sorted into analog or digital storage areas within the pixels [7,11,23]. After a 10–100 ms exposure time, a number of images with photons detected at different time/phase delays will be recorded. An on-board field-programmable gate array or a computer can process this information to provide distance measurements. ToF technologies can then provide single-shot real-time imaging systems for high speed FLIM, an application that is currently only available in specialist laboratories [8,33,34].

There are many similarities between the engineering of systems for ranging applications and fluorescence sensing. However, there is a fundamental difference that can hinder the use of ToF technologies for FLIM. The number of photons that can be emitted from a fluorophore is limited. Even the most photostable fluorophores can only emit photons at a rate limited by the inverse of the fluorescence lifetime. The photo-toxicity of light and the speed of biological processes further constrain the photon-budget available for bioimaging.

The dependence of range- or lifetime-estimates on photon-statistics is equivalent and has been described in several papers [5,6,20,35]. However, ranging applications benefit from higher signal-to-noise ratios compared to fluorescence. On the other hand, fluorescence imaging provides higher signal-to-background ratios relative to ranging applications. Some ToF technologies have been optimized to work in the presence of high signal-to-background ratios in order to operate outdoors [36]. However, on-pixel background suppression is less important for lifetime detection and may cause loss of information in FLIM applications. Furthermore, ToF technologies developed for ranging applications are too photon inefficient, providing non-ideal fill factors, quantum efficiencies and collection noise [7].

Fortunately, these shortcomings are not caused by technological or physical hard limits but only by the opportunities and investment available. On-pixel de-multiplexing, for instance, does not only alleviate motion blur artefacts, but also permits the detection of all photons during an acquisition cycle without the need for subsequent exposures. Therefore, ToF imagers with higher photon-efficiencies and pixel resolution will be available as soon as adequate investment from both private industry and public funding bodies is in place.

3. A Vision for the Future

Silicon-based FLIM cameras derived from ToF technologies may outperform tube-based detectors (photo-multiplier tubes (PMTs) and multi channel plates) within the next 5 to 10 years. In recent years, Hybrid PMT detectors have been adopted at a fast pace in fluorescence microscopy. These detectors provide high quantum efficiencies (∼40%) in the visible range thanks to GaAsP photocathodes with optimal timing specifications (∼100 ps instrument response function, no after pulsing and 10MHz maximum count rate). These specifications are hard to meet, but ToF technologies may be used to generate clusters of pixels equipped with on-chip FLIM capabilities to provide a large sensitive area (in the order of millimeters) and very high count rates (in the order of gigahertz) [11]. This will be possible if fill factor and quantum efficiencies can permit detection probability in excess of 40–50% therefore resulting in very fast, sensitive detectors capable of being integrated in microscopy setups without the use of expensive specialist electronics. However, the most competitive applications for ToF technologies are probably the replacement of multi-anode PMTs and multi channel plates. These technologies provide only low quantum efficiencies (∼10%) and, often, limited maximum count rates (∼1 MHz). Linear arrays of silicon-based lifetime-sensitive pixels are likely to outperform sensors that can be used in spectrographs or line scanning microscopes. Two-dimensional arrays could easily replace multi-channel plates used for wide-field FLIM detection. An ideal system would provide quantum efficiencies in excess of 50% (fill factor effects included) on the visible range (400–700 nm), with dead times lower than 100 ns, on pixel sizes of 10–20 μm. In order to enable scientific applications, we envisage that linear arrays should provide 64 pixels and two-dimensional arrays should have at least 256 × 256 pixels. To be commercially viable, probably these values should reach 128–256 and 512 × 512 pixels, respectively.

When integrated with inexpensive light sources such as light emitting diodes or laser diodes, these systems will provide extremely cost-effective and user-friendly tools. For instance, wide-field real-time FLIM could be available as an add-on for standard fluorescence microscopes [8,32], but the technology may benefit from flow and image cytometry application to industrials such as automated product inspection. Moreover, the most recent developments in analysis techniques, aimed at the simplification of data analysis and interpretation, properly tailored to the requirements of the biomedical user, could also permit a deeper penetration of these technologies [31]. However, ToF technologies and new camera systems equipped with “smart pixels” may not be just an alternative to current FLIM technologies.

Recently, high content imaging has begun to impact the scientific community. Automated microscopes are used to characterize complex biological processes with high throughput capabilities [1,37]. This enables the screening of large libraries of genes and molecules. The robustness and inherent standardization of FRET assays may significantly benefit these research programs. Automated microscopy already benefits basic research, target identification programs and validation of lead compounds using cell based assays. Furthermore, it is likely that many toxicity assays and other tests for drug candidates will gradually move from clinical trials to cell based assays in order to reduce the high attrition rates during the last phases of drug discovery, often caused by toxicity and inefficacy of the candidate compounds [38].

Among the translational applications of biophysical imaging modalities that ToF can support, there is also diagnostics. Many laboratories have demonstrated the increased contrast that fluorescence lifetime imaging and other techniques can provide, for instance, for tumour diagnostics [39,40]. Discrimination of pathological versus healthy tissue, by detection of the natural fluorescence of tissue or by novel fluorescent markers will be useful for histopathology, guided biopsies, endoscopy and fluorescence assisted operations.

In our laboratories, we are developing microscopes capable of multiplexing the many properties of light. Detection of polarization and spectral information can be achieved with the use of optics [41], but the timing properties of light require pico/nano-second sensitive pixels [11]. The integration of smart pixels and dedicated optics can provide an extremely versatile optical detector that can be reconfigured on-pixel to detect the features of biological interest. With these new sophisticated tools capable of classifying the optical signatures of biochemical processes, we will be able to implement effective tools for multiplexing biochemical reactions or classifying pathological signatures within healthy tissue.

With the appropriate resources invested in the development of the next generation ToF technologies dedicated to fluorescence detection, it will be possible not just to provide better FLIM systems, but also to lay down the first milestone for the development of radically novel detection systems that could impact significantly on the biomedical community and the health care system.