Application of Adaptive Optics in Ophthalmology

Abstract

The eye, the photoreceptive organ used to perceive the external environment, is of great importance to humans. It has been proven that some diseases in humans are accompanied by fundus changes; therefore, the health status of people may be interpreted from retinal images. However, the human eye is not a perfect refractive system for the existence of ocular aberrations. These aberrations not only affect the ability of human visual discrimination and recognition, but restrict the observation of the fine structures of human eye and reduce the possibility of exploring the mechanisms of eye disease. Adaptive optics (AO) is a technique that corrects optical wavefront aberrations. Once integrated into ophthalmoscopes, AO enables retinal imaging at the cellular level. This paper illustrates the principle of AO in correcting wavefront aberrations in human eyes, and then reviews the applications and advances of AO in ophthalmology, including the adaptive optics fundus camera (AO-FC), the adaptive optics scanning laser ophthalmoscope (AO-SLO), the adaptive optics optical coherence tomography (AO-OCT), and their combined multimodal imaging technologies. The future development trend of AO in ophthalmology is also prospected.

1. Introduction

The eye is one of the most important organs for humans to understand their external environment. Most of the knowledge and memory stored in the brain is acquired through the eyes. Further, some diseases, such as diabetes, high blood pressure, atherosclerosis, etc., can be diagnosed by examination of the fundus, which provides a way to interpret the health status of the body. However, it is not easy to detect early retinopathy and fundus lesions, which may cause irreversible visual impairment and even blindness if not treated in time. Fundus imaging is generally used for detecting and diagnosing human diseases that influence the retina. Some kinds of retinal imaging technologies, such as optical coherence tomography (OCT) [1], flood-illumination fundus camera (FC) [2], and scanning laser ophthalmoscopy (SLO) [3], can obtain living human retinal images, but none of them can provide details at the cellular level due to the resolution limitations imposed by aberrations in the human eye. Adaptive optics (AO) provides a solution for ophthalmic imaging with high resolution, which is close to the diffraction limit, to accurately obtain the structural and functional information of retina.

The idea of AO comes from astronomical observations [4], where AO is used to compensate the dynamic wavefront distortion caused by atmospheric turbulence and allows a telescope to achieve diffraction-limited imaging from the ground. Subsequently, AO has been rapidly developed and applied in the military, astronomy, industrial, civil and medical fields. AO in ophthalmology started in the early 1990s [5], and was mainly employed in three imaging modalities, including FC, SLO and OCT. Since then, it has achieved huge progress in the last ~25 years. Like the atmosphere, the human eye is optically imperfect: it suffers from wavefront aberrations whose pattern and amount differ between individuals. In ophthalmic medicine, AO is used to measure and correct the wavefront aberrations in the human eyes and optical systems through a closed-loop control (detection–control–correction), improving the lateral optical resolution from 10–15 μm to ~2 μm, enabling retinal imaging with cellular-level resolution [6,7,8,9,10]. These improved images can help doctors to detect early retinopathy and fundus lesions, determine the cause and course of eye disease, and evaluate the subsequent treatment. The combination of AO and retinal imaging devices is of great significance and has the potential to become a general tool for the early and clinical detection and diagnosis of some retinal diseases.

In this paper, the principle of AO in correcting human eye aberrations is introduced. Several AO-based ophthalmic imaging technologies, including adaptive optics fundus camera (AO-FC), adaptive optics scanning laser ophthalmoscope (AO-SLO), adaptive optics optical coherence tomography (AO-OCT), and their combined multimodal AO imaging techniques are illustrated. The applications of the AO retinal imaging techniques in ophthalmic medicine are reviewed. Furthermore, the development trend of AO in ophthalmology is discussed.

2. Principles and Methods

2.1. Wavefront Aberration in Human Eyes

The human eye is mainly composed of the cornea, the anterior chamber, the lens, the vitreous body, the retina, etc. The retina is a light-sensitive layer with a sophisticated structure that lines the inner surface of the eye. This layer receives images coming through the eye’s lens and sends the processed signals along the optic nerve to the brain. Human eyes are not ideal refractive systems, and they are sensitive to the wavelength and intensity of light. When light enters the human eye, it produces chromatic and monochromatic aberrations. Chromatic aberrations can be overcome by using a single-wavelength light source. However, monochromatic aberrations are wavelength-independent and are affected by many complex factors, including variations in the shape of the tear film, cornea, and lens [11]. Monochromatic aberrations are commonly divided into low-order and high-order aberrations. Low-order aberrations can be corrected using spherical and cylindrical lenses; but higher-order aberrations can significantly reduce lateral resolution and degrade retinal image quality with increasing pupil size of the human eye [12,13]. In daily life, people can see objects clearly because the human eye can limit the effect of aberration on vision by adjusting the size of the pupils. These wavefront aberrations usually limit the resolution of human eyes to ~10–15 μm, far from the theoretical lateral resolution of ~2 μm according to the diffraction limit formula, r = 0.61λ/NA (r is the resolution, λ is the imaging wavelength, and NA is the numerical aperture).

A comparison of wavefront shape between an ideal eye and a normal eye is shown in Figure 1. In the ideal eye, the light reflected from the fundus forms a flat wavefront after passing through the normal refractive medium. The aberration in the normal eye causes the reflected light from the fundus to form a distorted wavefront, which limits the resolution of the conventional fundus imaging techniques [6].

2.2. Basic Principles of Adaptive Optics

The AO system is an automatic control system that typically comprises a wavefront sensor, a wavefront controller, and a wavefront corrector. By detecting, controlling, and correcting the optical wavefront in real time, the AO system can actively adapt to the changes of external condition and maintain a good working state [14]. Modern AO retinal imaging systems can be divided into two types according to their different wavefront detection methods. One is based on geometric optics, wherein it uses wavefront sensors to directly measure the distorted wavefront, also known as the direct method. The corresponding AO system is referred to as hardware AO (HAO), which is usually composed of a wavefront sensor such as Shack-Hartmann wavefront sensor (SHWS), a compensator such as deformation mirror (DM) or liquid crystal spatial light modulator (LC-SLM), and the closed-loop control software between them. The direct method has the merits of high correction speed and detection accuracy, but it suffers from hardware complexity and high cost, relatively low spatial resolution, and limited dynamic range; moreover, a good “guide star” is necessary for the samples. The other is based on the iterative algorithm to indirectly deduce the wavefront through a series of images, also known as the indirect method, including sensorless AO (SAO) and computational AO (CAO) [8]. The indirect methods do not need an additional wavefront sensor, so the optical system is greatly simplified; their major disadvantage is their low speed (seconds to minutes) due to the collection of many images [15], leading to at least 10× slower speed compared to direct method (milliseconds).

Figure 2 shows the schematic diagram of a sensor-based AO system for retinal imaging [7,16]. Light focuses on the retina after passing through the cornea, lens, vitreous body, and other eyeball tissues. The reflected light from the retina passes through the cornea again and forms a distorted wavefront. A wavefront sensor is used to record the ocular wavefront aberrations and calculate the distortion degree. Then, the wavefront aberration is transmitted to the wavefront controller and converted to a control signal. This signal drives the adaptive mirror to produce the corresponding deformation for the correction of distorted wavefront. Finally, the corrected flat wavefront is acquired, and the high-resolution image is acquired by a CCD camera. Sensor-based AO is optimized for a single depth within the sample and is sensitive to system misalignment, irregularly shaped pupils, cloudy corneas, or other ocular opacities may obscure the detection of the ocular aberrations.

2.3. Sensorless AO and Computational AO

Compared with HAO, which uses a wavefront sensor and wavefront shaping device for wavefront aberrations correction, SAO eliminates a wavefront sensor and uses the properties of the image itself to estimate the wavefront quality [17,18,19,20,21]; CAO eliminates both the wavefront sensor and wavefront shaping, further omits the need to iterate a control algorithm while imaging, and modifies the phase of the data to correct optical aberrations [22,23,24,25]. Both SAO and CAO exploit optimization algorithms for aberration correction, which reduces the complexity and cost of the AO system and offers versatility at the expense of wavefront correction speed. Meanwhile, they require image stability or phase stability during the collection time.

Many SAO control algorithms and image quality metrics have been described and evaluated [18,26,27,28,29,30,31]. They can mainly be divided into two categories according to operating principle. One is to seek the optimum solution (control voltages or aberration modes) in line with desired system metrics from the numerous solutions (mostly random); the other is to seek the analytic relation between system metrics and the aberrations to attain the approximate solutions [18]. A stochastic parallel gradient descent (SPGD) algorithm was used in SAO-SLO for retinal imaging [17]. Moreover, a lot of retinal tracking and image registration algorithms were applied in the AO-SLO (see Section 3.2.5). SAO-OCT retinal imaging was demonstrated by the implementations of SPGD [20], coordinated search (CS) algorithm [19,32], hill climbing algorithm [33], and data-based online nonlinear extremum-seeker (DONE) algorithm [34], etc. Recently, some deep learning methods [35], such as deep reinforcement learning [36,37] and convolutional neural networks (CNNs) [38,39], have been applied in SAO systems in combination with SLO or OCT. Many of these features and aberration correction algorithms used in SAO could be implemented in CAO [40]. For example, an SPGD algorithm was introduced into CAO-OCT for automated fast computational aberration correction [40].

3. Application of Adaptive Optics in Ophthalmology

In 1994, Liang et al. measured wavefront aberrations of the human eye with SHWS, and reconstructed the actual wavefront via wavefront estimation with Zernike polynomials [5]. Then, they built an AO-FC for retinal imaging for the first time in 1997 [41]. However, the early AO retinal imaging system was only used for laboratory research, instead of clinical ophthalmology. With the development of laser, detector, corrector and other related techniques, AO has been applied to ophthalmology in combination with FC, SLO and OCT, endowing both structural and functional retinal imaging with greatly improved imaging performance metrics, such as resolution, imaging speed, aberration detection and correction ability. These AO-based techniques can be used to image retina with a resolution close to the diffraction limit, and are expected to become effective auxiliary tools for the examination and diagnosis of eye diseases in clinic [8,13].

3.1. Adaptive Optics Fundus Camera (AO-FC)

FC has been widely applied in the clinical diagnosis of ophthalmic diseases and is mainly used for large-scale retinal imaging and retinal fluorescein angiography by employing conventional flood-illumination and high-sensitivity CCD detection, with the advantages of simplicity of hardware, ease of use, and intuitive imaging. However, the CCD sensor detects light from everywhere, including scattered light from the anterior segment and other adjacent layers, which results in low image contrast of the retina. In addition, since traditional FCs cannot correct eye aberration according to individual differences and dynamic changes, their lateral resolution is limited to 10 μm or more.

AO can measure and correct the dynamic aberrations of human eyes and improve the lateral resolution of retinal imaging. In 1997, the first AO-FC was developed in Williams’ lab at the University of Rochester [41], which used a krypton arc flash lamp to flood illuminate the retina, a SHWS to measure the eye’s wavefront aberration, a DM to correct the wavefront aberrations and a scientific-grade CCD to acquire images of the retina, as shown in Figure 3. This system was demonstrated with a lateral resolution of ~2 μm, which was close to the value of 1.9 μm expected from diffraction alone, enabling the imaging of microscopic structures as small as the size of single cells in the living human retina. However, AO-FC is limited by a low axial resolution (~300 μm), which reduces cone photoreceptor contrast and decreases repeatability of longitudinal cone density measurements [42,43].

The spatial resolution, imaging field of view (FOV) and imaging speed of AO-FC are constantly improving [7,44,45]. On the basis of flood-illumination configuration, a commercially available AO-FC system, the RTX1 from Imagine Eyes (Orsay France), was developed and has been popularly used in retinal imaging for investigating parafoveal cone photoreceptors and retinal vasculature in normal subjects as well as eye diseases [46,47,48,49,50,51,52,53,54,55,56,57]. The RTX1 is based on a noncoherent flood-illuminated design with a central illumination wavelength of 850 nm, an FOV of 4 × 4° (1.2 × 1.2 mm on the retina), and a focusing range of 1600 μm that covers a retina thickness of ~300 μm, allowing for high-resolution (~2 μm) imaging of the retina. In 2012, Lombardo et al. [46] studied the variations in image optical quality of the eye and the sampling limit of resolution of the cone mosaic with axial length in young adults by using an RTX1 system. In 2015, Gocho et al. [48] reported the improvement of foveomacular cavities and spoke-wheel pattern retinoschisis observed by spectral-domain OCT (SD-OCT) and RTX1 AO-FC in a patient with X-linked retinoschisis (XLRS) treated with topical dorzolamide. AO images showed the detailed microstructure of spoke-wheel pattern foveoschisis and their improvement during a follow-up period, which was more helpful for clarifying the pathology of the foveoschisis in XLRS than with SD-OCT. In 2016, Soliman et al. [49] used an RTX1 AO-FC to assess cone density as a marker of early signs of retinopathy in patients with type II diabetes mellitus. It was found that in patients with type II diabetes, the loss of photoreceptors in AO images might be positively correlated with the severity of retinopathy, and the loss of photoreceptors was more obvious in patients with advanced retinopathy. Representative images of cone density, cone spacing, and Voronoi across study groups were shown in Figure 4. In 2018, Legras et al. [50] studied the distribution of cone density, spacing and arrangement in adult healthy retinas of 109 subjects at 2°, 3°, 4°, 5° and 6° of eccentricity along four meridians using an RTX1 AO-FC. In 2019, Markan et al. [51] used an RTX1 AO-FC to study photoreceptor changes after a successful macular hole surgery. They found the migration or shifting of cells from parafoveal retina toward the center that indicated the postoperative recovery of vision. In addition, the cells nearest to the hole margin (at 2° eccentricity) shifted more compared with cells that were further away. Nakamura et al. [52] used SD-OCT, fundus autofluorescence (FAF) imaging, and AO-FC to observe the long-term changes of retinal pigment epithelium (RPE) in the eyes of patients with Vogt-Koyanagi-Harada disease. SD-OCT and FAF could qualitatively study the thickness changes of the RPE layer, while high-resolution RTX1^TM AO-FC was able to be used to quantitatively observe the improvements in the elevation or thickening of the RPE layer. In 2020, Potic et al. [54] used an RTX1 AO-FC to measure the change of photoreceptor density in patients with retinal detachment and studied its relationship with macular lesion. Ochinciuc et al. [55] analyzed cone density, cone mosaic, and FAF images in patients with focal laser-treated central serous chorioretinopathy (CSC). FAF and cone mosaic images were obtained in all patients with an RTX1 AO-FC. The results showed that the presence of hypoautofluorescent lesions and the duration of pathology were negative prognostic factors in CSC. In 2021, Cheng et al. [56] investigated the relationship between visual resolution and cone parameters in eyes with different levels of best corrected visual acuity (BCVA), and obtained the images of the cone photoreceptors at 1.5° from the fovea using an RTX1 AO-FC. The results showed that the greater cone density and smaller cone spacing at the parafovea were found in eyes with BCVA of 20/12.5 or better, as compared to that in eyes with BCVA of 20/16. In 2022, Baltă et al. [57] investigated the retinal microcirculation in diabetic patients using AO ophthalmoscopy (RTX1^TM) and optical coherence tomography angiography (OCTA). The two techniques were demonstrated to provide complementary and useful information about the retinal microvasculature from early onset of diabetic disease.

In summary, AO-FC is an effective tool for studying the retinopathy of human eyes and has been widely applied in scientific experiments and clinical treatment to study color vision and cone classification [58,59], observe cone photoreceptors [46,50,51,53,54,56], image retinal vasculature [57,60], and track changes of cell groups [49,52], etc. AO imaging with a flood-illuminated system can acquire the entire retinal image in a relatively short time on the order of a few milliseconds, which minimizes the effect of eye movement. However, the entire frame is captured concurrently from backscattered light from the retina and choroid, which limits the axial resolution (~300 μm) of images and reduces the image contrast as compared to SLO-based systems [13].

3.2. Adaptive Optics Scanning Laser Ophthalmoscope (AO-SLO)

3.2.1. Basic Operation of AO-SLO

SLO was invented and developed in the 1980s by Webb et al. [61,62,63] and is widely regarded as an excellent choice for clinical imaging with superior contrast and resolution. Like a scanning laser microscope, the image in a SLO is generated over time by recording the scattered light from a focused spot as it is raster scanned across the region to be imaged. As such, it does not collect an image using a film or a CCD array. Instead, the intensity of each pixel is recorded using a more sensitive detector, such as photomultiplier tubes (PMT) or avalanche photodiodes (APD), and the location of each pixel is inferred by outputs from the scanning mirrors [64,65]. Compared to FCs, the SLOs are relatively insensitive to image degradation owing to scatter in the optics, and they also have the ability of optical sectioning in the retina, by employing a confocal pinhole conjugate to the retina. Similar to FCs, ocular aberration is a major limitation to the application range of SLOs.

With the combination of AO and SLO, AO-SLO can overcome the influence of wavefront aberrations and realize high resolution retinal imaging [66,67,68]. A simplified light path diagram for AO-SLO is shown in Figure 5 [64]. The AO-SLO system relays light from the laser source through a series of lenses and mirrors to the eye. An acousto-optic modulator (AOM) is used to modulate the laser beam to project patterns into the retina. The pupil is imaged onto the vertical scanner (VS), the horizontal scanner (HS), the DM and the SHWS lenslet array, and the Shack Hartman spot pattern is recorded by a CCD. Scatter light from the retina is detected by a PMT after passing through the confocal pinhole. When the confocal pinhole is optimized, the lateral and axial resolutions for a 6 mm pupil and 600 nm light are 1.9 μm and 33 μm, respectively. With larger pupils or shorter wavelengths, the resolution improves further. Owing to its confocal nature, AO-SLO is widely used for visualizing different layers within the retina such as photoreceptors, nerve fibers, retinal pigment epithelium [7]. In recent years, the performance of AO-SLO system, including resolution, field of view, and imaging speed, has been further improved.

3.2.2. Eye Tracking Integration into AO-SLO

The typical FOV of AO systems designed for human use is about 1~3° [41,66,69], which is consistent with the necessary image resolution/pixelation required to elucidate cones, microvasculature, and other structures at the cellular level. This design provides high magnification and can minimize system aberrations but restricts the identification of the exact retinal locus of the high-resolution view in relation to clinically observed changes. In an AO-SLO system, the small FOV is mainly limited by the human eye’s isoplanatic angle and the physical limits of its scanning mirrors and relay optics [70]. Moreover, eye motion is a major impediment to the efficient acquisition of high-resolution retinal images with AO-SLO. In 2010, Ferguson et al. [71] developed an AO-SLO by incorporating a wide-field line-scanning ophthalmoscope and a closed-loop optical retinal tracker. The wide-field imager and large-spherical-mirror optical interface design, as well as a large-stroke DM, enabled the AO-SLO image field to be corrected at any retinal coordinates of interest in a field of >25°. Eye tracking can significantly improve stable overlap and efficiency of sequential AO-SLO image capture by limiting the magnitude of eye movement excursions. In 2014, Yang et al. [72] implemented a robust closed-loop optical stabilization system with digital registration in one AO-SLO system. A 2-axis tip/tilt mirror was utilized for both slow scanning and optical stabilization. Closed-loop optical stabilization reduced the amplitude of eye-movement related-image motion by a factor of 10–15. Both optical stabilization and digital image registration synergistically corrected the residual eye motion with an accuracy of ~0.04–0.05 arcmin RMS (~0.20–0.25 μm) with typical fixational eye motion for normal observers. However, this approach failed in patients with poor fixation, because eye motion caused the FOV to drift substantially. Zhang et al. [73] solved this problem by tracking eye motion at multiple spatial scales by optically and electronically integrating a wide-FOV SLO (WF-SLO) with an AO-SLO. The multi-scale approach had a large stabilization range of ± 5.6° when implemented with fast tip/tilt mirrors, and the system was capable of reducing image motion by a factor of ~400, on average. The integrated WF-SLO and AO-SLO imaging system successfully addressed several major problems limiting the clinical utility of AO-SLO, including its small FOV, reliance on patient fixation for targeting imaging, and substantial post-processing time. In 2015, Sheehy et al. [74] built a system that combined a tracking SLO (TSLO) and an AO-SLO, resulting in both optical (hardware) and digital (software) eye-tracking capabilities. The hybrid system used the TSLO for active eye-tracking at a rate up to 960 Hz for real-time stabilization of the AO-SLO system; by correcting for high amplitude, low frequency drifts of the eye (RMS to an average of 0.06 arcmin), the active TSLO eye-tracking system enabled the AO-SLO system to capture high-resolution retinal images over a larger range of motion (FOV of 3.5° in experiments) than previously possible with just the AO-SLO imaging system alone.

3.2.3. Split Detection Approaches of AO-SLO

Recently, several non-confocal AO-SLO modalities have been developed, which collect scattered light surrounding the point of focus that is lost in confocal systems [75,76,77,78,79]. The utilization of multiple-scattered light photoreceptor imaging expanded rapidly and was investigated as a potential biomarker for therapeutic interventions in some hereditary retinal degenerations [80]. Non-confocal split-detector AO-SLO [77] employs two incoherent detectors to capture the equally-divided multiple-scattered light and has been proved to provide complementary information to that obtained from confocal AO-SLO [77,81,82,83,84]. In 2014, Scoles et al. [77] proposed and demonstrated a novel non-confocal split-detector AO-SLO technique for visualization of the cone photoreceptor mosaic. In that arrangement, the waveguided light from the photoreceptor outer segment (confocal) and the multiple-scattered light from the inner segment (split-detector) could be visualized simultaneously and in perfect spatial registration. Image sequences were collected at the center of the fovea and from 1° to 20° visual angle lateral (temporal) to fixation using a 1.0° and 1.5° square field of view. Figure 6 shows the confocal and split-detector AO-SLO images of the photoreceptor mosaic in a patient with achromatopsia at 0.4° and 2° from fixation. The results proved that split-detector imaging provided the first direct in vivo evidence of substantial remnant cone structure in patients with achromatopsia. Then, Sun et al. used confocal and non-confocal split-detector AO-SLO to examine cone photoreceptor structure in retinitis pigmentosa (RP) and Usher syndrome [81] and study photoreceptor disruption and vision loss associated with central serous retinopathy [82]. In 2019, Sajdak et al. [83] imaged the photoreceptor mosaic in the tree shrew retina in vivo and ex vivo. It was found that higher cone density was measured ex vivo compared with in vivo measurements. In 2021, Sredar et al. [84] recorded the quadrant reflectance confocal and non-confocal split-detector AO-SLO images of the photoreceptor mosaic in a subject with congenital achromatopsia (ACHM) and a normal control, by using various circular and annular apertures.

3.2.4. Handheld Designs of AO-SLO

The size and complexity of AO-SLO systems limit imaging to cooperative adult patients. To extend the application of AO-SLO to new patient populations, DuBose et al. [85] designed the first portable confocal handheld AOSLO (HAOSLO) system in 2018. By incorporating a novel computational wavefront sensorless AO algorithm and custom optics, the HAOSLO was miniaturized to weigh less than 200 g with dimensions of 10.3 cm × 5.3 cm × 14.4 cm, and it was used to image the cone photoreceptors in dilated supine young children and adult subjects. In 2020, Hagan et al. [86], from the same group, built the first compact and handheld multimodal (M-)HAOSLO (216 g, 12.3 cm × 5.3 cm × 14.4 cm) for simultaneous confocal and non-confocal split-detection (SD) imaging of cone photoreceptors in the human eye. The M-HAOSLO system allowed for more flexible and comprehensive patient data collection, which would be useful in acquiring high-resolution retinal images in clinical practice.

3.2.5. Algorithms and Applications of AO-SLO Retinal Imaging

Besides hardware correction, many data analysis approaches also play an important role in eliminating motion distortions and increasing the image signal-to-noise ratio (SNR). The image processing and data extraction steps for AO-SLO images, such as reference frame selection or identification of cone photoreceptor cells, often require some manual intervention, which is highly subjective and time consuming. Automated and semi-automated approaches for reference frame selection [87,88,89], motion extraction and registration [90,91], montaging [92], and cone localization [38,93,94,95] have been developed, which are of great significance for enhancing the performance of AO-SLO retinal imaging.

Table 1. Algorithms for improving system performance.

Year/Authors	Algorithm	Description	Advantages
2015/Chen et al.	Harris-SIFT [88]	Harris-SIFT algorithm matches feature vectors; random sample consensus (RANSAC) algorithm verifies the matching accuracy to obtain the motion vector estimation; the fixed frame compensation method is used to realize the motion compensation.	The algorithm could effectively eliminate jitter and enhance the contrast of video image.
2017/Salmon et al.	Automated reference frame selection (ARFS) [89]	ARFS comprises two main modules: distortion detection to select the least distorted frame(s) from an image sequence and motion tracking to allow selection of multiple reference frames from distinct spatial locations.	ARFS outperformed expert observers in selecting minimally distorted reference frames in AOSLO image sequences.
2018/Davidson et al.	MultiDimensional recurrent neural network (MDRNN) [94]	A powerful deep learning framework is used for automatic localization of cone photoreceptors in AO-SLO split-detection images.	The approach was demonstrated to be the most robust, most accurate, and appreciably faster algorithm for automatic cone localization in healthy and Stargardt afflicted retinas.
2018/Cunefare et al.	Late fusion dual mode convolutional neural networks (LF-DM-CNN) [38]	A new deep learning-based approach combines information from the confocal and non-confocal AO-SLO models to detect cones in subjects with achromatopsia.	The method outperformed the state-of-the-art automated techniques and is on a par with human grading.

lists four typical algorithms that can improve the image resolution or imaging speed of the AO-SLO system. It should be noted that these retinal tracking and image registration algorithms are applicable to not only AO-SLO, but also other AO-based retinal imaging systems.

AO-SLO is an effective tool for studying the retinopathy of human eyes, and has been widely applied in scientific experiments and clinical treatment. The AO-SLO has allowed researchers to identify cone and rod photoreceptor cells [39,77,96,97,98], retinal pigment epithelial cells [99], retinal blood cells [100,101], retinal ganglion cells [78,102], and nerve fibers [67,103]. Moreover, the AO-SLO has been successfully used to detect and evaluate a variety of eye diseases, including central serous retinopathy [82], subretinal drusenoid deposits (SDD) [104,105], retinal vasculopathy [106], retinitis pigmentosa (RP) [81,107,108], glaucoma [109,110], idiopathic macular telangiectasia [111], diabetic retinopathy (DR) [112,113], hypertension retinopathy [114], acute zonal occult outer retinopathy (AZOOR) [115], cancer-associated retinopathy (CAR) [116], etc.

In summary, AO-SLO is a promising technology that can obtain high-resolution and high-contrast retinal images, as well as possessing optical sectioning capability. Various emerging technologies, such as introducing the eye tracking technique to compensate for the impact of eye motion on retinal imaging, using non-confocal split-detector imaging to provide complementary information for confocal AO-SLO, and replacing the point scanning with line scanning to reduce the complexity of the system and enhance the scanning speed, have been integrated to improve the imaging performances of AO-SLO. Moreover, many effective algorithms, including machine learning and deep learning, have been successfully used for AO-SLO-based data processing for further improving the image quality. To be industrialized and applied to the clinical practice, it still has high requirements for system miniaturization and low cost.

3.3. Adaptive Optics Optical Coherence Tomography (AO-OCT)

OCT was first demonstrated as a non-destructive retinal imaging tool in 1991 [1]. Because Fourier domain OCT (FD-OCT), which can be further divided into spectral domain OCT (SD-OCT) and swept source OCT (SS-OCT), can offer a significant advantage in acquisition speed without degradation of sensitivity and resolution compared to the time domain OCT (TD-OCT) technique, it became more prevalent in ophthalmology and has been clinically applied to evaluate retina, vitreous, choroid, optic nerve, etc. [117,118,119]. Benefiting from the unique features of the interferometric measurement method, the axial resolution of OCT is determined by the source bandwidth (BW), not the focusing optics. The high axial resolution (a few μm) enables OCT to visualize and characterize all the main cellular layers in the human retina. However, the lateral resolution of OCT in the eye is poor—typically no better than 15 μm—due to a small imaging pupil and the ocular aberrations.

The combination of AO with OCT can improve the lateral resolution greatly and achieve high-resolution volumetric imaging, which makes it possible to visualize highly transparent cells and measure some of their internal processes within the retina. Moreover, AO-OCT could provide both optical amplitude and phase information [120]. The early attempts to implement AO correction in SD-OCT imaging were reported by Zhang et al. [121] and Zawadzki et al. [122], based on free space parallel SD-OCT and fiber-based SD-OCT, respectively. With dilated pupils >6 mm and distortion compensating AO as well as an improved axial resolution, these instruments could achieve lateral and axial resolutions up to 3 μm and 2~3 μm in the eye, respectively, which enabled the cellular level measurement [16]. A typical AO SD-OCT optical system was shown in Figure 7 [123] and it consisted of four modules: source channel, sample channel, reference channel, and detection channel. The illumination source was a broadband superluminescent diode (SLD). The AO system was integrated into the sample channel.

AO-OCT generally adopts a SHWS to measure the ocular aberrations directly and in real time [121,122,123]. However, various parameters, including the clinical situation, irregular pupil and retinal structure, high transparency of the eye, cataracts, and other factors, will affect the accuracy and reliability of the wavefront sensor data. Wavefront sensorless AO-OCT (SAO-OCT) does not rely on direct wavefront measurement, but depends on a depth-resolved image-driven optimization algorithm, which offers less system complexity compared with sensor-based AO design. Additionally, SAO-OCT does not need to define conjugate planes, which allows it to work reliably near non-planar retinal structures.

Numerous experiments have shown that SAO-OCT can be used to obtain retinal images with high resolution and high speed [19,20,34,37,124,125,126]. In 2015, Wong et al. [19] developed a SAO-OCT system for in vivo imaging of the photoreceptor layer in the human retina at several eccentricities. Due to SAO’s compensation, the photoreceptor visibility was improved, and the photoreceptor mosaic could be observed even at eccentricities as small as 1°. In 2017, Polans et al. [20,124] designed a wide-field (>70°) OCT combined with the SAO system, which was capable of acquiring aberration-corrected images without mosaicking. A fast SAO algorithm was employed to enable wavefront correction at the speed of individual B-scans (~20 ms) rather than volumes. Then, the system was used to enhance the visualization of anatomical and pathological features in the peripheral retina for neurologic diseases. In 2017, Reddikumar et al. [125] reported an SAO-OCT system with a 2.8-mm beam diameter and image-based optimization and a low-cost AO module containing a motorized Badal optometer for defocus correction and a liquid crystal device for vertical and oblique astigmatism correction. The system could achieve a lateral resolution of 11.5 μm over a width of at least 15° × 15°, and could correct the defocus ranging from −4.3 D to +4.3 D and vertical and oblique astigmatism ranging from −2.5 D to +2.5 D. In 2020, Camino et al. [126] built a SAO-OCTA instrument with an intermediate NA to produce depth-resolved angiograms with high lateral resolution and SNR over a 2 × 2 mm FOV and with a focal spot diameter of 6 μm. AO correction with the ability to optimize focusing at arbitrary retinal depths could be achieved in 1.35 s and the instrument could image the parafoveal circulation within 3 s.

CAO-OCT can correct optical aberrations and achieve volumetric aberration-free retinal imaging by numerically modifying the phase of the OCT data without the use of hardware AO components. In addition to lower cost and simpler system configurations, the post-processing nature of computational imaging also allows more flexible measurement and image improvement. CAO-OCT has been applied to in vivo imaging of the human retina [40,127,128,129]. South et al. [130] demonstrated a combined HAO and CAO system, achieving improved resolution as compared to HAO and improved SNR as compared to CAO. HAO + CAO may broaden the potential applications of computational imaging methods.

AO-OCT has been applied in normal and pathological retinal imaging for structural and functional measurements. The applications mainly include imaging or classification of photoreceptor [34,39,131], imaging and characterizing of RPE cells [132,133], imaging and quantifying of ganglion cells (GCs) [134] (as shown in Figure 8), imaging of macrophage distribution and dynamics [135], imaging and characterizing of lamina cribrosa within optic nerve head [136,137], retinopathy of prematurity [19], imaging of the outer layer of the retina in the macular area [20], visualization of retinal vessels [124], optic neuropathy [123], hereditary eye diseases, and the surgical treatment and follow-up of eye diseases.

Moreover, some approaches to improve versatility of AO-OCT have been reported, such as combing with polarization-sensitive OCT (PS-OCT) for more specific characteristics of tissue structure and function [138], and introducing reflective mirror-based line-scan to AO-OCT for high imaging speed [139,140].

In summary, AO-OCT can achieve 3D retinal images with high resolution and high speed, aberration corrected either by hardware or software, or both. It has played an extremely important role in promoting ophthalmic medicine, helping to visualize the microscopic morphology of the retinal structure, and allowing monitoring of disease progression or treatment.

3.4. Multimodal Adaptive Optics Retinal Imaging Techniques

AO-based multimodal retinal imaging can provide complementary information and enable resolution of various cell types for retinal depth imaging, and is of great clinical relevance, since microscopic retinal structures become visible in living human eyes [105,141,142,143,144,145,146,147,148,149,150,151]. Multimodal AO systems integrating AO-SLO and AO-OCT were first developed for retinal imaging by the groups led by Zawadzki [141] and Hammer [142,143], respectively. Both of them used FD-OCT, but with variations of SD-OCT and SS-OCT, respectively. The former could acquire OCT and SLO in vivo images of the human retina at ≤3.5 μm lateral resolution (~840 nm (110 nm BW) for OCT and ~680 nm (10 nm BW) for SLO, ~6.7 mm pupil diameter). The latter were capable of collecting AO enhanced SLO images to resolve cones within 0.5° of the fovea and AO enhanced OCT images with improved penetration for identification of all major retinal layers (~1070 nm (79 nm BW) for OCT, ~750 nm (14 nm BW) for SLO, ~830 nm (26 nm BW) for LSO and ~915 nm for retinal tracking (RT) beams). Figure 9 shows the schematic diagram of the multimodal AO system [142,143]; its output included SLO, OCT, LSO, and HSWS videos, and its performance was tested on human subjects and rodents. In 2013, Meadway et al. [144] developed a dual-mode AO-based retinal imaging system that combined SLO and OCT. The two imaging modes were switched by a flip mirror (FM). The system demonstrated the in vivo images from healthy and diseased retinas, and then it was used to study microstructure of subretinal drusenoid deposits [105]. In 2016, Salas et al. [146] proposed a compact multimodal imaging prototype combining AO-FC and AO-OCT in a single instrument. AO-FC was able to obtain AO fundus images with a FOV of 4 × 4° and a frame rate of 10 fps. In AO-OCT mode, the imaging speed was 200 kHz A-scan rate, and the lateral and axial resolutions were 4 µm and 5 µm, respectively. Hafner et al. (in 2018) [147] and Karst et al. (in 2019) [148] reported the integration of AO-FC and AO-OCT, where the AO-OCT prototype was incorporated into an RTX1 AO-FC. They studied the dynamic changes and analyzed 3D morphological details of retinal microaneurysms in diabetes, respectively. In 2018, Liu et al. [149] presented a novel multimodal AO (mAO) system that combined two complementary approaches, AO-SLO and AO-OCT, in one instrument with flexible alternation of scanning modes and independent focus control. The mAO system’s imaging performance was demonstrated through the visualization of cells in their mosaic arrangement across the full depth of the retina in three human subjects. In 2019, Wahl et al. [150] proposed a multimodal sensorless AO en-face retina imaging system composed of OCT, OCTA, confocal SLO and fluorescence detection. The system performance was studied by observation of dynamics of microglial cell branching and volumetric cellular imaging of microglia throughout the inner retina on wild type mice and transgenic mice with GFP-labeled cells. In 2021, Bower et al. [151] developed an imaging platform combining multimodal AO-SLO with AO-OCT for visualization of the human retinal pigment epithelial mosaic. The results showed that by combining information from all modalities, inferences about normal vs. disrupted cellular structure could be validated, thus improving the reliability, accuracy, and overall consistency of in vivo assessment of the human RPE mosaic.

In summary, multimodal AO retinal imaging is a powerful tool for capturing images of the retina. Extraction of information from retinal cells is optimal when their optical properties, structure, and physiology are matched to the unique capabilities of each imaging modality, which indicates the great clinical potential of multimodal AO retinal imaging for early diagnosis and treatment of eye disease.

4. Summary and Prospects

The eye is not only a window to the mind, but also a reflection of the health state of the body. Retinal diseases, which can cause vision loss and even blindness, have become one of the main causes of blindness and attracted extensive attention from many experts and scholars. Moreover, some retinopathy symptoms are related to human diseases, such as diabetes, tumor and kidney disease, etc. Therefore, it is of great significance to realize retinal imaging with high resolution and high speed. However, the aberrations of living human eyes show obvious individual differences and change with time; therefore, it is difficult to obtain retinal images at the cellular level using traditional clinical methods.

AO is used for real-time aberration compensation to improve image or beam quality in an optical system with dynamic aberrations. The use of AO-based optical imaging in ophthalmology has greatly changed the way we observe the human eye both at the cellular and metabolic level. By correcting the individual optical aberrations, AO offers a lateral resolution of ~2 μm, which is unachievable for traditional retinal imaging techniques (lateral resolution: ~10–15 μm). Therefore AO-based retinal imaging technology not only enables scientists to resolve fine retinal structures but also improves the clinical interventions for different eye diseases. By exploiting flood-illumination and CCD detection, AO-FC provides a direct and effective method for imaging the retina and diagnosing fundus diseases, and can help doctors to determine the treatment plan in time; however, its axial resolution (~300 μm) and image contrast are relatively poor due to the backscattered light from the retina and choroid. AO-SLO can obtain retinal images at the cellular level and in different retinal layers with an axial resolution of tens of microns by employing a confocal pinhole conjugate to the retina. AO-OCT not only inherits the advantages of OCT technology, such as non-invasiveness, depth-resolved cross-sectional imaging, high sensitivity, and detection speed, but can also provide a high-resolution 3D image (both lateral and axial resolution on the order of a few microns) of the retinal microstructures. The combination of multiple AO modalities increases the complexity of the optical system, but can provide complementary and optimal information for retinal imaging; multimodal AO shows great potential for clinical use.

At present, AO retinal imaging technology is not yet fully mature, and the relevant instruments and equipment are still in the process of design and development, and more exploration and experiments are needed before they can be applied in clinical ophthalmology. Meanwhile, the instruments and equipment are always expensive, and operators are required to have excellent skills and data processing capabilities. These characteristics make it challenging to popularize these systems in clinical ophthalmology. In the future, the application of AO in ophthalmic medicine may have the following development trends: (1) Miniaturization. The equipment in the laboratory stage is often large, which brings a lot of inconvenience in clinical use. Since sensors, calibrators, and other devices are becoming smaller and lighter, AO-based devices are expected to develop towards higher integration and smaller volume. (2) Low costs. Expensive systems will increase the burden of patients. With the increasing maturity of AO technology, the costs of mass production will be decreased, which will promote clinical applications. (3) Easy operation. The future systems will use highly integrated designs and provide user-friendly software, allowing doctors and nurses to operate it without much professional knowledge. (4) High resolution. High resolution has always been the pursuit of imaging systems. Although the introduction of AO has greatly improved the resolution of the system, there are still many restricting factors, such as the low spatial resolution and limited dynamic range of SHWS. In the future, the resolution of the system is expected to be further improved by the advances of hardware and software. (5) Multi-modality. Different types of ophthalmic imaging methods can obtain complementary information about the fundus tissues related to different aspects and levels. For example, the multi-mode system composed of OCT and SLO can quickly locate the target and simultaneously achieve retinal structure and function imaging, which meets the requirements of rapid and high accuracy in clinical imaging. (6) From structure to function. Changes in the structure and function of retinal cells may serve as sensitive biomarkers of disease onset. While in vivo studies focus on structural changes, functional ones may better capture cell health because they have more direct connection to cell physiology. Cell distribution, motility and changes are potentially sensitive indicators of cell health. In the future, AO-based retinal imaging approaches are to be applied in developing novel disease biomarkers and exploring new pathways in disease progression.

In summary, the application of AO in ophthalmology will inevitably become one of the important means to help doctors to explore the pathogenesis of various eye diseases and to follow up patients, and play a positive role in promoting the development of innovative devices and treatment procedures in refractive surgery and retinal diagnostics.