Multi-Modal Ptychography: Recent Developments and Applications

Abstract

Utilisation of partially coherent X-rays in Coherent X-ray Diffraction Imaging (CDI) and X-ray ptychography have brought intense research and developments for the past few years. Generally, data reconstructions convergence can be achieved by considering to incorporate partially coherent X-rays in ptychographic iterative reconstructions algorithms. Recently, exploration of both X-ray probe and sample modes have been studied and it was suggested that sufficient constraints are required to achieve accurate data reconstructions due to the mathematical inseparability of multiple sample modes that are superimposed into a single set of ptychographic intensity patterns. Here we review the multi-modal ptychography that can be used as a new emerging technique to investigate dynamical sample systems with partially coherent X-rays by recent experimental and simulated demonstrations.

1. Introduction

Significant progresses have been made during the past few years in applying coherent X-ray diffraction imaging (CDI) and X-ray ptychography in materials sciences and soft matter materials. To introduce X-ray ptychography [1,2,3,4] in short, the technique is the elegant amalgamation of scanning X-ray microscopy (STXM) with CDI, where partially overlapping illumination provides additional real-space constraint that expands the sample of interest to arbitrarily-defined region with high-resolution in both absorption and phase contrasts. In CDI, a modified algorithm that includes ab initio correction of transverse partial coherence in measured data by using Lucy-Richardson (RL) deconvolution method has been shown to give better data reconstructions [5,6]. Thibault and Menzel [7] recently demonstrated that the ptychographic method provides enough redundancy in datasets to allow high-quality reconstructions through complete recovery of X-ray illumination modes (and object’s states in the case of dynamical sample). This work also opened up the exciting possibilities of information multiplexing [8] and dynamical systems imaging [9] with ptychography. Following results by Thibault, Menzel et al. and Batey et al. [7,8], Li et al. [10] has recently demonstrated the feasibility of recovery of multiple sample modes by applying extra constraints into the iterative algorithms, and they discovered the inability of recovery of individual sample modes due to the mathematical equivalence in mixing the initial sample modes in multi-modal ptychography. Individual sample mode could be recovered when additional constraints on sample were applied [10]. To understand better this for dynamical sample modes, we have simulated ptychographic data with both X-ray beam and sample modes to mimic experimental data when partially coherent X-rays are used. Implementations of both sample modes and X-ray beam modes could enable relatively higher X-ray flux on dynamical samples. This approach ultimately reduces ptychographic data acquisition time to detect dynamical sample states with few-nanometer spatial resolution in a relatively short experimental time.

Figure 1 [9] shows experimental reconstructions of X-ray orthogonal illumination modes when partially coherent X-ray illumination was adopted with a vibrating sample. Multiple sample modes due to vibration are transferred to probe modes, resulting to partially coherent illumination.

Li et al. [10] recently successfully performed multi-modal experiments with visible lights on complex test objects with partially coherent illuminations with several orthogonal modes. Their results demonstrate the feasibility of inversion of complex wavefunctions of both object and illumination using multi-modal approach without a priori information to start with. This could be trivially extended to X-rays to enable dynamical studies on condensed matter materials by adopting this technique. Figure 2 illustrates experimental work demonstrated by Li et al. [10] by using visible lights to test reconstruction quality of measured data by applying multi-modal ptychography algorithms.

2. Theory

Previously, researchers adopted RL deconvolution to accommodate spatial partial coherence into CDI [5,6] and ptychographic [11] iterative algorithms, and their mathematical formulations have been described in details in Ref [5,6,11]. To further develop the mathematical concept of partial coherence that suits ptychography, a different version of representation of partial coherence of electromagnetic waves has been suggested [7,9], which adopts the theorem that transverse and longitudinal partially coherent X-rays can be generally formulated equivalently by introduction of superposition states of multiple modes in both spatial and wavelength (spectra) regimes. In the spectral regime, multi-modal approach is also valid. Insertion device generated synchrotron X-rays always possess a certain amount of longitudinal incoherence, especially when no X-ray monochromators are present, the resulting X-rays are to be composed of significant broadening of the energy spectra. It is considered of general interest to utilize an undulator harmonic from X-ray intensity spectra that is generated by an insertion device because coherent intensity diffraction intensities can be mathematically recovered with iterative algorithms by applying multi-modal ptychography. The original research on broadband X-ray illumination in coherent diffraction imaging reported by Abbey et al. [12], include a priori calculated electromagnetic wave spectrum from a particular undulator harmonic, obtained by simulations of broadband X-rays, produced by an insertion device to guide the iterative algorithms in the reconstruction of the acquired CDI data. This approach was extended to Ptychography, both in visible [8] and X-rays regimes [13], with demonstrated enhanced experimental efficiencies achieved by allowing larger flux illumination on sample. Figure 3 shows reconstructions of test sample with multiple probe modes using broad bandwidth pink beam X-rays, and the reconstructions improved significantly when introducing multiple probe modes into the iterative algorithms. Figure 4 displays Multi-modal ptychography can be extended to electron regime, and a study demonstrates the applicability of probe modal decomposition using electron ptychography [14]. Figure 4 illustrates experimental reconstructions of both modulus and phase components of object, showing much improved data inversion quality when 16 probe modes are incorporated when compared to results with only a single probe mode in the ptychographic iterative algorithms. It is particularly important to note that recent development of fly-scan ptychography [15,16] introduces continuous motions of sample movements, thus, would result in blurring of measured diffraction intensities. To reconstruct fly-scan ptychography datasets, incorporation of partial coherence is required; multiple probe modes are artificially introduced into data processing to account for the continuous motions of samples. Figure 5 shows experimental reconstruction of several probe modes as a function of sample scanning speed in their reported study with a collimated visible-light laser source [15]. The advantages of fly-scan ptychography are two-fold. First, the overhead of sample stages could be minimized to reduce total measurement time, and secondly, the sample stage scanning motions are much more stable and reliable, comparing to the case of excessive acceleration and deceleration of sample stage motors during ptychographic data acquisition.

The number of sample modes are representatives of multiple sample states, sometimes they can be orthogonal to each other, in the same fashion to that of X-ray beam. Following the free-space propagation intensity equation derived from Equation (1) in the work of Clark et al. [9],

$$I\left(q\right)=\int \int B({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})S({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})exp[\frac{i\pi }{z\lambda }({\mathbf{r}}_{\mathbf{1}}^2-{\mathbf{r}}_{\mathbf{2}}^2)]\times exp[i({\mathbf{r}}_{\mathbf{1}}-{\mathbf{r}}_{\mathbf{2}})·q]d{\mathbf{r}}_{\mathbf{1}}d{\mathbf{r}}_{\mathbf{2}}$$

(1)

where $B({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})$ is the probe function, $S({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})$ is the object function, ${\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}}$ represent two arbitrary points separated in the free-space propagation direction, and q represents the reciprocal-space unit vector, and $\lambda$ is the wavelength of the illuminating X-rays. The mutual optical intensity (MOI) equations of both multiple X-ray beam $J_B({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})$ and sample modes $J_S({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})$ can be described as the following mathematical formulations, which are first proposed by Thibault and Menzel [7] :

$$J_B({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})=\sum _{n=1}^N{\eta }_n{B}_n\left({\mathbf{r}}_{\mathbf{1}}\right)B_n^{*}\left({\mathbf{r}}_{\mathbf{2}}\right)$$

(2)

$$J_S({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})=\sum _{m=1}^M{\eta }_m{S}_m\left({\mathbf{r}}_{\mathbf{1}}\right)S_m^{*}\left({\mathbf{r}}_{\mathbf{2}}\right)$$

(3)

where ${\eta }_n$ and ${\eta }_m$ represent the corresponding X-ray beam and sample modes weights individually according to specific experimental setup conditions and sample states respectively. n and m represent the number of modes in X-ray beam and sample respectively, and where * indicates complex conjugate operation, and N and M represent the total number of modes in X-ray beam and sample respectively.

When the illumination demonstrates a statistically stationary mutual intensity function which depends only on the difference ${\mathbf{r}}_{\mathbf{1}}-{\mathbf{r}}_{\mathbf{2}}$, the measured partially coherent intensity $I_{pc}\left(\mathbf{q}\right)$ adopts a particularly simple form:

$$I_{pc}\left(\mathbf{q}\right)=\mathcal{F}[J_B({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})·J_S({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})]=\mathcal{F}[\sum _{n=1}^N\sum _{m=1}^M{\eta }_n{B}_n\left({\mathbf{r}}_{\mathbf{1}}\right)B_n^{*}\left({\mathbf{r}}_{\mathbf{2}}\right)·{\eta }_m{S}_m\left({\mathbf{r}}_{\mathbf{1}}\right)S_m^{*}\left({\mathbf{r}}_{\mathbf{2}}\right)]$$

(4)

where $\mathcal{F}$ denotes Fourier transform of the product of X-ray beam and sample states.

Following Thibault’s parametrization of the classical Fourier projector in the difference-map [7], the following equation expresses the partial coherence projector, ${\mathsf{\Pi }}_F$

$${\mathsf{\Pi }}_F={\mathcal{F}}^{-1}\left[\sum _{n=1}^N\sum _{m=1}^M\mathcal{F}(2{\mathsf{\Pi }}_0[{\psi }_j^{m,n}\left(\mathbf{r}\right)]-{\psi }_j^{m,n}\left(\mathbf{r}\right))·\left(\frac{\sqrt{I^j\left(\mathbf{q}\right)}}{{\sum }_{n=1}^N{\sum }_{m=1}^M\mathcal{F}(2{\mathsf{\Pi }}_o[{\psi }_j^{m,n}\left(\mathbf{r}\right)]-{\psi }_j^{m,n}\left(\mathbf{r}\right))}\right)\right],$$

(5)

where ${\psi }_j$ represents the product of X-ray beam and sample wavefunctions of the number jth position of the ptychography scan in a 2D series.

Also, with the real-space projection probe update operation ${\mathsf{\Pi }}_0$ being implemented, analogously augmented to incorporate multi-modal formulation:

$$P_n\left(\mathbf{r}\right)=\frac{{\sum }_m{\sum }_j{O}_{m,n}^{*}(r+r_j)·{\psi }_j{\left(r\right)}^{(m,n)}}{{\sum }_n{\sum }_j{\left|O_{m,n}(r+r_j)\right|}^2},$$

(6)

and

$$O_m\left(\mathbf{r}\right)=\frac{{\sum }_n{\sum }_j{P}_{m,n}^{*}(r+r_j)·{\psi }_j{\left(r\right)}^{(m,n)}}{{\sum }_m{\sum }_j{\left|P_{m,n}(r+r_j)\right|}^2}.$$

(7)

where r and $r+r_j$ represent sample position under X-ray beam illumination at position r and the next position $r+r_j$ in the jth position of the ptychography scan in a 2D series.

3. Simulation Results

To simulate sample multiple states for ptychographic reconstructions, we performed reconstructions using the first 5 orthorgonal modes of canonical Lena test image given by Singular Value Decomposition (SVD) algorithm [17,18]. Simulated data in which the Lena object $\left(L\right)$ under illumination was the real-part only canonical Lena test image of size 512 × 512 pixels. The orthogonal modes in both probe and object are particular ranks of the singular values of the image matrices. Selecting more numbers of singular values produces higher numbers of ranks of the matrices. The orthogonal modes can represent real modes in materials systems, and they can be scientifically interesting. Electrical or magnetic field with mutually orthogonal modes could be present in waveguides. The first two orthogonal modes have the highest proportions of weights, and the weights become much less for orthorgonal modes with increasing numbers of singular values. We used SVD to decompose a probe simulated by propagating illumination after a circular pinhole into 5 orthogonal modes, and the related reconstructions of the probe, which are displayed in Figure 6. The first 5 orthogonal modes of canonical Lena test image by SVD and the related reconstructions are depicted in Figure 7. Both probe and object orthogonal modes simulated by SVD are used to simulate multi-modal ptychographic diffraction intensities to test the feasibility of information retrieval by the reconstruction algorithms. Multi-modal diffraction intensities are simulated by performing summation over all the 25 individual coherent diffraction intensities that result from 5 orthogonal modes of both probe and object test pattern. The overlap ratio in our simulations was selected to be around 85%, and the total number of scan positions in the ptychographic datasets is 323, and the ptychographic scan was performed with a spiral geometry to intentionally break the possible translation symmetry. Only the first orthogonal mode of simulated probe is fed into the reconstruction algorithms, the remaining modes of the probe and all sample modes are unknown when starting the reconstructions. The reconstruction of simulated data was performed with 1000 iterations. The reconstructed images in Figure 6i and Figure 7i are compared to Figure 6f and Figure 7f respectively, where superposition of 5 orthogonal modes of both probe and sample are calculated to compare the reconstructed and simulated probe and sample. The central region of the superposition images of Figure 7a–e is selected for simulation analyses. Comparison of the superposition of the central region of the image (Figure 7f) to the reconstructed results (Figure 7i) on sample is performed. The reconstructed images in Figure 6i and Figure 7i are compared to Figure 6f and Figure 7f by using Real-space error metric [6] to be 0.0176 and 0.0154 respectively. The reconstructions of first 5 modes do not match the original 5 modes, because the reconstruction fails to recover separated modes. In real experiments, usually Poissonian or Gaussian noises can be present in the measured datasets. These noises can be effectively modelled and taken into account by using The maximum-likelihood (ML) formulation for ptychography [7,19].

4. Discussion

Multi-modal ptychography could be potentially used to study extremely short or large time-scale dynamics of materials sciences systems, and works well when dynamical time-scale of sample is faster than that of a single ptychographic scan point in a 2D scan series. As long as there is enough statistics in a single ptychographic scan point in 2D scan series, the reconstructions convergence for multi-modal ptychography can be achieved. The estimated ratio between average time-scale of single scans position and the object dynamics should be at least around 1 to 2. Ideally, more cycles of ptychography data of dynamical objects should be collected to ensure good data reconstructions with reasonable statistics. These limiting factors should be carefully considered in future experiments, and measurements should be performed with caution to be able to investigate different materials sciences dynamical systems.

5. Conclusions

We here report examinations on multi-modal (in both sample and probe) ptychography. Both experimental and simulated reconstructions of multi-modal ptychographic datasets have suggested that multi-modal ptychography is not able to directly retrieve individually separate sample and probe modes, instead, the technique is capable to recover superposition states of both sample and probe initial modes, simultaneously, with a very good agreement with correct results. The recovered final average image of either sample or X-ray probe is a superposition state of all the sample or X-ray probe modes. We envisage multi-modal X-ray ptychography can be potentially applied to dynamical sample systems using partially coherent X-rays with few-nanometer spatial resolution in the near future when diffraction-limited synchrotron radiation facilities become available worldwide.