Physics-Based TOF Imaging Simulation for Space Targets Based on Improved Path Tracing

Abstract

Aiming at the application of close-up space measurement based on time-of-flight (TOF) cameras, according to the analysis of the characteristics of the space background environment and the imaging characteristics of the TOF camera, a physics-based amplitude modulated continuous wave (AMCW) TOF camera imaging simulation method for space targets based on the improved path tracing is proposed. Firstly, the microfacet bidirectional reflection distribution function (BRDF) model of several typical space target surface materials is fitted according to the measured BRDF data in the TOF camera response band to make it physics-based. Secondly, an improved path tracing algorithm is developed to adapt to the TOF camera by introducing a cosine component to characterize the modulated light in the TOF camera. Then, the imaging link simulation model considering the coupling effects of the BRDF of materials, the suppression of background illumination (SBI), optical system, detector, electronic equipment, platform vibration, and noise is established, and the simulation images of the TOF camera are obtained. Finally, ground tests are carried out, and the test shows that the relative error of the grey mean, grey variance, depth mean, and depth variance is 2.59%, 3.80%, 18.29%, and 14.58%, respectively; the MSE, SSIM, and PSNR results of our method are also better than those of the reference method. The ground test results verify the correctness of the proposed simulation model, which can provide image data support for the ground test of TOF camera algorithms for space targets.

1. Introduction

In recent years, time-of-flight (TOF) imaging technology has been widely used in ground robot positioning and navigation, pose estimation, 3D reconstruction, indoor games, and other fields due to its advantages in structure and performance. Significantly, researchers are promoting TOF imaging technology in spatial tasks such as spatial pose estimation and relative navigation [1,2,3,4,5,6]. However, due to the particularity of the space environment, it is difficult to obtain the imaging results of the actual TOF camera before formulating the space mission scheme, planning mission content, and designing the related algorithms, so it is impossible to evaluate the algorithm capability and ensure the smooth implementation of the mission [7]. The imaging simulation method can provide data input for the back-end algorithm test of the space-based TOF camera. Nevertheless, it is different from the imaging simulation methods of the visible light camera [8,9,10,11,12], infrared camera [13,14,15], and radar [16,17], there are few imaging simulation methods of TOF cameras because the imaging principle is essentially different. Therefore, it is of great significance to develop the imaging simulation method of the TOF camera for space targets.

For the imaging simulation method of the TOF camera, the research status is as follows. References [18,19] proposed a real-time TOF simulation framework for simple geometry based on standard graphics rasterization techniques. This method only considers the influence of errors such as dynamic motion blur and flying pixels and does not consider the influence of the background environment, so it is only suitable for indoor scenes. Reference [20] presented an amplitude modulated continuous wave (AMCW) TOF simulation method using global illumination based on the bidirectional path tracing method for indoor scenes such as kitchens. References [21,22] established physics-based TOF camera simulation methods, respectively. In order to evaluate two alternative approaches in continuous-wave TOF sensor design, reference [21] focused on realistic and practical sensor parameterization. Based on the reflective shadow map (RSM) algorithm, reference [22] introduced the bidirectional reflection distribution function (BRDF) data of materials, which has the physical imaging characteristics of actual materials, but this method also does not consider the influence of background illumination. Reference [23] proposed a pulsed TOF simulation method based on Vulkan shader and NVIDIA VKRAY ray tracing for indoor scenes. It can be seen that most of these existing TOF imaging simulation methods are only for indoor scenes, without considering the influence of background illumination, and there is no research on the imaging simulation method of TOF cameras specifically for space targets.

The main difference between the space environment and the indoor environment is the influence of sunlight. The sunlight will affect the signal-to-noise ratio and even cause the detector’s supersaturation. Therefore, the camera’s hardware must consider the suppression of background illumination (SBI). Many TOF detectors considering the SBI function have been developed [24,25,26,27,28]. At the same time, for the TOF camera simulation, SBI must also be considered, and the corresponding simulation model must be developed. Reference [29] constructed a theoretical model of SBI for PMD detectors, which can effectively characterize the detector’s ability to suppress background illumination.

To sum up, this paper proposes a physics-based imaging simulation method of TOF cameras for space targets based on improved path tracing, aiming at the space application of TOF cameras. The main contributions of this method are as follows:

(1)

An improved path tracing algorithm is developed to adapt to the TOF camera by introducing a cosine component to characterize the modulated light in the TOF camera.

(2)

The background light suppression model is introduced, and the physics-based simulation is realized by considering the BRDF model fitted by the measured data in the near-infrared band of space materials

(3)

A ground test scene is built, and the correctness of the proposed TOF camera imaging simulation method is verified by quantitative evaluation between the simulated image and measured image.

2. Materials and Methods

2.1. Imaging Principle of TOF Camera

Based on the homodyne detection principle, the AMCW TOF camera measures the distance by measuring the cross-correlation between the reflected light and reference signals. The camera first transmits the near-infrared optical signal modulated by a sine wave. The optical signal is reflected by the target surface and received by the infrared detector. The phase delay of the received signal relative to the transmitted signal is calculated to calculate the target distance information. The specific principle is shown in Figure 1.

It is assumed that the transmitted infrared optical signal is $g(t)=a\cos (2\pi f_{0}t)$, its amplitude is $a$, $f_0$ is the signal modulation frequency, and the received optical signal $s(t)$ is

$$s(t)=a_r\cos (2\pi f_{0}t+\phi )+b$$

(1)

where $a_r$ is the amplitude of reflected signal light, $\phi$ is the phase delay caused by target distance, and $b$ is the offset caused by ambient light. Then, the cross-correlation between the transmitted optical signal and the received optical signal is:

$$c_{\tau }(\phi )=s★g=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{-T/2}{\overset{T/2}{\int }}s(t)g(t+\tau )dt}$$

(2)

where $\tau$ is the time delay and $★$ is the correlation operation symbol.

In order to recover the amplitude $a_r$ and phase $\phi$ of the reflected light signal, four sequence amplitude images are collected generally, which are defined as:

$$\begin{array}{ccc}{C}_i=c_{{\tau }_i}(\phi ),& {\tau }_i=i\cdot \frac{\pi }{2\times 2\pi f_0},& i\in \{0,1,2,3\}\end{array}$$

(3)

Then, the phase delay $\phi$, reflected signal amplitude $a_r$, and offset $b$ can be obtained as:

$$\phi =\arctan \left(\frac{C_3-C_1}{C_0-C_2}\right)$$

(4)

$$a_r=\frac{1}{2a}\sqrt{{\left(C_3-C_1\right)}^2+{\left(C_0-C_2\right)}^2}$$

(5)

$$b=\frac{1}{4}(C_0+C_1+C_2+C_3)$$

(6)

Finally, the distance $d$ between the TOF camera and the scene is:

$$d=\frac{1}{2}{c}_{light}\frac{\phi }{2\pi f_0}$$

(7)

The purpose of imaging simulation is to obtain the distance $d$ and the intensity of the reflected signal, which is related to the characteristics of the target material, background, the TOF camera, and so on.

2.2. Imaging Characteristic Modeling

2.2.1. Target Material Characteristics Modeling

In order to achieve physics-based simulation, it is necessary to introduce the reflection characteristics of the actual material of the target surface. The BRDF is usually used to express the reflection characteristics of the material surface. As shown in Figure 2a, on the surface element $dA$, the incident light direction is $({\theta }_i,{\varphi }_i)$, and the observation direction is $({\theta }_r,{\varphi }_r)$, where $\theta$ and $\varphi$ represent the zenith angle and azimuth angle, respectively, and $\stackrel{\wedge }{Z}$ represents the normal direction of the surface. The BRDF is defined as the ratio of the radiance $d{L}_r({\theta }_i,{\varphi }_i,{\theta }_r,{\varphi }_r)$ emitted along the direction $({\theta }_r,{\varphi }_r)$ to the irradiance $d{E}_i({\theta }_i,{\varphi }_i)$ of the measured surface incident along the direction $({\theta }_i,{\varphi }_i)$, and the formula is as follows.

$$f_r({\theta }_i,{\varphi }_i,{\theta }_r,{\varphi }_r)=\begin{array}{cc}\frac{d{L}_r({\theta }_i,{\varphi }_i,{\theta }_r,{\varphi }_r)}{d{E}_i({\theta }_i,{\varphi }_i)}& (s{r}^{-1})\end{array}$$

(8)

Yellow thermal control material and silicon solar cells are two primary surface materials of space targets. In this paper, their BRDF data are measured by the REFLET-180 BRDF measuring instrument as shown in Figure 2b. The specific material samples and some corresponding measurement results are shown in Figure 3.

At the same time, the above measured BRDF data are theoretically modeled using the microfacet BRDF model [14,15]. The microfacet BRDF model includes the specular reflection term and the diffuse reflection term, and the specular reflection term is the Torrance–Sparrow BRDF model [30]. The definition of the microfacet BRDF model is as follows.

$$f_r({\theta }_i,{\varphi }_i,{\theta }_r,{\varphi }_r)=\frac{k_s}{4\cos {\theta }_i\cos {\theta }_r}DFG+\frac{k_d}{\pi }$$

(9)

where $k_s$ is the specular reflection coefficient, $k_d$ is the diffuse reflection coefficient, $D$ is the micro surface distribution factor, $F$ is the Fresnel factor [31], and $G$ is the geometric attenuation factor. Their specific definition is as follows.

$$\{\begin{array}{l}D=e^{-{\tan }^2{\theta }_h/{\alpha }^2}/\left(\pi {\alpha }^2{\cos }^4{\theta }_h\right)\\ F=F_0+\left(1-F_0\right){\left(1-\cos \gamma \right)}^n\\ G=\min \left(1,2\cos {\theta }_h\cos {\theta }_r/\cos \gamma ,2\cos {\theta }_h\cos {\theta }_i/\cos \gamma \right)\end{array}$$

(10)

where $\alpha =\sqrt{2}\sigma$ and $\sigma$ are the root mean square slope of the microfacet, $F_0$ is the Fresnel coefficient at vertical incidence, and $n$ is the undetermined coefficient.

Since the wavelength of the TOF camera light source used in this paper is 850 nm, the parameters of the BRDF model at 850 nm are fitted, and the fitting error is expressed by the following formula [32,33].

$$e_{fit}=\frac{{\displaystyle \sum _{{\theta }_i}{\displaystyle \sum _{{\theta }_r}{\left[f_{r-model}\cdot \cos \left({\theta }_r\right)-f_{r-measured}\cdot \cos \left({\theta }_r\right)\right]}^2}}}{{\displaystyle \sum _{{\theta }_i}{\displaystyle \sum _{{\theta }_r}{\left[f_{r-measured}\cdot \cos \left({\theta }_r\right)\right]}^2}}}$$

(11)

where $f_{r-measured}$ represents the measured BRDF value, $f_{r-model}$ represents the fitted BRDF value, the parameter fitting results are shown in

Table 1. Fitting results of BRDF model parameters at 850 nm.

	${\mathit{k}}_{\mathit{s}}$	${\mathit{k}}_{\mathit{d}}$	$\mathit{n}$	$\mathit{\sigma }$	${\mathit{F}}_{\mathbf{0}}$	${\mathit{e}}_{\mathit{f}\mathit{i}\mathit{t}}$
Thermal control material	0.649	0.081	8.51	0.014	0.775	2.6%
Silicon solar cells	0.350	0.053	2.22	0.010	0.021	5.2%

, and the visualization of fitting results is shown in Figure 4.

2.2.2. Background Characteristics Modeling

Space targets run in the Earth’s orbit. The radiation in the imaging band of the TOF camera is mainly composed of direct solar radiation, solar radiation reflected by the Earth, solar radiation reflected by the Moon, and stellar radiation. Since the stellar radiation is minimal compared to other radiation, the stellar radiation at the target can be ignored. The surface radiation of the target is shown in Figure 5.

(1)

The irradiance generated by direct solar radiation at the target is:

$$E_S={\displaystyle \underset{{\lambda }_1}{\overset{{\lambda }_2}{\int }}\frac{\frac{c_1}{{\lambda }^5}{(e^{c_2/\lambda T_S}-1)}^{-1}\cdot R_S^2}{R_{S-E}^2}}d\lambda$$

(12)

where $\lambda$ is the wavelength in µm; $c_1$ is the first blackbody radiation constant; $c_2$ is the second blackbody radiation constant; $T_S$ is the solar radiation temperature, and $T_S=5900$ K; $R_S$ is the solar radius, and $R_S=6.5955\times {10}^5$ km; $R_{S-E}$ is the distance between the Sun and the Earth; ${\lambda }_1~{\lambda }_2$ is the observation band of the TOF camera.

(2)

Assuming that the space target is in a high Earth orbit and the Earth is assumed to be a diffuse sphere, the irradiance generated by solar radiation reflected by the Earth at the target is approximate as follows.

$$\begin{array}{l}{E}_E=\frac{E_S\cdot {\rho }_E\cdot R_E^2}{\pi {(R_E+H_{TE})}^2}{\displaystyle \underset{{\theta }_E-\frac{\pi }{2}}{\overset{\frac{\pi }{2}}{\int }}{\displaystyle \underset{-\frac{\pi }{2}}{\overset{\frac{\pi }{2}}{\int }}{\cos }^{3}B\left(\cos {\theta }_E{\cos }^{2}l+\sin {\theta }_E\sin l\cos l\right)}}dBdl\\ =\frac{4E_S\cdot {\rho }_E\cdot R_E^2}{3\pi {(R_E+H_{TE})}^2}\left[\cos {\theta }_E\left(\frac{\pi }{2}-\frac{{\theta }_E}{2}+\frac{\sin 2{\theta }_E}{4}\right)+\sin {\theta }_E\left(\frac{1}{4}-\frac{\cos 2{\theta }_E}{4}\right)\right]\end{array}$$

(13)

where ${\rho }_E$ is the average albedo of the Earth and ${\rho }_E=0.35$; $R_E$ is the radius of the Earth and $R_E=6370$ km; $H_{TE}$ is the height of the target from the ground; ${\theta }_E$ is the angle between vector ${\overrightarrow{v}}_{Sun-Earth}$ and vector ${\overrightarrow{v}}_{Earth-Target}$, and the value range is $[0,\pi ]$.

(3)

Similarly, assuming that the Moon is a diffuse sphere, the irradiance generated by solar radiation reflected by the Moon at the target is:

$$\begin{array}{l}{E}_M\approx \frac{E_S\cdot {\rho }_M\cdot R_M^2}{\pi R_{TM}^2}{\displaystyle \underset{{\theta }_M-\frac{\pi }{2}}{\overset{\frac{\pi }{2}}{\int }}{\displaystyle \underset{-\frac{\pi }{2}}{\overset{\frac{\pi }{2}}{\int }}{\cos }^{3}B\left(\cos {\theta }_M{\cos }^{2}l+\sin {\theta }_M\sin l\cos l\right)\cdot }}dBdl\\ =\frac{4E_S\cdot {\rho }_M\cdot R_M^2}{3\pi R_{TM}^2}\left[\cos {\theta }_M\left(\frac{\pi }{2}-\frac{{\theta }_M}{2}+\frac{\sin 2{\theta }_M}{4}\right)+\sin {\theta }_M\left(\frac{1}{4}-\frac{\cos 2{\theta }_M}{4}\right)\right]\end{array}$$

(14)

where ${\rho }_M$ is the average albedo of the Moon and ${\rho }_M=0.12$; $R_M$ is the radius of the Moon and $R_M=1738$ km; $R_{TM}$ is the distance between the target and the center of mass of the Moon; ${\theta }_M$ is the angle between vector ${\overrightarrow{v}}_{Sun-Moon}$ and vector ${\overrightarrow{v}}_{Moon-Target}$, and the value range is $[0,\pi ]$.

2.2.3. SBI Characteristics Modeling

The TOF sensor is commonly the photonic mixer device (PMD). The PMD is a two-tap sensor, and the structure diagram is shown in Figure 6a [34]. The interference of background light on the actively modulated light leads to the premature saturation of the quantum well of the pixel, so that less reflected active light containing depth information is detected, which will lead to increased noise and a reduced signal-to-noise ratio (SNR). The schematic diagram of the influence of background light on PMD is shown in Figure 6b [35]. Therefore, designing a system that can effectively reduce the impact of background illumination is an important job.

The suppression of background illumination (SBI) developed by PMDTec has been successfully applied to TOF cameras such as CamCube. The SBI is an in-pixel circuitry that subtracts ambient light, which prevents the pixels from saturating. The manufacturer did not publish the specific details of the SBI compensation circuit, but reference [36] summarizes the response relationship of the A and B channels of the PMD pixel to the exposure time through experiments, as shown in Figure 7. The PMD pixel will produce a photoelectric response to both the signal light and the background illumination during the exposure time. In the linear region, the SBI is not activated. When the number of electrons in the quantum well A or B reaches $n_{SBI,start}$, the SBI is activated. As the exposure time continues to increase, it will enter the SBI Limit region, and the data of this pixel is invalid at this time.

Reference [29] established the SBI model shown in Figure 8 by analyzing the response relationship of the TOF camera to illumination intensity or exposure time. This SBI model is also used in this paper, which can be described as follows. The charge stored in two quantum wells $\sum$ is continuously compared with a reference value $n_{SBI,start}$, and once the stored charge in one of the quantum wells exceeds this value, that is, the difference $n_{\Delta }$ between the stored charge and the reference value $n_{SBI,start}$ is positive, a compensation process will be triggered. Two compensation currents are injected into the two quantum wells during the compensation process, respectively, and the number of charges contained in the compensation currents is approximately the same as that of $n_{\Delta }$. After compensation, the quantum well containing more electrons is reset to $n_{SBI,start}$, and the charge of the other quantum well is set to a value lower than $n_{SBI,start}$.

The compensation process does not lose any critical information, as the phase delay can be estimated by keeping only the difference between the charges in the two quantum wells. As shown below, each sub-amplitude image in Equation (4) is the difference between the amplitude images of channels A and B.

$$C_i=C_{iA}-C_{iB}$$

(15)

where $C_{iA}$ and $C_{iB}$ are amplitude images output by channels A and B, respectively.

As shown in the figure above, both the background illumination and the SBI process introduce additional Poisson noise. The Poisson distribution is given by

$$P_{\lambda }=\frac{{\lambda }^k}{k!}{e}^{-\lambda }$$

(16)

where $\lambda$ describes the mean of the values, which is here the number of generated electrons. $P_{\lambda }$ is the probability of detecting k electrons for a given $\lambda$.

2.3. Imaging Simulation Modeling

In order to obtain the simulated image of the actual target, an AMCW TOF camera imaging simulation model based on path tracing is proposed in this paper. Firstly, establish the space three-dimensional target scene, simulate the TOF camera to transmit the modulated light signal to the target scene, and obtain the radiance image of the space target scene through the path tracing method. Secondly, the radiance image is coupled with the imaging chain factors such as platform motion, optical system, and detector, and the influence of background light is suppressed through the SBI module. Then, four frames of amplitude images are obtained through time sampling. Finally, the final depth and grey image are obtained through Formula (4) to Formula (7). The specific simulation model is shown in Figure 9.

2.3.1. Improved Path Tracing Algorithm of the TOF Camera

The general path tracing algorithm is suitable for the imaging simulation of visible light and infrared cameras. However, since the light signal in the TOF camera is modulated, there is a cosine component that varies with the propagation distance. Therefore, to apply the path tracing algorithm to the TOF camera, an improved path tracing algorithm is developed by introducing a cosine component to characterize the modulated light in the TOF camera, and the improved algorithm is shown as follows.

Define $T\left(g\left(t\right),d\right)$ to represent the propagation distance d [m] of the optical signal $g(t)=b+a\cos (2\pi f_{0}t)=g{(t)}^{-}+real\left(g{(t)}^{\cos }\right)$ without attenuation, and its form is as follows.

$$\begin{array}{c}T\left(g\left(t\right),d\right)=b+a\cos \left(2\pi f_{0}t+2\pi f_0\frac{d}{c}\right)\\ =g{(t)}^{-}+real\left(g{(t)}^{\cos }\cdot e^{\mathrm{i}\Psi d}\right)\end{array}$$

(17)

where $g{(t)}^{-}=b$ represents the DC component, $g{(t)}^{\cos }=a{e}^{\mathrm{i}(2\pi f_{0}t)}$ represents the cosine component, and $real()$ represents the real part of the imaginary number and $\Psi =\frac{2\pi f_0}{c}$.

The light source $L$ of the TOF camera is assumed to be a uniform point light source, then the light intensity $I_L$ is as follows.

$$I_L\left(t\right)=\frac{P_L}{{\Omega }_L}\left[1+\cos \left(2\pi f_{0}t\right)\right]$$

(18)

where $P_L[\mathrm{W}]$ represents the power of the light, ${\Omega }_L$ represents the solid angle, and $f_0$ represents the modulation frequency of the optical signal.

The illuminance $E_L\left(t,d\right)$ at the micro-plane perpendicular to the light propagation direction at $d[\mathrm{m}]$ away from the light source is:

$$E_L\left(t,d\right)=T\left(I_L\left(t\right)/d^2,d\right)$$

(19)

For a point $P$ in the scene shown in Figure 10, the illuminance generated by direct lighting at $P$ is $E_{L\to @P}(t,d_{L\to P})$.

$$E_{L\to @P}\left(t,d_{L\to P}\right)=E_L\left(t,d_{L\to P}\right)\cdot \cos {\theta }_{P\to L}$$

(20)

where $d_{L\to P}$ is the distance between the light source $L$ and the point $P$, and ${\theta }_{P\to L}$ is the included angle between the vector ${\overrightarrow{v}}_{P\to L}$ of $P$ pointing to $L$ and the normal vector ${\overrightarrow{n}}_P$ of the surface at point $P$.

The direct illumination radiance $L_{@P\to S}^{direct}(t)$ generated by the infrared light source along the direction of $P\to S$ at point $P$ is:

$$L_{@P\to S}^{direct}(t)=E_{L\to @P}(t,d_{L\to P})\cdot f_{L\to P\to S}$$

(21)

where $f_{L\to P\to S}$ represents the BRDF of the surface. For the irradiance $E_{L^{\prime }\to @P}(t,d_{L^{\prime }\to P})$ of ambient light sources $L^{\prime }$ such as the Sun, the Earth, and the Moon, as shown in Formulas (12)–(14), the cosine component of $E_{L^{\prime }\to @P}(t,d_{L^{\prime }\to P})$ is 0, only the DC component. Therefore, in the derivation process, this paper only gives the case of the active light of the TOF camera. When encountering ambient light, it only needs to change the cosine component related to ambient light to 0.

In addition to the direct illumination of light sources, the light signal reflected by other surfaces will also affect the radiance at point $P$, as shown in Figure 11. We call this part indirect illumination radiance denoted by $L_{@P\to S}^{indirect}\left(t\right)$, which is defined as follows.

$$L_{@P\to S}^{indirect}\left(t\right)={\displaystyle {\int }_{\Omega }{f}_{P^{\prime }\to P\to S}{L}_{P^{\prime }\to @P}\left(t\right)\cos \left({\theta }_i\right)d{\varpi }_i}$$

(22)

where $L_{P^{\prime }\to @P}\left(t\right)$ represents the radiance generated by $P^{\prime }$ at point $P$, $\Omega$ represents the solid angle of these surfaces that contribute to the radiance at point $P$, ${\varpi }_i$ represents the micro solid angle in the $\overrightarrow{{\omega }_i}$ direction, and $d{\varpi }_i$ is defined as follows.

$$d{\varpi }_i=\frac{\cos \left({\theta }_o\right)dA\left(P^{\prime }\right)}{d_{P^{\prime }\to P}^2}$$

(23)

Since the radiation energy of light in the scene is conserved, the radiance at $P$ can be associated with the radiance at another point $P^{\prime }$.

$$L_{P^{\prime }\to @P}\left(t\right)=T\left(L_o\left(isect\left(P,{\overrightarrow{\omega }}_i\right),-{\overrightarrow{\omega }}_i,t\right),d_{P^{\prime }\to P}\right)$$

(24)

where $isect\left(P,{\overrightarrow{\omega }}_i\right)$ calculates the first intersection $P^{\prime }$ between the light propagating along the ${\overrightarrow{\omega }}_i$ direction from point $P$ and the scene. $dA\left(P^{\prime }\right)$ represents the microfacet at the intersection $P^{\prime }$.

Substituting Equation (23) into Equation (22), we can get:

$$L_{@P\to S}^{indirect}\left(t\right)={\displaystyle {\int }_A{f}_{P^{\prime }\to P\to S}{L}_{P^{\prime }\to @P}\left(t\right)\frac{V\left(P\leftrightarrow P^{\prime }\right)\left|\cos \left({\theta }_i\right)\right|\left|\cos \left({\theta }_o\right)\right|}{d_{P^{\prime }\to P}^2}dA\left(P^{\prime }\right)}$$

(25)

where $A$ is all surfaces of the scene, and $V\left(P\leftrightarrow P^{\prime }\right)$ represents the visibility between points $P^{\prime }$ and $P$. If the two points are visible to each other, $V\left(P\leftrightarrow P^{\prime }\right)=1$; otherwise, $V\left(P\leftrightarrow P^{\prime }\right)=0$ Let.

$$G\left(P\leftrightarrow P^{\prime }\right)=\frac{V\left(P\leftrightarrow P^{\prime }\right)\left|\cos \left({\theta }_i\right)\right|\left|\cos \left({\theta }_o\right)\right|}{d_{P^{\prime }\to P}^2}$$

(26)

Then

$$L_{@P\to S}^{indirect}\left(t\right)={\displaystyle {\int }_A{f}_{P^{\prime }\to P\to S}{L}_{P^{\prime }\to @P}\left(t\right)G\left(P\leftrightarrow P^{\prime }\right)dA\left(P^{\prime }\right)}$$

(27)

Combining direct radiance and indirect radiance, the radiance $L_{@P\to S}(t)$ at the point $P$ can be expressed as follows.

$$\begin{array}{l}{L}_{@P\to S}\left(t\right)=L_{@P\to S}^{direct}\left(t\right)+L_{@P\to S}^{indirect}\left(t\right)\\ =E_{L\to @P}\left(t,d_{L\to P}\right)\cdot f_{L\to P\to S}+{\displaystyle {\int }_A{f}_{P^{\prime }\to P\to S}{L}_{P^{\prime }\to @P}\left(t\right)G\left(P\leftrightarrow P^{\prime }\right)dA\left(P^{\prime }\right)}\end{array}$$

(28)

After the radiance $L_{@P\to S}\left(t\right)$ propagate distance $d_{P\to S}$, the radiance $L_{P\to @S}(t)$ at the camera sensor $S$ is:

$$L_{P\to @S}(t)=T\left(L_{@P\to S}(t),d_{P\to S}\right)$$

(29)

The path tracing algorithm [37,38] based on Monte Carlo is utilized to solve the Equation (28), which is approximated as follows.

$$L_{@P_1\to S}(t)\approx \frac{1}{N}{\displaystyle \sum _{k=1}^N{\displaystyle \sum _l^{NumLight}{\displaystyle \sum _{n=1}^{MaxDepth}{L}_{MC}^k\left({\overline{p}}_n\right)}}}$$

(30)

where $MaxDepth$ represents the maximum bounce times of light, $NumLight$ represents the number of light sources, $N$ represents the number of samples per pixel, $L_{MC}^k\left({\overline{p}}_n\right)$ represents the Monte Carlo estimation of the kth sample, as shown below.

$$\begin{array}{l}{L}_{MC}\left({\overline{p}}_n\right)=real\{\frac{E_{L\to @P_n}^{—}(t,d_{L\to P_n})\cdot f_{L\to P_n\to S}\cdot f_{P_n\to P_{n-1}\to P_{n-2}}G\left(P_n\leftrightarrow P_{n-1}\right)}{p_A(P_n)}\\ \times \left({\displaystyle \prod _{i=1}^{n-2}\frac{f_{P_{i+1}\to P_i\to P_{i-1}}\cdot V(P_i\to P_{i+1})\cdot \left|\cos {\theta }_i\right|}{p_{\omega }(P_i\to P_{i+1})}}\right)\\ +\frac{E_{L\to @P_n}^{\cos }(t,d_{L\to P_n})\cdot f_{L\to P_n\to S}\cdot e^{\mathrm{i}\Psi d_{P_n\to P_{n-1}}}{f}_{P_n\to P_{n-1}\to P_{n-2}}G\left(P_n\leftrightarrow P_{n-1}\right)}{p_A(P_n)}\\ \times \left({\displaystyle \prod _{i=1}^{n-2}\frac{e^{\mathrm{i}\Psi d_{P_{i+1}\to P_i}}\cdot f_{P_{i+1}\to P_i\to P_{i-1}}\cdot V(P_i\to P_{i+1})\cdot \left|\cos {\theta }_i\right|}{p_{\omega }(P_i\to P_{i+1})}}\right)\}\end{array}$$

(31)

where $p_A(P_n)$ is the sampling probability function at $P_n$ point, which is defined as $p_A(P_n)=\frac{1}{{\displaystyle {\sum }_n{A}_n}}$, ${\displaystyle {\sum }_n{A}_n}$ is the sum of the surfaces of the scene, and $p_{\omega }(P_i\to P_{i+1})$ is the sampling probability density function in the $P_i\to P_{i+1}$ direction.

The irradiance $E_{pixel}$ received by the pixel of the TOF sensor is as follows.

$$E_{pixel}(t)=T\left(L_{@P_1\to S}(t),d_{P_1\to S}\right)\cdot \tau \cdot \frac{\pi }{4}\cdot {\left(\frac{1}{\mathrm{F}/\#}\right)}^2{\cos }^4\alpha$$

(32)

where, $\tau$ represents the transmittance of the AMCW TOF camera, $\mathrm{F}/\#$ represents its f-number, and $\alpha$ represents the angle between the optical axis and the vector ${\overrightarrow{v}}_{S\to P_1}$.

2.3.2. Imaging Link Impact Modeling

As shown in Figure 9, the target scene radiation forms the conversion process of “radiation-voltage-grey” after passing through each component module of the TOF camera. In this process, the signal is affected by the optical system, detector, circuit processing unit, and platform, which is reflected in the image as the dispersion effect on the radiation image. These dispersion effects can be described by each module’s modulation transmission function (MTF), as shown in Figure 12.

Then, the last output simulation image [39] is:

$$Imag{e}_{sim}=IFFT(FFT(Imag{e}_{Rad})\cdot MT{F}_{opt}\cdot MT{F}_{\det }\cdot MT{F}_e\cdot MT{F}_{mot})+Noise$$

(33)

where $FFT$ represents fast Fourier transform; $MT{F}_{opt}$ represents the MTF of the optical system; $MT{F}_{\det }$ represents the MTF of the detector; $MT{F}_e$ represents the MTF of the signal processing circuit; $MT{F}_{mot}$ represents the MTF of the platform motion; $Noise$ represents the noise of the image. MTF models of different processes can be found in references [8,17,40].

3. Results

In order to verify the correctness of the proposed TOF camera imaging simulation method for space targets, the ground experiment scene shown in Figure 13 is built. Considering that it is difficult to simulate the radiation of the Earth and the Moon on the ground, and the influence of these radiations is minimal, the ground experiment only considers the direct solar radiation. The experimental scene comprises a satellite model, a TOF camera, a turntable, a black background, and a solar simulator. The surface of the satellite model is mainly composed of yellow thermal control materials and silicon solar cells, and the TOF camera is placed horizontally on a tripod. The TOF camera used in this paper is the PMD CamBoard camera, and its performance indexes are shown in

Table 2. The performance indexes of PMD CamBoard.

Indexes	Value
Resolution	224 × 171 pixel
Wavelength of light source	850 nm
Field angle	62° × 45°
Focal length ($f/dx,f/dy$)	(208.33, 208.33)
Aperture	2 mm
Acquisition time per frame	4.8 ms typical at 45 fps
Average power consumption	300 mW

. The solar simulator used is the Newport Oriel Sol3A, and its performance indexes are shown in

Table 3. Some performance indexes of the Newport Oriel Sol3A Solar Simulator, Model 94123A.

Indexes	Value
Illumination area	305 mm × 305 mm
Maximum angle of incidence	(half angle) < ±0.5°
Typical power output	100 mW/cm2 (1 SUN), ±20% Adjustable
Uniformity	<±2%
Spectral match	9.7–16.1% (800–900 nm)

. The experimental background is a dark background composed of black light-absorbing flannel, and the light beam emitted by the solar simulator completely covers the target model.

During the experiment, all light sources in the room were turned off, the solar simulator was turned on, the output power of the solar simulator was set to the typical power (1 SUN), and the PMD camera was used to capture the images of the target model. Due to the size of the satellite model and the limited illumination area of the solar simulator, to make the beam of the solar simulator cover the target model entirely, the satellite model is inclined at a certain angle. At the same time, the TOF imaging simulation method proposed in this paper was used to obtain the corresponding simulation image according to the experimental conditions. Other simulation parameters used are shown in

Table 4. Other simulation parameters used.

Indexes	Value
Size of satellite body	20 × 20 × 20 cm
Size of solar panel	63 × 35 cm
$d_{cam-sat}$	1.5 m
${\theta }_{cam-panel}$	60°
${\theta }_{solarSim-panel}$	10°
$n_{SBI,start}$	36,500

. The measured images and the simulated images are shown in Figure 14. In order to facilitate the comparison with the simulated image, the background area of the measured image is eliminated manually.

In Table 4, $d_{cam-sat}$ represents the distance between the PMD TOF camera and satellite body center, ${\theta }_{cam-panel}$ represents the angle between the optical axis of the camera and the plane of the solar panel, and ${\theta }_{solarSim-panel}$ represents the angle between the beam of the solar simulator and the plane of the solar panel. According to references [41,42], the value of $n_{SBI,start}$ is about 36,500.

In order to quantitatively evaluate the simulation effect, the mean, variance, mean square error (MSE), structural similarity (SSIM), and peak signal-to-noise ratio (PSNR) of the grey image and depth image are calculated. The smaller the MSE, the larger the SSIM, and the larger the PSNR, indicating the more realistic the simulation results are. These indexes are shown in

Table 5. Comparison of indexes between measured images and simulated images.

Index		Measured Mage	Ref. [22]’s Results	Our Results	Ref. [22]’s Error	Our Error
Grey	Mean	17.79	16.73	18.25	5.96%	2.59%
	Var	1411.67	1347.53	1358.06	4.54%	3.80%
	MSE	—	1788.35	1782.60	—	—
	SSIM	—	0.70	0.72	—	—
	PSNR	—	15.60	15.62	—	—
Depth	Mean	403.04	750.28	476.75	86.16%	18.29%
	Var	2.95 × 105	4.15 × 105	3.38 × 105	40.68%	14.58%
	MSE	—	5.46 × 105	4.45 × 105	—	—
	SSIM	—	0.80	0.85	—	—
	PSNR	—	38.95	39.85	—	—

, where bold indicates better results. At the same time, the depth image is converted into a point cloud and denoised. The point cloud before and after denoising is shown in Figure 15.

4. Discussion

As shown in Figure 14, our simulated grey and depth images are very similar to the measured grey and simulation images in visual effect. The gray image simulated by the method of reference [22] is also very similar to the measured gray image. However, the corresponding simulated depth image is quite different from the measured depth. The specific reason is that this method does not consider the error introduced by background lighting, resulting in low depth noise. At the same time, as shown in Figure 15, the point cloud coincidence degree of our method is better than that of reference [22]. In addition, it can be seen from Table 5 that the relative error of the grey mean, grey variance, depth mean, and depth variance is 2.59%, 3.80%, 18.29%, and 14.58%, respectively, which is better than the results of the method in reference [22]. Furthermore, our method’s MSE, SSIM, and PSNR results are also better than those of the method in reference [22].

5. Conclusions

This paper proposed a physics-based AMCW TOF camera imaging simulation method based on the improved path tracing for space targets based on the analysis of space background environment characteristics and the TOF camera imaging mechanism. Firstly, the BRDF data of the yellow thermal control material and the silicon cell at 850 nm was measured, the parameters of the microfacet BRDF model were fitted, and the fitting error was less than 5.2%. Secondly, an improved path tracing algorithm was developed to adapt to the TOF camera by introducing a cosine component to characterize the modulated light in the TOF camera. Then, the imaging link simulation model considering the coupling effects of the BRDF of materials, SBI, optical system, detector, electronic equipment, platform vibration, and noise was established. Finally, the ground experiment was carried out, and the relative error of the grey mean, grey variance, depth mean, and depth variance was 2.59%, 3.80%, 18.29%, and 14.58%, respectively. At the same time, our method’s MSE, SSIM, and PSNR results were also better than those of the reference method. The experimental results verify the correctness of the proposed simulation method and can provide image data support for the ground test of TOF camera algorithms for space targets.