Differentiable Automated Design of Automotive Freeform AR-HUD Optical Systems

Abstract

The automotive augmented reality head-up display (AR-HUD) system projects critical driving information directly into the driver’s line of sight, enhancing driving safety, user experience, and navigation efficiency. However, due to the intrinsic asymmetry of vehicle windshields, existing optical configurations are difficult to use as effective design starting points. The asymmetric transmission region of the windshield causes the AR-HUD optical system to deviate significantly from the YOZ plane, increasing the complexity of system design and optimization. To address these challenges, this paper proposes an automated design method for automotive AR-HUD optical systems. Given the windshield geometry and system design specifications, a normal-guided iterative construction method is first employed to generate a high-performance initial optical structure with low distortion. Subsequently, differentiable ray tracing combined with optimization algorithms is employed to further improve system performance. Based on the proposed method, an AR-HUD optical system with a 130 mm × 50 mm eye-box and a 13° × 4° field of view was designed. The design results indicate that the maximum optical distortion is 0.51%. At five sampled eye positions within the eye-box, the MTF exceeds 0.5 at the spatial frequency of 6 lp/mm, and the dynamic distortion remains below 5.36′. Finally, a complete experimental prototype was established, and the experimental results verified the feasibility and effectiveness of the proposed automated design method.

1. Introduction

Head-up display (HUD) systems have been widely adopted in modern vehicles to present essential driving information, such as navigation cues and vehicle status, directly in front of the driver. By allowing drivers to access critical information without looking down at the instrument cluster, HUD systems reduce visual distraction and help mitigate inattentive driving, thereby enhancing driving safety [1,2,3,4]. With the rapid development of intelligent and connected vehicle technologies, augmented reality head-up display (AR-HUD) systems have attracted increasing attention in recent years [5,6]. Compared with conventional HUD systems, AR-HUD systems function as advanced human–machine interfaces by integrating information from cameras, radar, and high-precision maps. Virtual images are projected onto the windshield and spatially registered with real-world objects, such as lanes, vehicles, and pedestrians, enabling intuitive augmented visual cues including navigation arrows, lane-level guidance, and hazard warnings. These features significantly enhance drivers’ situational awareness and driving comfort.

A typical automotive AR-HUD optical system consists of an irregularly shaped windshield and multiple freeform reflective surfaces. The windshield, acting as an optical combiner, introduces considerable aberrations due to its complex geometry and nonuniform curvature. To compensate for these aberrations and to project the image generated by the picture generation unit (PGU) to a virtual image plane several meters in front of the driver, multiple freeform mirrors are commonly employed. Because automotive windshields exhibit strong asymmetry and large geometric variations across vehicle models, AR-HUD optical systems usually require customized designs. However, existing industrial design workflows largely rely on experience-driven trial-and-error procedures, which require substantial manual effort and expert intervention. As a result, the design efficiency is low, and the final imaging performance strongly depends on the designer’s experience.

To improve design efficiency and reduce reliance on trial-and-error procedures, various automated freeform optical design methods have been proposed [7,8,9]. Representative approaches include the Simultaneous Multiple Surface (SMS) method [10,11] and the partial differential equations (PDEs) method [12,13,14]. While effective, these methods are generally limited to systems with a single freeform surface or a small field of view (FOV). To address systems with multiple freeform surfaces, Yang et al. [15,16,17] proposed the construction–iteration (CI) method, which iteratively constructs optical surfaces at sampled points. Building on these foundational works, several studies have explored automated design strategies specifically for automotive AR-HUD systems. For instance, Fan et al. utilized the CI framework to generate initial AR-HUD configurations [18], and Yang et al. developed an automatic continuous optimization workflow [19]. Despite these advancements, AR-HUD systems reported in the literature often exhibit relatively large optical and dynamic distortion. Consequently, distortion-correction integrated circuits or complex warping-based pre-distortion algorithms are typically required to compensate for virtual-image distortion [20,21], increasing system cost and implementation complexity. Moreover, the dependence on commercial optical design software generally restricts the flexibility of the design workflow, making it difficult to implement fully integrated, end-to-end optimization strategies. To address these challenges, differentiable ray-tracing-based optimization methods have emerged as a powerful paradigm in optical design [22,23,24,25,26,27], utilizing gradient information for efficient end-to-end optimization. While promising results have been demonstrated in rotationally symmetric systems and off-axis systems with YOZ-plane symmetry, their application to automotive AR-HUDs remains relatively limited. These systems typically involve highly asymmetric freeform surfaces and stringent geometric constraints, presenting unique challenges for automated design and optimization.

In this paper, we propose an automated and differentiable design method for wide-field automotive AR-HUD optical systems. The proposed framework consists of two stages. First, starting from a simple planar initial configuration, a normal-guided iterative construction strategy is employed to automatically generate a freeform initial structure with low distortion based on a prescribed image point distribution. Second, a differentiable ray-tracing framework is introduced to trace sampled rays across the entire eye-box and to formulate an optimization loss function. By combining automatic differentiation with nonlinear least-squares optimization, the optical system parameters are optimized in an end-to-end and continuous manner, further improving imaging quality and reducing optical distortion. The feasibility of the proposed method is validated through the fabrication and experimental evaluation of a full-scale AR-HUD prototype.

2. Method

AR-HUD system is a typical multi-eye-pupil optical system, in which a large eye-box is defined as the entrance pupil to cover all possible eye-pupil positions, thereby ensuring superior imaging quality for the driver at arbitrary eye positions. In this section, an automated design method is proposed for the development of a high-performance and wide–field AR-HUD optical system. The proposed method consists of two main stages. First, sampled rays over the full FOV are traced throughout the entire eye-box to construct an initial AR-HUD configuration based on a planar optical system. Subsequently, all sampled rays are traced using differentiable ray tracing, and the AR-HUD optical system is iteratively optimized by combining this process with the Levenberg–Marquardt (LM) algorithm. The overall flowchart of the proposed automated design method is shown in Figure 1.

2.1. Construction Method of Initial Structures

To achieve a high-performance AR-HUD optical system, a well-designed initial configuration is crucial for ensuring optimization convergence. In this section, a Normal-Guided Iterative Construction (NIC) method is proposed for generating the initial optical structure, as illustrated in Figure 2. The NIC method is developed based on the modal surface reconstruction framework commonly used in deflectometry [28]. By fitting the ideal surface normals of the sampled rays, freeform surfaces are progressively constructed to approximate the target optical configuration. Prior to this process, pupil sampling over the entire eye-box and FOV sampling must be defined to ensure high imaging quality across the full eye-box range. As shown in Figure 3a, since the AR-HUD optical system is fully asymmetric, K sampled field points are selected over the entire rectangular FOV. The entire eye-box is treated as the entrance pupil of the optical system, and M rays are uniformly sampled over the rectangular pupil, as depicted in Figure 3b. Consequently, the total number of characteristic rays to be traced is $N_{\mathit{r}ays}=K\times M$.

As illustrated in Figure 3, during the construction of the freeform surface $S_i$, let $Q_{i,j}$ denote the intersection point of the j-th sampled pupil ray of the i-th field with the preceding surface. The intersection point with surface S_i is defined as $P_{i,j}$, and the corresponding target image point is $T_{i,j}$. Consequently, the normalized ideal normal vector at $P_{i,j}$ can be calculated as:

$${\widehat{\mathit{n}}}_{i,j}={\displaystyle \frac{\overrightarrow{Q_{i,j}{P}_{i,j}}-\overrightarrow{P_{i,j}{T}_{i,j}}}{\left|\overrightarrow{Q_{i,j}{P}_{i,j}}-\overrightarrow{P_{i,j}{T}_{i,j}}\right|}}$$

(1)

As an example, for an XY polynomial surface, the sag function and its gradient expressions can be written as:

$$\left\{\begin{array}{c}g(x,y)={\displaystyle \sum _{m=0}^M{\displaystyle \sum _{n=0}^N{a}_{mn}}}\cdot x^m{y}^n\\ {\displaystyle \frac{\partial g}{\partial x}}={\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=0}^{N}m\cdot a_{mn}}}\cdot x^{m-1}{y}^n\\ {\displaystyle \frac{\partial g}{\partial y}}={\displaystyle \sum _{m=0}^M{\displaystyle \sum _{n=1}^{N}n\cdot a_{mn}}}\cdot x^m{y}^{n-1}\end{array}\right.$$

(2)

To ensure consistency between the actual surface normals at the ray–surface intersection points on surface $S_i$ and the corresponding ideal normals, the following optimization objective function is formulated by enforcing normal constraints at all sampled points $P_{i,j}$:

$${\mathit{a}}^{\ast }=\arg \underset{\mathit{a}}{\min }{\displaystyle \sum _{i=1}^K{\displaystyle \sum _{j=1}^M\left[{\left(\frac{\partial g}{\partial x}+\frac{{\widehat{n}}_{x,i,j}}{{\widehat{n}}_{z,i,j}}\right)}^2+{\left(\frac{\partial g}{\partial y}+\frac{{\widehat{n}}_{y,i,j}}{{\widehat{n}}_{z,i,j}}\right)}^2\right]}}$$

(3)

where $\mathit{a}=\left\{a_{mn}\right\}$ denotes the coefficients of XY polynomial defining the freeform surface $S_i$, and ${\widehat{n}}_{x,i,j}$, ${\widehat{n}}_{y,i,j}$, and ${\widehat{n}}_{z,i,j}$ denote the x-, y-, and z-components of the ideal normal at point $P_{i,j}$, respectively. The objective function in Equation (3) is formulated to minimize the squared residual between the surface gradients and the target slopes defined by the normal vectors.

The intersection points used to compute the initial ideal surface normals are obtained from the intersections between the sampled rays and the initial planar surface $S_i$. After the first construction step, the sampled rays intersect the freeform surface $S_i$ at shifted locations, causing the corresponding intersection points $P_{i,j}$ to deviate from their initial positions. To address this discrepancy, an iterative procedure is employed to update the intersection points $P_{i,j}$ and recompute the associated ideal surface normals. With successive iterations, the surface geometry gradually converges toward the target configuration, ultimately yielding a smooth and continuous freeform surface that closely approximates the desired optical performance.

For AR-HUD optical systems with a wide FOV, a single freeform surface is generally insufficient to achieve high imaging quality across all fields while maintaining low optical distortion. Therefore, an additional freeform secondary mirror is often introduced to further improve the imaging performance. As illustrated in Figure 4, in an AR-HUD optical system incorporating a freeform secondary mirror, the construction of the primary mirror is based on the ideal image points defined on the intermediate image plane, whereas the construction of the secondary mirror relies on the ideal image points on the final image plane. To properly control the optical power distribution between the primary and secondary mirrors, the positions of the intermediate image points are scaled. This scaling relationship is defined as:

$${[x\prime ,y\prime ,z\prime ]}^T=\rho {[x,y,z]}^T$$

(4)

where ${[x,y,z]}^T$ denotes the coordinates of the ideal image point on the final image plane, while ${[x\prime ,y\prime ,z\prime ]}^T$ represents the coordinates of the ideal image point on the intermediate image plane. The parameter $\rho =L_1/L_2$ is defined as the surface factor, in which L₁ and L₂ denote the distance from the intermediate image plane to the freeform secondary mirror and the distance from the final image plane to the secondary mirror, respectively, as illustrated in Figure 4.

The construction of the entire initial optical configuration can be divided into two sequential stages: the construction of the freeform primary mirror and the construction of the secondary mirror. First, the freeform primary mirror is generated based on the ideal image points defined on the intermediate image plane, where the primary mirror is represented by an XY polynomial of a specified order. Subsequently, the secondary mirror is constructed using the intersection points of the sampled rays on the primary mirror, the corresponding reflected ray directions, and the ideal image points defined on the final image plane.

2.2. Parameter Optimization Method Based on Differentiable Ray-Tracing

The initial configuration of each freeform surface in the AR-HUD system can be obtained using the design procedure described in Section 2.1. However, due to fitting errors and the simplified specification of the initial positions of the two freeform surfaces, the optical system generated by the NIC method does not yet constitute an optimal and compact layout. In addition, non-negligible discrepancies remain between the actual image points and their corresponding ideal image points.

This section aims to present an automated optimization framework for iteratively refining the AR-HUD optical system, with the goal of improving imaging quality while satisfying geometric constraints. As illustrated in Figure 5, the proposed method is based on constructing a residual vector through differentiable ray tracing and efficiently computing the gradients of the residuals with respect to the optical system parameters via automatic differentiation [29]. Using these gradients, the complete Jacobian matrix is assembled to perform damped least-squares optimization. The entire optimization process is implemented using modern differentiable programming tools, such as PyTorch (version 2.4) [30].

Differentiable computation of the intersections between rays and freeform surfaces remains a challenging problem within differentiable ray tracing frameworks. The main difficulty arises from the fact that ray–surface intersections are typically solved using numerical iterative methods, while the backward pass must simultaneously ensure gradient accuracy, numerical stability, and memory efficiency. A ray can be expressed in parametric form as $\mathbf{o}+t\mathbf{d}$, where $\mathbf{o}$ denotes the ray origin, $\mathbf{d}$ is the direction vector, and $t$ represents the propagation distance. Research by Wang et al. [23] introduces an efficient strategy to compute the ray–surface intersection parameter $t$ using Newton’s method. During differentiable computation, the intermediate iterations of the Newton solver are neither explicitly stored nor backpropagated. Instead, only the final converged solution $t^{*}$ is retained, and its gradients with respect to the system parameters are computed directly. This approach is equivalent to applying implicit differentiation to the ray–surface intersection condition, thereby avoiding the unrolling of the iterative process. As a result, the memory consumption during backpropagation is significantly reduced, and the computational efficiency of differentiable ray tracing is substantially improved.

As illustrated in Figure 6, after multiple reflections within the freeform optical system, the chief ray of the $i$-th sampled field intersects the image plane at point $P_{i,0}$, while the non-chief rays intersect the image plane at points $P_{i,j}$. According to Snell’s law, the intersection position $P_{i,j}$ of each sampled ray on the image plane (expressed in the local coordinate system) is a highly nonlinear function of the independent variables $\mathit{\theta }$ of the optical system. These variables include the coefficients a of each freeform surface and the corresponding positional parameters $\mathit{\phi }$. In practical optical systems, aberrations are generally unavoidable. As a result, the intersection points $P_{i,j}$ deviate from the corresponding chief-ray intersection point $P_{i,0}$. Let $\mathrm{\Delta }{x}_{i,j}$ and $\mathrm{\Delta }{y}_{i,j}$ denote the differences between $P_{i,j}$ and $P_{i,0}$ along the $x$- and $y$-directions, respectively, in the local $xyz$ coordinate system of the image plane. Consequently, $\mathrm{\Delta }{x}_{i,j}$ and $\mathrm{\Delta }{y}_{i,j}$ are also highly nonlinear functions of the system variables. These quantities are defined as follows:

$$\left\{\begin{array}{c}\mathrm{\Delta }{x}_{i,j}=f_{i,j}(\mathit{\theta })\\ \mathrm{\Delta }{y}_{i,j}=g_{i,j}(\mathit{\theta })\end{array}\right.$$

(5)

In addition, the chief ray of each sampled field may not converge to its ideal image point $T_i$. On the image plane, the deviation between the actual chief-ray intersection point and the ideal image point is commonly used to characterize image distortion. For an ideal, distortion-free optical system, the deviation between $P_{i,0}$ and $T_i$ should be zero. Similarly, the distortion associated with each field of view is also a nonlinear function of the independent variables $\mathit{\theta }$. Accordingly, the distortion of the $i$-th sampled field is defined as:

$$dis{t}_i=h_i(\mathit{\theta })$$

(6)

In off-axis asymmetric optical systems, optical components may intersect or block the optical path, leading to undesired ray occlusions. Since a feasible optical system should avoid such occlusions, the concept of optical path obstruction is introduced to detect and eliminate unwanted overlaps along the optical path. As illustrated in Figure 6, two primary types of optical path obstruction can occur in the AR-HUD optical system. The first type occurs when the signed distance d₁ from the feature point Q to the ray segment AB is negative, while the second type occurs when the signed distance d₂ from the feature point C to the ray segment GB is negative. For the first case, the sign of the distance is determined by the relative orientation of the vectors $\overrightarrow{AQ}$ and $\overrightarrow{AB}$, where the sign is defined according to the angle between these two vectors. The magnitude of the optical path obstruction is defined as:

$$d_s=u_s(\mathit{\theta })$$

(7)

where s denotes the $s$-th type of optical path obstruction, and $d_s$ represents the corresponding obstruction magnitude, which is also a nonlinear function of the independent variables $\mathit{\theta }$.

In practice, $d_s$ is constrained within a prescribed range to avoid optical path obstruction while maintaining a compact system layout. To ensure both differentiability of the overall optimization framework and effective elimination of optical path obstructions, a constraint term is introduced for the $s$-th obstruction, which is defined as:

$$z_s=\mathrm{ReLU}(d_{\min ,s}-d_s)+\mathrm{ReLU}(d_s-d_{\max ,s})$$

(8)

where ReLU(x) = max (0, x) denotes the Rectified Linear Unit function, which applies a differentiable penalty only when $d_s$ violates its prescribed boundaries.

Subsequently, the AR-HUD optical system design can be formulated as a set of weighted nonlinear equations, in which all imaging errors and optical path obstruction constraints are expressed as differentiable functions of the optical system parameters $\mathit{\theta }$. Specifically, all residual terms are concatenated to form a single residual vector, which can be written as:

$$\mathit{r}(\mathit{\theta })=\left[\begin{array}{c}{w}_i\times f_{i,j}(\mathit{\theta })\\ w_i\times g_{i,j}(\mathit{\theta })\\ {\kappa }_i\times {\mathit{r}}_i(\mathit{\theta })\\ v_i\times z_s(\mathit{\theta })\end{array}\right]$$

(9)

where $w_i$, ${\kappa }_i$, and $v_i$ are weighting coefficients used to balance the relative importance of different residual terms.

Therefore, the above problem can be formulated as a nonlinear least-squares optimization problem. By adjusting the optical system parameters $\mathit{\theta }$, the objective is to minimize the combined residuals arising from imaging deviations of all sampled rays and the optical path obstruction constraints. The optimization loss function is defined as:

$$\mathcal{L}(\mathit{\theta })={\displaystyle \sum _{i=1}^n{\mathit{r}}_i}{(\mathit{\theta })}^2$$

(10)

The resulting optimization problem is a typical nonlinear least-squares problem, which can be solved using either the Gauss–Newton method or gradient descent. However, the Gauss–Newton method may become unstable in highly nonlinear regions, while gradient descent often suffers from slow convergence. The LM algorithm provides a compromise between these two approaches by combining convergence stability and computational efficiency, and is therefore adopted in this work. Since the entire ray-tracing process is differentiable, the complete optical ray tracing can be regarded as a forward propagation. With the automatic differentiation capability provided by the PyTorch framework, the Jacobian matrix can be obtained directly. The Jacobian matrix J is defined as:

$$\mathit{J}=\left[\begin{array}{ccc}{\displaystyle \frac{\partial {\mathit{r}}_1}{\partial {\mathit{\theta }}_1}}& \dots & {\displaystyle \frac{\partial {\mathit{r}}_1}{\partial {\mathit{\theta }}_n}}\\ ⋮& \ddots & ⋮\\ {\displaystyle \frac{\partial {\mathit{r}}_N}{\partial {\mathit{\theta }}_1}}& \dots & {\displaystyle \frac{\partial {\mathit{r}}_N}{\partial {\mathit{\theta }}_n}}\end{array}\right]$$

(11)

The iterative update rule of the LM algorithm is given by:

$$\left({\mathit{J}}^T\mathit{J}+\lambda \mathit{D}\right)\mathrm{\Delta }\mathit{\theta }=-{\mathit{J}}^T\mathit{r}$$

(12)

where $\lambda$ is the damping factor, which is adaptively adjusted during the optimization process to balance the search directions of gradient descent and the Gauss–Newton method. The matrix $\mathit{D}=\mathrm{diag}({\mathit{J}}^T\mathit{J})$ is introduced to normalize the scaling differences of the residuals across different dimensions. The optical system parameters are then updated according to:

$${\mathit{\theta }}_{n+1}={\mathit{\theta }}_n+\mathrm{\Delta }{\mathit{\theta }}_n$$

(13)

The overall optimization process is performed in a progressive manner by gradually increasing the polynomial order of the freeform surfaces. The iterative optimization terminates once the maximum predefined order for each freeform surface is reached.

3. Design Example

To validate the feasibility of the proposed method, a dual-freeform AR-HUD optical system is designed as an illustrative example. The designed system features a FOV of 13° × 4° and a virtual image distance (VID) of 10 m. The detailed design specifications are summarized in

Table 1. Specifications of the AR-HUD system.

Parameter	Specification
Field of view	13° × 4°
Virtual image distance	10 m
Wavelength	Visible (400 nm~780 nm)
Eye-box size	130 mm × 50 mm
PGU size	80 mm × 27 mm

. The PGU has a resolution of 1102 × 372 pixels, with a projected image size of 80 mm × 27 mm. The corresponding pixel size is 72.6 μm.

An initial optical system consisting of two tilted and decentered planes and a windshield surface is first constructed. The initial planar system is illustrated in Figure 7. A biased FOV of 2° in the vertical direction is adopted in the system. Although partial ray occlusions are present for edge rays in the initial planar configuration, these occlusions are gradually eliminated during the subsequent iterative optimization process.

The construction of the primary mirror is first performed using the NIC method. During the construction process, a fourth-order XY polynomial is employed in each iteration to fit all characteristic points. The AR-HUD optical system after completing the primary mirror construction is shown in Figure 8a. Figure 8b illustrates the iterative process of the primary mirror construction. It can be observed that the average RMS spot radius converges after only three iterations. The full FOV RMS spot radius distribution and the grid distortion of the system after the primary mirror construction are presented in Figure 8c,d, respectively. The average RMS spot radius is 332.07 μm, and the maximum distortion is 6.88%.

After the construction of the primary mirror is completed, the above procedure is repeated to construct the secondary mirror. To ensure that the secondary mirror remains convex, the surface factor $\rho$ is set to 0.9. The AR-HUD optical system after completing the secondary mirror construction is shown in Figure 9a. Figure 9b illustrates the iterative process of the secondary mirror construction. After the secondary mirror is constructed, the full FOV RMS spot radius distribution and the grid distortion are shown in Figure 9c and Figure 9d, respectively. The average RMS spot radius is 300.46 μm, and the maximum distortion is reduced to 1.44%. Compared with the results obtained after the primary mirror construction, the grid distortion is significantly improved. This configuration therefore serves as a suitable initial structure for subsequent iterative optimization.

After completing the construction of the initial configuration, the entire AR-HUD optical system is further optimized. During the optimization process, the polynomial orders of both the primary and secondary freeform surfaces are gradually increased from fourth order to sixth order. At each polynomial order, 15 optimization iterations are performed. The corresponding optimization convergence curve is shown in Figure 10a. It can be observed that the loss function decreases rapidly when fourth-order polynomials are used, while the decrease becomes much slower and gradually converges when fifth- and sixth-order XY polynomials are employed. Therefore, the maximum polynomial order used in the iterative optimization is set to six. The optimized AR-HUD optical layout is shown in Figure 10b. Figure 10c,d present the final full FOV RMS spot radius distribution and the grid distortion, respectively. After optimization, the average RMS spot radius of the AR-HUD system is reduced from 300.46 μm to 99.81 μm, while the maximum distortion is decreased from 1.44% to 0.52%. Notably, the extremely low grid distortion indicates that the AR-HUD designed using the proposed method does not require a dedicated warping chip to perform complex pre-distortion processing. Consequently, the projected virtual image exhibits no noticeable distortion.

For AR-HUD optical systems with a large eye-box, multiple eye-pupil positions must be sampled for performance evaluation in order to better reflect practical usage conditions. As illustrated in Figure 11a, five eye pupils (E1–E5), each with a diameter of 5 mm, are sampled across the entire eye-box. The pupil center coordinates are located at (0 mm, 0 mm), (−62.5 mm, 22.5 mm), (−62.5 mm, −22.5 mm), (62.5 mm, 22.5 mm) and (62.5 mm, −22.5 mm). As shown in Figure 11b, for each sampled eye-pupil position, both the RMS spot radius and the geometric spot radius are smaller than the corresponding Airy disk radius, indicating diffraction-limited or near diffraction-limited performance. The MTF curves are shown in Figure 11c. For all sampled eye-pupil positions, the MTF values at the spatial frequency of 6 lp/mm remain above 0.5, demonstrating high imaging quality across the entire eye-box.

Following the evaluation of imaging performance, the dynamic distortion of the AR-HUD system is further investigated. During the optical design process, rays are traced from the virtual image plane toward the PGU plane. In actual operation, however, the image is generated on the PGU plane and then reflected sequentially by the secondary mirror, the primary mirror, and the windshield before entering the human eye to form a virtual image. To accurately evaluate the dynamic distortion under realistic operating conditions, the entire optical system is therefore reversed, with the PGU plane defined as the object plane and the virtual image plane defined as the image plane. Under this configuration, a fixed grid pattern is displayed on the PGU, and the resulting virtual images formed at different eye-pupil positions are analyzed. Dynamic distortion is defined as the difference between the virtual-image grid formed at the central eye position and those formed at off-axis eye positions, which can be expressed as follows:

$${\mathit{D}}_{\mathrm{dyn}}=\underset{i}{\max }\left(\arctan {\displaystyle \frac{\mathrm{\Delta }{\mathit{r}}_i}{\mathrm{VID}}}\right)$$

(14)

where $\mathrm{\Delta }{\mathit{r}}_i$ denotes the distance between the $i$-th virtual image grid point at an edge eye position and the corresponding $i$-th grid point at the center eye position.

Figure 12 illustrates the dynamic distortion of the edge eye-pupil positions relative to the central eye position. It can be observed that the virtual image grid formed at the edge eye positions exhibits only minor deformation compared with that formed at the central eye position. The maximum dynamic distortion is limited to 5.39′. This result indicates that the proposed AR-HUD optical system maintains high image consistency across the entire eye-box, with no noticeable virtual-image shift during eye movements.

To further evaluate the practical manufacturability of the optimized AR-HUD optical system, a comprehensive tolerance analysis is conducted. The primary tolerance sources of the system are categorized into two types: surface figure tolerances and assembly tolerances. Specifically, the surface errors of the primary and secondary mirrors are simulated by applying random perturbations to their freeform polynomial coefficients. Taking into account the actual physical dimensions of the mirrors and current manufacturing capabilities, the added RMS surface errors are set to 15 μm and 5 μm, respectively. Furthermore, the assembly tolerances for each optical element are introduced by applying spatial pose perturbations to their local-to-global coordinate transformation matrices. The specific allocation ranges of the assembly tolerances for each element are detailed in

Table 2. Allocation of assembly tolerances for the optical elements.

Optical Element	Tilt	Decenter	Thickness
Windshield	0.3°	5 mm	3 mm
Primary mirror	0.1°	1 mm	1 mm
Secondary mirror	0.1°	1 mm	1 mm
PGU	0.1°	1 mm	1 mm

.

A total of 1000 Monte Carlo tolerance simulations were performed using uniform random sampling, and the statistical results are presented in Figure 13. Under the given tolerance perturbations, the maximum RMS spot radius across all sampled eye-box positions is strictly less than 46.81 μm at a 90% cumulative probability. Furthermore, the dynamic distortion at the edge eye positions remains below 7.07′. These values fully satisfy the practical application requirements of automotive AR-HUD systems.

4. Experimental Results and Discussion

To validate the AR-HUD optical system designed using the proposed method, both the primary and secondary mirrors were fabricated using a five-axis single-point diamond turning (SPDT) process. The dimensions of the primary mirror are 347.6 mm × 115.0 mm, while those of the secondary mirror are 152.5 mm × 58.0 mm. After fabrication, surface profile measurements were conducted on the manufactured mirrors using an FF2000 three-dimensional surface profiler developed by Suzhou Raphael Optoelectronics Technology Co., Ltd. (Suzhou, China) [31]. This instrument is specifically designed for the measurement of optical freeform surfaces and provides a measurement accuracy of ±3 μm. The measured surface errors of the fabricated mirrors are shown in Figure 14. For the primary mirror, the peak-to-valley (PV) and RMS surface errors are 32.3 μm and 9.2 μm, respectively. For the secondary mirror, the corresponding PV and RMS surface errors are 10.67 μm and 2.39 μm, respectively.

The complete experimental prototype is shown in Figure 15. A full-scale automotive windshield is used in the experiment and is mounted on an optical table via a custom-designed support structure. The primary mirror, the secondary mirror, and the PGU are fixed on the same optical platform and aligned with the windshield. A camera is positioned at the center of the designed eye-box to simulate the virtual image observed by the human eye.

To experimentally assess the imaging quality of the proposed AR-HUD system, a series of preliminary tests were performed. Initially, a uniform dot-matrix pattern was projected onto the PGU, as depicted in Figure 16a. Figure 16b shows the corresponding image captured by a camera positioned at the center of the eye-box. By extracting the centroid coordinates of the dot matrix, the maximum distortion at the central eye box was determined to be 1.46%. Furthermore, a resolution test pattern was introduced on the PGU. As illustrated in Figure 16c, horizontal and vertical fringes with a spatial frequency of 6 lp/mm were evaluated across nine sampled field positions. The resulting virtual image, captured at the eye-box center, is presented in Figure 16d. Quantitative analysis based on the extracted intensity distribution curves reveals that the fringe contrasts for the central and marginal fields of view are 0.53 and 0.39, respectively, at 6 lp/mm. In summary, the optical module demonstrates satisfactory virtual image performance at the central eye position, which is generally consistent with the simulation results.

To further assess the virtual image quality within the eye-box during eye movements, dot-matrix patterns at the peripheral eye positions were recorded, and the corresponding dynamic distortions were quantified, with specific values presented in

Table 3. Measured dynamic distortion at different eye-box positions.

Eye Position	E2	E3	E4	E5
Dynamic distortion	5.05′	6.31′	9.08′	12.49′

. Notably, the dynamic distortions at positions E4 and E5 are relatively pronounced, marginally surpassing the anticipated tolerance simulation results. This deviation can be largely ascribed to the conventional manufacturing process of the windshield, which lacks precision machining. As a result, it inherently exhibits more substantial surface figure errors than the primary and secondary mirrors, which adversely impacts the final imaging quality at the peripheral viewing positions. Moving forward, the surface measurement and quality control of the windshield’s clear aperture will be a critical focus. Accurately characterizing these complex surface figure errors through high-precision metrology and incorporating the measured data back into the optical design model for reverse compensation will be essential for further optimizing the system’s performance. Nevertheless, to evaluate the system’s practical display capabilities, a practical user interface image was loaded onto the PGU. A camera simulating the human eye was employed to capture the corresponding virtual images across the five pupil positions (E1–E5). As depicted in Figure 17, despite the aforementioned numerical distortions at the peripheral boundaries, the captured virtual images remain visually clear and highly recognizable at both the central and peripheral eye positions. This indicates that the proposed optical module can already provide a satisfactory visual experience for practical AR-HUD applications.

5. Conclusions

In this study, an automated design method for automotive AR-HUD optical systems is proposed and experimentally validated. The proposed method combines NIC-based initial structure generation with optimization driven by differentiable ray tracing. Using the proposed design framework, high-performance AR-HUD optical systems can be rapidly generated under given optical layouts and design constraints. To validate the method, an AR-HUD optical system with two freeform surfaces was designed and evaluated. The results demonstrate that the designed AR-HUD system maintains high imaging quality with low dynamic distortion throughout the eye-box. Furthermore, a full-scale AR-HUD experimental prototype was established, and functional tests were conducted. The experimental results show reasonable agreement with the design and simulation analyses, suggesting the engineering feasibility of the proposed method. Overall, the proposed automated design approach provides an effective solution for the development of high-performance AR-HUD optical systems and can be extended to other freeform optical systems.