A Learnable Viewpoint Evolution Method for Accurate Pose Estimation of Complex Assembled Product

Featured Application

This study can be applied for the camera pose estimation and appearance quality inspection of large-sized complex assembled products that require high reliability, adaptability, and accuracy simultaneously.

Abstract

Balancing adaptability, reliability, and accuracy in vision technology has always been a major bottleneck limiting its application in appearance assurance for complex objects in high-end equipment production. Data-driven deep learning shows robustness to feature diversity but is limited by interpretability and accuracy. The traditional vision scheme is reliable and can achieve high accuracy, but its adaptability is insufficient. The deeper reason is the lack of appropriate architecture and integration strategies between the learning paradigm and empirical design. To this end, a learnable viewpoint evolution algorithm for high-accuracy pose estimation of complex assembled products under free view is proposed. To alleviate the balance problem of exploration and optimization in estimation, shape-constrained virtual–real matching, evolvable feasible region, and specialized population migration and reproduction strategies are designed. Furthermore, a learnable evolution control mechanism is proposed, which integrates a guided model based on experience and is cyclic-trained with automatically generated effective trajectories to improve the evolution process. Compared to the ${1.69}^{°},55.67$mm of the state-of-the-art data-driven method and the ${1.28}^{°},77.67$mm of the classic strategy combination, the pose estimation error of complex assembled product in this study is ${0.23}^{°},23.71$mm, which proves the effectiveness of the proposed method. Meanwhile, through in-depth exploration, the robustness, parameter sensitivity, and adaptability to the virtual–real appearance variations are sequentially verified.

1. Introduction

In the high-end equipment manufacturing industry, accurate multi-view pose estimation of complex products is an essential step for subsequent appearance quality inspection. Taking an aerospace product as an example, a piece of equipment may consist of multiple sub-modules, including dozens or even hundreds of intricate parts (complex assembled products). To ensure integrity and reliability, the product needs to be repeatedly compared with 3D models and process documents from different viewpoints to cover all visible elements to avoid defects during its assembly stage. The complexity of structure and requirements to make this work are still completed manually in most cases. In this context, if there is a method to obtain the pose of the current viewpoint, it will greatly improve the efficiency of the process. Based on the estimated pose, the 3D model can be adjusted to the same perspective as the real object so as to extract the corresponding region of interest (ROI) and filter irrelevant context [1,2]. In such a way, the reference provided by a virtual image can greatly simplify inspection and evaluation (whether it is for the operator or algorithm) while avoiding the difficulty of building an end-to-end multi-target detector. The inherent logic of pose estimation is to solve the problem of constructing a virtual–real feature connection in a scenario with diverse noise and continuously adjust incorrect correspondences through correct correspondences (e.g., feature point). For vision technology that utilizes sensor perception and algorithm judgment, this seemingly simple task for humans is quite difficult, and the challenges can be summarized as follows:

(1) Industry characteristics: In the realm of high-end equipment manufacturing, the interpretability and dependability of methods are important considerations. Although it is possible to train a specific neural network for pose estimation through a large amount of data, blindly adhering to the big data-driven learning paradigm in the pursuit of adaptability may render it challenging to address failures, abnormalities, and re-improvement, ultimately impeding promotion [3].

(2) Task features: These assembled products with high structural complexity may involve dozens or even hundreds of components, typically assembled through human or human–machine collaboration. Under such operation scenarios, it is difficult to collect enough high-quality samples at once or accurately pre-plan the imaging pose. The task features of a small sample, multiple components, and coarse initial pose greatly increase the difficulty of pose estimation/adjustment in common vision-based applications, e.g., AR/VR-based projection [4] mobile robots based on viewpoint pre-planning [5,6].

(3) Product appearance: As shown in Figure 1, the appearance features of the target and background captured from different viewpoints are quite different, and the multi-layer stacking between objects will further enhance the difference. Due to perspective effect and distortion, whether the extremely small-sized target located at the far end can be successfully recognized is sensitive to the accuracy of pose estimation. It is common that there are differences in features and visual styles between the process model and physical object. Additionally, the surface treatment of machined products can result in sparse texture and high color consistency that are not conducive to feature extraction.

Technically, camera pose estimation focused in this study is the absolute pose estimation (APE). Research in this direction mainly includes six categories [7,8,9]:

(1) APE based on cooperative target can be further subdivided into manual markers [10,11], auxiliary projection [12], laser tracker [13], and structured light measurement [14]. In these methods, easily identifiable features are supplemented to the target manually or by automated devices to alleviate the difficulty of feature extraction and analysis during APE. This strategy is commonly used in high-precision tasks, such as calibration, and is also suitable for large structural parts with smooth surfaces and simple structures. Furthermore, with the support of a movable robot vision platform [15], APE can be transformed into matching and stitching of a 3D point cloud.

(2) APE based on geometric features includes two steps: feature extraction and relationship construction of 2D pixels and 3D coordinates (2D–D) and optimization-based APE estimation. Manual feature [16,17,18] and learning-based approaches [19] are usually adopted in the first step, in which two key issues need to be considered, namely, the adaptability of 2D features and the robustness of 2D–3D relationship evaluation. The representative optimization method for APE estimation is EPnP [20] and some other related studies [21,22,23,24]. In the estimation stage, it is essential to reduce the interference of outliers on the 2D–3D relationship and avoid getting trapped in the wrong local optimal solution. Compared with other technical routes, strategy (2) can obtain the best APE accuracy, but it is limited by the complexity of the scene to obtain enough 2D–3D pairs with correct relations at one time. Moreover, the iterative mode is easily affected by the latest observations, making the final convergent result significantly biased.

(3) APE based on mapping and retrieval is one of the commonly used strategies in the field of simultaneous localization and mapping (SLAM), emphasizing the robustness of scene feature encoding and the ability to describe differences between different viewpoints [25,26]. When the accuracy requirement is not high, the pose can be obtained by retrieving the viewpoint closest to the current feature in the map library; otherwise, the result of strategy (3) will be adopted as the initial value for strategy (2) [27].

(4) APE based on deep learning (DL) relies on a deep convolutional neural network (CNN) to perform pose estimation in an end-to-end manner. CNN-based architectures represented by classic research PoseNet series [28,29] do not require feature engineering and have a strong adaptability to feature variation. Nevertheless, there is still a gap in reliability and accuracy between strategy (4) and strategy (2) [30]. Moreover, some state-of-the-art studies indicate that close performance can be achieved in public datasets such as LineMod, LineMod-O [31,32,33], and MegaPose [34] and can adapt to unseen objects through rendering and comparison, but they all require ensuring model accuracy, fewer targets, and balanced sizes, which is significantly different from the complex assembled product.

(5) Hybrid and hierarchical strategies include methods based on functional hybrid training of multiple CNN models [35,36] and hierarchical approximation from coarse to fine [37]. Research in strategy (5) is devoted to addressing the adaptability of pose estimation to changeable feature context in open multi-scene. The accuracy achieved depends partly on the architecture and trick design of CNN and partly on the geometric feature-based APE algorithm adopted in the backend.

Through the literature review above, it can be found that the compatibility of accuracy and adaptability in pose estimation of complex assembled products under free view hinges on building an interpretable hierarchical architecture while maintaining the advantages of empirical design and learning paradigm. However, the interpretability and consistency of DL-based methods limit their application in manufacturing, while traditional APE lacks local–global collaborative control in matching and optimization mechanisms and relies on fixed empirical parameters in adaptability, resulting in quite strict requirements for initial pose in complex assembled products. Therefore, it is necessary to deeply integrate traditional APE- and DL-based paradigms in terms of execution mechanism, operation rule, and interactive strategy, laying the foundation for consistency of results, local–global feature adaptability, and traceability of exceptions, rather than simply concatenating the two and directly applying them.

Driven by this motivation, this paper proposes a novel, learnable viewpoint evolution algorithm for high-accuracy pose estimation of complex products in high-end equipment manufacturing. Compared with existing APE methods, the contribution can be summarized as follows:

(1)

Reconstruct the camera pose estimation into the evolution of parameter population to generate an interpretable fine-grained iterative framework;

(2)

To balance exploration and optimization, feasible region constraint, parent migration, and reproduction strategies are designed for local–global collaborative control;

(3)

Propose a guided-model-based hierarchical cyclic effective trajectory learning mechanism to reduce the empirical dependence of control and enhance adaptability.

2. Proposed Method

As shown in Figure 2, the method includes two stages, which can be divided into five modules. The initial stage is used to generate the initial pose, including the extraction of significant features, viewpoint sampling and projection, feature matching and viewpoint searching, virtual image rendering, and initialization of the evolution stage. The significant feature refers to the geometric feature (e.g., points, lines, and curves) that are well visible and distinguishable from different viewpoints. Virtual viewpoints are generated following the spherical or ellipsoidal rule [38]. The rough pose is provided by the sampled virtual viewpoint with the best similarity to the significant feature of the real image. The feature matching and similarity evaluation are performed by a weighted point pair matching algorithm [39]. Some CNN-based coarse pose estimators with a small number of samples are also suitable choices for obtaining the initial pose.

The evolution stage is redesigned as population initialization and migration, offspring generation, viewpoint quality assessment and selection, pose feasible region construction and update, and a learnable evolution function optimization inspired by importance sampling in reinforcement learning [40].

2.1. Viewpoint Performance Assessment

For each viewpoint $vp$, the performance ${vp}^q$ reflects the average similarity of virtual–real ROI of all visible components. The geometric feature is adopted to overcome texture sparsity. Inspired by [41], as shown in Figure 3, a self-adaptive evaluation line-based feature matching and evaluation algorithm that balances structural stability and multiple considerations is designed to enhance robustness against virtual–real differences, as well as uneven correspondence. For each visible component $w$, where $w$ belongs to the set of all visible components $W$ under $vp$, the virtual–real feature is denoted as $\left\{{\mathit{p}}^w,{\mathit{x}}^w\right\}$ and ${\mathit{p}}^w$ is projected from 3D coordinate ${\mathit{P}}^w$. ${\mathit{p}}_l^w={\{\mathit{p}}_{l,1}^w,\dots {\mathit{p}}_{l,m}^w\}$ represents the set of points on the $l$-th projected line segment of ${\mathit{p}}^w$, and the same applies to ${\mathit{x}}_r^w$.

For any ${\mathit{p}}_{l,m}^w$ in ${\mathit{p}}_l^w$, a rectangular search region with changeable size along the normal direction is expanded, where the size depends on $\mathsf{\sigma }=E^s\left({vp}^q\right),{vp}^q$ is the performance of the parent view of $vp$, and $E^s$ denotes the evolution function of the search factor $\mathsf{\sigma }$. In the rectangular region, any point belonging to ${\mathit{x}}_r^w$ can be selected as a candidate matching point for ${\mathit{p}}_{l,m}^w$ as long as the angle between ${\mathit{p}}_l^w$ and ${\mathit{x}}_r^w$ satisfies the preset range. After searching, a redundant candidate point set ${\tilde{\mathit{x}}}^w$ of ${\mathit{p}}^w$ is established. Theoretically, there is always an ideal transformation ${\mathbf{T}}_{2\mathrm{D}}$ that can minimize the error of $\left\{{{\mathbf{T}}_{2\mathrm{D}}\mathit{p}}^w,{\mathit{x}}^w\right\}$ with the constraints provided by root $vp$. During the evolution stage, the translational part in ${\mathbf{T}}_{2\mathrm{D}}$ is the main factor that narrows the gap between $\left\{{\mathit{p}}^w,{\mathit{x}}^w\right\}$. Based on the above assumptions, the quadrant is divided uniformly to roughly determine the potential direction of translation, and a pair of matching points is defined as $\left\{{\mathit{p}}_{l,m}^w,{\mathit{x}}_{r,n}^w\right\}$. According to

$$arccos\left({[\mathit{x}}_{r,n}^w-{\mathit{p}}_{l,m}^w\right]\left[\begin{array}{c}1\\ 0\end{array}\right]/‖{\mathit{x}}_{r,n}^w-{\mathit{p}}_{l,m}^w‖],$$

(1)

the candidate set of ${\mathit{p}}^w$ in each slope range can be obtained. For each slope range, the number ratio $ra$ and the distance variance ${var}^{res}$ between the candidate set and ${\mathit{p}}^w$ can be counted. The direction advantage is defined as $norm\left(ra\right)+(1-norm({var}^{res}\left)\right)$, where $norm$ represents normalization operation (all slope ranges), and candidate matching points are reassigned to ${\mathit{p}}^w$ in order of advantage. During the reassignment process, if there is a one-to-many situation, the candidate point belonging to the slope range that reduces the variance of the current matching distance is preferentially selected. Afterward, we can obtain $\left\{{\mathit{p}}^w,{\check{\mathit{x}}}^w\right\}$, ${cp}^{ra}$, and ${cp}^{res}$, where ${\check{\mathit{x}}}^w$ represents an incomplete set, that is, part of $\mathit{x}$ finds no corresponding $\mathit{p}$, ${cp}^{ra}$ is the number ratio, and ${cp}^{res}$ denotes the distance average of all matched pairs.

Evaluation criteria that rely solely on ${cp}^{res}$ are easily affected by background noise, incomplete geometric features, and virtual–real differences. Therefore, a ridge function mapping strategy is further embedded with ${cp}^{ra}$ as an additional evaluation factor. Inspired by inverse square root unit (ISRU) [42], the mapping is defined as follows:

$${ISRU}^{\prime }\left({cp}^{ra}\right)=\frac{1}{2}\left(\frac{1.07\left(2·nf·{cp}^{ra}-nf\right)}{\sqrt{1.5+{\left(2·nf·{cp}^{ra}-nf\right)}^2}}+1\right),$$

(2)

where $nf=5$ is the transformation factor of ${cp}^{ra}$. Here, (2) shows a parameter combination with suitable performance, which allows ${cp}^{ra}$ to be less sensitive to extreme ranges. Afterward, define the measurement point ${\mathit{m}\mathit{p}}_1={\left({cp}^{\mathrm{r}\mathrm{e}\mathrm{s}0},0,{vp}^{\mathrm{q}0}\right)}^{\mathrm{T}},{\mathit{m}\mathit{p}}_2={\left({cp}^{\mathrm{r}\mathrm{e}\mathrm{s}0},1.0,{vp}^{\mathrm{q}1}\right)}^{\mathrm{T}}$ and the fixed point ${\mathit{f}\mathit{p}}_1={\left({cp}^{\mathrm{r}\mathrm{e}\mathrm{s}0},1.0,1.0\right)}^{\mathrm{T}},{\mathit{f}\mathit{p}}_2={\left({root}^{\mathrm{r}\mathrm{e}\mathrm{s}0},0.0,0.0\right)}^{\mathrm{T}}$, where ${cp}^{\mathrm{r}\mathrm{e}\mathrm{s}0}=0$ describes the ideal average distance error. For the discrimination of the result after mapping, let ${cp}^{\mathrm{r}\mathrm{e}\mathrm{s}0}=0.1\mathrm{*}{root}^{\mathrm{r}\mathrm{e}\mathrm{s}0}$, where ${root}^{\mathrm{r}\mathrm{e}\mathrm{s}0}$ is the average distance error of the root $vp$. The ridge function $rf\left({cp}^{ra},{cp}^{res},{vp}^q\right)$ can be planned as

$$\left\{\begin{array}{c}{[\left({\mathit{f}\mathit{p}}_1-{\mathit{m}\mathit{p}}_1\right)\times \left({\mathit{f}\mathit{p}}_2-{\mathit{m}\mathit{p}}_1\right)]}^{\mathrm{T}}\left[{\left({cp}^{res},{cp}^{ra},{vp}^q\right)}^{\mathrm{T}}-{\mathit{m}\mathit{p}}_1\right]=\mathbf{0},{cp}^{ra}\le {rf}_{T1}\\ {[\left({\mathit{f}\mathit{p}}_1-{\mathit{m}\mathit{p}}_2\right)\times \left({\mathit{f}\mathit{p}}_2-{\mathit{m}\mathit{p}}_2\right)]}^{\mathrm{T}}\left[{\left({cp}^{res},{cp}^{ra},{vp}^q\right)}^{\mathrm{T}}-{\mathit{m}\mathit{p}}_2\right]=\mathbf{0},{cp}^{ra}>{rf}_{T2}\end{array}\right.$$

(3)

where ${rf}_{T1},{rf}_{T2}$ are the boundary of the transition region and ${rf}_{T1}={rf}_{T2}=({cp}^{res}-{root}^{\mathrm{r}\mathrm{e}\mathrm{s}0})/({cp}^{\mathrm{r}\mathrm{e}\mathrm{s}0}-{root}^{\mathrm{r}\mathrm{e}\mathrm{s}0})$. The transition region can similarly be set to be plane, making the mapped ${vp}^q$ smoother. $\{{vp}^{\mathrm{q}0},{vp}^{\mathrm{q}1}\}$ are a set of indicators adopted to adaptively modulate the propensity of ${cp}^{ra}$ and ${cp}^{res}$. Intuitively, it may be more suitable to set ${vp}^{\mathrm{q}1}>{vp}^{\mathrm{q}0}$ at the beginning and ${vp}^{\mathrm{q}1}<{vp}^{\mathrm{q}0}$ at the end.

In this way, the four aspects of structural stability, error direction, number of matching points, and distance error can be integrated. For the $w$-th component, let ${vp}^q{|}_w$ denote the result mapped by (3). The white numbers in the second row of Figure 3 represent ${cp}^{res}$ (first row) and ${vp}^q{|}_w$ (second row). To maintain consistency with the gradient direction of distance error, let ${vp}^q{|}_w=1-{vp}^q{|}_w$ in Figure 3. For some matching states that ${cp}^{res}$ cannot effectively distinguish, ${vp}^q{|}_w$ provides reliable discrimination. The viewpoint performance ${vp}^q$ can be calculated according to ${vp}^q=\mathcal{Q}\left(vp\right)=(1/W){\sum }_w^W\left({vp}^q{|}_w\right)$. If the $W$ and ${\mathit{P}}^w$ obtained by root $vp$ are not updated in time during the evolution stage, the evaluation will encounter greater viewpoint ambiguity and differences in geometric features. Therefore, by configuring different rendering frequencies, the adaptability of the proposed method with respect to the degree of virtual–real difference is deeply evaluated in Section 3.

2.2. Feasible Region Construction and Update

2.2.1. Problem Planning

Projection matrix (or pose) $\mathbf{T}$ contains seven variables if the representation approach of quaternion and translation is employed. The crossover and mutation in the traditional genetic algorithm are too exploratory to ensure the optimization of $\mathbf{T}$. The commonly used numerical optimization is limited by the observed $\{\mathit{x},\mathit{p},\mathit{P}\}$ and prone to fall into local optimum. Thus, it is expected to provide a range constraint in the solution space for parent ${\mathcal{P}}_i$ to children ${\mathcal{C}}_i$ based on observed $\{\mathit{x},\mathit{p},\mathit{P}\}$, which can be continuously updated and contracted to the ideal solution with the adjustment of $\mathit{x},\mathit{P}$ during the evolution. The range in this study is defined as pose feasible region $\mathcal{F}$, and for ${\mathcal{F}}_i,{\mathcal{F}}_i=\left\{\mathbf{T}\right|F_i^{{\delta }_i,I}\left(\mathbf{T}\right)\le 0,\mathbf{T}\in \mathrm{S}\mathrm{E}\left(3\right)\}$, where

$$F_i^{{\delta }_i,W}\left(\mathbf{T}\right)=\sum _w^W{‖{\mathit{x}}^{w\prime }-\pi \left(\mathbf{T}{\mathit{P}}^w\right)‖}_2^2-{{\delta }_i}^2.$$

(4)

${\mathit{x}}^{w\prime }$ represents the pixel coordinate corresponding to the best ${vp}^q{|}_w$ achieved by the $w$-th visible component in the previous $i$ round. ${\delta }_i={E^g(\overline{vp}}^{q,i}),{\overline{vp}}^{q,i}=(1/P_i){\sum }_k^{P_i}\left({vp}_k^q\right),{vp}_k\in {\mathcal{P}}_{i+1}$, ${\mathcal{P}}_{i+1}$ is the initial parent selected from $\{{\mathcal{C}}_i,{\mathcal{P}}_i^{\prime }\}$ in the $i$-th round, and ${\mathcal{P}}_i^{\prime }$ refers to ${\mathcal{P}}_i$ after migration to ${\mathcal{F}}_i$. $E^g$ is the evolution function of the tolerance factor for constructing $\mathcal{F}$. To provide a feasible constraint to the pose search, the expression of $\mathcal{F}$ should be solved. This expression problem can be formulated as

$${min}_{\mathbf{T}}\left\{\sum _w^W{‖{\mathit{x}}^{w^{\prime }}-\pi \left(\mathbf{T}{\mathit{P}}^w\right)‖}_2^2+\left(-\frac{1}{\dot{t}}\right)\log \left(-F_i^{{\delta }_i,W}\left(\mathbf{T}\right)\right)\right\},$$

(5)

where $-\mathrm{l}\mathrm{o}\mathrm{g}(-F_i^{{\delta }_i,W}\left(\mathbf{T}\right))$ is the logarithmic barrier function and $\dot{t}$ denotes the approximation factor to the indicative function. This is an optimization technique that uses the interior point method to remove inequality constraints. In practice, however, the optimization process of (5) is too complicated. Given different initial values, repeated iterations and optimizations of $\dot{t}$ are computationally expensive. To this end, the assumption of structural stability is again applied as a basis for simplification. Specifically, only the matching relation corresponding to the best advantage in $\left\{{\mathit{p}}^w,{\tilde{\mathit{x}}}^{w\prime }\right\}$ is retained to sparse ${\mathit{x}}^{w\prime }$ to reduce the scale of problem (5). To alleviate the tedious optimization work, a combined strategy based on boundary projection and sparse trajectory approximation is provided below.

2.2.2. Initialization

As shown in Figure 4, an ellipsoidal rule [38] is used to sample the initial viewpoint pose, denoted as $\mathcal{V}$. Due to the small rotation of the in-plane during machine or manual imaging, the rotation of the out-of-the-plane can be regarded as a strong prior for constructing the initial viewpoint network.

2.2.3. Boundary Projection

For each $\mathbf{T}$ of $vp,vp\in \mathcal{V}$, the projection to the boundary of ${\mathcal{F}}_i$ can be planned as

$${argmin}_{{\mathbf{T}}^{\perp }}\left\{{f_d(\mathbf{R}}^{\perp },\mathbf{R},{\mathit{t}}^{\perp },\mathit{t}\right)\},\hspace{1em}s.t.F_0^{{\delta }_0,W}\left({\mathbf{T}}^{\perp }\right)=0,$$

(6)

where $\{{\mathbf{R}}_v^{\perp },{\mathit{t}}_v^{\perp }\}\in {\mathbf{T}}^{\perp }$ is the projection result. The function $f_d\left(·\right)=‖\mathit{q}\otimes {{\mathit{q}}_v}^{-1}-{[{\mathbf{0}}_{3\times 1},1]}^{\mathrm{T}}‖+‖\mathit{t}-{\mathit{q}\otimes {{\mathit{q}}_v}^{-1}\mathit{t}}_v‖$ is designed to measure the distance between poses, where $\mathit{q}$ is the quaternion representation of $\mathbf{R}$ and $\otimes$ defines the multiplication of quaternions. The objective function (6) is a small-scale with equality-constrained optimization problem, and the solution of each iteration can be approximately equivalent to

$$min{f}_1\left(\mathbf{T}\right)=\frac{1}{2}∆{\mathbf{T}}^{\mathrm{T}}{\nabla }^{2}f\left({\mathbf{T}}^k\right)∆\mathbf{T}+\nabla f\left({\mathbf{T}}^k\right)∆\mathbf{T},\hspace{1em}s.t.\nabla F_0^{{\delta }_0,W}\left({\mathbf{T}}^k\right)∆\mathbf{T}+F_0^{{\delta }_0,W}\left({\mathbf{T}}^k\right)=0,$$

(7)

according to SQP [43], where $f_d\left({\mathbf{T}}^k\right)$ is the objective function of (8) and ${\mathrm{*}}^{\mathrm{T}}$ is the transpose operation. Then, (7) can be rewritten as

$$minL\left(∆\mathbf{T},\mathit{\lambda }\right)=\frac{1}{2}∆{\mathbf{T}}^{\mathrm{T}}{\mathbf{H}}_f∆\mathbf{T}+{\mathbf{J}}_f∆\mathbf{T}+{\mathit{\lambda }}^{\mathrm{T}}({\mathbf{J}}_F∆\mathbf{T}+F_0^{{\delta }_0,W}\left({\mathbf{T}}^k\right)),$$

(8)

at ${\mathbf{T}}^k$, $∆\mathbf{T}={\mathbf{T}}^{k+1}⊝{\mathbf{T}}^k$ is the variable to be solved, and $\mathit{\lambda }$ is the Lagrange multiplier. It should be noted that the above calculation involves the derivation of $\mathbf{T}$ and the update of $∆\mathbf{T}$. However, the $\mathbf{R}$ of $\mathbf{T}$ is only closed for multiplication, not for addition and subtraction. Therefore, a logarithmic mapping needs to be performed on $\mathbf{T}$ to obtain $\xi =\left[\begin{array}{c}\varphi \end{array},\mathit{\rho }\right]\in {\mathbb{R}}^6,\mathit{\rho }\in {\mathbb{R}}^3,\varphi \in \mathfrak{s}\mathfrak{o}\left(3\right)$ based on the representation of a Lie algebra. $⊝$ is the subtraction defined on $\xi$. Let $f\left(·\right)=‖{\mathbf{T}}_q‖+‖{\mathbf{T}}_t‖,$ where ${\widehat{\mathbf{T}}}_q,{\widehat{\mathbf{T}}}_t$ are their respective unit vectors. Then, the ${\mathbf{J}}_f$ of (8) can be expanded as

$${\mathbf{J}}_f=\left[\frac{\partial f_1}{\partial \delta \mathit{q}},\frac{\partial f_1}{\partial \delta \mathit{t}}\right]=\left[{{\widehat{\mathbf{T}}}_q}^{\mathrm{T}}\frac{\partial {\mathbf{T}}_q}{\partial \delta \mathit{q}}+{{\widehat{\mathbf{T}}}_t}^{\mathrm{T}}\frac{\partial {\mathbf{T}}_t}{\partial \delta \mathit{q}},{{\widehat{\mathbf{T}}}_t}^{\mathrm{T}}\frac{\partial {\mathbf{T}}_t}{\partial \delta \mathit{t}}\right],$$

(9)

The related research is mature, and hence, the calculation process of $\mathit{\rho }$, $\varphi$, and (9) is omitted. After completing the optimization, the results can be recovered from $\mathfrak{s}\mathfrak{o}\left(3\right)$ to $\mathrm{S}\mathrm{E}\left(3\right)$ by exponential mapping. The rSQP [44] is recommended if the scale of ${\mathit{x}}^{w\prime }$ and $\mathit{x}$ in other studies is still large.

2.2.4. Sparse Trajectory Approximation

Let the projected $\mathcal{V}$ be ${\mathcal{V}}^{\perp }$. For each $vp,vp\in {\mathcal{V}}^{\perp }$, an unconstrained objective function can be constructed as

$$f_2\left(\mathbf{T}\right)=\mathit{min}\left\{\sum _w^W{‖\mathcal{H}\left({\mathit{x}}^{w\prime }-\pi \left(\mathbf{T}{\mathit{P}}^w\right)\right)‖}_2^2\right\},$$

(10)

where $\mathcal{H}(·)$ is the kernel function to enhance the robustness of outliers, and Huber is selected in this study. Taking ${\mathcal{V}}^{\perp }$ as the initial state, (17) is solved in parallel based on the LM strategy, where the pose change can be recorded as ${tra}^h=\left\{{tra}_{{\mathbf{T}}_0^{\perp }},{tra}_{{\mathbf{T}}_1^{\perp }},\dots ,{tra}_{{\mathbf{T}}_v^{\perp }}\right\},{tra}_{{\mathbf{T}}_v^{\perp }}=\left\{{\mathbf{T}}_v^{\perp },{\mathbf{T}}_v^1,{\mathbf{T}}_v^2\dots ,{\mathbf{T}}_v^V\right\}$, and $V$ is the number of iteration steps of the $v$-th trajectory.

Let the optimization direction of all trajectories in the $z$-th step be $\left\{{∆\mathbf{T}}_0^z,{∆\mathbf{T}}_1^z,\dots ,{∆\mathbf{T}}_v^z\right\}$, and define the trajectory affinity as

$${\mathcal{A}}_{m,n}^z=f_d{(\mathbf{T}}_m^z,{\mathbf{T}}_n^z)·{(∆\widehat{\mathbf{T}}}_m^z{∆\widehat{\mathbf{T}}}_n^z),$$

(11)

Afterward, the horizontal interpolation is evaluated for $\left\{{\mathbf{T}}_1^z,{\mathbf{T}}_2^z,\dots ,{\mathbf{T}}_v^z\right\}$. For all ${\mathcal{A}}_{m,n}^z,m,n\in {tra}^h$,

$${tra}^v=\left\{li\left({\mathbf{T}}_m^z,{\mathbf{T}}_n^z,\frac{1}{2}\left(f_d({\mathbf{T}}_m^{z+1},{\mathbf{T}}_m^z)+f_d({\mathbf{T}}_n^{z+1},{\mathbf{T}}_n^z)\right)\right)|{\mathcal{A}}_{m,n}^z>\frac{1}{v}{\sum }_m{\sum }_{n=m+1}{\mathcal{A}}_{m,n}^z\right\}.$$

(12)

Let $\mathcal{t}={(f}_d{(\mathbf{T}}_m^{z+1},{\mathbf{T}}_m^z)+f_d{(\mathbf{T}}_n^{z+1},{\mathbf{T}}_n^z))/(2f_d{(\mathbf{T}}_m^z,{\mathbf{T}}_n^z\left)\right)$, then $li\left(·\right)$ can be expressed as a spherical linear interpolation operation on $\mathit{q}$ and $\mathit{t},$ respectively, in the following way.

$${li}_{\mathit{q}}=\frac{\mathit{sin}\left[\left(1-\mathcal{t}\right)\theta \right]{\mathit{q}}_m+\mathit{sin}\left(\mathcal{t}\theta \right){\mathit{q}}_n}{sin\left(\theta \right)},{li}_{\mathit{t}}={\mathit{t}}_m+\mathcal{t}({\mathit{t}}_n-{\mathit{t}}_m).$$

(13)

Through the above work, the discrete approximation ${tra}_i=\{{tra}_i^h,{tra}_i^v\}$ of ${\mathcal{F}}_i$ can be obtained.

During evolution, once the historical best ${vp}^q{|}_w$ of any component is reset due to the generation of a child $vp$, the pose feasible region update can be selectively started according to the change in the ${vp}^q$. ${\mathcal{F}}_{i+1}$ is updated from ${\mathcal{F}}_i$, and the update work can be summarized into four steps, including the following: (a) simplifying ${\mathit{x}}^w$ based on structural stability to obtain ${\mathit{x}}^{w\prime },{\mathit{x}}^{\prime }$; (b) determining ${\delta }_{i+1}$ and $F_{i+1}^{{\delta }_{i+1},W}$ and selecting the support $vp$ according to ${vp}^q$ from $\left\{{\mathcal{P}}_i,{\mathcal{C}}_i\right\}$; (c) calculating the projection of the support $vp$ to the boundary by Formulas (6)–(9); and (d) solving the optimization problem (10) and performing affinity-based interpolation with (11)–(13).

2.3. Parent Migration and Offspring Generation

As shown in Figure 5, a parent migration mechanism is proposed to ensure that the population performance can be gradually improved with the update of $\mathcal{F}$. The migration operation is defined as ${\mathcal{P}}_{i+1}\to {\mathcal{P}}_{i+1}^{\prime },$ and ${\mathbf{T}}_v^{\mathrm{*}}$ of ${vp}_k^{\prime }\in {\mathcal{P}}_{i+1}^{\prime }$, which denotes the pose of the migrated parent, can be expressed as

$${\mathbf{T}}_v^{*}={\mathbf{T}}_v^{rand\left[{0,(1-\lambda }_k)·v_k\right]},{\mathbf{T}}_v\in {tra}_{{\mathbf{T}}_v^{\perp }},{\lambda }_k=E^t\left({vp}_k^q\right),$$

(14)

where $rand$ means to randomly pick an integer in the range $\left[{0,(1-\lambda }_k\right)·v_k]$ and $E^t$ is the evolution function of the migration factor ${\lambda }_k$. To further align the exploration with the actual situation, we performed rotation correction on the randomly generated values.

Sampling $vp$ is highly exploratory while performing strictly according to the optimization direction lacks exploration. To address these problems, an offspring generation strategy based on the pose feasible region constraint is designed.

The Jacobian matrix of (10) is ${\mathbf{J}}_F\in {\mathbb{R}}^{I\times 7}$. For any ${vp}_k^{\prime }$ and its ${\mathbf{T}}_v^{\mathrm{*}}$, the next feasible point within the trust region $r^{\mathrm{*}}$ can be calculated from $\left({{\mathbf{J}}_F}^{\mathrm{T}}{\mathbf{J}}_F+r^{\mathrm{*}}\mathbf{I}\right)∆=-{{\mathbf{J}}_F}^{\mathrm{T}}{f}_2\left({\mathit{q}}_v^{\mathrm{*}},{\mathit{t}}_v^{\mathrm{*}}\right)$. The basic direction is defined as

$$\mathcal{d}=\left\{{\mathcal{d}}_{\mathit{o}}|\left({{\mathbf{J}}_{\mathit{o}}}^{\mathrm{T}}{\mathbf{J}}_{\mathit{o}}+r^{*}\mathbf{I}\right){\mathcal{d}}_{\mathit{o}}=-{{\mathbf{J}}_{\mathit{o}}}^{\mathrm{T}}{f}_2\left({\mathit{q}}_v^{\mathit{*}},{\mathit{t}}_v^{\mathit{*}}\right),{\mathbf{J}}_{\mathit{o}}={\mathbf{J}}_F{\mathbf{I}}_{\mathit{o}=0},\mathit{o}\in \mathfrak{o}\right\},$$

(15)

where ${\mathbf{I}}_{\mathit{o}=0}$ indicates that the corresponding column of $\mathbf{I}$ is assigned to 0 according to the gradient clipping configuration $\mathfrak{o}$. For example, $\mathit{o}=\{\mathrm{5,6},7\}$ means that only the gradient of rotation is considered. In this study, $\mathfrak{o}$ includes maintaining the gradient of ${\mathit{t}}_v^{\mathrm{*}}$ and clipping ${\mathit{q}}_v^{\mathrm{*}}$ in four dimensions and maintaining the gradient of ${\mathit{q}}_v^{\mathrm{*}}$ and clipping ${\mathit{t}}_v^{\mathrm{*}}$ in three dimensions. The estimated amount of gradient descent can usually be expressed as ${\mathcal{l}}_{\mathit{o}}=-{{\mathcal{d}}_{\mathit{o}}}^{\mathrm{T}}{{\mathbf{J}}_{\mathit{o}}}^{\mathrm{T}}{f}_2\left({\mathit{q}}_v^{\mathrm{*}},{\mathit{t}}_v^{\mathrm{*}}\right)-\frac{1}{2}{{\mathcal{d}}_{\mathit{o}}}^{\mathrm{T}}{{\mathbf{J}}_{\mathit{o}}}^{\mathrm{T}}{\mathbf{J}}_{\mathit{o}}{\mathcal{d}}_{\mathit{o}}$. The probability of generating offspring in each basic direction is described as

$$p_{\mathit{o}}=\mathrm{I}\left({\mathcal{l}}_{\mathit{o}}\right)\left(1-p^e\right)\frac{{\mathcal{l}}_{\mathit{o}}}{{\sum }_{\mathit{o}\in Cu{\mathfrak{o}}^e}{\mathcal{l}}_{\mathit{o}}}+\left(1-\mathrm{I}\left({\mathcal{l}}_{\mathit{o}}\right)\right)p^e\left|\frac{{\mathcal{l}}_{\mathit{o}}}{{\sum }_{\mathit{o}\in {\mathfrak{o}}^e}{\mathcal{l}}_{\mathit{o}}}\right|,$$

(16)

where $\mathrm{i}\mathrm{f} {\mathcal{l}}_{\mathit{o}}>0$, $\mathrm{I}\left({\mathcal{l}}_{\mathit{o}}\right)=1$, otherwise, $\mathrm{I}\left({\mathcal{l}}_{\mathit{o}}\right)=0$. $p^e$ is the pre-allocated probability of exploration, and ${\mathfrak{o}}^e=\left\{\mathit{o}|{\mathcal{l}}_{\mathit{o}}\le 0,\mathit{o}\in \mathfrak{o}\right\},$ $Cu{\mathfrak{o}}^e$ is the complementary set of ${\mathfrak{o}}^e$. To prevent the imbalance of $p_{\mathit{o}}$ due to too few exploration directions, it is recommended to apply secondary adjustments according to the ratio of the ${\overline{p}}_{\mathit{o}}$ in the two sets.

As shown in Figure 5, once the basic direction ${\mathcal{d}}_{\mathit{o}}$ is selected, a cone-based disturbance is further performed on the rotation and translation, respectively. For simplicity, $\tau =E^a\left({{vp}^{\prime }}_k^q\right)$ and $s=E^e\left({{vp}^{\prime }}_k^q\right)$ are adopted to represent the disturbance range of angle range and scale, respectively, where ${E^a,E}^e$ are the corresponding evolution functions. Therefore, the perturbed direction ${\mathcal{d}}_{\mathit{o}}^{\prime }=rand\left\{\mathcal{C}\left(\mathbf{T}\right)\right\}-{\mathbf{T}}_v^{\mathrm{*}}$, where $\mathcal{C}\left(\mathbf{T}\right)$ is described as

$${\widehat{\mathcal{d}}}_{\mathit{o}}\left(\mathbf{T}-\left({\mathbf{T}}_v^{*}+{\mathcal{d}}_{\mathit{o}}\right)\right)=0,s.t.{‖\mathbf{T}-\left({\mathbf{T}}_v^{*}+{\mathcal{d}}_{\mathit{o}}\right)‖}_2^2\le {tan}^2\left(\tau \right){‖{\mathcal{d}}_{\mathit{o}}‖}_2^2$$

(17)

${\widehat{\mathcal{d}}}_{\mathit{o}}$ is the unit vector, $rand\left\{\mathcal{C}\left(\mathbf{T}\right)\right\}$ is adopted to randomly sample from the distribution $\mathcal{C}\left(\mathbf{T}\right)$. After ${\mathcal{d}}_{\mathit{o}}^{\prime }$ is determined, the pose ${\mathbf{T}}_v^{\mathrm{*}}$ can be updated through ${{\mathbf{T}}_v^{\mathrm{*}}=\mathbf{T}}_v^{\mathrm{*}}+{s\left({\sum }_{j=1}^V\left|{\mathbf{T}}_v^j-{\mathbf{T}}_v^{j+1}\right|\right)\mathcal{d}}_{\mathit{o}}^{\prime },{\mathbf{T}}_v^j\in {tra}_{{\mathbf{T}}_v^{\perp }}$ according to $s$ and the cumulative change in pose.

To alleviate the unpredictability of actual gradient descent caused by the introduction of $\left\{\tau ,s\right\}$, a relaxed projection approach based on $\mathcal{F}$ is established. The projection position of ${\mathbf{T}}_v^{\mathrm{*}}$ is determined by the hyperplane supported by its three neighbors in $\mathcal{F}$. The hyperplane is defined as ${\mathbf{A}}^{\mathrm{T}}(\mathbf{T}-\mathit{b})=0,$ and the relaxed factor is denoted as $\mathsf{\alpha }=E^p\left({{vp}^{\prime }}_k^q\right)$. ${vp}_{k,\mathit{o}}^{\prime }$, who is the offspring of ${vp}_k^{\prime }$, can be calculated after relaxed projection through

$$\begin{gathered} {\mathbf{T}}_v^{\mathrm{*},c}={{({\mathbf{T}}_v^{\mathrm{*}}+{\mathcal{d}}_{\mathit{o}}^{\prime })+rand(0,\mathsf{\alpha })(\mathbf{T}}_v^{\mathrm{*}}}_{\perp {tra}_i}-({\mathbf{T}}_v^{\mathrm{*}}+{\mathcal{d}}_{\mathit{o}}^{\prime })), \\ {{\mathbf{T}}_v^{\mathrm{*}}}_{\perp {tra}_i}={\mathbf{A}}^{\mathrm{T}}{\left({\mathbf{A}}^{\mathrm{T}}\mathbf{A}\right)}^{-1}{\mathbf{A}}^{\mathrm{T}}\mathit{b} \end{gathered}$$

(18)

according to the projection theorem, where $\mathsf{\alpha }$ is adopted to control the constraint strength and $E^p$ is the evolution function of $\mathsf{\alpha }$.

2.4. A Learnable Evolutionary Function Optimization

The evolution functions mentioned above include $\delta ={E^g(\overline{vp}}^q)$, $\lambda =E^t\left({vp}_k^q\right)$, $s=E^e\left({{vp}^{\prime }}_k^q\right)v_k$, $\tau =E^a\left({{vp}^{\prime }}_k^q\right)$, $\mathsf{\alpha }=E^p\left({{vp}^{\prime }}_k^q\right)$, and $\mathsf{\sigma }=E^s\left({{vp}^{\prime }}_k^q\right)$, which are employed to manage evolution parameters and processes based on the performance of the population. The evolution function is initialized as a linear function, and the abscissas of the head and tail are taken as ${\overline{vp}}^{q0}$ and $min\{{\epsilon \overline{vp}}^{q0},1.0\}$, where $\epsilon$ is the termination threshold. ${\delta }_0$ can be referenced by $\frac{1}{N_i}{\sum }_k^{N_i}{cp}^{res},{vp}_k\in {\mathcal{P}}_1$,${\delta }_{end}=0.0$. The ordinates of other parameters need to be empirically provided; see Section 3 for details.

As shown in Figure 6, a learning mechanism for optimizing the evolution function is proposed. The learning stage consists of three main modules and is designed as a hierarchical cyclic process. In the data collection module, each image of the dataset is repeatedly tested according to the current evolution function, and the evolution trajectory $\mathcal{T}\in \mathcal{T}$ of the viewpoint with the best effectiveness in each test is collected. The effectiveness of the $i$-th round $k$-th viewpoint is defined as

$${\mathsf{\varrho }}_k^i={vp}_{i,k}^q+{\sum }_{j=i}({{vp}_{j,k}^{\prime }}^q+p_{\mathit{o}}{vp}_{j+1,k}^q)$$

(19)

which can be regarded as a reverse tracking of ${vp}^q$ of the trajectory generated by the final output viewpoint. For each node in $\mathcal{T}$, $\{{vp}^q,\delta ,\mathsf{\sigma },\dot{\lambda },\dot{s},\dot{\tau ,}\dot{\mathsf{\alpha }}\}$ are saved to complete the trajectory collection, where $\dot{\mathrm{*}}$ represents the actual sampled value. Afterward, $\left\{\delta ,\mathsf{\sigma },\dot{\lambda },\dot{s},\dot{\tau ,}\dot{\mathsf{\alpha }}\right\}$ are normalized to prevent overwhelming effects due to the magnitude differences. It can be found that the core purpose of evolution function optimization is to build the mapping relation between the observed population performance and process adjustment through high-quality results. Intuitively, there should be a better way of exploratory decay than six separate descent strategies with fixed slopes.

The uneven performance of $\mathcal{T}$ (there is a large variance of ${vp}^q$) generated by linear function will make further learning difficult to converge. To this end, a guided model based on multi-model fusion is designed to provide a reasonable transition for data distribution learning. The linear model can be extended as $e=a_b\left({{vp}^q}^b\right)+c_b,e\in {\mathcal{E}}_0$, where $b$ controls the downward trend of the evolution curve, and $e$ is the normalized result of each parameter in ${\mathcal{E}}_0$. Given the head and tail points, $a_b,c_b$ can be modeled as functions of $b$. Considering the correlation between parameters, factor fusion is performed as

$${\sum }_{e\in {\mathcal{E}}_0}\left\{a_{b_e}\right({{vp}^q}^{b_e})+c_{b_e}\}={\mathit{e}}^{\mathrm{T}}\mathit{W}\mathit{e}$$

(20)

where $\mathit{W}$ is the correlation matrix. In such a way, the relationship between ${vp}^q$ and ${\mathcal{E}}_0$ can be comprehensively modeled by (20). The design of this guided model is derived from an empirical understanding of the evolution process from coarse-to-fine and a priori knowledge of the interaction of parameters. For example, the correlation between $s$ and $\tau$ should be prioritized over the correlation between $s$ and other parameters.

Furthermore, let the $b_e$ of each parameter be initialized as a constant $\mathrm{b}0,\mathrm{b}0>1.0$. The fitting of (20) is executed with ${\mathcal{T}}_0$ as the dataset to obtain the guided model. Next, the guided model is applied to testing and data collection as a temporary evolution function to prepare ${\mathcal{T}}_1$, which implies that the distribution state of the data will be naturally constrained. A neural network with ResNet18 as the backbone and four fully connected layers as the output is employed for completing the learning of complex nonlinear mapping relationships through ${\mathcal{T}}_1$. The mapping model ${{\mathcal{E}}_1=\mathrm{E}}_{MLP}\left({\mathit{v}\mathit{p}}^q\right)$, where the input is the ${vp}^q$ set of five parents, and the output is ${\mathcal{E}}_1\in {\mathbb{R}}^{6\times 1}$. Let ${\mathcal{T}}_{{\mathcal{E}}_1}$ be the new dataset obtained according to the mapping model. Obviously, the construction of ${\mathrm{E}}_{\mathrm{M}\mathrm{L}\mathrm{P}}$ is affected directly by the value of $b_e$, which symbolizes the strictness of the intermediate guidance strategy. For the automation of the whole learning process, the value of $b_e$ is set with reference to $e$ following the principle of decreasing integral. For $b1$,

$$\int (e\left(\mathrm{b}0\right)-e(b1\left)\right)d{(vp}^q)=\int {∆}_{pr}\left(e\right(1.0)-e(\mathrm{b}0\left)\right)d{(vp}^q)$$

(21)

Then for $bi$,

$$\int e\left(bi\right)d{(vp}^q)=\int e\left(\mathrm{b}0\right)-{i·∆}_{pr}\left(e\right(1.0)-e(\mathrm{b}0\left)\right)d{(vp}^q)$$

(22)

In this study, $\mathrm{b}0=1.5,{∆}_{pr}=0.5$, and the standard deviation of the output offspring performance after repeated evolution of each image is denoted as ${std}_{{\mathsf{\varrho }}^{end}}$. The hierarchical cyclic learning process is shown in Figure 6, which can be specifically expressed as $\mathrm{b}0\to {\mathcal{T}}_1\to {\mathrm{E}}_{\mathrm{M}\mathrm{L}\mathrm{P}}\to {\mathcal{T}}_{{\mathcal{E}}_1}\to \sum {std}_{{\mathsf{\varrho }}^{end}}\to \mathrm{b}1\to \dots$. Each update of ${\mathrm{E}}_{\mathrm{M}\mathrm{L}\mathrm{P}}$ is based on the previous one, and the ${\mathrm{E}}_{\mathrm{M}\mathrm{L}\mathrm{P}}$ corresponding to the one with the best $\sum {std}_{{\mathsf{\varrho }}^{end}}$ is taken as the final learned evolution function.

3. Experiment and Results

Section 3.1 shows the preparation work. The cutting-edge DL-based method and classical VPS method are compared in Section 3.2. The robustness of the proposed method to pose deviation and the advantage of enabling the learning mechanism are verified in Section 3.3. Section 3.4 conducts parameter sensitivity analysis to provide a basis for parameter selection; Section 3.5 compares different strategies for offspring generation; the adaptability of the virtual–real difference is presented in Section 3.6.

3.1. Preliminary Works

(1) Experimental platform. For this study, a complex assembled product with 113 components was constructed, including 27 different types of equipment, 10 parts (e.g., craft cap), 14 plugs, and 62 screws. The appearance of this product is inconsistent from different viewpoints. Some equipment and parts were surface-treated to exhibit low texture and high color consistency.

(2) Parameter configuration. Migration factor $\left[{\lambda }_0{,\lambda }_{end}\right]=[0.5,0.05]$, scale disturbance factor $\left[s_0{,s}_{end}\right]=[0.10,0.001]$, angle disturbance factor $\left[{\tau }_0{,\tau }_{end}\right]=[{20}^{°},0.0]$, relaxed factor $\left[{\mathsf{\alpha }}_0{,\mathsf{\alpha }}_{end}\right]=[0.3,0.0]$, search factor $\left[{\mathsf{\sigma }}_0{,\mathsf{\sigma }}_{end}\right]=[25.0,5.0]$, and termination threshold $\epsilon =4.0$, $\left\{{vp}^{\mathrm{q}0},{vp}^{\mathrm{q}1}\right\}=\{0.1,0.7\}$. The number of parent nodes is 10, and the number of child nodes is 5.

(3) Dataset. As shown in Figure 7, the dataset consists of about 240 viewpoints, which were captured by a hand-held industrial camera and a mobile robot vision system, respectively. The working distance is about 1.8 m, the view width is about 1.5 m, and the resolution is 3264 $\times$ 2448. The ground truth was calculated through artificial markers and manual refinement (the beige base is only used to calculate ground truth).

(4) Evaluation Indicator. The pose of complex products in the manufacturing industry mainly serves subsequent inspection, AR-based projection, or robot operation. Therefore, the positioning accuracy of each component can better reflect the practical significance of pose accuracy. For this reason, in addition to the widely used pose deviation (mean translation error ${\overline{t}}_e$ and mean angular error ${\overline{a}}_e$), we have also modified IoU as follows:

$$l_{\mathrm{*}}\left({\overline{IoU}}_e\right)=\frac{1}{\left|l_{\mathrm{*}}\right|}{\sum }_{w\in l_{\mathrm{*}}}\mathrm{I}\mathrm{o}\mathrm{U}\left(rect\right({\mathit{p}}^{w,8}),rect({\mathit{p}}_{gt}^{w,8}\left)\right),{\mathit{p}}^{w,8}=\pi \left(\mathbf{T}{\mathbf{P}}^{w,8}\right)$$

(23)

to better distinguish performance, where ${\mathbf{P}}^{w,8}$ is the 3D 8-point box coordinate of component $w$, $l_{\mathrm{*}}$ represents a set of elements ($\left|l_{\mathrm{*}}\right|$ is the number) belonging to Range * (Range I, II, III, plugs, and skews).

3.2. Performance of Pose Estimation

Three commonly used solutions in APE were employed for comparison, including direct estimation, iterative estimation, and hierarchical approach. Each solution was supported by a different combination of the most common and classic algorithms. Specifically, the methods for virtual–real feature matching consist of nearest neighbor principle (S0), normal-based search [14] (S1), fusion of Turkey estimation and color [45] (S2), region-based method [16] (S3), iterative point matching [39] (S4), and global strategy FRICP [46]. The optimized part can be arbitrarily selected from EPnP [20], EPPnP, CEPnP [22], ASPnP [31], PPnP [23], and MLPnP [24]. DL-based methods employed in hierarchical strategy include PoseNet2 [29], single-shot pose [47], and state-of-the-art estimator PVNet [30] and MegaPose [34]. Another part of the latest study [31,32] has also undergone research. However, in the case of insufficient sample size/form and dense auxiliary supervision signals, these structurally complex networks are difficult to train sufficiently and prone to overfitting. Therefore, considering the different assumptions of the research, they were not compared for the sake of fairness.

The control group settings are as follows: (a) Control group I. The virtual viewpoint sampling and matching method described in Section 2 was employed to perform the rough positioning. Only the direct estimation scheme with the best performance is listed. (b) Control group II. Iterative version of Group I. For each optimization algorithm, the combination with the best performance and the combination with the method in Section 2.1 (VPS) were compared. (c) Control group III. Different feature-matching algorithms were embedded into the proposed viewpoint evolution.

The group applying the deep learning method was denoted as Group DL, which includes the following: (a) DL-A. The rough pose was adjusted by PoseNet2 with a different number of bounding box labels. The bounding boxes of two devices from Range I were set as labels in DL-A1, and the same 2 boxes from Range I, together with another 4 boxes of devices from Range II, were set as labels in Group DL-A2. (b) DL-B. The single-shot pose network was trained with the labels of DL-A1, and its output was used for PnP estimation to obtain the rough pose. The label was modified from a bounding box to an 8-point bounding box. (c) DL-C. The estimated result of DL-B was adopted as the rough pose for Group II; DL-C1 represents the combination with best performance, and DL-C2 represents the combination with VPS. (d) DL-D. PVNet is employed, whose semantic labels were generated from the minimum bounding rectangle of significant geometric features. (e) DL-E. MegaPose is selected, where the base is removed from the 3D model, and the bounding box prompt is manually set.

For all DL Groups A–D, about 120 real samples and 1200 virtual samples were prepared for model training. The configurations of the proposed method were as follows: (a) Proposed-A. Evolution function learning was disabled in this group, while only one rendering according to the rough pose was allowed. (b) Proposed-B. Evolution function learning and multiple rendering were enabled in this group, and the estimated result of DL-B was accepted as a rough pose.

The results are listed in

Table 1. Camera pose estimation performance comparison.

			Range I (${\overline{\mathit{I}\mathit{o}\mathit{U}}}_{\mathit{e}}$)	Range II (${\overline{\mathit{I}\mathit{o}\mathit{U}}}_{\mathit{e}}$)	Range III (${\overline{\mathit{I}\mathit{o}\mathit{U}}}_{\mathit{e}}$)	Plugs (${\overline{\mathit{I}\mathit{o}\mathit{U}}}_{\mathit{e}}$)	Screws (${\overline{\mathit{I}\mathit{o}\mathit{U}}}_{\mathit{e}}$)	Average Error
								${\overline{\mathit{a}}}_{\mathit{e}}$ (°)	${\overline{\mathit{t}}}_{\mathit{e}}$(mm)
Rough pose			0.64	0.57	0.34	0.11	0.02	6.03	185.71
Group I	S4 + ASPnP		0.74	0.60	0.43	0.13	0.03	3.61	134.74
Group II	FRICP + EPnP		0.77	0.63	0.48	0.15	0.04	1.89	110.77
	EPnP	S3	0.81	0.69	0.53	0.22	0.08	1.83	96.92
		VPS	0.86	0.75	0.62	0.32	0.18	1.44	85.23
	CEPnP	S3	0.79	0.66	0.51	0.19	0.06	1.95	98.20
		VPS	0.86	0.75	0.62	0.32	0.18	1.44	86.07
	RPnP	S3	0.77	0.64	0.49	0.17	0.04	2.02	102.32
		VPS	0.86	0.75	0.62	0.31	0.18	1.43	86.29
	ASPnP	S3	0.82	0.74	0.62	0.29	0.12	1.64	90.67
		VPS	0.87	0.75	0.63	0.33	0.19	1.39	84.86
	EPPnP	S3	0.84	0.75	0.61	0.28	0.12	1.59	92.45
		VPS	0.87	0.75	0.62	0.32	0.19	1.42	83.29
	MLPnP	S3	0.81	0.71	0.55	0.23	0.09	1.73	95.10
		VPS	0.86	0.75	0.62	0.32	0.18	1.41	85.44
Group III	S0		0.89	0.81	0.72	0.49	0.30	0.97	63.41
	S1		0.92	0.87	0.79	0.60	0.49	0.63	47.90
	S2		0.92	0.87	0.80	0.60	0.47	0.71	47.33
	S3		0.93	0.89	0.83	0.65	0.51	0.59	44.78
	S4		0.97	0.95	0.90	0.81	0.65	0.26	33.05
Group DL	DL-A1		0.77	0.65	0.46	0.15	0.03	2.14	115.67
	DL-A2		0.79	0.67	0.50	0.19	0.05	1.93	103.52
	DL-B		0.68	0.57	0.35	0.12	0.02	5.32	167.42
	DL-C1		0.85	0.75	0.61	0.32	0.15	1.57	84.18
	DL-C2		0.88	0.81	0.65	0.38	0.22	1.28	77.67
	DL-D		0.85	0.74	0.60	0.29	0.11	1.79	89.45
	DL-E		0.88	0.77	0.61	0.34	0.19	1.69	55.67
Proposed-A			0.97	0.95	0.89	0.79	0.63	0.27	35.24
Proposed-B			0.98	0.97	0.94	0.85	0.71	0.23	23.71

, and the performance is shown in Figure 8. The areas that can reflect the difference in accuracy are marked, and the higher the brightness, the more accurate. Overall, it can be summarized as the following points: (a) the effectiveness of the proposed method can be proved; (b) non-DL methods are still the first choice for high-precision applications; (c) DL module with weak annotation can be used to provide a suitable initial pose under insufficient dataset; (d) the possibility of escaping from the local optimum in feature matching is the main factor affecting the performance.

The difficulty of the problem is evident in Figure 8a, and the results of Group I and the reasons are shown in Figure 8b. Unpredictable background diversity and various overlapping states pose challenges to the construction of correspondence in the matching process under dense noise, while virtual–real geometric difference further increases the difficulty of noise identification. Moreover, under some special initial viewpoints, there is an uneven deviation in the virtual–real correspondence, which makes it more difficult to explore the correct direction in local large-deviation areas with dense noise. Take the upper plug in the sixth row of the sub-image as an example. There is a large spatial difference between the plug and all visible components that provide the 2D–3D pairs for pose estimation. Therefore, even if these visible components are well positioned, the ${\overline{IoU}}_e$ of the plug may still be less than 0.5. Through the results of Screws ${\overline{IoU}}_e$, it can be proved that a satisfactory camera pose cannot be achieved only by direct estimation. Since the local iteration mode of S4 brings more exploration to the virtual–real feature matching of each component, it shows the best performance and is listed in Table 1.

In terms of matching strategy, S3, which simultaneously fuses edge, texture, and color, is the best choice among S0–S4. The matching and weight adjustment of S4 is based on S0, so it is easily interfered with by noise and has no advantage in Group II. Compared with S0–S4, the strategies of structural stability and advantage direction in VPS effectively improve feature matching, especially in the reduction in angle error. From the perspective of the matching mechanism, the conventional iterative mode lacks exploration and cannot be solved by simply increasing the search range. Once the components with sparse texture stack seriously and the initial deviation is large, the matching process is easily affected, which further leads to a local optimal result (e.g., Figure 8b). Therefore, as shown in the hard example in Figure 8a, no matter how many iterations, there is no obvious difference between the obtained pose and the rough pose. Similarly, iterative global matching tends to maintain the majority of good (minor deviation) while sacrificing bad (large deviation), resulting in cumulative deviation.

On the other hand, the applied optimization methods usually assume that the noise follows a Gaussian distribution in their studies. Therefore, it is possible for the method to ignore outliers and escape from the local optimal state only when the distribution of noises in 2D–3D is close to the Gaussian assumption. In practice, a more common situation is that the final matching results tend to exhibit a biased state due to ignoring the support of geometric features in low-quality matching regions. For example, as shown in the result of Group II in row 5 and row 2, the projection result is skewed by the estimated pose to the side with suitable 2D–3D pairs. According to the results shown by the combination with VPS, the performance difference between the optimization algorithms is not obvious.

The results of Group III demonstrate the robustness and improved ability of the proposed method to feature-matching algorithms. Due to the iterative local exploration ability, S4 in Group III shows excellent performance, even surpassing Proposed-A. However, this approach is less efficient and has insufficient practical significance. For the DL group, the amount of dataset is the primary issue. From the results of DL-A1 and A2, DL models always tend to provide suitable results near fully trained viewpoints, while its estimation performance degrades greatly when the target participating in training is occluded. Therefore, the improvement in the DL-A2 with more labels is not significant compared with DL-A1. Transforming the pose estimation into a two-stage method of significant geometric point regression and PnP, DL-B effectively alleviates the unreliability of end-to-end estimation and is a recommended rough posed acquisition strategy. Supported with DL-B, DL-C further brings better performance to Group II, which fully illustrates the importance of the initial pose. Despite providing precise target boxes, DL-E still cannot directly adapt to complex products that are invisible.

3.3. In-Depth Analysis of the Proposed Method

3.3.1. Verifying Robustness to Initial Pose Deviation

In this section, the repeatability and robustness to initial pose deviation of Proposed-A and -B were tested. The DL-B method was employed in the rough positioning process, and the test was repeated 20 times for each image. Furthermore, different max Gaussian image noises were added to the coordinates of the two 8-point bounding boxes recognized by DL-B.

In

Table 2. The robustness to initial pose deviation.

	Noise-0		Noise-2		Noise-4		Noise-6		Noise-8		Noise-10		Noise-12
${\overline{\overline{a}}}_d(°)$	5.32		5.94		5.74		6.12		7.75		8.97		11.74
${\overline{\overline{t}}}_d(°)$	167.42		178.22		183.28		187.53		194.13		226.56		277.29
	A	B	A	B	A	B	A	B	A	B	A	B	A	B
${\overline{\overline{a}}}_e(°)$	0.26	0.23	0.28	0.24	0.31	0.27	0.34	0.26	0.46	0.34	0.59	0.43	0.78	0.71
${\overline{a}}_e\_std(°)$	0.18	0.13	0.17	0.14	0.21	0.15	0.23	0.22	0.29	0.24	0.34	0.29	0.47	0.46
${\overline{a}}_e\_min$	0.11	0.10	0.10	0.09	0.14	0.11	0.13	0.10	0.15	0.13	0.20	0.15	0.17	0.18
${\overline{a}}_e\_max$	0.54	0.44	0.61	0.50	0.75	0.59	0.78	0.58	0.93	0.77	1.07	0.92	1.35	1.23
${\overline{\overline{t}}}_e$ (mm)	32.66	23.71	31.54	22.97	33.46	25.61	37.35	28.38	45.52	34.27	51.63	42.57	73.83	66.34
${\overline{t}}_e\_std$ (mm)	17.14	12.87	16.82	13.21	19.52	12.79	23.63	18.75	26.48	27.82	31.57	32.11	41.63	39.69
${\overline{t}}_e\_min$	10.27	6.42	9.17	8.36	9.49	7.94	14.49	8.34	13.53	11.75	17.93	13.56	17.31	14.57
${\overline{t}}_e\_max$	54.53	43.08	59.31	42.24	61.40	47.75	64.37	51.85	71.59	73.33	83.23	77.59	92.64	88.62
$\overline{iter}$	12.48	8.14	12.75	8.21	12.69	8.17	12.95	8.39	13.44	8.80	12.95	8.91	14.27	9.05

, ${\overline{\overline{a}}}_e$ represents the average value of ${\overline{a}}_e$ in all datasets, ${\overline{a}}_e\_std$ is the standard deviation of all ${\overline{a}}_e$, ${\overline{a}}_e\_min$ and ${\overline{a}}_e\_max$ denote the extreme value of ${\overline{a}}_e$, and the same is true for ${\overline{\overline{t}}}_e,{\overline{t}}_e\_std,{\overline{t}}_e\_min$, and ${\overline{t}}_e\_max$. It can be clearly seen that the proposed method exhibits the ability to adapt to rough poses within the error range of ${6.12}^{°},187.53$ mm. Moreover, as shown by the results of ${\overline{a}}_e\_std$ and ${\overline{t}}_e\_std$ in Noise-0, it is reasonable to believe that the proposed method has suitable repeatability, and this repeatability can be maintained until the noise exceeds the tolerance (e.g., Noise-10). As shown in Figure 9, the pose estimation process was visualized according to the number of iteration rounds. It can be found that the rough pose with large error requires more exploration, which means that the effective update of the feasible region decreases and the value of $\overline{iter}$ tends to increase. The update lag of $\mathcal{F}$ reduces the probability of occurrence of offspring with suitable ${vp}^q$ before termination, thereby increasing ${\overline{a}}_e\_max$ and ${\overline{t}}_e\_max$. Due to the existence of evolution function optimization and multiple rendering mechanisms, the performance gap between -A and -B is obvious, which can also be found by the $1-{vp}^q$ and projection errors of -A and -B with the same $iter$. Once the error of the initial pose exceeds the acceptable range, such as Noise-12, the performance of both groups is destroyed. Presently, it is necessary to modify the method parameters to improve the degree of exploration, such as increasing the initial ellipsoid size, $s_0$, ${\tau }_0$, and ${\mathsf{\sigma }}_0$.

3.3.2. The Advantage of Learning Mechanism

To further explore the performance difference between Proposed-A and -B during the evolution, an additional metric is designed as

$${\overline{a}}_e\left(i,*\right)=\frac{1}{\left|Data\right|}{\sum }_{Data}\frac{1}{\left|*\right|}{\sum }_{*}{a}_e\left(i\right),*={\mathcal{P}}_{i}or{\mathcal{P}}_i^{\prime }$$

(24)

where ${\mathcal{P}}_i$ and ${\mathcal{P}}_i^{\prime }$ represent the $i$-th round of parents and the migrated parents. Then, let ${\stackrel{=}{a}}_e\left(i,\mathrm{*}\right)$ be the mean of ${\overline{a}}_e\left(i,\mathrm{*}\right)$ for all repetitions (${\stackrel{=}{t}}_e\left(i,\mathrm{*}\right)$ is the same), and the results are shown in Figure 10.

Overall, the final performance and improvement speed of ${\overline{\overline{a}}}_e$ in -B is significantly higher than that of -A, which indicates that a more reasonable parameter reduction approach is conducive to the high-quality update of $\mathcal{F}$, indirectly increasing the probability of local difficult components escaping the dilemma. Comparing ${\overline{\overline{a}}}_e,{\overline{\overline{t}}}_e$ with their respective ${\stackrel{=}{a}}_e\left(i,\mathrm{*}\right),{\stackrel{=}{t}}_e\left(i,\mathrm{*}\right)$, it can be observed that appropriate reproductive strategy can provide suitable local exploration for the composition of the optimal viewpoint and make ${\overline{\overline{\mathrm{*}}}}_e<{\stackrel{=}{\mathrm{*}}}_e\left(i,{\mathcal{P}}_i^{\prime }\right)$. However, as the iterations progress, the absence of fine control for -A leads to the inability to strictly constrain the exploration. Meanwhile, the lack of timely rendering results in viewpoints with high ${vp}^q$ not necessarily exhibiting smaller errors. Therefore, in the later stage, the optimal viewpoint of -A is perturbed and invalid updated, causing fluctuations in ${\stackrel{=}{t}}_e$, even greater than ${\stackrel{=}{t}}_e\left(i,\mathrm{*}\right)$. Proposed-B shows better result stability compared with -A, suggesting that many ineffective explorations could be avoided, and a smaller $iter$ can be obtained. Therefore, the Proposed-B was employed in all subsequent experiments.

3.4. Parameter Sensitivity Analysis

Important parameters include the number of parents and children and a series of initial values {${\lambda }_0,{\mathsf{\sigma }}_0,s_0,{\tau }_0{,\mathsf{\alpha }}_0$} related to the initialization of the evolution function. Sensitivity analysis was first carried out on the number of parents and children, and then an orthogonal experiment was performed for {${\lambda }_0,{\mathsf{\sigma }}_0,s_0,{\tau }_0{,\mathsf{\alpha }}_0$}. Then, based on the results of the orthogonal experiment, the main factors are extracted using the mean effect plot, and further analysis is performed.

As shown in Figure 11, $P5C5$ indicates that the number of parents and children is both 5. It should be noted that the application of guided model (20) requires additional sub-viewpoints when $P<6$, while the $P$ of the parent still follows the configuration during offspring generation. In theory, more parents means greater global exploration, and more children means stronger local exploration. When $P$ is fixed, the performance first increases and then decreases with the increase in $C$ from the results in Figure 11b. This is because local exploration requires a reasonable combination with global exploration, so blindly expanding $C$ will inject too many low-quality attempts, harming the updating of $\mathcal{F}$ and the evolution process. In terms of process, excessive $C$ makes it difficult to maintain the early advantages of evolution and premature convergence to the low-performance range. When $C$ is fixed, the performance gradually stabilizes with the increase in the $P$. However, from a process perspective, although the results have improved, more fluctuations have emerged during this period, resulting in delayed convergence. To balance accuracy and efficiency, $P10C5$ was adopted for subsequent experiments.

Letting $s_0\in \left\{\mathrm{0.15,0.125,0.1,0.075,0.05}\right\},{\tau }_0\in \left\{{35}^{°},{30}^{°},{25}^{°},{20}^{°},{15}^{°}\right\}$, ${\lambda }_0\in \left\{\mathrm{0.9,0.8,0.7,0.6,0.5}\right\},{\mathsf{\alpha }}_0\in \left\{\mathrm{0.6,0.5,0.4,0.3,0.2}\right\},{\mathsf{\sigma }}_0\in \left\{\mathrm{35,30,25,20,15}\right\}$, the orthogonal experiments with 5 factors and 5 levels were performed, and the results are shown in

Table 3. Orthogonal experiment of initial parameters.

Config	${\overline{\overline{\mathit{a}}}}_{\mathit{e}}$ (°)	${\overline{\overline{\mathit{t}}}}_{\mathit{e}}$(mm)	Config	${\overline{\overline{\mathit{a}}}}_{\mathit{e}}$ (°)	${\overline{\overline{\mathit{t}}}}_{\mathit{e}}$(mm)
{1,1,1,1,1}	0.7118	38.6272	{3,4,5,1,2}	0.3289	31.6075
{1,2,3,4,5}	0.5432	30.9155	{3,5,2,4,1}	0.6245	31.4838
{1,3,5,2,4}	0.5148	32.5514	{4,1,3,5,2}	1.0717	40.5826
{1,4,2,5,3}	0.4505	30.4453	{4,2,5,3,1}	0.3486	25.7130
{1,5,4,3,2}	0.5673	32.5823	{4,3,2,1,5}	0.6769	51.0665
{2,1,5,4,3}	0.5493	26.0468	{4,4,4,4,4}	0.2366	24.5641
{2,2,2,2,2}	0.5637	35.1910	{4,5,1,2,3}	0.3477	31.0679
{2,3,4,5,1}	0.5201	21.0529	{5,1,2,3,4}	0.6443	27.2303
{2,4,1,3,5}	0.4341	59.3825	{5,2,4,1,3}	0.5959	43.7068
{2,5,3,1,4}	0.4369	49.7276	{5,3,1,4,2}	0.5311	38.3228
{3,1,4,2,5}	0.3744	37.6432	{5,4,3,2,1}	0.5638	32.4311
{3,2,1,5,4}	0.2759	32.6803	{5,5,5,5,5}	0.3495	35.0093
{3,3,3,3,3}	0.2489	31.9432	\	\	\

.

Moreover, to further clarify the impact of each factor, we drew a corresponding 5-factor 5-level main effect plot based on Table 3. For ease of calculation, the pose error was merged as $norm\left({\overline{\overline{a}}}_e\right)+norm\left({\overline{\overline{t}}}_e\right)$, and the results are provided in Figure 12. In terms of value range (the first row), it can be found that $\{{\lambda }_0,{\mathsf{\sigma }}_0\}$ have a strong influence on the performance (0.3597, 0.3940) and tend to provide the best results at levels 4 and 5. This is because ${\lambda }_0$ is a variable that controls the degree of global exploration and ${\mathsf{\sigma }}_0$ determines the search scope of the viewpoint quality evaluation. With the assistance of the method mechanism, an appropriate increase in $\{{\lambda }_0,{\mathsf{\sigma }}_0\}$ can indeed contribute to early exploration. As the projection operation based on ${\mathsf{\alpha }}_0$ in offspring generation will provide additional limitations for exploration, ${\{s}_0,{\tau }_0\}$ has the least impact on the performance due to the close effect of ${\mathsf{\sigma }}_0$ at levels 3 and 4 and the fact that increasing ${\mathsf{\sigma }}_0$ will increase the range of viewpoint evaluation (reduce efficiency). Moreover, the balance of efficiency and performance depends on the hardware devices and application scenarios. In this study, the configuration with $P10C5$ and {3, 4, 5, 4, 3} was ultimately selected and provided.

3.5. Appearance Difference Adaptability

This section verified the adaptability of the proposed method to the appearance difference (e.g., product at different stages or lightweight model) and viewpoint difference between the virtual model and real product. The setting of appearance difference is shown in Figure 13, where A, D, and E were not equipped with cables, one device belonging to Range II was removed in B1, one virtual component belonging to Range II was removed in B2, and C = B1 + B2. Additionally, the visual style of Groups A–C was different from the standard, and the part dimensions of Groups D and E were modified, as shown in Figure 13, respectively. The dataset was adjusted according to the above changes, and the experimental results are listed in

Table 4. Appearance difference adaptability.

	A-Dataset	B1-Dataset	B2-Dataset	C-Dataset	D-Dataset	E-Dataset
${\overline{\overline{a}}}_e(°)$	0.33	0.43	0.39	0.75	0.27	0.28
${\overline{\overline{t}}}_e$ (mm)	30.50	44.49	50.67	69.59	28.92	26.16
Proposed-B with Standard 3D Model
${\overline{\overline{a}}}_e(°)$	0.25	0.26	0.26	0.25	0.23	0.25
${\overline{\overline{t}}}_e$ (mm)	24.71	25.71	26.09	26.65	24.34	23.37

.

The results proved the adaptability of the proposed method to state differences, especially to visual feature changes caused by the differences in model rendering style. Moreover, pose errors of Groups A, D, and E reflect that sparse context changes such as component size modification have less impact on the performance than rendering style changes. Although the error of Group C is close to ${0.75}^{°}$, 69.59 mm, the proposed method still has an advantage over the existing methods listed in Table 1. The impact of a missing part is significant, and the reasons are shown in Figure 14. The current object has a high similarity with the component behind it in this viewpoint, and mismatching is considered a high-quality correspondence. Moreover, when occluded components participate in matching, their features are extremely different from the region that contains a missing part but rather highly similar to the noise. Therefore, the coupling of the above situations prevents anomalies from being intercepted by VPS, causing viewpoints that should have been discarded to always participate in feasible region construction and evolution, resulting in continuous accumulation of errors.

The viewpoint difference was adjusted by setting the total rendering times of the evolution process. For example, Group 2 means that only the first two rounds enabled rendering. The virtual image rendering was performed based on the viewpoint with the best ${vp}^q$ in each round. The upper limit of iteration rounds was set to 11, and the results are shown in

Table 5. Adaptability of the difference between virtual and real viewpoints.

	1	2	3	4	5	6	7	8	9	10	Each
${\overline{\overline{a}}}_e(°)$	0.26	0.24	0.23	0.22	0.23	0.22	0.21	0.22	0.23	0.23	0.23
${\overline{\overline{t}}}_e$ (mm)	27.24	26.40	27.12	26.11	25.48	25.26	24.51	23.96	23.74	23.70	23.71
$\overline{ite}$ (round)	9.12	8.73	8.81	8.93	8.25	7.97	7.93	8.11	8.13	8.14	8.14

. By comparing the results of Group 1 in Table 5 with those of Proposed-A in Section 3.2, it can be further found that the value of evolution function optimization. The results of Groups 1~3 prove the adaptability of the proposed method to the differences between virtual and real viewpoints and the geometric feature inconsistency caused by them. The pose estimation accuracy is positively correlated with the rendering times. By observing the change in $\overline{iter}$ from Groups 4 to 11, we can find that rendering times are not the more, the better. Since suitable results usually appear in the 6th to 9th rounds and then tend to be stable, the difference between virtual and real features brought by additional rendering will stimulate more exploration, which will lead to fluctuation in results and an increase in $\overline{iter}$. Therefore, the balance between accuracy and performance should be determined based on practical applications.

3.6. Practical Application Cases

The application cases in two real manufacturing scenarios are shown in Figure 15.

The first row shows a large-sized truss-type product painting task, and the second row introduces the assembly and gluing of porous structural components assisted by AR. The core task of both cases is pose estimation, and their respective problems correspond to a part of the experimental platform. The test results show that the proposed method can improve the initial error of the former from ${5.0}^{°}$, 80 mm to ${0.8}^{°}$, 17 mm, and increase the hole projection deviation ${\overline{IoU}}_e$ of the latter from <0.1 to >0.5. Some valuable setting strategies and potential limitations analysis are as follows.

(1) Setting strategies. (a) Divide object set based on assembly relationships and part sizes, and define a rule for sub-object to gradually participate in evolution; (b) given an empirical viewpoint, obtain the initial pose through the strategies provided in this article or studies [48,49,50]; (c) considering both uniform distribution and matching residuals, determine the constraint features for feasible region initialization; (d) choose to adjust VPS or replace it with other matching strategies based on the object structure; (e) select the number of termination evolution rounds based on observations; and (f) choose whether calibration optimization is needed based on the characteristics of the scene.

(2) Potential limitations. (a) No significant structural features (such as flexible bodies). Lack of initial pose will result in the inability to initialize the feasible region. One possible solution is to adopt a method like MegaPose [34] that can adapt to unseen objects to provide an initial pose. (b) Excessive initial pose deviation is always a challenge. For possible failures, we recommend some possible methods from an engineering perspective to constrain the initial pose, including planning viewpoint, empirical adjustment, fixtures, etc. (c) Equipment failure. The coverage of the field of view and the rationality of the depth of field needs to be given priority consideration during the planning phase.

4. Discussion and Conclusions

In this study, we present a novel attempt at the integration of knowledge-based agent models and data-driven deep learning for accurate pose estimation of complex products. The pose estimation is reconstructed as an evolution process. To solve the problem of large-scale solution space optimization, virtual–real matching considering structural stability, evolvable feasible region constraint, and adaptive population migration and reproduction are proposed. To reduce the empirical dependency in parameter tuning rules, enhance robustness and promote rapid convergence, a guided-model-based hierarchical cyclic effective trajectory learning mechanism is proposed and embedded. Compared with the best result ${1.28}^{°},77.67$ mm and ${1.69}^{°},55.67$ mm in classic and latest solutions under different existing method configurations, the proposed method enjoys better performance, the pose error can reach ${0.23}^{°},23.71$ mm, and the cause of abnormal results can be retraced.

Moreover, through experiments and analysis, we have found the following:

(1)

A hierarchical architecture from coarse to fine is a practical approach to image-based pose estimation while maintaining high accuracy and reliability;

(2)

Consistent with previous studies [28,29,30,35], cascaded DL module and geometric optimization can effectively balance adaptability and accuracy;

(3)

Some practical problems, such as limited samples, complex structure, and multiple elements, make it difficult to directly perform complex advanced DL-based networks. On the contrary, indirect models that only require weak annotation are easier to deploy in practical applications;

(4)

For global or divide-and-conquer architectures based on matching, metric, and optimization, the coordination of exploration and optimization requires more detailed regulation in the pose estimation of complex assembled products.

From the perspective of application value, through case studies and validation, it shows that the method will be beneficial for non-serious measurement applications that require camera pose in the assembly process of some complex products, such as AR projection, robot-assisted control, appearance inspection, etc.

In terms of potential limitations, it is undeniable that many specialized designs and appropriate initial poses need to be ensured. In practical applications, we recommend planning the imaging pose network or employing robot imaging for known scenarios in advance to reduce the search space. Moreover, we plan to carry out further validation in more manufacturing scenarios to identify more possible practical challenges (e.g., actual missing parts) and adopt CNN-based strategies to represent feasible regions.