An Investigation on a Virtual Assembly System for Structural Experiments

Abstract

The complexity of marine structural experimental devices is usually attributed to the boundary conditions and loads applied to the objects. As a result, the complexity of the device and the volume of the objects make strict requirements on the experimental designs and assembly of these devices. In this study, a virtual assembly system for a structural laboratory (VAL) is developed in the Unity 3D environment, and collision detection algorithms are derived based on bounding box models and mesh models. This research realized the parametric modeling or importing of 3D objects and reconstructed pressure loading experimental scenarios in a 3D environment. The algorithm can automatically select a detection method according to the geometric object, and every assembly step is recorded and visualized. The system can effectively simulate the assembly of a structural experimental device. Moreover, the 3D file importing interface, the rendering of transporting tracks, and interaction detection algorithms can support the construction of a virtual scenario for more experiments.

1. Introduction

1.1. Background

The cost of marine structural experiments is usually high, and repeating structural experiments is difficult. In the preparations for structural experiments, the arrangements of the layout and assembly might lead to interference and collision. As a result, feasible methods for simulating experimental processes and evaluating plans are necessary. As 3D virtual methods can simulate the physical properties and kinematic properties of entities from real experimental scenarios, virtual processes that characterize real experimental scenarios show potential for application. This research aims to explore the application of a virtual 3D assembly to the design of marine structural experiments.

Previous experimental studies on load-bearing capacities use typical forms of experimental scenarios, including equipment, objects, and laboratory environments [1,2,3,4]. Researchers have performed many studies on virtual modeling and assembly that focus on the technique of virtual modeling and accurate assembly for engineering structures. Xue et al. propose an advanced rigid–flexible hybrid assembly deviation analysis (RFHA) method for aero-structural shape deviation [5]. Cai et al. [6] researched the dimensional quality of sheet metal assembly and developed a rigid–compliant hybrid variation modeling methodology for sheet metal assembly. For models generated by nonlinear finite element (NFEA) software, feature point gaps and kinematic formulations can be proposed to compute the rigid motions for any FE nodes behind the mating surface in the assembly sequence [7]. Yang et al. proposed a three-dimensional contour-measuring algorithm that employs binocular multi-line laser sensing and further developed a virtual assembly system for aircraft engine rotors [8].

By driving CAD and CAE software using scripts, the modeling and analysis of serialized experimental objects can be automatically completed. A common method is to generate parameterized geometric models in CAD systems and then import them into CAE software through file formats such as .step, .iges, and .x_t. This method may face problems such as feature loss and topology errors during the model conversion process. Katsoulis et al. developed a parametric modeling method based on T-spline [9]. Papanikolaou [10] explored the parametric modeling of ships and combined the process with digital siblings and multi-objective optimization. Du et al. [11] employed an online-training artificial neural network mechanism and a multi-stage parametric modeling method for the optimization of ship resistance. Khan et al. [12] derived a parametric modeling method for hull design. Li et al. [13] evaluated the ultimate strength of hulls through parametric modeling based on sets.

Virtual assembly is an effective method for simulations of complex systems. Jin et al. [14] established advanced virtual assembly methodologies for complex mechanical structures through the application of digital twin technology. Jiang et al. [15] derived a virtual assembly method based on point clouds and building information models, which realized assembly error detection. Point cloud models can serve as an effective medium for assembly error detection [16]. The Jacobian–Torsor model can be employed to calculate the assembly accuracy of complex structures [17]. Zhu et al. [18] developed a collision detection program based on a bounding box algorithm and a spatial triangle intersection detection algorithm. The feasibility and effectiveness of virtual pre-assembly collision detection programs have been verified through examples. Jiang et al. [19] derived a virtual assembly method based on an oriented bounding box (OBB) algorithm and a Devillers and Guigue algorithm, and a differential triangular surface algorithm was also employed. Li et al. [20] researched visualization methods using NFEA results in a visual 3D environment.

Researchers also employ digital twins (DT) and virtual–real fusion, aiming to strengthen the capability of 3D virtual models. Ángel et al. proposed a framework of processing digital twins (DTs) in the shipbuilding scenario, highlighted the application of robotic manufacturing units and verified the robotized small pre-assembly units [21]. Gao et al. presented a DT framework that integrates condition monitoring, virtual verification, and intelligent decision-making of the deep-sea Argo float, and proposed a computationally efficient decision-support method by constructing a behavior–performance mapping [22]. Xu et al. constructed a digital twin system for deep-water well construction tailored to offshore energy development, coupled the optimization of conductor installation and the soaking phases, and overcame traditional single-stage limitations [23]. Liu et al. established a digital twin system to assist the coordination control in deep-sea mining scenarios, performed real-time simulations to predict the spatial configuration of the umbilical cables [24]. Zhang et al. established a simulation model of a water-lubricated stern bearing system, and realized the real-time visualization of the structural performance [25].

1.2. Research Gap and Contributions

The above studies concentrate on the assembling of complex entities for mass production, which focus on the accuracy of installations. The objectives of marine structural experiments include stiffened plates, hull girders, cabin sections and so on, and the sizes of the models vary in certain ranges. Comparing with other assembled objects, each experimental layout differs from each other, and employs different supporters and constraints. Moreover, the continuous installation of the connecting parts results in spatial interference in different dimensions. Collision between entities might happen in every assembly step. As a result, this research aims to realize the recording, optimization and the dynamic collision detection of each assembly step.

1.3. Research Objective

This study aims to develop a virtual assembly system for a structural laboratory (VAL) based on Unity3D, and realize the full process of virtual assembly, including parametric 3D modeling, model importation, transportation, collision detection, and installation. VAL supports importing models in multiple industrial formats, including .igs, .stp, and .x_t, and provides basic operation functions such as model transportation, rotation, scaling, deletion, and copying. In every virtual assembly step, the recording of tracks and the detection of collisions are performed simultaneously.

2. Methodology

2.1. Modeling of the Experimental Scenario

In this research, the virtual scenario of ultimate strength experiments on marine structures is constructed, including the laboratory, the experimental equipment, and the experimental objects. In the construction of the 3D environment, 3ds Max 2018 and Substance Painter 2019.3.3 are used. The 3D modeling of the laboratory scene includes three steps: 3D geometric modeling, surface mapping and texture mapping, and virtual 3D rendering. The virtual models are expected to have precise geometric structures, realistic material features, and good interactive performance. Moreover, VAL also supports importing external 3D models with proper formats (.igs, .stp, .x_t).

Initially, 3D geometric modeling (3ds Max code employed) is performed based on measured geometric data of the objects. The laboratory space has been captured by wide-angle scanners, and precise dimensional data and spatial layout information have been processed. Based on the measured data, the geometrical model of the laboratory scene, consisting of architectural structures such as walls, doors, windows, and floors, experimental equipment, and experimental objects, is constructed. The sizes of 3D entities are fully consistent with the actual spatial layout of the laboratory. For instance, the thickness of walls, the positions and dimensions of doors and windows, and the height of the ceiling are all precisely measured and modeled. Fixed equipment in the laboratory, such as pressure cylinders, strain gauges, and pressure sensors, is also modeled according to their actual dimensions.

The modeling process aims to adjust the models, ensuring that the distribution of vertices and edges is reasonable and avoiding excessive triangular faces or irregular geometric shapes. For complex structures, the model surfaces are smoothed and enhanced. During the modeling process, the unit settings of the model are consistent with the unit system of Unity3D (2019.4.29f1c2), which is typically set to 1 unit = 1 m, to avoid scaling issues in the importation.

Surface unwrapping is performed on the model after the geometric modeling is completed, i.e., the surface of a 3D model is expanded into a 2D plane, which is the foundation of texture mapping. During the unwrapping process, special attention is paid to key parts of the equipment, to ensure that the mapping of these parts is clear and convenient for subsequent texture mapping.

After the illumination rendering is completed, the model optimization is also performed, including reducing the number of polygons, optimizing textures, and reducing draw calls. In Unity3D, the tool called Mesh Simplifier is employed to reduce the number of polygons in the model, and the Level of Detail (LOD) method is used to automatically switch between models of different levels of detail based on distance.

Finally, examinations and adjustments on effects and interactive performance of the model are conducted. Materials, lighting, and model parameters are adjusted as required to ensure that the final effect aligns with expectations.

2.1.1. 3D Geometric Modeling

The preparation stage is the first step of modeling, which includes collecting reference materials, determining modeling objectives, and selecting an appropriate modeling method. This step involves on-site photography, photoshop texture processing, and obtaining CAD vector base maps. On-site photography involves using a wide-angle digital camera to capture real images of the laboratory, equipment, and experimental objects. Photoshop texture processing involves processing the on-site photography data to meet the requirements for exterior texture mapping. And the CAD vector base maps can guide the distribution of points in a virtual 3D environment.

In the construction of a 3D entity, a CAD vector base map of the real entity is firstly imported, then the outlines are drawn, and the heights of the entities are set. The experimental scenario consists of box, cylindrical, and spherical entities. As a result, the rotational modeling and loft modeling are employed. When adding surface textures, a modifier is employed to adjust the UV coordinates to ensure that the texture is correctly mapped to the model surface.

The modeling stage consists of the import of base images, the creation of basic shapes, the addition of details, and the adjustment of the model. In the construction of a 3D entity, a CAD vector base map of the real entity is firstly imported, then the outlines are drawn, and the heights of entities can also be modified, then the basic shape of the entity is generated. The CAD vector base image is imported to guide the creation of the 3D models, the vector image can be traced with lines, and then the shape of the object is created.

In the construction of the experimental scenarios, Boolean operations are used to create complex geometric shapes by combining or cutting multiple basic geometric primitives to form intricate structures, including deleting faces, merging vertices, and adjusting edges.

The optimization stage is a crucial phase for ensuring the quality and performance of the models, which is primarily achieved by reducing the number of polygons and the smoothing of the surface. Entities with more polygons are firstly selected, and the polygons are reduced by adjusting the percentage of vertices. Then the appearance of the surface is softened by an automatic smoothing algorithm and setting a threshold.

2.1.2. Processing of Surfaces

The properties of objects’ surfaces are modeled by the texture method, which replaces the outer point cloud with simulated surface models. In the surface modeling, all the operations are saved in the form of layers, allowing the adjustment or deletion of any layer at any time without affecting other layers. Intelligent materials are based on geometric analysis and can automatically generate texture effects according to the shape and surface characteristics of the model.

The anchor point function allows us to establish a causal relationship between layers, enabling the effect of one layer to influence other layers. For example, designers can create a scratch layer and then transfer its effect to a rust layer through the anchor point, and the rust will automatically appear. This function realizes a true programmatic and non-destructive workflow. The physical characteristics of preset materials are created based on procedural technology, which can adjust parameters to suit different models and scenes. For 3D virtual environments, interaction between light and materials in the real world is required.

The 3D scenario of the laboratory can be seen in Figure 1. In 3D virtual assembly scenarios, the geometric characteristics, surface properties and functional properties of the laboratory are reflected in the virtual environment.

2.2. Importation of 3D Files

As the simulation of the assembly is realized in the Unity 3D environment, industrial model files from different software programs are converted into Unity3D-recognizable formats and imported into the system. In this research, VAL supports importing of three formats: .igs, .stp, and .x_t. The initial Graphics Exchange Specification (.igs) is a commonly used 3D model format for data exchange between CAD systems. The .x_t format is a 3D solid design file format based on the Parasolid kernel. The .stp format is a more advanced 3D model format that supports more complex geometric shapes and assembly relationships, which can be directly imported and rendered by Unity 3D.

As shown in Figure 2, in this research, the IGS format can be imported directly through the “ImportFile” method of “AssetDatabase” in Unity, relying on Unity’s built-in resource management mechanism. Two conversion modes are designed for X_T (Parasolid) format.

An external conversion tool “Process” from Unity is employed to convert X_T to FBX format. Then the FBX format is imported into Unity through the “ImportFBX” function. In the programming of the algorithm, the command parameter for conversion is designed as “import {x_tPath} export {fbxPath}”. For files from Abaqus, an independent Python script is employed. Parasolid files are firstly opened through Abaqus’ Python API (mdb.openParasolid). 3D solid components are created and repaired automatically, and finally written into IGES files.

2.3. Parametric Modeling of Experimental Objects

In this research, a script for parametric modeling of experimental objects (stiffened plates) has been developed, which controls the shape and properties of geometric models by defining key model parameters. The experimental objects can be automatically updated based on user-input parameters, and the consistency of model topology is maintained. A direct correlation has been established between design variables and geometric models.

Secondary development of ABAQUS based on the Python script interface has been conducted, and ABAQUS kernel commands are invoked by Python scripts to achieve automatic execution of creating components, defining material properties, assembling instances, setting analysis steps and interactions, applying loads and boundary conditions, meshing, submitting jobs, and extracting results.

In parametric modeling, users can select typical test object types and enter parameters in the system’s UI. VAL can automatically process relevant data and call modeling software to generate parameterized models. Users can view and perform subsequent model processing, simulation analysis, and other operations in the modeling software (Abaqus 2020). Finally, experimental objects and simulation calculation result files are generated, and relevant files can be imported into the system. Based on the parameterized script, typical experimental objects including stiffened plates with T-shape stiffeners can be created.

2.4. Design of Interference Detection Algorithm

This study aims to realize the simulation of assembling structural experimental devices in the virtual environment, and the capability of assembling step simulation is the core function. For this purpose, a multi-level detection strategy is derived, which can adjust collision detection in each assembly step.

The design of the algorithm is depicted in Figure 3. The algorithm employs ray detection and collision detection: rays are generated by camera models at the user’s location, and when the rays touch the models, the models are checked out and marked. Each assembly step by users aims to install an entity, and consists of several transportation steps. In each new transportation step, a boundary region is divided around the camera and all entities are traversed, if the boundary of an entity is not within this area, the collider of the entity will be disabled. Otherwise, the collider will be activated and perform collision detection with the assembled entity. Collision detection is realized by detecting the intersection of collision bodies generated on the surfaces of different models to identify collision, which contains boundary control and model interference calculation.

The box collider is suitable for objects with regular shapes, while the mesh collider is suitable for models with complex shapes. The complexity of the model is judged by fitting the surface of the entity. For marine structural experimental scenarios, if the surface can be well fitted by the box, the box collider will be applied in the detection. On the contrary, if other shapes such as cylindrical or spherical surfaces are employed, the entity will be judged as a complex model.

In order to improve the efficiency of collision detection, a leveled optimization strategy is adopted in the designing of the algorithm. The primary detection aims to troubleshoot the objects to be detected. In this process, a bounding region around the transported entity is divided, and entities within this area are regarded as potential colliding entities. Simplified detection of the colliding entities aims to judge and analyze the intersection of the entities. The bounding box of the entities to be measured is projected onto the coordinate axis, and the projections on the coordinate axes are compared. If the projections intersect, a collision is detected. For entities with complex shapes, detailed detection is based on the intersection of triangular mesh patches, and through this stage of detection, object detection can be fully determined.

The mathematical model of colliders can be described as follows:

For entities with regular and symmetric shapes, the collision body is assumed to be a cuboid which can exactly surround the entity, and the vertex set of collision body V can be expressed as:

$$V=\left\{v_i\in {ℝ}^3\right\}.$$

(1)

The size of the collision body is calculated by coordinates of the vertices ${\overrightarrow{p}}_{\max }$ and ${\overrightarrow{p}}_{\min }$, and the collision body is controlled by center point $\overrightarrow{c}$ and the range of the point cloud $\overrightarrow{s}$:

$$\left\{\begin{array}{c}{\overrightarrow{p}}_{\max }=\left(\max x_i,\max y_i,\max z_i\right)\\ {\overrightarrow{p}}_{\min }=\left(\min x_i,\min y_i,\min z_i\right)\end{array}\right..$$

(2)

$$\overrightarrow{c}=({\overrightarrow{p}}_{\min }+{\overrightarrow{p}}_{\max })/2.$$

(3)

$$\overrightarrow{s}={\overrightarrow{p}}_{\max }-{\overrightarrow{p}}_{\min }.$$

(4)

And the collision body $B$ can be expressed as:

$$B=\left\{(x,y,z)\left|\left|x-c_x\right|\right.\le s_x/2,\left|y-c_y\right|\le s_y/2,\left|z-c_z\right|\le s_z/2\right\}.$$

(5)

The mesh collision bodies with complex surfaces are expressed as triangular panel set $Mesh$, which can be expressed as:

$$Mesh=\left\{triangular pan{e}_l T_i\left|\left\{a_i,b_i,c_i\right\}\right.\subset T_i\right\}.$$

(6)

where $a_i,b_i,c_i$ denotes the vertexes of triangular panel $T_i$, while the coordinates of them are calculated by the global coordinate system of the 3D space.

For simple entities, the detection of collision is realized based on bounding boxes, including axis-aligned bounding boxes and oriented bounding boxes, which can be described as follows:

Axis-aligned bounding boxes (AABBs) are prepared for quickly eliminating obviously disjoint entities. AABB assumes that the bounding box of each entity is axis-aligned in the local coordinate system, the center of the bounding box coincides with the origin of the object coordinate system, and only translation is permitted in the space; as a result, the bounding box remains aligned with the axis and keeps constant in size. The global position $P_i$ of the bounding box is defined as the global coordinate of the bounding box’s center, and a size vector $\overrightarrow{h}$ is defined as follows:

$$\begin{array}{c}\overrightarrow{h}=(h_x,h_y,h_z)\\ h_i=s_i/2(i=x,y,z)\end{array}.$$

(7)

where $s_i$ denotes the components of $\overrightarrow{s}$.

The bounding box of two entities can be expressed as:

$$\begin{array}{l}{B}_1=\left\{x\in {ℝ}^3\left|\left|x-P_{1x}\right|\right.\le h_{1x},\left|\left|y-P_{1y}\right|\right.\le h_{1y},\left|\left|z-P_{1z}\right|\right.\le h_{1z}\right\}\\ B_2=\left\{x\in {ℝ}^3\left|\left|x-P_{2x}\right|\right.\le h_{2x},\left|\left|y-P_{2y}\right|\right.\le h_{1y},\left|\left|z-P_{2z}\right|\right.\le h_{2z}\right\}\end{array}.$$

(8)

The criterion for intersection between two entities can be expressed as:

$$\forall j\in \left\{x,y,z\right\},\left|p_{1j}-p_{2j}\right|\le h_{1j}+h_{2j}.$$

(9)

The oriented bounding box (OBB) is prepared for rotations and overlaps in assembly steps, which is defined by a bounding box and a displacement matrix $M_i$. The bounding box can be expressed as:

$$B_i=\left\{M_i·u\left|u\in {ℝ}^3,\left|u_x\right|\right.\le h_{i,x},\left|u_y\right|\le h_{i,y},\left|u_z\right|\le h_{i,z}\right\}.$$

(10)

The displacement matrix contains translation and rotation, and the detection of collision is realized by transmitting the local coordinates of entity 1 into the local coordinates of entity 2:

$$T=M_1^{-1}·M_2.$$

(11)

As a result, the directional bounding box of entity 1 changes into a local axis-aligned bounding box, while the directional bounding box can be derived through the matrix T. And the intersection between two entities is detected by checking the overlap of axes.

When the surface projections of two entities overlap, their bounding boxes may be offset from each other along a certain pair of edges. The local axis of entity 1 is selected as a vector group $\overrightarrow{a_1}$, while the local axis of entity 2 is selected as the vector group $\overrightarrow{a_2}$, representing matrix T’s rotation vectors. As shown in Equation (12), the product of two vector groups ($\overrightarrow{a_1}\times \overrightarrow{a_2}.$) can derive 9 vectors, representing directions simultaneously perpendicular to an edge of box A and an edge of box B. The vectors are employed to capture the edge–edge separation between two entities that cannot be detected solely by surface normal vectors.

$$\overrightarrow{a_{1i}}\times \overrightarrow{a_{2j}}, i,j\in \left\{1,2,3\right\}.$$

(12)

$\Delta$ denotes the half-length projection of detected entity, and the half-length projection of entity 1 is defined as:

$${\Delta }_1=h_{1x}\left|a_{1x}\right|+h_{1y}\left|a_{1y}\right|+h_{1z}\left|a_{1z}\right|.$$

(13)

The position of the center of entity 2 in the local coordinate system of entity 1 can be calculated as follows:

$$t=T·c_2.$$

(14)

And the half-length projection of entity 2 is defined as:

$${\Delta }_2=\left|\overrightarrow{h_2}·\left(R_2^T\overrightarrow{a_i}\right)\right|.$$

(15)

The center distance projection $d_i$ is calculated by:

$$d_i=\left|\overrightarrow{a_i}·t\right|.$$

(16)

If $d_i>{\Delta }_1+{\Delta }_2$ is not checked out in any axis, interaction between two bounding boxes is detected.

2.5. Virtual Assembly

2.5.1. Virtual Assembly Perspectives

In this research, various perspectives have been developed, including first-person perspective, free-flying perspective, and fixed-observation-point perspective. The first-person perspective provides an immersive experience, suitable for visiting laboratory scenes. Free-flying perspective provides the ability to explore a wide range of scenes, suitable for overall observation. Fixed-observation perspective can be used to observe the real-time status of the experimental equipment during the experiment. View control is achieved through a keyboard and mouse. Operators can use the keys to control the movement of the view directions and vertical movement, and use the mouse to drag left, right, up, and down to control horizontal and vertical rotation.

In order to improve the smoothness of viewing perspectives, rendering optimization strategies are adopted. Based on the viewing distance, the rendered objects are managed in a hierarchical manner. Level of Detail (LOD) technology is applied to models at long distances, using low-polygon-count versions to reduce rendering cost. The rendering efficiency of the closest object is firstly prioritized, while objects that are far away and within the coverage range of the viewing angle are of secondary priority. Objects outside the viewing range are rendered statically.

2.5.2. Track Managing Algorithm

In this research, a track managing algorithm for virtual assembly is derived. The design of the algorithm is depicted in Figure 4. For each entity, every movement (translation and rotation) will be recorded as a transportation step, and the locations of the entity will be imported into the optimization calculation of the assembly steps of the device. The recorded track points are rendered by the Bezier method, and the prediction of the transportation step will be given according to translation and rotation degrees of freedom.

The location of an entity at a discrete time $t_k$ can be assumed as $p_k(x_k,x_k,x_k)$, and the track recorder maintains a fixed-length storage queue with a capacity of $N$ (in this research, $N=100$). The current point is constantly generated and recorded; if the length of the queue exceeds the preset upper limit $N$, the earliest point will be deleted. As a result, the final track point set $p_k$ can be expressed as follows:

$$p_k=\left\{p_{k-N+1},p_{k-N+2},\dots \dots ,p_k\right\}.$$

(17)

A polyline curve is firstly derived by connecting the stored trajectory points in sequence with straight-line segments. If the current trajectory point set is not full, the last real point will be selected to calculate an extrapolation point $p_{ext}$ based on the entity’s current forward unit vector $\overrightarrow{f}$ and an approximate step size $st$:

$$p_{ext}=\left\{p_{last}+st·\overrightarrow{f}\right\}.$$

(18)

This point will be added to the tail of the temporary point set as a simple linear extrapolation for predicting the future short-term motion trajectory of an object, and the point set for track rendering in each assembly step $p_t$ can be expressed as follows:

$$p_t=\left\{p_{k-N+2},\dots \dots ,p_k,p_{ext}\right\}.$$

(19)

Segmented cubic Bezier curves and the recursive De Casteljau method are used to improve trajectory continuity:

An n-th Bezier curve is defined by n + 1 control points $p_0,p_1,\dots \dots ,p_n$; the curve points $B(t)$ can be expressed as follows:

$$B(t)={\displaystyle \sum _{i=0}^n\left(\begin{array}{c}n\\ i\end{array}\right)}{(1-t)}^{n-i}{t}^i{p}_i\begin{array}{c},\end{array}t\in \left[0,1\right].$$

(20)

In this research, the degree of the Bezier curve is set as $n=3$, and the group of sampling points $Grou{p}_j$ can be expressed as follows:

$$Grou{p}_j=\left\{p_{4j},p_{4j+1},p_{4j+2},p_{4j+3}\right\}\begin{array}{c},\end{array}j=0,1,\dots \dots ,\left[{\displaystyle \frac{m-1}{4}}\right].$$

(21)

For each group, sample at a step size $△t=0.1$ on $t\in \left[0,1\right]$; 11 curve points (0.0, 0.1, 0.2, …, 1.0) can be obtained.

For a given sequence of control points $p_0,\dots \dots ,p_n$ and parameter $t$, the recursive calculation process can be expressed as follows:

$$p_i^{(0)}=p_i\begin{array}{c},\end{array}i=0,1,\dots \dots ,n.$$

(22)

For $r=1,2,\dots \dots ,n i=0,\dots \dots ,n-r$:

$$p_i^{(0)}=p_i, i=0,1,\dots \dots ,n.$$

(23)

Finally, $B(t)=p_0^{(n)}$.

3. Results

3.1. Construction of the Virtual Assembly System

3.1.1. Architecture of the System

As shown in Figure 5, this virtual assembly system adopts a layered architecture, including the following levels:

The user interface layer processes interactions with users, including model library interface, scene operation interface, and driving control interface. Users can browse and select models, perform scene operations, and control driving movements through this layer.

The business logic layer processes users’ operation requests, coordinating interactions between various modules, and implementing control and management of the assembly process. This layer includes functional modules such as model management, assembly process control, and collision detection management.

The data access layer stores, retrieves, and transforms model data. This layer includes functional modules such as experimental object database management, model format conversion, and database access.

The core engine layer provides core functions such as 3D rendering, physical simulation, and interactive control based on the Unity3D engine. This layer is the foundation of the system and provides technical support for upper-level functions.

The hardware interface layer is responsible for interacting with external hardware devices, such as input devices, display devices, etc. This layer enables the system to adapt to different hardware environments and provide diverse interactive experiences.

3.1.2. Database of Experimental Objects

As shown in Figure 6, the experimental object database is established to store and manage 3D models available for virtual assembly. In this research, the prefabricated component classification management method is adopted to realize the function of the experimental object database.

The Prefab system of Unity3D is an efficient way to manage experimental objects. By encapsulating models, components, and scripts into prefabricated components, rapid instantiation and reuse of models can be achieved. In this system, each experimental object corresponds to a prefabricated component, which includes necessary components such as the model, colliders, rigid bodies, and related interaction scripts.

Figure 5. The structure of the virtual assembly system.

Figure 5. The structure of the virtual assembly system.

In order to realize the quick query of the desired model, the experimental object database adopts a multi-level classification mode. Classification levels include usage classification (commonly used types of experimental subjects) and functional classification (experimental objects and experimental equipment). Users can quickly locate the desired model through a hierarchical menu or search box.

Each prefabricated component contains metadata information such as model name, size, material, and other attributes. These metadata can be dynamically read and modified through scripts, providing necessary parameter information for the assembly process. At the same time, metadata also supports fast filtering and sorting functions for models.

In this system, model files (such as IGS, STP, X_T formats, etc.) are stored in the local file system, while metadata is stored in the system database for quick querying and management. After the selection of the model, the system loads the model into the scene through prefabricated component instantiation. This approach improves loading efficiency and ensures model consistency and repeatability.

Figure 6. UI of the experimental object database. (a) Selection of objects; (b) detailed information of the object.

Figure 6. UI of the experimental object database. (a) Selection of objects; (b) detailed information of the object.

3.1.3. Track Managing Mechanism

As shown in Figure 7, track managing is realized through the algorithm, which is programmed and run in the Unity 3d environment. Through the algorithm, drawing of lines, dynamic creation, color/width adjustment, material application, and vertex control are realized. The algorithm supports track point marking, listing, and rendering, and the assembly track is optimized and updated through the Bezier curve model. In this research, the core content of track managing can be described as follows:

(1)

Dynamic line drawing is adopted to control the starting position and final position of the object being transported. The color and width of the line can mark the spaces influenced by the object, and the real-time updating of vertex positions can record the important position in every assembly step.

(2)

Customizable materials and light rendering of the transportation track can be employed to distinguish each assembly step and identify possible interferences in the assembly process.

(3)

The interactive application that combines mouse clicking and manual input is employed to realize functions such as freehand drawing and path tracking.

3.2. Effects of Parametric Modeling

As shown in Figure 8, parameters of experimental objects can be assigned to VAL, and a Python script (programmed by Python 3.11.0) is employed to generate geometric models for experimental objects and import the models into the 3D environment automatically. The parameters for flat objects mainly include the length, width and number of the plate, the thicknesses, and the properties of stiffeners. To generate geometrical models with fewer variables, parameters are firstly filtered, then the input parameters are stated. Users can generate models with driving parameters (marked in Figure 8). The parametric characterization of experimental objects can generate multiple types of formats, covering the common parts for marine engineering structures.

3.3. Virtual Assembly Operations

Users can generate experimental objects through parametric modeling or importing models from outer software. As shown in Figure 9, the virtual assembly process starts after the experimental object appears in the laboratory scenario. The transportation of a model instance serves as the basic step of the virtual assembly, and the first step is the verification of the model. As shown in Figure 10 and Figure 11, users can click the model instance to activate the motion function, including translation and rotation.

After the verification of the experimental object, users can activate the transportation equipment (crane for transportation) and start the virtual assembly process. As shown in Figure 10, the connection of the crane and experimental object is defined as the entry of the experimental object. Users can control the crane with in-plane moving commands and vertical moving commands.

VAL can record each movement command and render it in a 3D environment. As a result, the track of every assembly step is stored, and the tracks serve as the guidance of the collision detection algorithm. As shown in Figure 12, VAL can manage every assembly step and calculate bounding boxes iteratively according to the environment around the object. The detection of collision is applied in every movement of the model instance, and the upcoming collision will be highlighted.

3.4. Validation of Functions

To examine the stability of the system, VAL is firstly activated and kept running for 12 days, which is enough to prepare a structural experiment. In the virtual assembling of experimental devices, 3D entities of equipment are imported into VAL, as shown in Figure 13. In the converting process, the geometric dimensions of the entities, details of frames, elbow plates and connecting holes, and the surface properties are fully inherited by Unity.

As shown in Figure 14, the locations of an entity (represented by the center of the entity) and the angular displacements can be monitored and adjusted in every transportation step. If interference is detected, VAL will send a collision alert to the UI, and the entities related to the collision will be highlighted. The final position will not be recorded until the collision disappears. As shown in the figure, when a collision warning occurs, the distance between the collision planes of two models is detected through the ruler in the virtual space, i.e., the amplitude of the interference can be calculated by the current location and the sizes of the entity. In this validation case, the monitored location of the entity is (6.56757 m, 1.64156 m, −20.0242 m); i.e., the accuracy of the collision detection is less than 3 mm.

An axial ultimate loading experiment for stiffened plate is selected as a validating case for VAL. The experimental device can be seen in Figure 15. As shown in Figure 16, users can firstly perform free assembly in the 3D virtual environment. The connecting parts of experimental objects can be represented by point constraints between entities in the 3D space. In the free assembly, VAL will record the installing steps (positions and transportation steps) of each entity, and the transportation step predicted by algorithm is combined with translation/rotation commands (by magnified rendering). The design of UI combines motion control with track prediction to reduce the cost of calculation and storage, and the transportations are quantified by positions, translating distances and rotation angles.

The optimization of the assembly is performed according to the properties of structural laboratories, the crane and supporting equipment. In every assembly step, the operations are divided into transportation steps and installation steps. Entities which connect with ground bolts (supporters and brackets) of the laboratory are installed preferably, and their locations will affect the installing steps of other entities, and then the entities which connect with supporters and brackets are installed; i.e., entities are always installed after their supporting entities. Moreover, vertical installations are performed after horizontal translation. For each case, the horizontal translating distances of each step are calculated to reduce the frequency of collision detection.

As shown in Figure 17, the optimized assembly of the validation case contains four steps. An installing region is firstly selected for the layout, and the entities are translated to the region above the ground bolts. Secondly, the experimental object, hydraulic cylinder, and constrictors are translated to their supposed horizontal positions. Then the supporting brackets are translated and installed. The vertical installing of the supported entities is performed finally. In traditional assembling of experimental devices, each entity is transported and installed in one step, and the assembly contains reciprocating movements and continuous adjustments of cranes. However, the moving tracks of cranes are simplified according to the assembly steps, and the assembly time is reduced by 10–20 min, as fewer adjustments of cranes are needed.

4. Conclusions

To optimize the assembling of experimental devices, a virtual assembly system concentrating on marine structural experiment scenarios has been constructed in a 3D virtual environment. Collision detection supported by space controlling methods and collision bodies has been derived. Based on script programming and interface construction, parameterized generation and importation of 3D models are realized. The system realizes the optimization of assembly steps of the experimental device, and the following conclusions can be drawn:

(1)

According to the connection between different parts, the assembly of each structural experimental device is divided into different assembly steps; every assembly step consists of transportation/installation steps of each entity. A bounding area model is employed to activate collision bodies, which is influenced by the distances between entities.

(2)

For the validation case in this research, the arrangements and adjustments of the transportations and installations (distances and locations) are simulated and checked in the system, and four assembly steps are conducted, with no collision bodies activated.

(3)

Spatial interferences result from accumulated installation of entities. The optimizing strategy can be concluded by reducing the activations of collision bodies by adjusting the transportation distances and locations in each assembly step, through which the scales of the bounding areas can also be efficiently reduced.

(4)

Serialized experimental objects can be generated through the model driven by users’ input parameters, which is suitable for designing experimental plans. A cross-software 3D generating interface is implemented by combining a parameter-driven generator and format convertor.

In this research, box models and meshes are selected as collision bodies. Future work should involve complex surfaces that commonly appear in experimental scenarios such as hull girders. As a result, it is necessary to employ different topology shapes to fit the surfaces of every entity and generate bounding models to reduce the calculation complexity of virtual simulation.