Intelligent Assembly of Prefabricated Concrete Column Joints Based on Multimodal Transformer and Digital Twin ^†

Abstract

In the intelligent assembly of prefabricated concrete column joints based on a multimodal transformer and digital twin, a finite element model is constructed using a digital twin to map physical entities from multiple dimensions, accurately set the material constitutive model, and rationally divide the mesh. An assembly feature integration based on multimodal transformer is proposed to achieve multimodal collaborative optimization. Two connection methods are set: integrated concrete columns with irregularly shaped composite beams and modular concrete columns with separately combined composite beams. The example results show that in the stress monitoring, the stress in the contact area between the connecting steel plate and concrete is 11.8–13.5 mpa, with uniform distribution; the stress around the bolt holes was 17.5–19.2 MPa, within a controllable range. Three-dimensional laser scanning showed that the actual and design coordinates of each component deviated very little, with the maximum deviation of the concrete column being 0.3 mm, meeting the stringent requirements of actual engineering for assembly accuracy and reliability.

1. Introduction

In the field of building engineering, prefabricated concrete column joints are key parts, and the assembly quality and performance affect the stability, safety, and durability of the entire structure. However, traditional assembly methods have limitations such as difficulty in ensuring assembly accuracy, reliance on manual experience, and lack of real-time monitoring and feedback, which restrict large-scale promotion [1,2,3]. Therefore, it is necessary to explore efficient, precise, and intelligent assembly technologies.

Therefore, this paper proposes an intelligent assembly method for prefabricated concrete column joints based on multimodal transformer and digital twin. This method utilizes a multimodal transformer to fuse and analyze various data from the assembly process, and uses a digital twin to construct a virtual assembly model to achieve real-time mapping and dynamic optimization, providing a new solution for high-quality assembly.

2. Intelligent Assembly Design of Prefabricated Concrete Column Nodes Based on Multimodal Transformer and Digital Twin

2.1. Constructing a Finite Element Model of Prefabricated Concrete Column Joints Based on Digital Twins

This paper relies on professional software to carry out the construction of its digital twin finite element model. Concrete, connecting end plates, stiffeners, and other materials have regular shapes and thicknesses [4,5]. Using three-dimensional solid elements, a precise two-dimensional diagram is first drawn and then stretched into a three-dimensional solid. Longitudinal load-bearing steel bars, transverse stirrups, and anchoring steel bars are slender and mainly subjected to axial forces [6,7]. Three dimensional truss elements are selected, and a plan view is drawn on the component interface. The software automatically generates a three-dimensional truss model.

Based on the concept of digital twin, the finite element model is deepened and expanded, and a comprehensive model architecture covering five dimensions including “physical entity, virtual entity, twin data, service, and link” is proposed. Among them, the virtual entity (VE) constructed based on the physical entity (PE) is composed of a geometric model (G), physical model (P), behavioral model (B), and rule model (R) [8]. These four models comprehensively map the physical entity from different dimensions, and their mapping relationship can be expressed by the following formula:

$$VE=f\left(G,P,B,R\right)={\alpha }_{1}G+{\alpha }_{2}P+{\alpha }_{3}B+{\alpha }_{4}R$$

(1)

where $f$ is the digital twin function, ${\alpha }_1$, ${\alpha }_2$, ${\alpha }_3$, ${\alpha }_4$ are the weight coefficients of each model in the virtual entity construction, and ${\alpha }_1+{\alpha }_2+{\alpha }_3+{\alpha }_4=1$.

In the selection of material constitutive models, in order to ensure the reliability of finite element analysis results, different material properties must be accurately set. Concrete adopts a specific constitutive relation based on specifications and engineering experience; steel uses an ideal elastoplastic model, with its stress–strain relationship set according to the corresponding conditions:

$$\sigma =E\epsilon +VE$$

(2)

where $\sigma$ represents stress, $E$ represents elastic modulus, and $\epsilon$ represents strain. Once the stress reaches the yield strength, the stress remains constant, while the strain continues to increase.

When meshing, a strategy is selected based on the geometry and stress characteristics of each component. Components with regular shapes and uniform stress use larger mesh sizes to improve computational efficiency; complex stress areas and stress concentration regions [9] use smaller mesh sizes to ensure accuracy. Reasonable meshing lays a solid foundation for subsequent assembly.

2.2. Integrating Prefabricated Concrete Column Node Assembly Features Using Multimodal Transformer

This chapter proposes a feature integration method (MT-JAM) for prefabricated concrete column node assembly based on the multimodal transformer structure. On the basis of the existing finite element model, an innovative feature extraction and integration architecture is designed to construct a transformer integration architecture that integrates multimodal information. MT-JAM introduces a multimodal transformer module parallel to the finite element feature extraction module, combining structural mechanical features with multimodal information [10]. The overall architecture of MT-JAM is illustrated in Figure 1, with a layered progression: the bottom layer obtains mechanical features, the middle layer integrates multimodal features, and the top layer outputs the integrated assembly features.

Figure 1. Overall architecture of MT-JAM.

To enhance the ability of the multimodal transformer to capture assembly features, a unique feature weight adjustment mechanism was designed. A specific weight calculation function was designed to dynamically adjust the self-attention calculation weights based on the impact of different modal features on assembly performance.

Given the finite element mechanical eigenvector as $F_1$, the geometric eigenvector as $F_2$, and the material eigenvector as $F_3$, the comprehensive eigenvector $F$ is calculated as:

$$F=w_1{F}_1+w_2{F}_2+w_3{F}_3$$

(3)

where $w_1$, $w_2$, $w_3$ are the weight coefficient of each modal feature, and its value is determined by the following formula:

$$w_i={\displaystyle \frac{e^{t_i}}{{\displaystyle \sum _{i=1}^3{e}^{t_i}}}}$$

(4)

where $e$ is the base of the natural logarithm, $t_i$ is a parameter related to the importance of each modal feature.

This integration method breaks through the limitation of traditional single modal feature, realizes multimodal collaborative optimization, improves the reliability and stability of node assembly, and provides a comprehensive and accurate basis for subsequent performance evaluation and optimization design.

3. Examples

3.1. Example Preparation

The experiment was conducted using Visual Studio Code (1.60.0) as the development environment, Python 3.9 as the programming language, and the PyTorch 1.9.1 framework as the foundation. Model training and testing were performed using NVIDIA RTX 3090 graphics cards (NVIDIA, Santa Clara, CA, USA) at the university’s high-performance computing center. The dataset used in the experiment is as follows: Dataset A contains 2000 groups of mechanical property test data, covering different section sizes, reinforcement forms and loading conditions; Dataset B includes node geometry information, material properties and construction deviation records, with a total of 1500 node instances; Dataset C simulates a variety of boundary conditions and damage scenarios, with a scale of 3000 groups of samples. The large complex dataset includes the first three, a total of 8000 groups of samples, with additional multi-sensor monitoring data and environmental variables in the actual project. The multimodal transformer model was trained for 600 cycles on three datasets, and 250 cycles on large and complex datasets. Adamw optimizer was used with momentum coefficients of 0.92 and 0.998. The batch data size is set to 2, the initial learning rate is 0.00008, the learning rate is attenuated by 12% every 60 cycles, and the transformer branch dropout is set to 25% to prevent over fitting.

A 700 mm × 1000 mm × 1500 mm base was installed at the bottom of the column for stable connection to the testing machine platform. C25 commercial concrete was used for pouring, with 14 longitudinal reinforcing bars of diameter 18 mm and Φ10@120 stirrups. The stirrup spacing in the column base reinforcement zone was 60 mm, and the height was 600 mm. Q355 steel plates were selected, and the connection was made using CO₂ gas shielded welding with ER50-6 welding wire.

Concrete strain gauges should be placed in the middle of the post-cast concrete on the outer side of the pier column, and the strain gauges should span the joints between the post-cast concrete and the pre-cast pier column, and between the post-cast concrete and the cast-in-place pier cap. Ten concrete strain gauges should be installed on each pier column. The monitoring location on the concrete surface is shown in Figure 2. The strain gauge used is model BMB150-12AA.

Figure 2. Monitoring locations on concrete surface.

This experiment used LD-300 (range ±150 mm), LD-150 (range ±75 mm), and LD-80 (range ±40 mm) wire-type displacement gauges to automatically collect and record displacement data. Three horizontal displacement gauges were installed at the top of the column loading head, and three at the bottom of the column. One displacement gauge was placed at the bottom of the column, at the first and second column heights, and at the third and fourth column heights to detect whether horizontal slippage occurred in the steel and concrete sections.

3.2. Example Result Analysis

As a key connector in the intelligent assembly of prefabricated concrete column nodes, the connecting steel plate is used to investigate its stress performance during assembly by analyzing stress monitoring data of the contact area between the connecting steel plate and the concrete and the area around the bolt holes. The stress data is shown in Figure 3.

Figure 3. Stress data during assembly.

As shown in Figure 3, the stress in the contact area between the connecting steel plate and the concrete is 11.8–13.5 MPa, and the data of each strain gauge does not fluctuate much. The stress in the area around the bolt holes is 17.5–19.2 MPa, which is higher than that in the contact area, but still within the reasonable stress range of the material. This demonstrates that intelligent assembly precisely controls bolt connections, ensuring manageable stress under load and preventing connection failure due to excessive stress.

Next, a 3D laser scanner is used to scan and measure the nodes after assembly to obtain the actual coordinate data of each component. The actual coordinates are compared with the design coordinates, and the results are shown in Table 1.

Table 1. Comparison results of actual coordinates and design coordinates of each component.

Component	Actual Coordinates	Design Coordinates	Deviation Value
concrete column	(1000.2, 800.1, 5000.3)	(1000.0, 800.0, 5000.0)	(0.2, 0.1, 0.3)
concrete beam	(1500.3, 1200.2, 5000.5)	(1500.0, 1200.0, 5000.0)	(0.3, 0.2, 0.5)
connecting steel plate	(1250.1, 1000.1, 5000.2)	(1250.0, 1000.0, 5000.0)	(0.1, 0.1, 0.2)
bolt	(1251.0, 1001.0, 5000.8)	(1250.5, 1000.5, 5000.5)	(0.5, 0.5, 0.3)
rebar coupler	(1250.3, 1000.3, 5000.4)	(1250.0, 1000.0, 5000.0)	(0.3, 0.3, 0.4)
embedded parts	(1250.4, 1000.4, 5000.6)	(1250.0, 1000.0, 5000.0)	(0.4, 0.4, 0.6)

Analysis of Table 1 reveals that the deviations in the X, Y, and Z directions of the concrete columns are only 0.2 mm, 0.1 mm, and 0.3 mm, respectively. The deviations of the rebar sleeves and embedded parts are reasonable, meeting the accuracy requirements for subsequent installation and connection.

4. Conclusions

Under the development of intelligent prefabricated buildings, the intelligent assembly technology of concrete column nodes based on multimodal transformer, and digital twins, the integration solves the problems of difficult control of traditional assembly accuracy and weak quality traceability; it is also able to monitor key parameters in real time and dynamically optimize assembly and warn risks.