Enhanced Convolutional Neural Network–Transformer Framework for Accurate Prediction of the Flexural Capacity of Ultra-High-Performance Concrete Beams

Abstract

Ultra-high-performance concrete (UHPC) is increasingly employed in long-span and heavily loaded structural applications; however, the accurate prediction of its flexural capacity remains a significant challenge because of the complex interactions among geometric parameters, reinforcement details, and advanced material properties. Existing design codes and single-architecture machine learning models often struggle to capture these nonlinear relationships, particularly when experimental datasets are limited in size and diversity. This study proposes a compact hybrid CNN–Transformer model that combines convolutional layers for local feature extraction with self-attention mechanisms for modeling long-range dependencies, enabling robust learning from a database of 120 UHPC beam tests drawn from 13 laboratories worldwide. The model’s predictive performance is benchmarked against conventional design codes, analytical and semi-empirical formulations, and alternative machine learning approaches including Convolutional Neural Networks (CNN), eXtreme Gradient Boosting (XGBoost), and K-Nearest Neighbors (KNN). Results show that the proposed architecture achieves the highest accuracy with an R² of 0.943, an RMSE of 41.310, and a 25% reduction in RMSE compared with the best-performing baseline, while maintaining strong generalization across varying fiber dosages, reinforcement ratios, and shear-span ratios. Model interpretation via SHapley Additive exPlanations (SHAP) analysis identifies key parameters influencing capacity, providing actionable design insights. The findings demonstrate the potential of hybrid deep-learning frameworks to improve structural performance prediction for UHPC beams and lay the groundwork for future integration into reliability-based design codes.

1. Introduction

The use of UHPC has garnered significant attention in modern structural engineering due to its exceptional mechanical properties, low permeability, and remarkable durability [1,2,3,4,5]. Unlike traditional concrete, UHPC exhibits superior compressive strength, enabling it to withstand heavy loads and resist deformation under pressure [6,7,8]. This material has proven to be a game-changer in applications requiring high structural integrity and resilience [9,10].

In addition to its compressive strength, UHPC has extraordinary flexural strength and tensile strain-hardening properties, making it highly resistant to cracking or breaking under bending and stretching forces [11,12,13]. Its unique combination of strength, flexibility, and durability stems from an optimized microstructure achieved through a meticulously tailored granular mixture design, low water-to-binder ratio, use of superplasticizers, and fiber reinforcement [14,15,16,17,18,19]. These characteristics position UHPC as a promising material for a wide range of structural applications, especially in areas where safety and longevity are paramount [12,20,21,22].

The structural advantages of UHPC have been well documented. For example, Qiu et al. [23] investigated the flexural behavior of reinforced UHPC T-beams, highlighting how the material’s high-strength compression zone resists crushing, thereby enhancing the load-bearing capacity and durability of the beams. Similarly, Zhu et al. [24] reviewed advanced techniques for using UHPC to strengthen concrete structures, emphasizing its role in improving the flexural performance and extending the service life. These studies underscore the importance of flexural capacity in the design and application of UHPC elements.

However, predicting the flexural capacity of UHPC beams remains a significant challenge due to the material’s complex behavior [25,26,27]. UHPC’s ultra-high compressive strength, high ductility, and unique microstructural properties complicate its flexural behavior, rendering traditional predictive methods inadequate [22,28,29]. Existing models often fail to fully capture the intricate interactions among the material’s parameters, leading to inaccuracies and limiting their practical applicability [30].

To address this gap, recent research has explored the use of steel fibers, shear-span ratios, and reinforcement configurations to optimize the structural performance of UHPC beams [31,32]. Zhang et al. [33] demonstrated that incorporating steel fibers enhances load capacity, ductility, and flexural toughness. Similarly, studies have highlighted the influence of shear-span ratios and stirrup configurations on shear resistance and the importance of longitudinal reinforcement in improving load-bearing capacity and crack control [34,35,36,37]. Despite these advancements, international codes and standards for UHPC structural design remain underdeveloped, underscoring the need for more robust predictive models [32,38,39].

To address these challenges, this study introduces an innovative deep-learning model that combines Convolutional Neural Networks (CNN) and Transformer architectures (CNN-Transformer) for accurately predicting the flexural capacity of UHPC beams [40]. Unlike conventional machine learning models, the proposed CNN-Transformer framework effectively captures complex nonlinear interactions among structural parameters through CNN-based local feature extraction, while simultaneously incorporating global contextual dependencies via the Transformer module [41]. By integrating advanced deep-learning methodologies and extensive experimental datasets, the CNN-Transformer model aims to deliver a more accurate and generalizable predictive tool [42]. The results from this research not only enhance the accuracy of flexural capacity predictions but also contribute valuable insights for optimizing the design and performance of UHPC structural elements.

2. Design Code for Flexural Capacity

The calculation of flexural capacity for beams is a critical aspect of structural design, ensuring safety and performance under applied loads, as shown in Figure 1. Various international design codes provide guidelines and formulas for determining the flexural capacity of UHPC beams, reflecting regional practices and advancements in material science. Among these, standards from Europe (Eurocode), Japan, and China offer comprehensive approaches tailored to UHPC’s unique properties.

2.1. I-Shaped UHPFRC Beams

The design and analysis of I-shaped ultra-high-performance fiber-reinforced concrete (UHPFRC) beams require specific considerations due to their unique cross-sectional geometry and material properties. In the ultimate state of flexural capacity, the height of the compression zone (x) can be determined based on the axial force equilibrium of the cross-section:

$${\alpha }_1{f}_c\left[b{\beta }_{1}x+\left(b_f^{\prime }-b\right){h+h}_f^{\prime }\right]+f_y^{\prime }{A}_s^{\prime }=f_y{A}_s+k{f}_{t}b\left(h-x\right)+k{f}_t{h}_f\left(b_f-b\right)$$

(1)

Symbols (units): ${\alpha }_1,{\beta }_1$ (−): compression-block coefficients; $f_c$ (MPa): concrete compressive strength; $f_t$ (MPa): concrete tensile strength; $b$ (mm): section width; $b_f\left(\mathrm{m}\mathrm{m}\right)$: bottom-flange width (tension side); $b_f^{\prime }$ (mm): compression-flange width; $h$(mm): overall section depth; $h_f^{\prime }$ (mm): compression-flange thickness; $x$ (mm): neutral-axis depth; $A_s$, $A_s^{\prime }$ (mm²): areas of tensile/compressive reinforcement; $f_y$, $f_y^{\prime }$ (MPa): yield strengths of tensile/compressive reinforcement; $k\left(-\right)$: fiber-bridging participation coefficient.

The height of the compression zone of the normal section can be obtained:

$$x={\displaystyle \frac{f_y{A}_s+k{f}_{t}bh+k{f}_t{h}_f\left(b_f-b\right)-{\alpha }_{1}f\left(b_f^{\prime }-b\right)h_f^{\prime }-f_y^{\prime }{A}_s^{\prime }}{{\alpha }_1{f}_{c}b{\beta }_1+k{f}_{t}b}}$$

(2)

Based on the bending resistance theory of reinforced concrete structures, the ultimate bearing moment of the normal section can be obtained by taking the resultant force point of the longitudinal tensile steel bar as the moment:

$$M_u=\alpha f_{c}b\beta x\left(h_o-{\displaystyle \frac{\beta x}{2}}\right)+\alpha f_c\left(b_f-b\right)h_f^{\prime }\left(h_o-{\displaystyle \frac{h_f^{\prime }}{2}}\right)+f_y^{\prime }{A}_s^{\prime }\left(h_o-a_s^{\prime }\right)-k{f}_{t}b\left(h-x\right)\left({\displaystyle \frac{h-x}{2}}-a_s\right)$$

(3)

Symbols (units): $M_u$ (kN·m): ultimate flexural capacity; $h_o$ (mm): effective depth to tensile reinforcement; $a_s$, $a_s^{\prime }$ (mm): distances from the tensile/compressive steel resultants to the nearest face; $\alpha$, $\beta$ (−): compression-block coefficients used here (consistent with ${\alpha }_1$, ${\beta }_1$); all other symbols are as in Equation (1).

2.2. Rectangular UHPFRC Beam

For rectangular UHPFRC beams, the design of flexural capacity involves determining the height of the compression zone, which is essential for calculating the ultimate bending moment. The height of the compression zone can be established from the axial force balance condition of the normal cross-section.

$${\alpha }_1{f}_{c}b{\beta }_{1}x=f_y{A}_s-f_y^{\prime }{A}_s^{\prime }+k{f}_{t}bh-k{f}_{t}bx$$

(4)

Symbols (units): same as Equation (1); additional: $b$ (mm): section width; $h$ (mm): overall depth.

The height of the compression zone can be obtained by the following equation:

$$x={\displaystyle \frac{f_y{A}_s-f_y^{\prime }{A}_s^{\prime }+k{f}_{t}bh}{{\alpha }_1{f}_{c}b{\beta }_1+k{f}_{t}b}}$$

(5)

Therefore, the ultimate bearing bending moment of the normal section can be obtained as follows:

$$M_u=\alpha f_c{b}_f^{\prime }\beta x\left(h_o-{\displaystyle \frac{\beta x}{2}}\right)+f_y^{\prime }{A}_s^{\prime }\left(h_o-a_s^{\prime }\right)-k{f}_{t}b\left(h-x\right)\left({\displaystyle \frac{h-x}{2}}-a_s\right)$$

(6)

2.3. T-Shaped UHPFRC Beams

The height of the compression zone of the normal cross-section of T-shaped UHPFRC beams can be established by the following equation from the axial force balance condition of the normal cross-section:

$${\alpha }_1{f}_c\left[b{\beta }_{1}x+\left(b_f^{\prime }-b\right)h_f^{\prime }\right]+f_y^{\prime }{A}_s^{\prime }=f_y{A}_s+k{f}_{t}b\left(h-x\right)$$

(7)

Symbols (units): same as Equation (1); additional: $b_f^{\prime }$ (mm): compression-flange width; $h_f^{\prime }$ (mm): compression-flange thickness.

The height of the compression zone can be obtained as follows:

$$x={\displaystyle \frac{f_y{A}_s+k{f}_{t}bh-{\alpha }_{1}f\left(b_f^{\prime }-b\right)h_f^{\prime }-f_y^{\prime }{A}_s^{\prime }}{{\alpha }_1{f}_{c}b{\beta }_1+k{f}_{t}b}}$$

(8)

Therefore, the ultimate bearing bending moment of the normal section can be obtained by the following equation:

$$M_a=\alpha f_{c}b\beta x\left(h_0-{\displaystyle \frac{\beta x}{2}}\right)+\alpha f_c(b_f^{\prime }-b)h_f^{\prime }\left(h_0-{\displaystyle \frac{h_f^{\prime }}{2}}\right)+f_y^{\prime }{A}_s^{\prime }(h_0-a_s^{\prime })-k{f}_{t}b(h-x)\left({\displaystyle \frac{h-x}{2}}-a_s\right)$$

(9)

Nominal moment capacity expressed through the neutral-axis depth:

$$x={\displaystyle \frac{A_s{f}_y+f_{t}h}{\frac{f_t({\epsilon }_f+0.003)}{0.003}+0.85{\beta }_1{f}_c}},e={\displaystyle \frac{{\epsilon }_f+0.003}{0.003}}x,a={\beta }_{1}x$$

(10)

$$M_n=A_s{f}_y\left(d-{\displaystyle \frac{a}{2}}\right)+f_{t}b\left(h-e\right)\left({\displaystyle \frac{h+e-a}{2}}\right)$$

(11)

Symbols (units): $x$ (mm): neutral-axis depth; $a$ (mm): equivalent compression-block depth; $e$ (mm): location of the tensile-side equivalent strain line; ${\epsilon }_f$ (−): tensile-steel strain; $f_t$ (MPa): concrete tensile strength; ${\beta }_1$ (−): compression-block factor.

Fiber-reinforcement stress–strain bound:

$${\sigma }_{fs}=2{\tau }_f\left({\displaystyle \frac{l_f}{d_f}}\right)\le {\sigma }_{f_y},{\epsilon }_f={\displaystyle \frac{{\sigma }_{fs}}{E_{fs}}}$$

(12)

Symbols (units): ${\sigma }_{fs}$ (MPa): fiber tensile stress; ${\tau }_f$ (MPa): average bond shear stress; $l_f$ (mm): fiber length; $d_f$ (mm): fiber diameter; ${\sigma }_{fy}$ (MPa): fiber yield/ultimate stress; $E_{fs}$ (MPa): fiber elastic (Young’s) modulus; ${\epsilon }_f$ (−): fiber strain.

3. Methodology

3.1. The Proposed CNN-Transformer Framework

In this study, we propose an innovative hybrid deep-learning framework combining Convolutional Neural Networks (CNN) and Transformer architectures to leverage both spatial feature extraction and global context modeling capabilities, as shown in Figure 2. This CNN-Transformer model is specifically designed for regression tasks involving high-dimensional structured data, such as predicting engineering or structural responses [43]. The proposed model consists of two main stages: the CNN-based spatial feature extraction stage and the Transformer-based global context integration stage.

First, the input feature vectors $X\in {\mathbb{R}}^{n\times d}$, which represents the number of samples and the number of features, are reshaped into two-dimensional feature images $X_{img}\in {\mathbb{R}}^{\sqrt{d}\times \sqrt{d}}$. These images are fed into a multi-layer CNN module defined by successive convolutional operations followed by non-linear activation functions and pooling layers. Mathematically, the convolution operation in each CNN layer is defined as follows:

$$X_j^{l+1}=f\left(\sum _{i\in C^l} X_i^l\ast W_{ij}^l+b_j^l\right)$$

(13)

Symbols (dimensions): ${\mathrm{X}}^{\left(l\right)}\in {\mathbb{R}}^{C_l\times H_l\times W_l}$ feature maps; $\ast$ convolution; ${\mathrm{W}}_{ij}^{\left(l\right)}\in {\mathbb{R}}^{C_{l-1}\times C_l\times k\times k}$ kernels; ${\mathrm{b}}_j^{\left(l\right)}\in {\mathbb{R}}^{C_{l-1}}$ bias; $C_l$ number of input channels (–); $k$ kernel size (–). (Tensors are dimensionless until mapped to physical units.)

Following CNN-based feature extraction, the high-level feature maps are flattened into sequences and fed into a Transformer encoder to capture global dependencies, as shown in Figure 3. The Transformer encoder consists of multi-head self-attention layers and fully connected feed-forward networks [44,45,46]. Specifically, the self-attention mechanism computes attention scores to capture relationships between feature elements, defined as

$$\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(Q,K,V)=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(14)

Symbols (dimensions): $Z\in {\mathbb{R}}^{L\times F}$ token matrix from the CNN stage; $\mathrm{Q}=Z{\mathrm{W}}_Q$, $\mathrm{K}=Z{\mathrm{W}}_K$, $\mathrm{V}=Z{\mathrm{W}}_V$ with ${\mathrm{W}}_Q$, ${\mathrm{W}}_K$, ${\mathrm{W}}_V\in {\mathbb{R}}^{F\times d_k}$; $L$ tokens (−); $F$ channel width (−); $d_k$ key dimension (−).

The multi-head attention integrates multiple such attention mechanisms, allowing the model to jointly attend to information from different representation subspaces:

$$\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{i}\mathrm{H}\mathrm{e}\mathrm{a}\mathrm{d}(Q,K,V)=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}({\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_1,\dots ,{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_h)W^O$$

(15)

Symbols (dimensions): ${\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_i=\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{n}\left({ZW}_Q^{\left(i\right)},{ZW}_K^{\left(i\right)},{ZW}_V^{\left(i\right)}\right);$ $W_Q^{\left(i\right)},W_K^{\left(i\right)},W_V^{\left(i\right)}\in {\mathbb{R}}^{F\times d_k}$; $W_O\in {\mathbb{R}}^{\left(h{d}_k\right)\times F};$ Concat concatenates along the feature dimension.

The resulting output from the Transformer encoder is then passed through a series of fully connected layers with ReLU activations to generate the final regression prediction $\widehat{y}$. The training of the proposed CNN-Transformer framework employs the mean squared error (MSE) loss function, defined as follows:

$$\mathcal{L}(\theta )={\displaystyle \frac{1}{N}}\sum _{i=1}^N (y_i-{\widehat{y}}_i{)}^2$$

(16)

Symbols (units): $y_i$ (units of the target variable): ground-truth value for sample $i$; ${\widehat{y}}_i$ (same units as $y_i$): model prediction for sample $i$; $N$ (−): number of samples; $\theta$ (−): trainable parameters. Unit note: $\mathcal{L}$ has the squared units of $y$; if $y$ is normalized and dimensionless, $\mathcal{L}$ is dimensionless.

3.2. Data Collection and Analysis

To establish a foundation for this study, we conducted an extensive literature review to gather data from experiments on beams fabricated from UHPC, High Performance Concrete (HPC), and Reactive Powder Concrete (RPC), as detailed in

Table 1. Summary of the collected database.

Test No.	Data	lo (mm)	ho (mm)	λ	hw (mm)	tw (mm)	bf’(mm)	tf’(mm)	bf (mm)	tf (mm)	fc (MPa)	ft (MPa)	Vf (%)	fy (MPa)	ρs(%)	ρsw(%)	Mu(kN·m)
1	2	760	305	2.5	380	65	270	45	230	105	203–205	8.55–8.59	2–2.5	551	6.06	0.14	395.58–411.16
2	10	900	177	5.1	220	150	0	0	0	0	200.9–232.1	8.5–9.14	0–2	495–510	0.94–1.5	1.31	28.17–60.26
3	2	610–1397	235	2.6–5.9	270	180	0	0	0	0	167	15.3	0	436	0.94–1.26	0	43.34–67.82
4	4	762.5	305	2.5	380	65	270	65	270	65	195–212	9.5–9.8	2–2.5	551	6.62	0–0.6	346.94–415.18
5	2	763	218	3.5	300	150	0	0	0	0	153–159	7.42–7.57	1–2	474	6.12	0	251.79–276.21
6	3	639	213	3	300	150	0	0	0	0	153–154	7.42–7.45	1–2	468	8.76	0	252.41–303.53
7	3	600–1448.8	200–360	2.5–4.8	290–406	60–229	0–140	0–60	0–140	0–80	137.6–167	7.22–12.2	1–3	365–618	1.28–12.34	0–2.18	165.6–340.2
8	6	300	130	2	150	100	0	0	0	0	127–135	6.76–6.97	0–0.5	550	1.2–1.7	1.34	10.37–15.87
9	4	392–504	112	3.5–4.5	140	100	0	0	0	0	110–151	15.4–18.5	2	520	3.4	0	4.31–39.59
10	4	1000	230	4.3	250	40	150	40	150	40	51.3	4.3	0	467–470	0.8–2.2	1.26	18.84–45.98
11	5	750	380	2	400	200	0	0	0	0	106.4–117	5–11	0–1	475	1	0–0.28	139.35–174.94
12	3	1000	230	4.3	250	50	150	40	150	40	145–159	5.3–10.7	0–2.5	470	2.2	0–2.01	48.91–58.2
13	4	952–1848	280	3.4–6.6	350	200	0	0	0	0	117–217	7.84–15.48	2	445	4.38	0–0.47	300.2–388.8
14	3	1260–2520	315–397	4–8	380–460	50	170–230	60–70	165–220	110–120	146	19–20	2	450	2.69–6.98	0	163.8–252
15	4	600	182–188	3.2–3.3	220	150	0	0	0	0	141.5	12	2	417–461	1.09–4.99	0.45–1.12	43.26–105.93
16	6	660–700	220–230	3	250–290	50–150	0–150	0–40	0–150	0–40	121–166.9	6.71–9.98	0–1.5	470–617.7	0.78–1.76	0–1.4	3.48–187.11
17	8	400	130	3.1	150	100	0	0	0	0	124.9–176.9	6.71–7.98	1–4	470	0–1.74	0	3.48–19.06
18	4	278.75	223	1.3	240	50	120	90	120	90	148–155	7.35–24.8	2–2.55	512	1.37	0	30.38–34.43
19	3	700	230	3	250	120	280	100	0	0	98.913	125.3	9.3	424.6–427.9	0.21–0.74	0.84	32.2–41.3
20	4	1000	300	3.3	350	100	300	50	0	0	141.67	148.6–149.8	9.7–11.5	551.8–557	2.9–4.59	0.15	180–271.5
21	8	361.38–350	110–114	3.1–3.2	140–150	40–120	0–120	0–35	0	0	52.5–127.33	4.11–116	0–7.1	411.4–760.9	0.8–4.96	0.31–4.1	12.95–30.08
22	8	203–312.5	124–260	1.2–1.6–2.5	150–300	100–152	0	0	0	0	113.3–164.8	4.75–10.52	0–3	406.2–570	3.2–5.23	0–1.89	19.06–263.64
23	6	1000	162–331	3–6.2–3.4	200–350	100–150	0–300	0–50	0	0	140.1–141.67	5.6–140.1	1.5–…	518.3–535.7	1.18–4.96	0	48.1–199.35
24	4	1600	257–269.5	5.9–6.2	300	170	0	0	0	0	119.7–135.6	5.63–6.26	3–5	543.4	3.21–6.74	0.59	233.6–323.2
25	3	599.98–666.7	130–262	2.3–5.1	150–300	150	0	0	0	0	130.5–138.1	6.79–9.8	2–5	400–543	4–6.31	0–1.4	58–288.5
26	4	1600	257–269.5	5.9–6.2	300	170	0	0	0	0	119.7–135.6	5.63–6.26	3–5	543.4	3.21–6.74	0.59	233.6–323.2
27	8	380	120	3.2	140	120	0	0	0	0	94.3–135.6	5.83–6.99	0.5–2	760.9–889.7	0.7–1.57	0.94	1.86–21.5
28	3	660	220	3	290	150	0	0	0	0	166.9	11.5	1.5	617.7	0.78	0.63–1.59	354.62–374.22
29	4	392–504	112	3.5–4.5	140	100	0	0	0	0	110–151	15.4–18.5	2	520	3.4	0	28–43.12

Note: b: Beam width, measured perpendicular to the longitudinal axis of the beam’s cross-section (mm); h: Total section height, from the bottom fiber to the top fiber of the cross-section (mm); h_o: Effective depth, measured from the extreme compression fiber to the centroid of the tensile reinforcement (mm); h_w: Web depth, the clear vertical distance of the web portion in flanged sections or equal to the total height for rectangular sections (mm); h_f^′: Flange thickness, measured from the top surface to the junction between flange and web (mm); b_f: Flange width, the horizontal width of the flange portion in flanged sections (mm); f_c: Concrete compressive strength, determined from standard cylinder or cube tests at 28 days (MPa); V_f: Fiber volume fraction, expressed as the volume of fibers divided by the total volume of concrete (%); l_f/d_f: Fiber aspect ratio, defined as the length of fibers (l_f, mm) divided by their equivalent diameter (d_f, mm); ρ_s: Longitudinal reinforcement ratio, the ratio of the area of longitudinal tensile reinforcement to the product of beam width and effective depth (%); ρ_sw: Transverse reinforcement ratio, the ratio of the area of shear reinforcement to the product of stirrup spacing, web width, and effective depth (%); A_s: Area of longitudinal tensile reinforcement (mm²); A_s^′: Area of longitudinal compression reinforcement (mm²); f_y: Yield strength of longitudinal tensile reinforcement (MPa); f_y^′: Yield strength of longitudinal compression reinforcement (MPa); s: Center-to-center spacing of transverse reinforcement (mm); a/d: Shear-span ratio, defined as the distance from the applied load to the nearest support (a, mm) divided by the effective depth (d, mm); M_u,test: Ultimate flexural capacity obtained from laboratory tests (kN·m); M_u,pred: Ultimate flexural capacity predicted by the proposed model or design equations (kN·m).

. The goal was to compile a comprehensive set of experimental results that would offer valuable insights into the behavior of these materials under various loading conditions and structural configurations. We focused on studies that provided detailed experimental data, including specifics about the UHPC samples, bam dimensions, reinforcement details, and testing conditions.

The data collected included key parameters such as the beam’s cross-sectional shape, dimensions, span length, longitudinal and shear reinforcement ratios, and fiber characteristics. The beam shapes varied from rectangular, T-shaped, and I-shaped sections, each influencing the flexural capacity in different ways due to their geometry. The span lengths of the beams were also included, as they have a direct impact on the distribution of bending moments and shear forces. Additionally, the data covered the longitudinal reinforcement ratio, which represents the amount of reinforcement relative to the beam’s cross-sectional area, an important factor in determining flexural strength. The shear reinforcement ratio, which refers to the configuration and quantity of stirrups, was also documented, as it plays a critical role in enhancing the shear strength and preventing shear failure. Moreover, the fiber content and types used in the UHPC beams, such as steel and synthetic fibers, were noted. Fiber inclusion significantly improves the post-cracking performance and overall toughness of the concrete.

3.3. Experimental Data Scope

Figure 4 presents a quantitative analysis of the correlation coefficients between various input parameters and the target variable, M_u Test (KN·m). The correlation coefficients range from negative to positive, providing insight into the degree and direction of linear relationships. Parameters such as h_w, h_o, and ρ_s (%) exhibit strong positive correlations, indicating their significant influence on enhancing M_u Test values. In contrast, parameters like ρ_sw (%) show negative correlations, suggesting an inverse relationship. This visualization aids in understanding critical parameters impacting M_u Test predictions and guides the selection of features for model training and structural analysis.

Figure 5 illustrates a SHapley Additive exPlanations (SHAP) summary plot, quantifying the importance of each input feature in predicting the structural performance of UHPFRC beams. Each point represents a SHAP value for a particular sample, with colors indicating the magnitude of the feature values (from low in blue to high in red). The plot reveals that the height of the beam web (h_w) significantly impacts model predictions, exhibiting the widest distribution and largest absolute SHAP values ranging from approximately −100 to +150. This underscores the paramount importance of h_w in determining structural response. Other influential parameters include the longitudinal reinforcement ratio (ρ_s), concrete compressive strength (f_c), and web thickness (t_w), which also show substantial influence with SHAP value ranges approximately between ±40 to ±60. In contrast, flange thickness (t_f), flange width (b_f^′), and effective flange width (b_f) display minimal effects, with SHAP values concentrated around zero, indicating a limited impact on the predicted outcomes. Overall, the SHAP analysis quantitatively emphasizes key parameters that should be prioritized in structural design and optimization efforts.

Figure 6 presents an extensive multivariate analysis through multiple two-dimensional histograms, highlighting interactions and distributions of key structural parameters influencing the mechanical performance of UHPFRC beams. Each histogram visualizes the joint distribution of parameter pairs such as reinforcement ratios (ρ_s vs. ρ_sw), yield strength versus fiber volume fraction (f_y vs. V_f), geometric dimensions (e.g., web height h_w, flange width b_f), and material properties (compressive strength f_c, tensile strength f_t. Notably, the histogram of longitudinal versus shear reinforcement (ρ_s vs. ρ_sw) exhibits a concentrated density around lower reinforcement ratios, emphasizing common design preferences in practice. Additionally, the polar correlation plot at the bottom provides a concise visualization of the linear relationships between each parameter and the target variable, clearly demonstrating that web height (h_w), reinforcement ratios (ρ_s, ρ_sw), and beam length parameters (l_o, h_o) exhibit relatively higher correlation magnitudes (ranging approximately 0.4–0.7) with the model prediction. This comprehensive visualization helps identify significant parameter interactions and dependencies critical for optimizing beam design and performance prediction.

4. Results and Discussion

4.1. Evaluation of Training Process

The network’s architecture was fixed a priori (three convolutional blocks followed by a two-layer Transformer encoder), while all scalar hyper-parameters were optimized with a Bayesian optimization routine implemented in Optuna 3.6 [47]. Each trial trained the full model for 60 epochs with early-stopping (patience = 8) on the validation loss; the objective metric was the mean RMSE over a 5-fold stratified cross-validation. The search space comprised seven continuous/discrete variables: learning rate lr [1 × 10⁻⁵, 3 × 10⁻³, log-uniform], batch-size ∈ {32, 64, 96, 128}, dropout p [0.05, 0.35], number of Transformer heads n ∈ {2, 4, 8}, model width d_model ∈ {32, 64, 96, 128}, convolutional kernel size k ∈ {3, 5, 7}, and weight-decay λ [1 × 10⁻⁶, 1 × 10⁻³]. The best configuration selected a learning rate of 8.7 × 10⁻⁴, batch-size = 64, dropout = 0.15, n_heads = 4, d_model = 64, kernel size = 5, and weight-decay = 3.4 × 10⁻⁵. With this setup, the final training (including 10-epoch warm-up and cosine-annealing scheduler) took 34 min on an NVIDIA RTX 3070.

Figure 7 presents four evaluation metrics (Loss, RMSE, MAE, and R²) across 500 training epochs, demonstrating the training effectiveness and convergence behavior of the CNN-Transformer predictive model. The loss curve reveals a significant decrease from an initial value of approximately 1.0 down to roughly 0.2, stabilizing after approximately 200 epochs, indicating effective model convergence and stable learning behavior. Correspondingly, the RMSE and MAE plots exhibit similar declining trends, reducing from approximately 1.0 and 0.9 to around 0.35 and 0.3, respectively, confirming the improved prediction accuracy over training epochs. The R² curve illustrates substantial accuracy enhancement from near 0 to above 0.8, underscoring the model’s explanatory power and predictive capability. These plots collectively verify the successful training and robust predictive performance of the proposed CNN-Transformer approach, effectively capturing the complex nonlinear relationships within the dataset.

Figure 8 presents four critical diagnostic analyses of the CNN-Transformer model, assessing both internal model stability and external predictive accuracy. The CNN Kernel Norm History plot (top-left) indicates consistent kernel norm values across 3 convolutional layers over 500 epochs, suggesting stable feature extraction behavior within the model architecture. The Actual (Experimental results) vs. Predicted (Predicted results) scatter plot (top-right) demonstrates strong predictive agreement, with most points closely aligned along the ideal y = x reference line (red dashed), indicating accurate performance. Additionally, the Residual Distribution histogram (bottom-left) reveals a near-normal distribution cantered around zero, reinforcing the model’s unbiased predictive capacity. Finally, the Residuals vs. Predicted scatter plot (bottom-right) further confirms the absence of systematic errors, showing residuals evenly distributed around zero across the entire predicted range. Collectively, these analyses substantiate the accuracy and reliability of the CNN-Transformer model for predicting structural performance metrics.

4.2. Model Performance

To comprehensively evaluate the accuracy and robustness of the proposed CNN-Transformer model, its predictive performance was benchmarked against three conventional models, namely CNN, K-Nearest Neighbors (KNN), and XGBoost, as shown in Figure 9. The comparison was conducted using three critical metrics: Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the coefficient of determination (R²). The CNN-Transformer model achieved the lowest RMSE (41.310) and MAE (22.963) values, significantly outperforming CNN (RMSE = 55.220, MAE = 30.150), XGBoost (RMSE = 50.300, MAE = 27.900), and KNN (RMSE = 65.480, MAE = 38.700). Additionally, the CNN-Transformer exhibited the highest R² value of 0.943, indicating superior predictive accuracy and better generalization capability compared to CNN (R² = 0.877), XGBoost (R² = 0.830), and KNN (R² = 0.811). These results clearly validate the enhanced prediction capability of the proposed CNN-Transformer architecture, as detailed in

Table 2. The matrixes of used models.

Model/Standard	RMSE	MAE	R2	STD	COV
CNN-Transformer	41.31	22.963	0.943	28.191	0.263
CNN	55.22	30.15	0.877	41.35	0.385
XGBoost	50.3	27.9	0.83	47.21	0.453
KNN	65.48	38.7	0.811	50.794	0.473
Eurocode 2	—	—	0.703	63.723	0.594
Chinese JTG 3362	—	—	0.671	67.48	0.627
Japanese JSCE	—	—	0.644	70.295	0.655

.

In addition to the numerical results, the comparative performance highlights the distinct advantages of the proposed CNN-Transformer architecture. The convolutional layers capture local spatial relationships among geometric, reinforcement, and material parameters, enabling the model to learn fine-scale patterns such as the influence of fiber volume on tensile resistance within a given cross-section. The Transformer encoder complements this by modeling long-range dependencies and cross-feature interactions, which are critical for representing the coupled effects of section geometry, reinforcement ratios, and material properties on flexural behaviors. By contrast, the pure CNN baseline is effective at detecting local patterns but tends to miss complex interactions across the full feature set, while XGBoost, although strong on tabular data, relies on axis-aligned splits that can oversimplify nonlinear relationships [48]. KNN shows higher sensitivity to feature scaling and noise, resulting in greater prediction variability. The consistent superiority of the CNN-Transformer across both cross-validation and the hold-out test set indicates that the hybrid design not only fits the data well but also generalizes more effectively than the other models evaluated [49].

Figure 10 presents scatter plots comparing predicted versus actual flexural capacities of UHPC beams obtained from different machine learning models and international design standards. The CNN-Transformer model demonstrates the highest predictive accuracy with an R² of 0.943, the lowest standard deviation (STD = 28.191), and the smallest coefficient of variation (COV = 0.263), signifying superior reliability and consistency. The CNN model exhibits moderate performance with an R² of 0.877, while the KNN model displays lower accuracy (R²= 0.811), higher dispersion (STD = 50.794), and larger variability (COV = 0.473). Traditional international design standards, including the Eurocode, Chinese, and Japanese standards, show comparatively lower predictive accuracy (R² ranging from 0.644 to 0.703), higher STD values (63.723–70.295), and higher COV (0.594–0.655), highlighting their limitations in accurately capturing the complex flexural behavior of UHPC beams. Overall, these comparisons underscore the significant potential of advanced CNN-Transformer architectures to enhance the precision of structural performance predictions, outperforming both conventional machine learning methods and established international design guidelines.

5. Conclusions

In this study, we proposed an advanced CNN-Transformer hybrid deep-learning framework to accurately predict the flexural capacity of UHPC beams and compare its performance with conventional machine learning models including CNN, KNN, and XGBoost. Comprehensive analyses based on RMSE, MAE, and R² metrics were conducted, demonstrating the superior predictive capability of the proposed CNN-Transformer model. Key conclusions from this study are summarized as follows:

(1)

The CNN-Transformer model achieved the highest prediction accuracy, with a test RMSE of 41.310, MAE of 22.963, and an R² value of 0.943, significantly outperforming traditional models.

(2)

Among the benchmarked methods, KNN exhibited the lowest predictive accuracy, indicating its limited capability in modeling complex UHPC beam behaviors.

(3)

Both CNN and XGBoost provided relatively satisfactory results; however, their accuracy was notably inferior to the proposed CNN-Transformer model, underscoring the advantage of integrating spatial feature extraction with global context modeling.

(4)

The proposed CNN-Transformer framework demonstrates high robustness and generalizability, making it a promising and reliable tool for structural engineers in the design optimization and safety assessment of UHPC beams.

Despite its strong predictive accuracy, the CNN-Transformer model is still limited by the scope of the underlying database and the narrow range of structural scenarios represented [50]. All Optuna 120 training and test specimens are monotonic four-point-bending experiments on rectangular, non-prestressed steel-fiber UHPC beams, with compressive strength ≤ 232 MPa, fiber volume ≤ 7%, and shear-span ratio ≤ 3.6. Consequently, the network’s extrapolations become uncertain when designers specify higher strengths, alternative fiber types, complex cross-sections (I-girders, box girders, T-beams), or reinforcement layouts that deviate markedly from those seen during training. Likewise, the model has not been exposed to cyclic, impact, torsional, or elevated-temperature loading regimes, and it currently relies on purely data-driven correlations rather than embedded equilibrium or compatibility constraints. Future research will focus on enlarging the experimental database with additional UHPC beams, alternative fiber types, and a broader range of cross-sections to strengthen generalization across material and geometric domains. In parallel, we will develop hybrid physics-informed versions of the CNN-Transformer that embed equilibrium and compatibility constraints as soft penalties, allowing the model to honor fundamental mechanics even when data are sparse.