RailDLA-Net: An Intensity-Aware Deep Local Aggregation Framework for Railway Point Cloud Semantic Segmentation

Highlights

What are the main findings?

• An intensity-aware deep local aggregation framework, RailDLA-Net, is proposed for semantic segmentation of railway point clouds.
• RailDLA-Net achieved 96.89% OA, 91.92% mAcc, and 88.44% mIoU on Rail3D, with strong performance on both major and sparse railway categories.

What are the implications of the main findings?

• Combining geometric structure, hierarchical context, and LiDAR intensity improves fine-grained semantic discrimination in complex railway scenes.
• The proposed framework shows practical potential for intelligent railway inspection and infrastructure management.

Abstract

Accurate semantic segmentation of railway point clouds is essential for intelligent railway inspection and infrastructure management. However, railway scenes contain elongated structures, locally similar objects, and severe class imbalance, while the discriminative value of LiDAR intensity is still insufficiently exploited. To address these issues, this study proposes RailDLA-Net, an intensity-aware deep local aggregation framework for railway point cloud semantic segmentation. The network adopts an encoder–decoder architecture and integrates a railway structure-guided residual local encoding module to capture local geometric variations, a hierarchical structural context aggregation module to model multi-level contextual dependencies, and an intensity-aware adaptive attention mechanism to enhance the discrimination of geometrically similar categories. In addition, a geometry–semantics collaborative objective is introduced to jointly optimize final predictions, intermediate semantic representations, and local spatial structures. Experiments were conducted on Rail3D and WHU-Railway3D datasets to evaluate RailDLA-Net under different railway scene conditions. The proposed method achieved 96.89% OA, 91.92% mAcc, and 88.44% mIoU on Rail3D, and 92.36% OA, 91.84% mAcc, and 83.63% mIoU on WHU-Railway3D. Compared with the baseline methods, RailDLA-Net showed competitive overall performance and improved the balance across categories, especially in more complex and imbalanced railway scenes. Ablation studies further demonstrated the effects of key configuration settings and the loss design. Together with the evaluations on two railway point cloud datasets, these results support the effectiveness of RailDLA-Net as an integrated railway-oriented framework for fine grained point cloud semantic segmentation.

1. Introduction

Railway infrastructure is an essential component of modern transportation systems, and its safety condition is directly related to transport efficiency, operational safety, and regional economic development [1,2,3]. With the development of mobile laser scanning (MLS) systems [4] and LiDAR technology [5], large-scale railway scenes can be rapidly and accurately acquired as 3D point clouds, providing a new data basis for railway asset management and intelligent inspection [6]. However, raw point clouds are usually composed of massive discrete 3D points without explicit semantic information, making them difficult to directly support component recognition, facility inventory, and risk analysis [7,8,9]. Point cloud semantic segmentation (PCSS) aims to assign a semantic category to each 3D point and is a key step in transforming geometric measurements into understandable railway scene information [10]. In railway scenes, this task requires distinguishing different categories such as rails, sleepers, ballast, ground, vegetation, catenary poles, signal facilities, and other auxiliary objects. Accurate semantic segmentation provides structural scene-level information for railway infrastructure management, intelligent inspection and the construction of digital-twin railways [11].

Although several studies have explored railway point cloud segmentation, this task still faces significant challenges [12,13,14,15]. Railway scenes have a typical linear spatial organization, where rails and catenary wires appear as elongated continuous structures, sleepers are periodically arranged, and ballast and ground are locally similar in geometry. Meanwhile, key facilities such as poles and signal devices usually contain fewer points and can be weakened by large-area background categories. In addition, variations in scanning distance, occlusion, point density, and laser intensity further increase the difficulty of semantic understanding in railway point clouds [7]. Early railway point cloud processing methods mainly relied on manual rules, geometric templates, or heuristic thresh-olds. These methods are interpretable and can be effective for specific railway lines, sensors, and target objects [16,17]. However, they are usually sensitive to parameter settings and scene assumptions, and their generalization ability can decrease when point density, acquisition platforms, or railway structures change. In contrast, deep learning methods can automatically learn geometric and semantic features from data and have therefore become an important direction for railway point cloud semantic segmentation [14].

In general point cloud understanding, PointNet [18] first demonstrated the feasibility of directly processing unordered point sets, but its ability to represent local geometric structures is limited. PointNet++ [19] enhanced multi-scale geometric representation through hierarchical sampling and local neighborhood learning, laying the foundation for subsequent point cloud semantic segmentation networks. Subsequently, methods such as KPConv [20] and RandLA-Net [21] improved large-scale point cloud processing from the perspectives of point convolution, local aggregation, and efficient sampling. In recent years, several research tendencies have emerged in point cloud semantic segmentation. One line of studies emphasizes complex local operators and attention mechanisms to improve fine-grained neighborhood representation [22,23]. Meanwhile, Transformer-based methods exploit self-attention to capture long-range dependencies and global contextual relationships in irregular point sets, providing an effective solution for large-scale scene understanding [24,25]. Another line of studies highlights the importance of network depth, training strategies, and lightweight structures for efficient point cloud semantic segmentation [26]. For railway scenes, relying solely on complex local operators may make it difficult to balance long-range linear-structure representation and large-scale point cloud processing efficiency [27]. Therefore, capturing railway micro-structures, hierarchical context, and radiometric intensity differences with low computational cost remains an important problem.

This study develops RailDLA-Net, a railway-oriented deep local aggregation framework for railway point cloud semantic segmentation. RailDLA-Net does not aim to propose a fundamentally new network paradigm. It adapts several effective components to the structural and radiometric characteristics of railway scenes. The residual local encoding module represents the fine-grained local geometry of railway components. The hierarchical structural context aggregation module combines boundary details, component-level relationships, and high-level scene semantics. The intensity-aware adaptive attention mechanism uses LiDAR intensity differences to guide local feature aggregation. A geometry–semantics collaborative objective jointly optimizes final predictions, intermediate semantic representations, and local spatial structures. This task-specific integration provides a practical framework for semantic understanding of complex railway point clouds. It also evaluates the usefulness of structural context and intensity-aware local aggregation in railway scenes.

2. Related Work

2.1. Railway Point Cloud Dataset

In recent years, advances in 3D laser scanning and deep learning have made point cloud data an important basis for 3D scene understanding. High-quality datasets are essential for developing and evaluating point cloud analysis methods [28]. Although many outdoor [29,30,31] and indoor [32,33,34] datasets have been proposed, most of them focus on general scenes and cannot fully satisfy the specific requirements of railway environments.

To address this gap, several railway point cloud datasets have been released. WHU-Railway3D [35] provides large-scale railway scenes covering urban, rural, and plateau environments, but suffers from severe class imbalance. RailPC [36] focuses on urban and mountainous railways and adopts a semi-automatic weak annotation pipeline, where a small subset is manually annotated first, and then a segmentation network is trained to annotate the remaining data, followed by manual review and correction. Rail3D [37] covers railway scenes from Hungary, France, and Belgium, supporting generalization evaluation, although its color and intensity attributes are not fully calibrated. RailPoints [38] targets sparse long-distance perception for trains, while ExpressRail [39] provides high-quality annotations for high-speed railway scenes. These datasets have promoted railway point cloud segmentation, but class imbalance, cross-scene generalization, and annotation cost remain major challenges. Detailed information is summarized in Table 1.

Table 1. Comparison of Point Cloud Datasets for Different Railway Scenarios. “w/o I” indicates whether the intensity attribute is included in the dataset.

Type	Name	w/o I	Scale	Points	Classes
MLS	WHU-Railway3D [35]	√	30 km	4614 M	11
MLS	RailPC [36]	√	25 km	3000 M	16
MLS	Rail3D [37]	√	5.8 km	288 M	9
TLS	RailPoints [38]	√	30 km	3 M	5
MLS	ExpressRail [39]		10 km	209 M	9

2.2. Point Cloud Semantic Segmentation

Point cloud semantic segmentation is a fundamental task in computer vision and has achieved significant progress with the development of deep learning [40]. PointNet [18] first enabled direct learning on unordered point clouds and laid the foundation for subsequent point-based methods. Existing deep learning methods can be broadly divided into several categories [41]. (1) MLP-based methods, such as PointNeXt [23] and PointVector [42], extract point-wise features using shared MLPs and aggregate local information through symmetric functions or vector representations. (2) Point convolution-based methods extend convolution to irregular point sets by designing local kernels or aggregation operators, with representative methods including RandLA-Net [21] and DeLA [43]. (3) Graph convolution-based methods represent point clouds as graphs and use GNNs to learn neighborhood context, such as SPH3D-GCN [44]. (4) Transformer-based methods exploit self-attention to capture long-range dependencies and global context, as seen in LitePT [24] and Point Transformer V3 [25]. (5) Mamba-based methods, including CamPoint [45] and PointSSM [46], have introduced state-space sequence modeling with linear complexity for efficient large-scale point cloud analysis.

In addition, weakly supervised methods [47,48,49,50,51] reduce annotation costs by using sparse or low-granularity labels together with label propagation and pseudo-label optimization strategies.

2.3. Existing Deep Local Aggregation Methods in 3D Point Clouds

Deep local aggregation has become an important strategy for 3D point cloud semantic segmentation. RandLA-Net [21] combines random sampling with local feature aggregation to improve the efficiency of large-scale point cloud processing. DeLA [43] designs local aggregation operators for irregular point sets and strengthens neighborhood feature representation. DeepLA-Net [26] explores very deep local aggregation networks and shows that lightweight local operators can be stacked to improve feature learning in 3D point clouds. These methods provide useful references for efficient local representation learning. The proposed RailDLA-Net follows the general idea of deep local aggregation, but adapts it to railway point cloud semantic segmentation. In RailDLA-Net, residual local encoding is used to represent fine-grained railway components. Hierarchical structural context aggregation is used to combine local details, component-level relationships, and scene-level semantics. Intensity-aware adaptive attention introduces relative LiDAR intensity differences into local aggregation. These adaptations are intended to better match the structural regularity and radiometric characteristics of railway scenes.

2.4. Point Cloud Semantic Segmentation in Railway Scenes

Railway scenes differ significantly from general urban or indoor scenes due to their long strip-like track layout, diverse line-side facilities, and severe class imbalance. These characteristics limit the direct applicability of general point cloud segmentation methods. Therefore, several studies have developed railway-specific semantic segmentation methods by incorporating railway prior knowledge and structural features. RailSeg [12] improves railway point cloud segmentation through local–global feature aggregation and contextual semantic regularization. SALAProNe [13] combines self-attention local aggregation with prototype-guided regularization for fine-grained high-speed railway segmentation and cross-scene generalization. Grandio et al. [14] improved PointNet++ with preprocessing strategies such as grid sampling, block partitioning, and data augmentation, and later introduced an image-rasterization-based filtering strategy for efficient fine-grained segmentation [38]. 3DGraphSeg [15] compresses large-scale railway point clouds into superpoint graphs and applies graph convolution for contextual segmentation. Despite these advances, existing methods still struggle to simultaneously handle the long and narrow railway layout, multi-scale railway objects, and severe class imbalance.

3. Method

3.1. Overview

Railway point clouds are characterized by a regular spatial layout. In such scenes, rails and overhead lines usually form long continuous structures, while sleepers and track-bed components appear as repeated patterns along the corridor. At the same time, many railway facilities are small and sparsely distributed, and some categories share similar local geometry. These properties make accurate segmentation dependent on both fine local representation and broader structural context. Based on this observation, RailDLA-Net is proposed as an intensity-aware deep local aggregation network for railway point cloud semantic segmentation, incorporating a railway structure-guided residual local encoding module, a hierarchical structural context aggregation module, an intensity-aware adaptive attention mechanism, and a geometry–semantics collaborative objective.

The proposed network is illustrated in Figure 1. Given an input point cloud $F_0\in {\mathbb{R}}^{N\times 4}$, where $N$ is the number of points, each point is represented by its 3D coordinates $\left(x, y,z\right)$ and a Lidar intensity $I$ value that is normalized to $[0, 1]$ using min-max normalization. Starting from $F_0$, the point cloud is progressively encoded through four downsampling stages. In each stage, farthest point sampling (FPS) first reduces the point resolution, and k-Nearest Neighbors (KNN) is then used to construct local neighborhoods around the sampled points. Positional encoding (PE) describes the relative spatial layout within each neighborhood, while intensity-aware adaptive attention (IA-Att) introduces local intensity differences into feature aggregation. The railway structure-guided residual local encoding module (RLE) further aggregates and updates the neighborhood features. Through these operations, the encoder produces four multi-scale feature maps, $F_1\left({\displaystyle \frac{N}{2}}, 64\right)$, $F_2\left({\displaystyle \frac{N}{4}}, 128\right)$, $F_3\left({\displaystyle \frac{N}{8}}, 256\right)$, and $F_4\left({\displaystyle \frac{N}{16}}, 512\right)$. These feature maps serve as railway structure-aware point features, as they encode both local geometric patterns and intensity-guided neighborhood relationships. The four stages contain 20, 20, 60, and 20 RLE blocks, respectively. This design helps preserve fine railway structures while gradually expanding the receptive field for broader scene context. During the decoder stage, the four feature maps are upsampled to the original point resolution using nearest-neighbor interpolation and transformed into the restored features $F_1^{\prime }\left(N, 64\right)$, $F_2^{\prime }\left(N, 128\right)$, $F_1^{\prime }\left(N, 256\right)$, and $F_1^{\prime }\left(N, 512\right)$, respectively. These restored features are then fed into the hierarchical structural context aggregation module to integrate features from different encoding stages. The aggregated feature is then processed by the classification head to produce the final semantic label for each point.

Figure 1. Overall architecture of the proposed RailDLA-Net for railway point cloud semantic segmentation.

3.2. Railway Structure-Guided Residual Local Encoding

Many railway objects show clear local structural patterns. Rails usually appear as continuous linear elements, sleepers are arranged repeatedly along the track, ballast has rough local surfaces, and catenary poles form slender vertical structures. These differences are often subtle within a small neighborhood, but they are important for distinguishing railway components. Therefore, a railway structure-guided residual local encoding module is designed to enhance local geometric representation and neighborhood-relative feature learning, as shown in Figure 2.

Figure 2. Detailed structure of the Railway Structure-Guided Residual Local Encoding module.

For the $s$-th encoding stage, the input point feature, coordinate, and intensity are denoted as $F_s\in {\mathbb{R}}^{N_s\times C_s}$, $P_s\in {\mathbb{R}}^{N_s\times 3}$, and $I_s\in {\mathbb{R}}^{N_s\times 1}$, respectively. Here, $N_s$ denotes the number of points and $C_s$ denotes the feature dimension at the $s$-th stage. As shown in Figure 2, the module first constructs local neighborhoods using coordinate-based KNN to obtain the neighborhood index $KNN\left(P_s\right)\in {\mathbb{R}}^{N_s\times K}$. Then, the same index is shared by the feature, coordinate, and intensity branches. Based on this index, the neighboring point features, coordinates, and intensity values are gathered as grouped features $F_{N_s}\in {\mathbb{R}}^{N_s\times K\times C_s}$, grouped coordinates $P_{N_s}\in {\mathbb{R}}^{N_s\times K\times 3}$, and grouped intensity $I_{N_s}\in {\mathbb{R}}^{N_s\times K\times 1}$, where K is the number of neighboring points. This shared grouping strategy ensures that the geometric, feature, and intensity relations are constructed over the same local neighborhood.

After neighborhood grouping, three local relation branches are built. In the feature branch, the vector feature difference $∆f_{ij}$ is computed between the grouped neighboring features $f_j$ and the expanded center features $f_i$:

$$∆f_{ij}=f_i-f_j, ∆f_{ij}\in {\mathbb{R}}^{N_s\times K\times C_s}$$

(1)

This difference describes the local feature variation between each center point $p_i$ and its neighbor points $p_j$.

In the coordinate branch, the relative coordinate offset is computed in the same way:

$$∆p_{ij}=p_i-p_j, ∆p_{ij}\in {\mathbb{R}}^{N_s\times K\times 3}$$

(2)

The offset tensor represents the relative spatial layout within each local neighborhood. It is then transformed into the structural positional embedding:

$${ PE}_{N_s}={\varphi }_p\left(∆p_{ij}\right), {PE}_{N_s}\in {\mathbb{R}}^{N_s\times K\times C_s}$$

(3)

where ${\varphi }_p\left(·\right)$ denotes the positional encoding function, implemented by a linear transformation followed by batch normalization and GELU activation. It maps the 3D coordinate offset from ${\mathbb{R}}^{N_s\times K\times 3}$ to the stage-wise ${\mathbb{R}}^{N_s\times K\times C_s}$.

In the intensity branch, the relative intensity difference is calculated as:

$$∆I_{ij}=I_i-I_j, ∆I_{ij}\in {\mathbb{R}}^{N_s\times K\times 1}$$

(4)

where $I_i$, $I_j$ denote the corresponding intensity of the center point $p_i$ and neighbor points $p_j$. The relative intensity difference is then fed into the intensity-aware adaptive attention to generate the intensity attention embedding:

$$∆I_{N_s}=\sigma \left({\varphi }_I\left(∆I_{ij}\right)\right), ∆I_{N_s}\in {\mathbb{R}}^{N_s\times K\times 1}$$

(5)

where ${\varphi }_I\left(·\right)$ denotes the intensity embedding function and $\sigma \left(·\right)$ denotes the sigmoid activation and GELU activation. This branch allows local radiometric differences to adjust the contribution of neighbor points during feature aggregation. When neighbor points have similar geometry but different intensity responses, the attention embedding can adaptively modulate their influence.

After the three branches are obtained, the vector feature difference $∆F_{N_s}$ and the structural positional embedding ${PE}_{N_s}$ are first combined by element-wise summation. The intensity attention embedding $∆I_{N_s}$ is then used to modulate the combined local representation. Weighted max pooling is applied over the K neighboring points:

$${ R}_{N_s}={MaxPool}_K\left(∆I_{N_s}⨀\left(∆F_{N_s}+{PE}_{N_s}\right)\right), { R}_{N_s}\in {\mathbb{R}}^{N_s\times C_s}$$

(6)

where $⨀$ denote denotes element-wise multiplication. The max-pooling operation then selects the most discriminative response from each local neighborhood, producing the local structural response $R_{N_s}$.

Finally, the local structural response is used to update the center-point feature through a residual connection, followed by a feed-forward refinement network:

$$F_s^{out}=FFN\left(R_{N_s}\right), F_s^{out}\in {\mathbb{R}}^{N_s\times C_s}$$

(7)

The FFN consists of batch normalization, linear projection, GELU activation, and another linear layer, as shown in Figure 2. This residual refinement keeps the input and output feature dimensions unchanged. The resulting $F_s^{out}$ is the railway structure-aware point feature of the s-th encoding stage. Through this process, local feature differences, structural positional embeddings, and intensity-guided neighborhood relationships are integrated into a unified point-wise representation. This improves the discrimination of fine railway components with similar geometry.

3.3. Hierarchical Structural Context Aggregation

Railway point clouds have a strong hierarchical structure. At the local scale, rails, sleepers, ballast, poles, and other facilities are distinguished by fine geometric details and neighborhood relationships. At a larger scale, these objects are not randomly distributed. Rails extend continuously along the corridor, sleepers appear repeatedly along the track, and poles, wires, and auxiliary facilities follow clear spatial arrangements. Therefore, reliable railway point cloud segmentation requires both local structural details and broader scene context.

Multi-level encoder features provide complementary representations for railway scene understanding. Features from shallow stages retain higher point resolution and are effective for preserving local geometric details, such as rail edges, sleeper boundaries, and small facility contours. In contrast, features from deeper stages have larger receptive fields, which helps represent the continuity of the railway corridor and the spatial relationships among trackside objects. Relying only on shallow features limits the representation of high-level semantic context, while using only deep features may weaken fine-grained railway structures due to progressive resolution reduction. Therefore, the hierarchical structural context aggregation (HSCA) module is introduced to integrate features from different encoding stages, as shown in Figure 3.

Figure 3. Structure of the Hierarchical Structural Context Aggregation module. GELU denotes the Gaussian Error Linear Unit activation function.

The encoder produces four railway structure-aware feature maps: $F_1\in {\mathbb{R}}^{{\displaystyle \frac{N}{2}}\times 64}$, $F_2\in {\mathbb{R}}^{{\displaystyle \frac{N}{4}}\times 128}$, $F_3\in {\mathbb{R}}^{{\displaystyle \frac{N}{8}}\times 256}$, and $F_4\in {\mathbb{R}}^{{\displaystyle \frac{N}{16}}\times 512}$. These features have different point resolutions and channel dimensions. In HSCA, each feature map is first processed by a $1\times 1$ convolutional projection implemented by a Linear-BN-GELU block. This operation adjusts the channel representation of each scale before resolution recovery. The projected features are then upsampled to the original point resolution $N$:

$$F_s^{\prime }=U\left(\psi \left(F_s\right)\right), s=1, 2, 3, 4$$

(8)

where $\psi \left(·\right)$ denotes the Linear-BN-GELU projection, and $U\left(·\right)$ denotes the upsampling operation implemented by nearest-neighbor interpolation, which restores each projected feature map from the downsampled point set to the original point set by assigning each original point the feature of its nearest point in the corresponding downsampled set. After projection and upsampling, the restored multi-scale features are denoted as $F_1^{\prime }\in {\mathbb{R}}^{N\times 64}$, $F_2^{\prime }\in {\mathbb{R}}^{N\times 128}$, $F_3^{\prime }\in {\mathbb{R}}^{N\times 256}$, and $F_4^{\prime }\in {\mathbb{R}}^{N\times 512}$. Because all four features have been restored to the same point number $N$, they can be aligned point by point. The four restored features are then concatenated along the channel dimension:

$$F_{cat}=Concat\left(F_1^{\prime },F_2^{\prime }, F_3^{\prime }, F_4^{\prime }\right), F_{cat}\in {\mathbb{R}}^{N\times \left(62+128+256+512\right)}$$

(9)

This concatenated feature contains structural information from all encoding levels. The shallow part provides detailed geometric boundaries, while the deeper part provides larger-scale semantic context. To fuse these heterogeneous features, $F_{cat}$ is further processed by another Linear-BN-GELU block:

$$F_{agg}=\psi \left(F_{cat}\right), F_{agg}={\mathbb{R}}^{N\times 256}$$

(10)

This operation transforms the concatenated feature $F_{cat}$ into the aggregated hierarchical structural context $F_{agg}$, which serves as a unified point-wise representation of multi-level structural context. This output provides the decoder with integrated features that combine fine geometric details from shallow stages and broader railway-scene context from deeper stages.

3.4. Intensity-Aware Adaptive Attention

LiDAR intensity provides radiometric information that complements the geometric representation of railway point clouds. In railway scenes, several categories are close to each other in space and may also show similar local geometry. For example, rails, sleepers, ballast, metallic facilities, vegetation, and ground can appear in adjacent neighborhoods, but their surface materials and reflectance responses are different. If local aggregation relies only on geometric offsets and feature differences, such radiometric distinctions may not be fully used. Therefore, intensity-aware adaptive attention (IA-Att) is introduced to guide the neighborhood aggregation process.

The IA-Att module follows the shared-neighborhood design described in Section 3.2. The same KNN index is used for the feature, coordinate, and intensity branches, so the intensity relationship is computed over exactly the same local neighborhood as the geometric and feature relationships. For each center point and its neighboring points, the relative intensity difference $∆I_{ij}$ has been defined in Equation (4), and the corresponding intensity attention embedding $∆I_{N_s}$ has been obtained in Equation (5).

During local aggregation, $∆I_{N_s}$ is used as a neighborhood-level modulation weight. Since its channel dimension is 1, it is broadcast along the feature dimension to match the combined local representation $∆F_{N_s}+{ PE}_{N_s}$. The modulated local response is then obtained through the weighted max-pooling operation in Equation (6). In this way, the intensity branch does not replace geometric encoding. Instead, it adjusts the contribution of each neighboring point before pooling, while the local structure is still described by the vector feature difference and the structural positional embedding.

This design has two advantages for railway point cloud segmentation. First, relative intensity differences describe local radiometric contrast within a neighborhood, which is more suitable than using absolute intensity alone because LiDAR intensity can be affected by scanning distance, incidence angle, and local acquisition conditions. Second, the attention embedding provides an adaptive weighting mechanism for neighboring points. When neighboring points have similar geometric layout but different intensity responses, their influence can be adjusted during aggregation. This is useful for distinguishing categories such as rail and ballast, sleeper and track bed, or metallic facilities and vegetation.

The output of IA-Att is therefore a set of intensity-guided neighborhood weights, rather than an independent semantic feature. These weights are integrated into the RLE module through Equation (6), and the resulting local structural response is further refined by the residual update in Equation (7). By embedding intensity information into local aggregation, RailDLA-Net can use both structural and radiometric cues while keeping the feature dimensions consistent with the encoder stage.

3.5. Loss Function

The category distribution of railway point clouds is usually highly imbalanced, where large background categories dominate the point number, while key facility categories are sparse and elongated. If only the final classification loss is used for supervision, the model tends to favor majority classes such as ground, vegetation, and ballast, weakening its ability to recognize rails, sleepers, and poles. To this end, this study constructs a geometry–semantics collaborative optimization objective to jointly constrain final predictions, intermediate semantic representations, and local spatial structures, as illustrated in Figure 4.

Figure 4. Geometry–semantics collaborative optimization strategy for RailDLA-Net.

First, class-weighted cross-entropy is adopted as the main segmentation loss.

$${\mathcal{L}}_{seg}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{c=1}^C}{\omega }_c{y}_{ic}\log p_{ic}$$

(11)

where $C$ denotes the number of classes, ${\omega }_c$ denotes the class weight of class $c$, and $y_{ic}$, $p_{ic}$ denote the ground-truth label and predicted probability, respectively. The class weights are determined according to class frequencies in the training set to increase the optimization contribution of minority railway facility classes.

Second, to improve the trainability of the deep network, auxiliary semantic supervision is introduced at multiple encoding stages. The output feature of each stage is mapped into the category probability space and supervised by cross-entropy loss with the ground-truth labels at the corresponding scale.

$${\mathcal{L}}_{sem}={\displaystyle \frac{1}{S}}{\displaystyle \sum _{s=1}^S}{\mathcal{L}}_{ce}^s$$

(12)

where $S$ denotes the number of encoding stages involved in auxiliary supervision, and ${\mathcal{L}}_{ce}^s$ denotes the semantic supervision loss at the $s$-th stage. This design encourages intermediate features to acquire category-discriminative ability earlier and reduces the attenuation of small-object semantic information during deep propagation.

Third, to preserve railway linear structures and local geometric relationships, this study introduces a spatial distribution constraint. This constraint requires relative relationships in the feature space to be consistent with neighborhood displacements in the real 3D space:

$${\mathcal{L}}_{spa}={\displaystyle \frac{1}{S}}{\displaystyle \sum _{s=1}^S}{\displaystyle \frac{1}{N_{S}K}}{\displaystyle \sum _{i=1}^{N_S}}\sum _{j\in {\mathsf{N}}_s\left(i\right)}{‖\left[g_s\left(F_j^s\right)-g_s\left(F_i^s\right)\right]-{\gamma }_s\left(p_j^s-p_i^s\right)‖}_2^2$$

(13)

where $F_i^s\in {\mathbb{R}}^{C_s}$ and $F_j^s\in {\mathbb{R}}^{C_s}$ denote the features of the center point and its neighboring point at the s-th stage, respectively; $p_i^s, p_i^s\in {\mathbb{R}}^3$ denote their 3D coordinates; $N_s$ denotes the number of points at the s-th stage; and K denotes the number of neighboring points. The function $g_s\left(·\right)$ is constructed by two linear layers with Batch Normalization and GELU activation, following the mapping:

$$g_s: {\mathbb{R}}^{C_s}\to {\mathbb{R}}^3.$$

Therefore, $g_s\left(F_j^s\right)$ and $g_s\left(F_i^s\right)$ have the same dimension as the coordinate displacement $p_j^s-p_i^s$, which makes the two relation terms compatible in Equation (13). The coefficient ${\gamma }_s$ is a stage-specific scale normalization factor used to normalize coordinate displacements at different resolutions. In the four-stage encoder, it is set to ${\gamma }_s\in \left\{{\displaystyle \frac{1}{1.6}},{\displaystyle \frac{1}{3.2}}, {\displaystyle \frac{1}{6.4}}, {\displaystyle \frac{1}{12.8}} \right\}$, corresponding to stages 1 to 4, respectively.

The spatial distribution constraint guides the network to learn feature distributions consistent with real railway geometric structures, which is particularly beneficial for preserving the structural continuity of rails, sleepers, and catenary facilities.

Finally, the overall loss function is defined as:

$$\mathcal{L}={\mathcal{L}}_{seg}+\alpha {\mathcal{L}}_{sem}+\beta {\mathcal{L}}_{spa}$$

(14)

In the early training stage, larger auxiliary-supervision weights provide stable optimization directions for the deep network. As training proceeds, the auxiliary-supervision weights gradually decrease, allowing the model to focus more on the final point-wise semantic prediction.

Through the collaboration of main segmentation supervision, hierarchical semantic supervision, and spatial distribution supervision, RailDLA-Net achieves more stable and fine-grained semantic segmentation in complex railway point cloud scenes.

4. Results and Analysis

4.1. Implementation Details

In this section, we evaluate the performance of RailDLA-Net on the Rail3D [37] and WHU-Railway3D [35] datasets. For the Rail3D dataset, the training, validation, and testing sets were divided at a ratio of 13:6:6. For the WHU-Railway3D dataset, the corresponding ratio was 3:1:1. The datasets were first partitioned at the scene level, and the raw LiDAR intensity values of each scene were then normalized to [0, 1] using min-max normalization.

All experiments were conducted on an NVIDIA^® GeForce RTX™ 5090 D GPU (NVIDIA Corporation, Santa Clara, CA, USA) machine using PyTorch (version 2.7.0) framework and CUDA acceleration (version 12.9). For the network configuration, RailDLA-Net contains four encoding stages. The feature dimensions of the four stages were set to 64, 128, 256, and 512, respectively. The corresponding numbers of residual local encoding blocks were set to 20, 20, 60, and 20. In each stage, KNN was used to construct local neighborhoods, and the number of neighboring points was set to K = 24. The grid sizes of the four stages were set to 0.08, 0.16, 0.32, and 0.64, respectively, and the maximum number of input points was set to 60,000. In the feature projection, positional embedding, intensity-aware attention, and feed-forward layers, Batch Normalization was used as the normalization layer, and GELU was used as the activation function. The model was optimized using the AdamW optimizer with an initial learning rate of $4\times {10}^{-3}$. A cosine learning rate decay strategy was adopted during training after 10 warm-up epochs, and the learning rate was gradually reduced until convergence. The batch size was set to 8, and the network was trained for 100 epochs.

To quantitatively analyze the performance of the proposed architecture, overall accuracy (OA), mean Accuracy (mAcc), per-class intersection over union (IoUs), and mean IoU (mIoU) are used as evaluation metrics as follows:

$$OA={\displaystyle \frac{{\sum }_{i=1}^n{TP}_i}{N}}$$

(15)

$$mACC={\displaystyle \frac{{\sum }_{i=1}^n{Acc}_i}{n}}$$

(16)

$${IoU}_i={\displaystyle \frac{{TP}_i}{{TP}_i+{FP}_i+{FN}_i}}$$

(17)

$$mIoU={\displaystyle \frac{{\sum }_{i=1}^n{IoU}_i}{n}}$$

(18)

where $TP$ denotes the number of true positive samples, $FP$ denotes the number of false positive samples, $FN$ denotes the number of false negative samples, $i$ denotes the $i$-th semantic class, $n$ denotes the number of total semantic classes and $N$ denotes the number of total points.

4.2. Experiment Results

Table 2 presents the quantitative results on the Rail3D dataset. RailDLA-Net achieves the competitive performance, with 96.89% OA, 91.92% mAcc, and 88.44% mIoU. Compared with Point Transformer V3, RailDLA-Net improves mAcc and mIoU by 1.77 and 0.90 percentage points, respectively. The relatively small mIoU margin may be related to the competitive performance of Point Transformer V3 and the relatively regular scene structure of Rail3D. Under this setting, RailDLA-Net obtains a slightly higher mAcc and comparable overall segmentation performance. Compared with DeepLA-Net, RailDLA-Net improves mIoU from 70.89% to 88.44%, indicating that the railway-oriented design contributes to the overall segmentation performance. However, Table 2 also shows that RailDLA-Net does not achieve the best IoU for every semantic category. This may be related to the different feature preferences of different methods and the large variations among railway-scene categories in geometric scale, point density, spatial continuity, and contextual dependency. For example, Point Transformer V3 obtains higher IoU values for poles, wires, signaling, fence, and building, while CamPoint performs better on ground and vegetation. In contrast, RailDLA-Net achieves the highest mAcc and mIoU among the compared methods and obtains the best IoU for installation, a sparse and challenging railway-related category. Therefore, these results support the conclusion mainly from the perspective of overall segmentation performance and category-level balance, rather than absolute superiority in each individual category.

Table 2. Quantitative results on the Rail3D dataset. Bold values indicate the best results achieved by RailDLA-Net.

Method	OA	mAcc	mIoU	Ground	Vegetation	Rail	Poles	Wires	Signaling	Fence	Installation	Building
DeLA [43]	95.71	70.22	55.49	95.06	90.35	81.8	47.51	45.37	9.97	84.56	34.54	10.25
Point Transformer (Version 3) [25]	96.55	90.15	87.54	94.73	90.96	86.95	95.67	99.44	96.41	89.86	34.68	99.17
CamPoint [45]	97.71	74.7	64.02	97.48	95.61	94.36	44.88	95.18	41.07	88.99	18.59	0
DeepLA-Net [26]	96.66	74.22	70.89	95.17	91.25	93.41	85.98	97.38	0	83.27	0	91.51
RailDLA-Net	96.89	91.92	88.44	95.27	91.73	94.22	93.47	98.52	88.87	88.08	48.65	97.14

To further evaluate the visual performance, Figure 5 presents qualitative comparisons between RailDLA-Net and the compared methods on representative Rail3D scenes. The results show that all methods can identify the main components in railway scenes, including ground, vegetation, rails, poles, and wires. In the highlighted regions, several compared methods produce fragmented predictions or local confusion near the boundaries between vegetation and ground, as well as around railway facilities. RailDLA-Net gives more continuous predictions for rails, poles, and installation objects, and reduces local misclassification in these areas. Figure 6 compares the error maps of RailDLA-Net and Point Transformer V3, which achieves the best mIoU among the compared methods in Table 2. The highlighted regions show fewer misclassified points along the railway corridor and adjacent background areas in the result of RailDLA-Net. However, errors remain near object boundaries and sparse facilities, indicating that detailed segmentation in railway scenes still requires further improvement.

Figure 5. Qualitative results on the Rail3D dataset. The blue circles indicate misclassified regions of RailDLA-Net, while the red circles indicate correctly classified regions. “PT” indicates the abbreviation of “Point Transformer”.

Figure 6. Visual comparison of segmentation error maps on the Rail3D dataset. Red points indicate misclassified regions, while blue points indicate correctly classified regions. The yellow dashed circles highlight regions near the railway tracks, where our method produces fewer misclassified points than the Point Transformer V3 method.

Table 3 reports the quantitative results on the WHU-Railway3D dataset. RailDLA-Net achieves the highest mAcc and mIoU, reaching 91.84% and 83.63%, respectively. Compared with Point Transformer V3, RailDLA-Net improves mAcc and mIoU by 5.26 and 3.47 percentage points, while its OA is 1.38 percentage points lower. This indicates that RailDLA-Net performs better in terms of average category accuracy, whereas Point Transformer V3 has higher overall point-wise accuracy. For class-wise IoU, RailDLA-Net obtains the best results on masts, support devices, building, and others, while Point Transformer V3 performs better on several linear structures and large background classes. These results show that RailDLA-Net improves average category performance on WHU-Railway3D, but further improvement is still needed for some individual categories.

Table 3. Quantitative results on the WHU-Railway3D dataset. Bold values indicate the best results achieved by RailDLA-Net.

Method	OA	mAcc	mIoU	Rails	Track Bed	Masts	Support Devices	Overhead Lines	Fences	Poles	Vegetation	Building	Ground	Others
DeLA [43]	83.31	80.58	67.09	64.78	68.63	77.4	67.11	87.45	56.16	57.29	79.97	60.51	79.1	39.51
Point Transformer (Version 3) [25]	93.74	86.58	80.16	86.44	84.48	86	66.58	98.09	89.8	80.15	99.18	35.97	96.04	59.31
CamPoint [45]	91.61	85.95	78.18	73.21	92.41	87.86	71.79	89.56	61.86	71.31	84.02	63.4	90.06	74.5
DeepLA-Net [26]	88.65	83.06	72.4	63.78	88.16	87.77	71.79	89.56	61.86	71.31	84.02	63.4	90.06	74.5
RailDLA-Net	92.36	91.84	83.63	71.26	91.89	94.81	84.43	93.45	63.91	78.35	84.14	81.12	89.13	87.44

The difference in improvement margins between Rail3D and WHU-Railway3D can be mainly explained by dataset characteristics and baseline performance. On Rail3D, Point Transformer V3 already performs well on several categories, so the improvement space for RailDLA-Net is relatively limited. The gain in Table 2 is therefore mainly reflected in overall category balance. In contrast, WHU-Railway3D contains more categories, denser railway facilities, and stronger similarity between adjacent components, which makes category discrimination more difficult. As shown in Table 3, RailDLA-Net brings clearer improvements for structure-dependent and context-related classes, such as masts, support devices, building, and others, which contributes to the larger gains in mAcc and mIoU.

Figure 7 presents qualitative results on the WHU-Railway3D dataset. Compared with Rail3D, WHU-Railway3D contains more semantic categories and denser railway facilities, including track bed, masts, support devices, overhead lines, fences, and poles. In these complex scenes, several compared methods show confusion among adjacent railway facilities or miss small structures around the track area. RailDLA-Net gives clearer predictions for masts, support devices, and surrounding background objects, which is consistent with its higher IoU values for these categories in Table 3. Figure 8 further compares the error maps of RailDLA-Net and Point Transformer V3. The result of RailDLA-Net contains fewer errors in the highlighted facility region, while Point Transformer V3 still shows lower error density in some large background areas. This visual comparison indicates that RailDLA-Net performs better on several facility categories in WHU-Railway3D, but its predictions for long linear objects and large background regions still require further improvement.

Figure 7. Qualitative results on the WHU-Railway3D dataset. The blue circles indicate misclassified regions of RailDLA-Net, while the red circles indicate correctly classified regions. “PT” indicates the abbreviation of “Point Transformer”.

Figure 8. Visual comparison of segmentation error maps on the WHU-Railway3D dataset. Red points indicate misclassified regions, while blue points indicate correctly classified regions. The yellow dashed circles highlight building points that are correctly classified by our method.

4.3. Ablation Studies

In this subsection, a series of ablation experiments were conducted to evaluate the effects of key parameter settings and loss components on the performance of RailDLA-Net.

Table 4 investigates the effect of grid size. The grid size of 0.08 achieves the best mIoU of 88.44%, indicating a better balance between preserving railway geometric details and reducing redundant local noise. A smaller grid size retains more points but may introduce redundancy, while a larger grid size may lose fine structural information.

Table 4. Effect of grid size on the performance of RailDLA-Net.

Grid Size	OA	mAcc	mIoU
0.04	96.25	72.61	68.26
0.08	96.89	91.92	88.44
0.12	96.92	89.74	87.01

As shown in Table 5, the neighborhood size K has a clear influence on segmentation performance. When K = 16, the neighborhood range is relatively limited, and the model achieves 70.02% mIoU. Increasing K to 20 improves the mIoU to 79.33%, indicating that a larger neighborhood helps capture more stable local geometric patterns. The best performance is obtained when K = 24, with 96.89% OA, 91.92% mAcc, and 88.44% mIoU. When K is further increased to 28, the mIoU decreases to 73.45%. This may be because an overly large neighborhood introduces points from adjacent semantic categories, especially around boundaries between rails, sleepers, ballast, and ground. Therefore, K = 24 is used as the default neighborhood size in this study.

Table 5. Sensitivity analysis of neighborhood size K on Rail3D.

K Value	OA	mAcc	mIoU
16	96.37	78.64	70.02
20	97	84.99	79.33
24	96.89	91.92	88.44
28	96.91	77.88	73.45

Table 6 compares different architecture scales. The Large architecture obtains the best performance, with 91.92% mAcc and 88.44% mIoU. This result shows that sufficient network capacity is important for capturing complex railway structures and high-level semantic context.

Table 6. Effect of architecture scale on the performance of RailDLA-Net, where the Small, Middle, and Large architectures are defined by the numbers of blocks as 0, [10, 10, 30, 10], and [20, 20, 60, 20], respectively.

Architecture	OA	mAcc	mIoU
Small	96.47	76.43	69.8
Middle	96.88	90.72	86.37
Large	96.89	91.92	88.44

Table 7 evaluates the influence of the maximum number of input points. The performance improves as the number increases from 30,000 to 60,000, while further increasing it to 75,000 reduces mAcc and mIoU. This suggests that 60,000 points provide a suitable trade-off between structural information preservation and redundant background interference.

Table 7. Effect of the maximum number of input points on the performance of RailDLA-Net.

Max Points	OA	mAcc	mIoU
30,000	96.05	69.45	63.52
45,000	96.8	73.98	70.75
60,000	96.89	91.92	88.44
75,000	97.11	87.5	83.16

Table 8 analyzes the contribution of different loss terms in the proposed optimization design. Using only the main segmentation loss provides a baseline mIoU of 85.68%. After adding $L_{sem}$, the mIoU increases to 87.84%, suggesting that hierarchical semantic supervision is helpful for learning multi-level semantic representations. When $L_{spa}$ is used without $L_{sem}$, the mIoU decreases to 72.96%, indicating that spatial distribution supervision may be less effective when it is not supported by sufficient semantic guidance. By combining the main segmentation loss, $L_{sem}$, and $L_{spa}$, the model achieves the highest mIoU of 88.44%. These results suggest that semantic auxiliary supervision and spatial distribution supervision can work complementarily in the proposed optimization design.

Table 8. Ablation study of different loss components in RailDLA-Net.

${\mathit{L}}_{\mathit{s}\mathit{e}\mathit{g}}$	${\mathit{L}}_{\mathit{s}\mathit{e}\mathit{m}}$	${\mathit{L}}_{\mathit{s}\mathit{p}\mathit{a}}$	OA	mAcc	mIoU
√			97.06	89.76	85.68
√	√		97.08	93.13	87.84
√		√	96.69	79.33	72.96
√	√	√	96.89	91.92	88.44

4.4. Cross-Scene Generalization

To evaluate the cross-scene transferability of different methods, we conduct generalization experiments on the Rail3D dataset. Specifically, all methods are trained on the Hungary scene and directly tested on the France scene, without using any France data during training. The quantitative results are reported in Table 9.

Table 9. Quantitative results of cross-scene generalization from the Hungary scene to the France scene on the Rail3D dataset.

Method	OA	mAcc	mIoU	Ground	Vegetation	Rail	Poles	Wires	Signaling	Fence	Installation	Building
DeLA [43]	94.48	51.59	41.86	86.93	93.02	48.48	76.74	71.6	0	0	0	0
Point Transformer (Version 3) [25]	87.53	24.36	22.96	83.51	85.03	0.05	25.25	12.78	0	0	0	0
CamPoint [45]	88.92	39.75	34.37	76.47	86.65	0.85	63.48	76.29	3.36	2.23	0	0
DeepLA-Net [26]	93.08	63.33	40.87	84.84	90.36	55.84	72.94	61.82	0	2.04	0	0
RailDLA-Net	94.04	56.42	46.6	86.61	92.66	44.7	86.38	73.15	35.86	0	0	0

As shown in Table 9, the proposed method achieves the highest mIoU of 46.60% in this setting. Compared with DeLA, DeepLA-Net, CamPoint, and Point Transformer V3, the mIoU is improved by 4.74%, 5.73%, 12.23%, and 23.64%, respectively. Although DeLA obtains a slightly higher OA, its lower mIoU suggests that the result may be influenced by dominant classes with large spatial coverage. In contrast, RailDLA-Net shows relatively balanced performance across categories, especially on railway objects such as poles, wires, and signaling. For the signaling class, RailDLA-Net obtains an IoU of 35.86%, while the compared methods achieve lower or zero IoU. These results suggest that RailDLA-Net can retain useful semantic discrimination when transferred to another scene. Since this experiment is also conducted on Rail3D, it provides supplementary evidence for the effectiveness of RailDLA-Net on Rail3D beyond the standard fixed split evaluation in Table 2. However, all methods still obtain zero IoU on several categories, including fence, installation, and building, indicating that generalization for sparse or scene specific categories remains difficult.

5. Discussion

The experiments on the Rail3D dataset show that RailDLA-Net achieves competitive segmentation accuracy for railway point clouds, especially in terms of mAcc and mIoU. However, accuracy alone is not enough for practical railway applications. The network has a deep architecture, which creates challenges for industrial deployment and large-scale processing of MLS data along long railway corridors. Therefore, besides segmentation accuracy, we also evaluated the efficiency of RailDLA-Net on Rail3D dataset, including parameter size, FLOPs, inference speed, and memory usage. The results are listed in Table 10.

Table 10. Comparison of model complexity and inference efficiency. “M” indicates Million.

Methods	Parameters (M)	FLOPs (G)	FPS (Points/s)	Memory (G)
DeLA [43]	7.08	0.96	10.3 M	7
Point Transformer (Version 3) [25]	46.17	2.66	0.36 M	11
CamPoint [45]	15.78	3.7	0.79 M	17
DeepLA-Net [26]	30.3	42.7	4.93 M	12
RailDLA-Net	33.36	45.2	0.42 M	15

As shown in Table 10, RailDLA-Net has 33.36 M parameters, requires 45.20 G FLOPs, processes 0.42 M points per second, and uses 15 G memory. Combined with the segmentation results in Table 2, these results show that RailDLA-Net achieves a reasonable balance between segmentation accuracy and model complexity on Rail3D. Compared with lightweight methods such as DeLA and CamPoint, RailDLA-Net requires more computation but obtains higher mAcc and mIoU, indicating that the deeper local aggregation design is beneficial for railway point cloud representation. Compared with Point Transformer V3, RailDLA-Net uses fewer parameters and achieves a slightly higher inference speed, while maintaining competitive segmentation performance. This suggests that the proposed framework does not simply increase model size, but uses a deep local aggregation structure to strengthen the representation of railway geometric details, component relationships, and scene context.

From the perspective of practical applications, RailDLA-Net is more suitable for offline railway point cloud analysis, high precision semantic mapping, infrastructure inventory, and inspection tasks, where segmentation quality is often more important than strict real-time inference. The method may be less suitable for onboard or real-time perception systems because its deep architecture still introduces relatively high FLOPs and memory consumption. For large-scale MLS data along railway corridors, partition-based inference can be used to reduce memory pressure during processing. Future work will focus on reducing computational cost through lightweight aggregation, efficient neighborhood search, dynamic point sampling, sparse computation, and model compression, while preserving the ability to model railway structures and contextual relationships.

6. Conclusions

This study proposed RailDLA-Net, an intensity-aware deep local aggregation framework for railway point cloud semantic segmentation. The framework combines railway structure-guided residual local encoding, hierarchical structural context aggregation, intensity-aware adaptive attention, and a geometry–semantics collaborative objective. These components are used to improve the representation of railway structures, local context, category imbalance, and use of LiDAR intensity. Experiments on Rail3D and WHU-Railway3D showed that RailDLA-Net achieved competitive segmentation results under different railway scene conditions, with 96.89% OA, 91.92% mAcc, and 88.44% mIoU on Rail3D, and 92.36% OA, 91.84% mAcc, and 83.63% mIoU on WHU-Railway3D. The ablation studies confirmed the contribution of the main parameter settings and loss components to the final performance. The cross-scene experiment further showed that RailDLA-Net retained useful segmentation ability under scene transfer, indicating its potential for broader railway scene analysis. Considering its computational and memory costs, the method is better suited to offline high-precision point cloud analysis, infrastructure inventory, and intelligent inspection than strict real-time deployment. In future work, more effective strategies for sparse object representation and boundary refinement will be explored to further improve fine-grained semantic understanding in complex railway scenes. Lightweight model design and broader cross-scene validation will also be considered to enhance the efficiency and robustness of the proposed framework.