LMFusion: Breaking the Computational Barrier for Multimodal Classification in Remote Sensing

Highlights

What are the main findings?

• We propose LMFusion, an efficient multimodal remote sensing classification framework that integrates linear-complexity cross-attention for bidirectional HSI-LiDAR feature interaction and Mamba-based state space modeling for spatial-spectral representation learning.
• Experimental results on multiple multimodal remote sensing datasets demonstrate that LMFusion achieves competitive classification performance while providing an effective feature fusion strategy for hyperspectral and LiDAR data.

What is the implication of the main finding?

• The proposed framework shows that effective multimodal feature interaction can be achieved with reduced computational burden, making multimodal remote sensing classification more suitable for resource-constrained scenarios.
• The introduced selective quantization-aware optimization further improves the compactness of the model, providing potential support for future deployment on low-bit or edge-computing hardware platforms.

Abstract

Multi-modal land cover classification plays an important role in remote sensing applications such as urban monitoring and environmental analysis. By integrating complementary information from hyperspectral imagery (HSI) and LiDAR data, multimodal learning can significantly improve classification performance. However, existing Transformer-based fusion methods often suffer from high computational complexity and inefficient cross-modal interaction modeling, which limits their applicability in resource-constrained scenarios. To address these challenges, we propose LMFusion, an efficient framework for multimodal feature learning. Specifically, LMFusion enables efficient bidirectional feature interaction through a linear-complexity cross-attention mechanism and enhances long-range spatial-spectral representation learning with Mamba-based state space modeling, thereby achieving effective multimodal dependency modeling with linear computational complexity. In addition, a selective quantization-aware optimization strategy is introduced to support multiple bit-width settings (down to 1-bit), yielding a more compact and efficient model while improving representation robustness under low-bit constraints. Extensive experiments on the Houston2013, MUUFL, and Augsburg datasets demonstrate the effectiveness of LMFusion. It achieves overall accuracies of 95.84%, 94.95%, and 99.05%, respectively, consistently outperforming representative multimodal classification methods and showing strong potential for accurate and efficient multimodal remote sensing classification.

1. Introduction

Multi-modal land cover classification is a fundamental problem in remote sensing and plays a crucial role in real-world applications, including urban planning, environmental monitoring, and military target recognition [1,2,3,4]. With the rapid development of Earth observation technologies, heterogeneous sensing modalities such as hyperspectral imagery, LiDAR, and SAR have become increasingly accessible. These modalities provide complementary information about the same scene, where spectral-rich data characterize material properties, while active sensing modalities capture geometric and structural cues that are robust to illumination variations and adverse atmospheric conditions [5,6,7,8]. Effectively integrating such heterogeneous information has therefore become a central challenge in remote sensing analysis [9,10,11,12].

Despite notable progress achieved by deep learning–based multi-modal approaches, several inherent challenges remain unresolved [2,13]. Remote sensing data are typically high-dimensional, which often leads to severe feature redundancy and increased computational burden [1,5,11]. Moreover, accurate land cover classification requires modeling long-range contextual dependencies as well as complex interactions across modalities over large spatial extents [5,11,14]. Existing convolutional, transformer-based, and hybrid frameworks have demonstrated strong representational capability, but they frequently suffer from high computational complexity, limited efficiency in cross-modal interaction modeling, and restricted deployability under constrained computational resources [13].

More specifically, while convolutional architectures excel at capturing local spatial–spectral patterns, their inherently limited receptive fields restrict the modeling of long-range dependencies across heterogeneous modalities. Transformer-based approaches alleviate this issue through global attention mechanisms; however, their quadratic computational complexity with respect to the number of tokens often results in substantial memory consumption and computational overhead when processing high-dimensional remote sensing data. Consequently, an efficient multimodal classification framework should not only enable effective cross-modal interaction, but also support long-range spatial–spectral dependency modeling under affordable computational complexity.

To address these limitations, recent studies have explored more efficient modeling paradigms that emphasize linear-complexity computation and structured sequence representations [7,10]. These approaches provide a promising direction for capturing long-range dependencies while mitigating the quadratic computational cost inherent in conventional global interaction mechanisms [8,12,15]. Nevertheless, in the context of multimodal remote sensing, it remains challenging to simultaneously achieve effective cross-modal interaction, computational efficiency, and deployment-friendly model design [6,7,10]. Existing works often investigate efficiency-oriented architectures and practical deployment considerations separately, leaving a gap between high-performance multimodal learning and real-world applicability [16,17].

Accordingly, LMFusion is designed to address this gap through three coordinated components. First, Linear Cross Attention is introduced to perform bidirectional information exchange between HSI and LiDAR features with linear-complexity computation, thereby improving cross-modal interaction efficiency. Second, SS2D-based state-space modeling is adopted to enhance long-range spatial–spectral dependency modeling without relying on computationally expensive global self-attention. Third, quantization-aware training is incorporated to improve the deployment potential of the framework under low-bit settings. These components jointly support the goal of achieving accurate, efficient, and deployment-friendly multimodal remote sensing classification.

In this work, we focus on efficient multi-modal land cover classification under realistic resource constraints, aiming to bridge the gap between expressive cross-modal representation learning and lightweight deployment. Our study is centered on the following aspects:

• We propose LMFusion, an efficient multimodal classification framework that integrates linear-complexity cross-modal interaction with Mamba-based long-range spatial–spectral modeling, enabling effective HSI–LiDAR feature fusion under limited computational budgets.
• We develop a quantization-aware optimization scheme for LMFusion, supporting ultra-low-bit deployment while preserving feature learning and improving model compactness and inference efficiency.
• Extensive experiments on multiple multimodal benchmark datasets demonstrate that the proposed method consistently outperforms representative state-of-the-art approaches, while ablation studies further verify the effectiveness of Linear Cross Attention, SS2D-based state-space modeling, and quantization-aware training.

2. Related Works

2.1. Multi-Modal Learning in Remote Sensing

With the rapid development of high-resolution remote sensing (RS) sensors, the integration of heterogeneous data sources, such as hyperspectral imagery (HSI), LiDAR, and SAR, has become an essential strategy for accurate land-cover classification [3,4,18]. Traditional multimodal fusion methods mainly rely on hand-crafted feature extraction, morphological profiles, or shallow feature-level fusion strategies, which can exploit complementary information but have limited representation capacity for complex spatial–spectral patterns and heterogeneous modality relationships [9,19]. CNN-based methods are widely used to extract local spatial and spectral features and have achieved promising performance in HSI–LiDAR classification [1,16]. However, convolutional operations are inherently local, which limits their ability to model long-range dependencies and complex cross-modal interactions.

Transformer-based fusion architectures have been introduced to capture global contextual information and modality correlations through attention mechanisms [11,12,20]. Nevertheless, standard attention-based fusion usually suffers from quadratic computational complexity with respect to the number of tokens, which restricts its efficiency and scalability for high-dimensional remote sensing data. Linearized attention mechanisms provide a promising solution by replacing standard softmax attention with kernel-based or feature-mapping-based formulations, reducing the complexity from $O\left(N^2\right)$ to $O\left(N\right)$ while preserving global interaction capability [21]. Liao et al. introduced MambaHL, a hybrid model combining state-space dynamics with convolutional backbones to better capture long-range dependencies while maintaining linear-time complexity. Building on the principles of recurrent-like modeling, state-space networks such as Mamba and its two-dimensional extensions (SS2D) reformulate sequence processing as continuous-time state evolution, parameterized by transition matrices (A,B,C). This formulation enables efficient modeling of spatial and temporal correlations without relying on dense attention operations. However, most of these works are restricted to unimodal vision or natural language data, leaving their potential for multimodal RS fusion largely unexplored [7].

To overcome the above limitations, recent research has sought to combine the efficiency of state-space modeling with cross-modal feature alignment for lightweight multimodal fusion [13]. These advances provide a promising foundation for lightweight cross-modal architectures that maintain both representational richness and computational efficiency. Motivated by these findings, our framework integrates Linear Cross Attention with SS2D-based state-space modeling to achieve bidirectional cross-modal interaction and long-range dependency modeling between HSI and LiDAR modalities under linear complexity. Different from existing multimodal fusion methods that mainly rely on CNN-based local feature extraction, Transformer-based global attention, or state-space modeling alone, the proposed LMFusion combines these two mechanisms into a unified lightweight framework, aiming to improve cross-modal interaction and long-range spatial–spectral representation learning with reduced computational cost.

2.2. Efficient and Lightweight Optimization

The ever-increasing resolution of RS data also raises practical challenges in model deployment and inference efficiency [10]. Deep multimodal models typically involve large parameter counts and memory footprints, hindering their use in real-time or embedded RS applications. Quantization techniques have thus emerged as a critical solution to compress models while maintaining accuracy [22,23]. Early post-training quantization (PTQ) methods directly discretized weights and activations but often resulted in severe performance degradation [22]. To mitigate this, Jacob et al. proposed Quantization-Aware Training (QAT), which incorporates fake quantization operations into the training pipeline to preserve gradient flow through discrete values [22]. Subsequent studies extended QAT to convolutional and Transformer-based architectures, enabling stable low-bit inference without noticeable loss in accuracy [24,25]. In the RS community, a few works have begun to investigate lightweight quantized models for spectral and spatial feature extraction [16,26]. Li et al. applied QAT to hyperspectral CNNs, demonstrating that low-bit models can significantly reduce computational costs while maintaining high classification precision. However, the integration of quantization with multimodal feature fusion remains underexplored-particularly when balancing long-range modeling, cross-modal consistency, and low-bit deployment [13,19,27]. In this context, our work adopts a Quantization-Aware Optimization strategy that jointly quantizes convolutional and linear layers during training, while selectively preserving full precision in cross-attention and state-space modules to ensure reliable cross-modal feature modeling [28].

In summary, existing multimodal RS methods have achieved notable progress in feature alignment and dependency modeling but still suffer from high computational overhead and limited deployability. Meanwhile, recent advances in state-space modeling and quantization provide opportunities for scalable and efficient architectures. Building upon these insights, our proposed framework integrates Linear Cross Attention, Mamba-based State Space Modeling, and Quantization-Aware Optimization, jointly enhancing cross-modal interaction, long-range reasoning, and lightweight inference for next-generation remote sensing classification.

3. Methodology

3.1. Problem Definition

In multimodal remote sensing classification, hyperspectral imagery (HSI) and LiDAR data provide complementary spectral and structural information for land-cover recognition. Given a pair of spatially aligned input patches, the hyperspectral and LiDAR inputs are represented as $X_h\in {\mathbb{R}}^{C_h\times H\times W}$ and $X_l\in {\mathbb{R}}^{C_l\times H\times W}$, respectively, where $C_h$ and $C_l$ denote the numbers of channels of HSI and LiDAR data, and H and W represent the spatial dimensions of the input patch.

In our experiments, the spatial size is fixed to $H=W=32$. The channel numbers vary across datasets such as Houston2013, MUUFL, and Augsburg.

The objective is to learn a multimodal classification function $\widehat{y}=f(X_h,X_l)$, where $f(·)$ denotes the proposed network and $\widehat{y}$ represents the predicted land-cover label.

3.2. Framework Overview

The overall architecture of the proposed framework is illustrated in Figure 1. The network is designed to jointly exploit the complementary information from hyperspectral imagery (HSI) and LiDAR data. The entire framework consists of three stages: initial feature encoding, cross-modal interaction modeling, and final classification. Specifically, the data flow starts from modality-specific Patch Embedding, proceeds through SS2D-based feature modeling and Linear Cross Attention-based cross-modal interaction, and finally reaches the classifier for land-cover prediction. The detailed step-by-step process is described below.

In Stage 1, the HSI and LiDAR inputs are first projected into a unified feature space through Patch Embedding layers. The embedded features of each modality are then processed by an SS2D block to perform preliminary feature modeling. Each SS2D block contains an SS2D-based state space modeling unit that captures spatial structural information and enhances local feature representations. The resulting modality-specific features are then used as the inputs to the following cross-modal interaction stage.

In Stage 2, the modality-specific features obtained from Stage 1 are first passed to a Linear Cross Attention module to model cross-modal interactions. In this module, a cross-attention operation allows features from HSI and LiDAR to exchange information. After this interaction, the features of each modality are further refined by another SS2D block. A second Linear Cross Attention module is then applied to further strengthen cross-modal fusion and establish deeper correlations between the two modalities.

In Stage 3, the fused features are further processed by an SS2D block for high-level semantic modeling. The resulting final representation is then fed into the classifier to generate the land-cover prediction.

To improve deployment efficiency on resource-constrained devices, a quantization-aware training strategy is adopted. Specifically, convolution and linear layers are quantized with multiple bit-width settings, enabling efficient inference while maintaining classification performance.

3.3. Linear Cross Attention

To capture cross-modal interactions between hyperspectral imagery and LiDAR features, we introduce a Linear Cross Attention module. Let $F_h\in {\mathbb{R}}^{N\times d}$ and $F_l\in {\mathbb{R}}^{N\times d}$ denote the HSI and LiDAR feature representations, respectively, where $N=H\times W$ is the number of spatial tokens and d is the feature dimension.

The query, key, and value matrices are obtained through linear projections as follows:

$$Q=F_h{W}_q,\hspace{1em}K=F_l{W}_k,\hspace{1em}V=F_l{W}_v,$$

(1)

where $W_q$, $W_k$, and $W_v$ are learnable projection matrices.

In conventional attention mechanisms [29], the attention output is computed as:

$$Z=\mathrm{Softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d}}}\right)V,$$

(2)

which requires computing pairwise token similarities and results in a quadratic complexity of $O\left(N^2\right)$.

To alleviate the quadratic computational complexity of conventional attention mechanisms, linear attention approximates the exponential similarity function using a kernel feature mapping $\varphi (·)$ [21,30]:

$$exp\left(Q_i^T{K}_j\right)\approx \varphi {\left(Q_i\right)}^T\varphi \left(K_j\right).$$

(3)

Thus, the attention output can be reformulated as:

$$Z_i={\displaystyle \sum _{j=1}^N}{\displaystyle \frac{\varphi {\left(Q_i\right)}^T\varphi \left(K_j\right)}{{\sum }_{j=1}^N\varphi {\left(Q_i\right)}^T\varphi \left(K_j\right)}}{V}_j.$$

(4)

By exploiting the associative property of matrix multiplication, the computation order can be reorganized as:

$$Z=\varphi \left(Q\right)\left(\varphi {\left(K\right)}^{T}V\right),$$

(5)

which reduces the computational complexity to $O\left(N{d}^2\right)$ while preserving effective cross-modal interaction between HSI and LiDAR features.

In practice, the proposed Linear Cross Attention module adopts a multi-head structure with 4 attention heads and incorporates residual connections for stable optimization. The feature dimensions of the three stages are set to 64, 128, and 256, respectively, and the hidden dimension is set to 128. These architectural settings are kept consistent across all datasets used in our experiments, while only the input channel numbers and the number of output classes are adjusted according to the dataset-specific HSI/LiDAR modalities and class definitions.

3.4. SS2D-Based Spatial State Space Modeling

To capture long-range spatial dependencies in multimodal remote sensing data, we introduce a spatial state space modeling module derived from the Mamba architecture [31]. Compared with convolutional neural networks, which primarily focus on local spatial correlations, state space models are capable of modeling global contextual relationships while maintaining linear computational complexity with respect to the sequence length [32]. Recent studies have further extended state space models to visual tasks by applying structured scanning strategies over spatial tokens, enabling efficient modeling of two-dimensional spatial dependencies and improving representation capability for high-dimensional visual data [33].

Given an input feature map $F\in {\mathbb{R}}^{H\times W\times C}$, the spatial dimensions are first reshaped into a sequence of spatial tokens, resulting in $X\in {\mathbb{R}}^{N\times d}$ with $N=H\times W$, where N denotes the number of spatial tokens and d represents the feature dimension. The token sequence is then processed using a selective state space model (SSM) [31].

The continuous-time state space formulation of the system can be defined as [32]:

$${\displaystyle \frac{dh\left(t\right)}{dt}}=Ah\left(t\right)+Bx\left(t\right),$$

(6)

where $h\left(t\right)$ denotes the hidden state, $x\left(t\right)$ represents the input signal, and A, B, and C are learnable parameters.

Following the discretization adopted in Mamba [31], the hidden state update at step t is computed as:

$$h_t={\overline{A}}_t{h}_{t-1}+{\overline{B}}_t{x}_t,$$

(7)

where the discretized transition matrices are defined as:

$${\overline{A}}_t=exp\left({\Delta }_{t}A\right),\hspace{1em}{\overline{B}}_t=\left({\int }_0^{{\Delta }_t}exp\left(A\tau \right) d\tau \right)B,$$

(8)

where ${\Delta }_t$ denotes an input-dependent step size computed as ${\Delta }_t=\mathrm{Softplus}\left(W_{\Delta }{x}_t\right)$, enabling adaptive state transitions conditioned on the input features.

To adapt the state space modeling to spatial feature maps, the SS2D module performs selective scanning over the flattened spatial tokens [33]. Let $X=\{x_1,x_2,\dots ,x_N\}$ denote the spatial token sequence. During the scan process, hidden states are propagated along the predefined spatial scanning order according to the state update rule above, allowing long-range spatial dependencies to be captured efficiently while maintaining linear complexity with respect to the token length.

Finally, the SS2D module is integrated into the feature modeling block using a residual formulation:

$$F_{out}=F_{in}+\mathrm{SS}2\mathrm{D}\left(\mathrm{LN}\left(F_{in}\right)\right),$$

(9)

which stabilizes the optimization process and improves the capability of modeling long-range spatial dependencies.

3.5. Quantization-Aware Training

To further improve deployment efficiency on resource-constrained devices, we adopt a quantization-aware training (QAT) strategy [34]. During training, low-bit quantization is simulated in the forward pass, allowing the network parameters to gradually adapt to quantization-induced noise while maintaining stable optimization dynamics [25].

In the forward pass, quantized weights and activations are used for computation, while the backward pass employs the straight-through estimator (STE) to approximate gradients of the quantization function:

$${\displaystyle \frac{\partial L}{\partial w}}={\displaystyle \frac{\partial L}{\partial w_q}},$$

(10)

where L denotes the overall training loss, w represents the original floating-point weight, and $w_q$ denotes the corresponding quantized weight.

To balance computational efficiency and classification performance, selective quantization is adopted [35]. Convolutional and fully connected layers are replaced by quantized counterparts during training, while other operations remain in floating-point precision to preserve numerical stability. For a selected weight bit-width $b_w$, the full-precision weight w of each quantized layer is first scaled, rounded to the nearest discrete level, clipped to the valid integer range determined by $b_w$, and then rescaled to obtain the quantized weight $w_q$. During forward propagation, $w_q$ and the corresponding quantized activations are used to compute the layer outputs, so that the network is trained under the same low-bit constraint as inference. During backward propagation, the underlying full-precision weights are retained and updated using STE, which allows gradients to pass through the non-differentiable rounding operation. Therefore, the selected bit-width determines the discrete quantization range used to obtain $w_q$, while the QAT process gradually adapts the full-precision parameters to the low-bit forward computation.

The detailed quantization-aware training procedure is summarized in Algorithm 1.

Algorithm 1 Quantization-Aware Training Procedure

Require: Pre-trained model M, weight bit-width $b_w$, activation bit-width $b_a$ Ensure: Quantized model $M_q$ 1: Initialize $M_q\leftarrow M$ 2: for each layer l in $M_q$ do 3:     if l is Conv2d or Linear then 4:         Compute the quantization scale $$s={\displaystyle \frac{max\left(\right|w\left|\right)}{2^{b_w-1}-1}}$$ 5:         Normalize weights $$w_n={\displaystyle \frac{w}{s}}$$ 6:         Map normalized weights to discrete levels $$\widehat{w}=⌊w_n⌉$$ 7:         Clip the quantized values to the valid integer range $$\widehat{w}=\mathrm{clip}\left(\widehat{w},-2^{b_w-1},2^{b_w-1}-1\right)$$ 8:         Rescale to obtain the quantized weight $$w_q=s·\widehat{w}$$ 9:         Replace the floating-point layer with its corresponding quantized counterpart 10:    end if 11: end for 12: Perform forward propagation using quantized weights and activations 13: Update parameters using backpropagation with STE 14: return  $M_q$

4. Experiments and Analysis

4.1. Data Description

To comprehensively evaluate the effectiveness of the proposed framework, experiments are conducted on three widely used multimodal remote sensing benchmark datasets: Houston2013, MUUFL Gulfport, and Augsburg. These datasets provide co-registered multimodal observations with pixel-level annotations for land-cover classification. They exhibit diverse characteristics in terms of spatial resolution, spectral dimensionality, auxiliary modalities, and class distributions, which enables a comprehensive evaluation of the proposed method. Therefore, these datasets are selected to evaluate whether LMFusion can generalize across different scene types, sensor configurations, and category distributions rather than being tailored to a single benchmark.

4.1.1. Houston2013 Dataset

The Houston2013 dataset was released as part of the 2013 IEEE GRSS Data Fusion Contest. It consists of co-registered hyperspectral imaging (HSI) and LiDAR data acquired over the University of Houston campus in Houston, Texas, USA. The hyperspectral data were collected using the ITRES CASI-1500 sensor (ITRES Research Limited, Calgary, AB, Canada), providing 144 spectral bands covering wavelengths from 0.38 µm to 1.05 µm. The LiDAR data are provided as a single-band elevation map representing surface height.

All images in the dataset have a spatial resolution of $349\times 1905$ pixels. The dataset contains 15 land-cover categories. The official training and testing partitions provided by the contest organizers are adopted in this work. The detailed numbers of training and testing samples for each category are summarized in

Table 1. A List of the Number of Training and Testing Samples for Each Class in the Houston2013 Dataset.

Land Cover	Train	Test	Land Cover	Train	Test
Background	662,013	652,648	Grass-healthy	198	1053
Grass-stressed	190	1064	Grass-synthetic	192	505
Tree	188	1056	Soil	186	1056
Water	182	143	Residential	196	1072
Commercial	191	1053	Road	193	1059
Highway	191	1036	Railway	181	1054
Parking-lot1	192	1041	Parking-lot2	184	285
Tennis-court	181	247	Running-track	187	473

.

4.1.2. Augsburg Dataset

The Augsburg dataset is a multimodal remote sensing dataset collected over the city of Augsburg, Germany, for land-cover classification tasks. It includes hyperspectral imaging (HSI), synthetic aperture radar (SAR), and LiDAR data, ensuring multimodal consistency across the observed scene.

The hyperspectral data were captured by the HySpex spectrometer (Norsk Elektro Optikk (NEO), Oslo, Norway) and consist of 180 spectral bands ranging from 400 nm to 2500 nm with a spectral resolution of 10 nm. The SAR data are obtained from Sentinel-1 using the Interferometric Wide Swath mode, providing dual-polarization (VV–VH). In addition, the LiDAR data provide elevation describing the surface structure of the scene.

All data were preprocessed to a spatial resolution of $332\times 485$ pixels with a ground sampling distance (GSD) of 30 m to ensure alignment across modalities. The dataset contains seven land-cover categories, including forest, residential areas, industrial areas, low vegetation, allotments, commercial zones, and water bodies. In total, the dataset contains 761 training samples and 77,533 testing samples. The detailed distribution of training and testing samples is provided in

Table 2. A List of the Number of Training and Testing Samples for Each Class in the Augsburg Dataset.

Land Cover	Train	Test	Land Cover	Train	Test
Background	160,259	83,487	Forest	146	13,361
Commercial Area	264	30,065	Residential Area	21	3830
Industrial Area	248	26,609	Low Plants	52	523
Allotment	7	1638	Water	23	1507

.

4.1.3. MUUFL Gulfport Scene Dataset

The MUUFL Gulfport Scene dataset was collected in November 2010 over the University of Southern Mississippi-Gulf Park Campus in Long Beach, MI, USA. The hyperspectral data were acquired using the ITRES Research Ltd. Compact Airborne Spectral Imager (CASI-1500) sensor. The hyperspectral image has a spatial resolution of $325\times 220$ pixels and originally contains 72 spectral bands, among which 64 bands are available for analysis. These bands span wavelengths from 375 nm to 1050 nm with a spectral interval of 10 nm. In addition, the LiDAR data provide elevation information with two channels describing surface height characteristics. The dataset includes 11 land-cover categories and a total of 53,687 labeled pixels. The official training and testing partitions are adopted. The detailed numbers of training and testing samples for each class are summarized in

Table 3. A List of the Number of Training and Testing Samples for Each Class in the MUUFL Dataset.

Land Cover	Train	Test	Land Cover	Train	Test
Background	68,817	20,496	Trees	1162	22,084
Grass-Pure	214	4056	Grass-Groundsurface	344	6538
Dirt-And-Sand	91	1735	Road-Materials	334	6353
Water	23	443	Buildings’-Shadow	112	2121
Buildings	312	5928	Sidewalk	69	1316
Yellow-Curb	9	174	ClothPanels	13	256

.

For the input configuration, all datasets are organized into two branches in the proposed framework. The HSI data are used as the spectral branch, while the auxiliary modality data are used as the structural branch. Each sample is cropped into a spatial patch with a size of $32\times 32$. Specifically, Houston2013 uses 144 HSI channels and 1 LiDAR channel with 15 land-cover classes; MUUFL uses 64 HSI channels and 2 LiDAR-related channels with 11 land-cover classes; Augsburg uses 180 HSI channels and co-registered SAR/LiDAR auxiliary data, including dual-polarization SAR channels and a LiDAR elevation channel, with 7 land-cover classes.

4.2. Evaluation Metrics and Parameter Setting

4.2.1. Evaluation Metrics

To quantitatively evaluate the classification performance of the proposed framework, four widely used metrics in remote sensing classification are adopted: Overall Accuracy (OA), Class Accuracy (CA), Average Accuracy (AA), and the Kappa coefficient.

Overall Accuracy (OA) measures the ratio of correctly classified samples to the total number of samples. Class Accuracy (CA) evaluates the classification accuracy of each individual category. Average Accuracy (AA) is defined as the mean of all class accuracies, which provides a more balanced evaluation under class-imbalanced conditions. The Kappa coefficient further measures the agreement between predicted labels and ground-truth labels while taking random agreement into account.

The corresponding definitions are given as follows:

$$OA={\displaystyle \frac{N_c}{N_a}},$$

(11)

$$C{A}_i={\displaystyle \frac{N_c^i}{N_a^i}},$$

(12)

$$AA={\displaystyle \frac{1}{C}}{\displaystyle \sum _{i=1}^C}C{A}_i,$$

(13)

$$\kappa ={\displaystyle \frac{OA-P_e}{1-P_e}},$$

(14)

where $N_c$ denotes the number of correctly classified samples, $N_a$ represents the total number of samples, and $N_c^i$ and $N_a^i$ denote the number of correctly classified samples and the total number of samples in the i-th category, respectively. C is the total number of land-cover categories. $P_e$ denotes the probability of random agreement and is computed as

$$P_e={\displaystyle \frac{{\sum }_{i=1}^C{N}_r^i{N}_p^i}{N_a^2}},$$

(15)

where $N_r^i$ and $N_p^i$ denote the ground-truth samples and predicted samples belonging to the i-th category, respectively.

4.2.2. Parameter Setting

The proposed framework is implemented in PyTorch 2.3.1 with CUDA 11.8 support. All experiments are conducted on an NVIDIA A100-SXM4-80GB GPU computing platform (NVIDIA Corporation, Santa Clara, CA, USA).

For all datasets, the input spatial patch size is fixed to $32\times 32$. The HSI and auxiliary modality channels are set according to the corresponding dataset configuration. Specifically, the Houston2013 dataset contains 144 HSI channels and 1 LiDAR channel with 15 land-cover categories, the MUUFL dataset contains 64 HSI channels and 2 LiDAR channels with 11 categories, and the Augsburg dataset contains 180 HSI channels and 4 auxiliary-modality channels with 7 categories.

During training, the AdamW optimizer is adopted with an initial learning rate of $1\times {10}^{-4}$ and a weight decay of $1\times {10}^{-2}$. The training batch size and test batch size are both set to 32. The model is trained for 200 epochs, and the dropout rate is set to 0.1. In addition, the cosine annealing learning-rate scheduler is employed to adjust the learning rate during training. The random seed is fixed to 3407, and the number of data-loading workers is set to 4.

Unless otherwise specified, the same training configuration is adopted for all datasets to ensure fair comparisons.

4.3. Ablation Study

To evaluate the contribution of each component in the proposed framework, we conduct a series of ablation experiments covering three aspects: modeling paradigm, cross-attention design, and quantization strategy. The results averaged over the Houston2013, MUUFL, and Augsburg datasets are summarized in

Table 4. Unified ablation study averaged over three datasets (Houston2013, MUUFL, and Augsburg).

CNN	Trans	Mamba	Linear CA	Quant	Bit	OA	AA	$\mathit{\kappa }$	Params	FLOPs	Size
A. Modeling Paradigm
✓						95.82	89.55	94.56	4.08 M	14.16 G	15.55 MB
	✓					96.45	91.91	95.37	4.05 M	85.75 G	15.46 MB
		✓				97.14	95.53	96.40	7.58 M	19.86 G	28.92 MB
B. Cross-Attention Design
		✓				97.14	95.53	96.40	7.58 M	19.86 G	28.92 MB
		✓	✓		FP32	96.61	94.97	95.82	7.59 M	18.30 G	28.96 MB
C. Quantization (based on Mamba + Linear CA)
		✓	✓	✓	1	97.19	95.65	96.44	7.59 M	18.30 G	0.91 MB
		✓	✓	✓	2	96.61	95.02	95.81	7.59 M	18.30 G	1.81 MB
		✓	✓	✓	4	96.69	95.09	95.91	7.59 M	18.30 G	3.62 MB
		✓	✓	✓	8	96.25	94.63	95.43	7.59 M	18.30 G	7.24 MB
		✓	✓	✓	16	96.73	95.18	95.95	7.59 M	18.30 G	14.48 MB

.

A.

Modeling Paradigm Analysis:

We first analyze the impact of different modeling paradigms. Specifically, we replace the Mamba module with CNN and Transformer counterparts while keeping the rest of the architecture unchanged. As shown in Table 4, the CNN-based model achieves an overall accuracy (OA) of 95.82%, while the Transformer-based model reaches 96.45%. In contrast, the Mamba-based model obtains a higher OA of 97.14% and significantly improves AA and Kappa. These results suggest that the Mamba-based sequence modeling mechanism is more effective in capturing long-range spatial–spectral dependencies compared with convolutional and Transformer-based counterparts. By leveraging selective state space modeling, the Mamba module can efficiently model global contextual relationships while maintaining linear computational complexity, which leads to improved discriminative feature representation for multimodal data.

B.

Cross-Attention Design Analysis:

We further investigate the effectiveness of the proposed Linear Cross-Attention mechanism. Compared with the conventional cross-attention module, the linear variant slightly reduces the OA from 97.14% to 96.61%, while maintaining comparable AA and Kappa. Although the linear cross-attention slightly reduces the OA from 97.14% to 96.61%, it significantly lowers the computational cost by reducing the FLOPs from 19.86 G to 18.30 G. More importantly, the linear attention formulation is more compatible with low-bit quantization, enabling efficient lightweight deployment while maintaining competitive classification performance.

C.

Quantization Analysis:

To evaluate the lightweight deployment capability of the proposed framework, we conduct quantization experiments with bit-widths ranging from 1-bit to 16-bit based on the Mamba + Linear CA configuration. As shown in Table 4, quantization significantly reduces the model size while maintaining competitive classification performance. For instance, under 1-bit quantization, the model size decreases from 28.96 MB to 0.91 MB, while the OA improves to 97.19%.

These results indicate that the proposed framework is robust to low-bit quantization and achieves substantial storage reduction with minimal impact on classification accuracy. In particular, the significant reduction in model size suggests strong potential for deployment on resource-constrained devices that support low-bit inference.

Although this study focuses on theoretical analysis and simulation-based evaluation, the quantization results indicate that the proposed framework maintains stable performance under aggressive bit-width reduction, which benefits practical lightweight deployment.

These results verify the effectiveness of each component in the proposed framework. The Mamba module strengthens feature representation capability, the linear cross-attention mechanism reduces computational overhead, and the quantization strategy substantially compresses the model size, enabling efficient multimodal classification with improved deployability.

5. Results

5.1. Comparisons with Previous Methods

In this section, the proposed method is compared with representative multimodal remote sensing classification approaches, including coupled_CNN [5], CCR-Net [36], LSAF [37], HCT [38], MHST [38], SSEFT [39], ExViT [40], and DSHF [41]. These methods represent different multimodal fusion paradigms, such as convolution-based architectures, attention-based fusion models, and transformer-based frameworks.

To clarify the comparison protocol, the baseline results reported in this section are obtained from two sources. When the experimental settings are consistent or directly comparable, results from the corresponding original publications or existing benchmark studies are adopted. For methods with publicly available implementations, we reproduce the results under the corresponding dataset settings whenever possible. Since some baseline methods do not provide complete implementation details or use different data splits, not all compared methods are re-run from scratch. This protocol improves the transparency of the experimental comparison while maintaining fair and reasonable comparability among representative multimodal classification methods.

All experiments are conducted on three widely used multimodal remote sensing benchmark datasets, including Houston2013, MUUFL, and Augsburg. Three evaluation metrics are adopted to assess the classification performance, namely overall accuracy (OA), average accuracy (AA), and the kappa coefficient ($\kappa$). The detailed class-wise classification results are reported in

Table 5. OA, AA and Kappa Coefficient on the Houston2013 Dataset (in %) by Considering HSI and LiDAR Data.

No.	Class	Coupled CNN	CCR-Net	LSAF	HCT	MHST	SSEFT	ExViT	DSHF	Ours
1	Healthy grass	82.81	88.41	83.10	97.34	92.18	83.38	82.91	87.56	98.34
2	Stressed grass	99.81	99.91	83.08	96.62	96.80	97.09	98.68	99.34	99.51
3	Synthetic grass	97.23	100.00	100.00	84.75	97.82	97.23	99.60	98.81	100.00
4	Tree	100.00	99.15	89.49	96.78	99.81	98.86	99.15	99.81	98.11
5	Soil	100.00	99.34	100.00	100.00	100.00	99.72	99.91	99.91	100.00
6	Water	100.00	95.80	100.00	96.50	96.50	91.61	99.30	93.71	100.00
7	Residential	92.35	94.96	92.72	82.09	95.52	91.32	96.08	83.96	98.69
8	Commercial	94.30	91.26	92.69	95.54	96.39	90.03	90.03	80.63	85.39
9	Road	92.73	92.63	97.07	90.84	87.72	80.36	86.12	64.68	91.06
10	Highway	85.14	79.63	68.44	58.88	82.24	55.31	72.97	96.72	76.7
11	Railway	98.86	94.21	89.94	97.53	98.01	84.82	88.99	78.75	100.00
12	Park lot 1	93.28	88.38	96.25	90.11	85.49	77.71	90.39	86.36	91.5
13	Park lot 2	89.47	77.19	89.12	97.19	92.98	61.75	90.18	88.77	99.18
14	Tennis court	100.00	96.76	100.00	100.00	94.33	99.76	99.60	98.79	100.00
15	Running track	98.94	99.79	100.00	100.00	97.04	92.18	95.14	100.00	100.00
	OA(%)	94.37	93.15	90.51	91.15	93.81	86.33	91.40	89.01	95.84
	AA(%)	94.99	93.16	92.13	92.28	94.19	86.41	92.60	90.52	95.9
	Kappa(%)	93.88	92.56	89.69	90.40	93.28	85.18	90.66	88.06	95.54

Bold indicates the best or tied-best result in each row.

,

Table 6. OA, AA and Kappa Coefficient on the Augsburg dataset (in %) by Considering HSI and LiDAR Data.

No.	Class	Coupled_CNN	CCR-Net	LSAF	HCT	MHST	SSEFT	ExViT	DSHF	Ours
1	Forest	92.96	92.06	97.21	94.23	98.92	91.83	93.51	97.06	99.85
2	Commercial Area	1.77	6.78	0.31	4.82	1.40	9.34	15.20	8.91	99.71
3	Residential Area	97.04	97.13	98.97	98.54	94.55	90.19	97.31	96.88	93.88
4	Industrial Area	76.66	62.09	30.31	43.79	64.33	52.22	64.26	14.83	99.65
5	Low Plants	96.10	84.20	95.97	95.33	86.04	88.29	87.57	98.01	92.25
6	Allotment	50.10	46.27	49.14	67.88	52.01	55.64	52.77	5.54	90.03
7	Water	30.66	26.94	11.94	52.09	34.37	8.63	27.01	47.51	94.78
	OA(%)	91.39	86.47	90.13	90.90	87.47	84.42	88.28	89.81	99.05
	AA(%)	63.61	59.35	54.84	65.24	61.66	56.59	62.52	52.68	95.74
	Kappa(%)	87.56	80.63	85.45	86.83	82.17	77.74	83.28	84.93	98.64

Bold indicates the best or tied-best result in each row.

and

Table 7. OA, AA and Kappa Coefficient on the MUUFL Dataset (in %) by Considering HSI and LiDAR Data.

No.	Class	Coupled_CNN	CCR-Net	LSAF	HCT	MHST	SSEFT	ExViT	DSHF	Ours
1	Trees	97.67	97.44	97.00	97.03	98.86	97.37	98.58	95.06	98.51
2	Grass-Pure	88.09	77.93	83.36	90.29	79.83	77.76	87.70	81.01	91.55
3	Grass-Groundsurface	88.99	82.90	90.79	90.07	79.63	83.13	90.96	72.90	90.14
4	Dirt-And-Sand	95.22	88.13	94.18	94.18	93.95	82.54	90.61	94.87	95.8
5	Road-Materials	97.32	96.49	95.88	93.86	94.84	95.17	94.73	90.41	96.47
6	Water	99.10	94.36	94.58	95.71	92.78	92.10	93.68	1.81	98.36
7	Buildings’Shadow	85.38	83.36	87.36	87.09	87.13	78.97	90.05	97.97	89.51
8	Buildings	98.08	97.93	98.14	96.61	97.12	96.78	97.76	92.70	98.04
9	Sidewalk	51.60	52.89	75.91	46.35	68.77	63.22	68.54	56.08	87.77
10	Yellow-Curb	10.34	2.30	13.22	18.97	28.16	26.44	23.56	0	92.73
11	ClothPanels	81.64	85.16	84.77	75.39	93.75	91.02	80.86	0	97.75
	OA(%)	93.65	91.50	93.70	92.95	92.43	91.17	94.37	88.65	94.95
	AA(%)	81.22	78.08	83.20	80.50	83.17	80.41	83.37	63.11	93.28
	Kappa(%)	91.59	88.74	91.68	90.69	89.92	88.32	92.54	85.15	93.28

Bold indicates the best or tied-best result in each row.

.

On the Houston2013 dataset, the proposed method achieves an OA of 95.84%, an AA of 95.90%, and a $\kappa$ value of 95.54%, which are the best results among the compared methods. Compared with coupled_CNN and CCR-Net, the proposed method improves the OA by 1.47% and 2.69%, respectively. In terms of class-wise performance, the proposed method achieves strong results across categories. For example, the Residential class achieves 98.69% accuracy, while the Synthetic Grass and Running Track classes both reach 100.00%.

On the Augsburg dataset, the proposed method achieves an OA of 99.05%, an AA of 95.74%, and a $\kappa$ value of 98.64%, which are the highest values among all compared methods. From Table 6, the proposed method achieves the best accuracy in several categories. For instance, Forest achieves 99.85%, while Commercial Area and Industrial Area achieve 99.71% and 99.65%, respectively.

On the MUUFL dataset, the proposed method achieves an OA of 94.95%, an AA of 93.28%, and a $\kappa$ value of 93.28%. Compared with CCR-Net and DSHF, the OA is improved by 3.45% and 6.30%, respectively. In addition, the proposed method achieves the best performance in several categories, including Grass-Pure, Dirt-and-Sand, Sidewalk, Yellow-Curb, and ClothPanels.

Computational Complexity Analysis:

In addition to classification accuracy, we further analyze the computational complexity of different methods in terms of parameter scale and model size.

For fair comparison, the complexity evaluation follows the Houston2013 dataset configuration, where the hyperspectral image (HSI) input is set to $1\times 144\times 32\times 32$ and the LiDAR input is set to $1\times 1\times 32\times 32$ with a batch size of 1.

The comparison results are summarized in

Table 8. Parameter and model size comparison of different multimodal classification methods.

Model	Params (M)	Model Size (MB)
MAHiDFNet	77.0	308.0
FusAtNet	36.9	147.6
SepG-ResNet50	14.7	58.8
Ours (FP32)	7.60	29.00
Ours (1-bit)	7.60	0.91

. As shown in the table, the proposed framework contains 7.58 M parameters with a model size of 28.92 MB under the full-precision (FP32) setting.

For several existing multimodal classification models, including MAHiDFNet, FusAtNet, SepG-ResNet50, the parameter statistics are directly reported from the benchmark comparison presented in the recent study CTPMSN [42]. Compared with these representative approaches, the proposed method achieves a significantly smaller parameter scale and storage cost while maintaining strong classification capability.

For instance, MAHiDFNet requires 77.0 M parameters, and FusAtNet contains 36.9 M parameters, both of which are considerably larger than the proposed model. Even compared with SepG-ResNet50 (14.7 M parameters), the proposed method still achieves a notable reduction in model complexity.

Quantization Analysis:

To further improve deployment efficiency, the proposed framework supports low-bit quantization during inference. As reported in Table 8, when applying 1-bit quantization, the model size can be reduced from 28.92 MB to 0.91 MB, achieving approximately 31× storage compression compared with the full-precision model.

It should be noted that the reported storage cost under low-bit quantization corresponds to the theoretical model size assuming ideal bit-level representation. In practical deployment scenarios, achieving such compression requires hardware platforms that support low-bit arithmetic operations.

Overall, the proposed framework maintains a moderate parameter scale while enabling substantial storage reduction through quantization, demonstrating its potential for lightweight deployment in remote sensing applications. It is worth noting that lightweight remote sensing classification requires not only a compact parameter scale, but also a balanced consideration of accuracy, memory footprint, and inference efficiency. In this regard, the proposed LMFusion provides a practical trade-off by combining linear-complexity cross-modal interaction with low-bit quantization. The Linear Cross Attention module reduces the computational burden of multimodal interaction, while quantization-aware training further decreases the storage requirement of the model. This design is particularly meaningful for remote sensing scenarios where onboard or edge devices have limited memory and computational resources.

5.2. Result Visualization

To provide a more intuitive comparison of classification performance among different methods, visualization results on the Houston2013, Augsburg, and MUUFL datasets are presented. Each land-cover category is represented by a distinct color to facilitate visual comparison.

On the Houston2013 dataset, several CNN-based approaches such as coupled_CNN and CCR-Net tend to produce fragmented predictions in complex urban regions. This phenomenon is particularly visible in areas containing residential blocks and dense road networks, where misclassified pixels frequently appear. Transformer-based methods improve spatial consistency to some extent; however, confusion between spectrally similar categories, such as Synthetic grass and Running track, can still be observed. In comparison, the proposed method produces more spatially continuous classification regions and clearer object boundaries, especially around urban structures such as highways and railways.

For the Augsburg dataset, which contains complex urban scenes with multiple land-cover categories, noticeable differences can be observed among the compared methods. Several existing models generate noisy predictions in built-up areas, especially in Commercial Area and Industrial Area. In contrast, the classification map produced by the proposed method shows more coherent regions and fewer scattered pixels. Large homogeneous areas such as Forest and Low Plants are also preserved more clearly.

On the MUUFL dataset, some categories share similar spectral characteristics, making them difficult to distinguish. For example, Sidewalk, Yellow-Curb, and ClothPanels often appear mixed in predictions of several baseline methods. As shown in the visualization results, the proposed approach produces more consistent classification patterns in these regions and maintains clearer structural details around roads and buildings.

The visualization results are shown in Figure 2, Figure 3 and Figure 4. The visual comparison further supports the quantitative results reported in the comparison tables. Compared with several baseline methods, LMFusion produces more spatially coherent classification maps and reduces scattered misclassified pixels in complex regions. This indicates that the proposed combination of Linear Cross Attention and SS2D-based state-space modeling is beneficial for preserving structural continuity while enhancing cross-modal feature representation. These results further demonstrate the effectiveness of the proposed framework for multimodal remote sensing classification.

6. Discussion

The experimental results demonstrate that LMFusion achieves a favorable balance between classification accuracy and computational efficiency in multimodal remote sensing classification. Compared with CNN-based methods, the proposed framework is better suited for capturing long-range spatial–spectral dependencies through SS2D-based state-space modeling. Compared with Transformer-based methods, Linear Cross Attention reduces the computational burden caused by pairwise token interaction while still enabling effective bidirectional information exchange between HSI and LiDAR features. This design allows the model to exploit complementary spectral and structural information without introducing excessive computational overhead.

The ablation results further show that each component contributes to the overall effectiveness of the framework. The Mamba-based modeling paradigm improves feature representation by capturing broader contextual dependencies. The Linear Cross Attention module provides a more efficient cross-modal fusion strategy than conventional attention, reducing FLOPs while maintaining competitive classification performance. In addition, quantization-aware training significantly decreases the model storage cost under low-bit settings, indicating that LMFusion has potential for lightweight deployment in resource-constrained remote sensing scenarios.

The visualization results also support the quantitative comparisons. LMFusion produces more spatially coherent classification maps and reduces scattered misclassified pixels in complex regions. This suggests that the combination of efficient cross-modal interaction and long-range dependency modeling is beneficial for preserving structural continuity and improving category discrimination, especially in scenes where different land-cover classes have similar spectral responses or fragmented spatial distributions.

Nevertheless, several aspects remain worthy of further investigation. Although the current experiments are conducted on three representative multimodal benchmark datasets, more large-scale scenes with different sensor configurations can be considered in future work to further evaluate the generalization ability of the framework. Moreover, the reported low-bit quantization results mainly demonstrate the theoretical storage reduction, while practical acceleration still depends on hardware support for low-bit arithmetic. Therefore, future work will further investigate real-device inference latency, memory consumption, and energy efficiency on edge-computing platforms.

7. Conclusions

In this paper, we propose LMFusion, an efficient multimodal classification framework for hyperspectral and LiDAR data. By enabling effective cross-modal interaction and long-range spatial–spectral dependency modeling under linear-complexity computation, the proposed method achieves a favorable balance between classification performance and computational efficiency. Experimental results on three benchmark datasets demonstrate that LMFusion consistently outperforms several representative methods. In addition, visualization and quantization analyses verify its ability to generate spatially coherent predictions and maintain robust representation capability under low-bit settings, indicating its potential for efficient deployment in resource-constrained remote sensing scenarios. Future work will further investigate hardware-aware optimization, including low-bit operator support, memory-access optimization, and inference acceleration on edge devices. In addition, we will extend the proposed framework to more heterogeneous remote sensing modalities, such as SAR, multispectral imagery, and multi-temporal observations, to further evaluate its scalability and robustness under more diverse application scenarios.