Selective State-Space Models with Adaptive Collaborative Awareness for Sequential Recommendation

Abstract

Sequential recommendation systems face challenges in integrating local sequential patterns with global collaborative information. While Transformers capture long-term dependencies through self-attention, they suffer from quadratic complexity. State-space models offer linear efficiency but are constrained by Markovian assumptions that limit their ability to model direct inter-item relationships. This paper addresses the expressiveness limitations of selective state-space models in capturing collaborative signals. We propose MCARec, which integrates selective state spaces with a dedicated collaborative awareness module. The key components include: (1) a lightweight attention mechanism that explicitly models item co-occurrence and transition patterns, enabling direct pairwise relationship modeling beyond the sequential bottleneck; (2) context-aware adaptive gating that dynamically balances sequential and collaborative features based on input context; (3) a lightweight architecture that enhances representational capacity while maintaining computational efficiency. On MovieLens-1M, a dataset characterized by dense user interactions, MCARec achieves improvements of 3.89% in HR@10, 5.52% in NDCG@10, and 6.97% in MRR@10 over Mamba4Rec, and 9.19%, 12.09%, and 8.45% respectively over SASRec (all $p<0.001$). Performance gains correlate with interaction density: substantial improvements on dense datasets diminish on sparser Amazon datasets (2–6% over SASRec in most metrics), while showing mixed results compared to Mamba4Rec on sparse datasets, suggesting that the collaborative awareness mechanism is most effective when sufficient co-occurrence signals are available. This work provides the first systematic analysis of how Markovian constraints in state-space models limit collaborative information utilization in recommendations. MCARec demonstrates that augmenting state-space models with explicit collaborative modeling significantly improves recommendation accuracy in dense interaction scenarios, offering a complementary approach to pure sequential or pure attention-based methods.

1. Introduction

Sequential recommendation aims to predict users’ next interactions by modeling the temporal dynamics of their behavior histories [1]. Unlike static collaborative filtering, sequential methods must capture how user preferences evolve over time, making them essential for modern platforms where user interests shift rapidly [2,3].

Early approaches based on Markov chains [4], RNNs [5,6], and CNNs [7] model sequential transitions but struggle with long-range dependencies or computational efficiency. Transformer-based methods such as SASRec [8] and BERT4Rec [9] address this through self-attention, enabling direct pairwise modeling across all positions, but incur quadratic complexity with sequence length. Recently, selective state-space models (SSMs), particularly Mamba [10], have emerged as efficient alternatives that achieve linear complexity while maintaining strong sequential modeling capability [11,12]. Hybrid architectures such as Jamba [13] and MaTrRec [14] further explore combining Mamba with Transformer layers.

However, Mamba’s architecture presents a fundamental limitation for recommendation tasks. Its recurrent state update ${\mathbf{h}}_t={\overline{\mathbf{A}}}_t{\mathbf{h}}_{t-1}+{\overline{\mathbf{B}}}_t{\mathbf{x}}_t$ compresses all historical information into a fixed-size state vector, and each position’s output depends only on its local context and this compressed history. As a result, Mamba cannot directly compute relationships between arbitrary item pairs—a capability central to collaborative filtering.

This limitation matters because collaborative signals are fundamental to recommendation quality. Consider a scenario where users who purchase a phone case frequently also purchase screen protectors and charging cables. This co-occurrence pattern is independent of sequential proximity—the items may appear at varying positions across different user histories. Attention mechanisms capture such patterns through an explicit $L\times L$ relationship matrix, but Mamba’s state-mediated information flow can only approximate them through lossy compression. Existing hybrid approaches [13,14] employ general-purpose attention layers without specifically targeting these collaborative relationships.

To address this, we propose MCARec, which augments a selective state-space encoder with a dedicated collaborative awareness (CA) module. The CA module employs lightweight single-layer attention to explicitly model item co-occurrence and transition patterns, while a context-aware adaptive gating mechanism dynamically balances sequential and collaborative features based on input context. This design preserves Mamba’s efficient sequential modeling while providing the direct pairwise relationship modeling that collaborative filtering requires.

Our main contributions are as follows:

• We analyze how the Markovian state bottleneck in selective SSMs limits collaborative information utilization, providing theoretical motivation for augmenting SSMs with explicit pairwise modeling.
• We propose MCARec, integrating a lightweight collaborative awareness module with context-aware adaptive gating into the Mamba architecture, where each layer combines sequential and collaborative modeling within a unified block design.
• We conduct extensive experiments demonstrating that MCARec achieves significant improvements on dense datasets (3–7% over Mamba4Rec, 6–14% over SASRec on ML-1M, $p<0.001$), with gains correlating with interaction density, providing practical guidance on when collaborative awareness mechanisms are most beneficial.

2. Preliminaries

2.1. Sequential Recommendation

Sequential recommendation systems predict users’ future preferences based on their interaction history. Let $\mathcal{U}=\{u_1,u_2,\dots ,u_{\left|\mathcal{U}\right|}\}$ be the set of users and $\mathcal{V}=\{v_1,v_2,\dots ,v_{\left|\mathcal{V}\right|}\}$ be the set of items. For user $u\in \mathcal{U}$, the interaction sequence ${\mathcal{S}}_u=\{v_1,v_2,\dots ,v_{n_u}\}$ is ordered chronologically, and the goal is to predict $v_{n_u+1}$.

2.1.1. Markov Chain Methods

Early sequential recommendation systems used Markov Chain (MC) models [15] based on the assumption that the current state depends only on the previous state: $P\left(v_t\right|v_{1:t-1})=P\left(v_t\right|v_{t-1})$. Factorized Personalized Markov Chains (FPMCs) [4] improved this by combining matrix factorization with Markov transitions, handling both personalization and sequential patterns. However, these methods are fundamentally limited by the first-order Markov assumption and cannot capture longer-range dependencies.

2.1.2. Recurrent Neural Network Methods

RNNs and their variants, such as LSTM [16] and GRU [17], capture temporal dependencies through recurrent hidden states. GRU4Rec [5] applies GRU to session-based recommendation, where the hidden state ${\mathbf{h}}_t$ is updated at each time step through gating mechanisms:

$${\mathbf{h}}_t=(1-{\mathbf{z}}_t)\odot {\mathbf{h}}_{t-1}+{\mathbf{z}}_t\odot {\tilde{\mathbf{h}}}_t$$

(1)

where ${\mathbf{z}}_t$ is the update gate and ${\tilde{\mathbf{h}}}_t$ is the candidate hidden state. While RNNs handle longer dependencies than Markov models, they suffer from sequential computation bottlenecks and gradient degradation over long sequences.

2.1.3. Attention-Based Methods

Transformer-based methods [18] revolutionized sequential recommendation by enabling direct pairwise relationship modeling. SASRec [8,19] employs self-attention to compute relationships between all position pairs:

$$\mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{softmax}\left({\displaystyle \frac{\mathbf{Q}{\mathbf{K}}^T}{\sqrt{d_k}}}\right)\mathbf{V}$$

(2)

BERT4Rec [9,20] further advances the field with bidirectional encoding and masked pre-training. These methods achieve strong performance through parallel computation and explicit pairwise modeling, but incur quadratic complexity $\mathcal{O}\left(L^2\right)$ with sequence length L, creating challenges for long user histories. Attention-based approaches have been extensively studied for recommendation, and alternative efficient architectures such as LinRec [21] have also been explored. This trade-off between expressiveness and efficiency motivates exploring state-space models as an alternative paradigm.

2.2. State-Space Models

2.2.1. Traditional State-Space Models

Linear Time-Invariant (LTI) state-space models provide a powerful framework for characterizing dynamic systems, with applications in control theory, signal processing, and time series analysis.

The state-space model (SSM) maps input sequence $\mathbf{x}\left(t\right)\in {\mathbb{R}}^D$ to output sequence $\mathbf{y}\left(t\right)\in {\mathbb{R}}^D$ through latent state $\mathbf{h}\left(t\right)\in {\mathbb{R}}^{d_{state}}$:

$${\mathbf{h}}^{\prime }\left(t\right)=\mathbf{A}\mathbf{h}\left(t\right)+\mathbf{B}\mathbf{x}\left(t\right)$$

(3)

$$\mathbf{y}\left(t\right)=\mathbf{C}\mathbf{h}\left(t\right)$$

(4)

where $\mathbf{A}\in {\mathbb{R}}^{d_{state}\times d_{state}}$, $\mathbf{B}\in {\mathbb{R}}^{d_{state}\times D}$, and $\mathbf{C}\in {\mathbb{R}}^{D\times d_{state}}$ are learnable matrices. Here $\mathbf{h}\left(t\right)$ is the hidden state vector capturing system dynamics and memory, $\mathbf{x}\left(t\right)$ is the input vector, and $\mathbf{y}\left(t\right)$ is the observable output.

For discrete sequences, the model must be discretized with step size $\Delta$:

$${\mathbf{h}}_t=\overline{\mathbf{A}}{\mathbf{h}}_{t-1}+\overline{\mathbf{B}}{\mathbf{x}}_t$$

(5)

$${\mathbf{y}}_t=\mathbf{C}{\mathbf{h}}_t$$

(6)

where $\overline{\mathbf{A}}=\exp (\Delta \mathbf{A})$ and $\overline{\mathbf{B}}={(\Delta \mathbf{A})}^{-1}(\exp (\Delta \mathbf{A})-\mathbf{I})·\Delta \mathbf{B}$. After converting from continuous form $(\mathbf{A},\mathbf{B},\mathbf{C})$ to discrete form $(\overline{\mathbf{A}},\overline{\mathbf{B}},\mathbf{C})$, the model can be computed as a linear recurrence, improving computational efficiency.

2.2.2. Structured State-Space Sequence Model (S4)

S4 [22] addresses computational inefficiencies of traditional state-space models through structured parameterization. By diagonalizing the state matrix, S4 reduces computational complexity while preserving expressive power.

The state matrix is parameterized as:

$$\mathbf{A}=\mathbf{F}{\mathbf{F}}^T+\lambda \mathbf{I}$$

(7)

The continuous-time state evolution is:

$$\mathbf{h}\left(t\right)=e^{\mathbf{A}t}\mathbf{h}\left(0\right)+{\int }_0^t{e}^{\mathbf{A}(t-\tau )}\mathbf{B}\mathbf{x}\left(\tau \right)d\tau$$

(8)

Here $\mathbf{F}$ is a low-rank factor matrix that captures essential data characteristics while reducing parameter space, and $\lambda$ prevents overfitting by controlling the eigenvalue spectrum of matrix $\mathbf{A}$.

S4 structures the state matrix $\mathbf{A}$ with HiPPO initialization [23] to improve long sequence modeling. S4’s key innovation is efficient parameterization that handles long-range dependencies with linear complexity. However, its fixed parameterization lacks adaptability for highly context-dependent sequential data.

2.2.3. Selective State-Space Model (Mamba)

Among recent SSM variants, S5 [24] simplifies the state-space layer design, and Mega [25] combines the moving average with gated attention. Building on S4, Mamba [10] introduces an input-dependent selection mechanism with a hardware-aware parallel algorithm. Unlike S4’s fixed parameterization, Mamba makes the SSM parameters functions of the input, enabling the model to selectively retain or discard information based on the current context. This selective mechanism significantly enhances the model’s ability to capture complex sequential patterns while maintaining computational efficiency.

The discretized state-space model in Mamba operates as follows:

$${\mathbf{h}}_t={\overline{\mathbf{A}}}_t{\mathbf{h}}_{t-1}+{\overline{\mathbf{B}}}_t{\mathbf{x}}_t$$

(9)

$${\mathbf{y}}_t={\mathbf{C}}_t{\mathbf{h}}_t$$

(10)

where ${\mathbf{h}}_t\in {\mathbb{R}}^{d_{state}}$ is the hidden state at time t, ${\mathbf{x}}_t\in {\mathbb{R}}^D$ is the input, and ${\mathbf{y}}_t\in {\mathbb{R}}^D$ is the output. The key innovation of Mamba is that the discretized state transition matrices ${\overline{\mathbf{A}}}_t,{\overline{\mathbf{B}}}_t$ and output projection ${\mathbf{C}}_t$ are computed based on the input ${\mathbf{x}}_t$ through learned transformations:

$${\mathbf{B}}_t={\mathrm{Linear}}_B\left({\mathbf{x}}_t\right)$$

(11)

$${\mathbf{C}}_t={\mathrm{Linear}}_C\left({\mathbf{x}}_t\right)$$

(12)

$${\Delta }_t=\mathrm{Softplus}(\mathrm{Parameter}+{\mathrm{Linear}}_{\Delta }\left({\mathbf{x}}_t\right))$$

(13)

$${\overline{\mathbf{A}}}_t,{\overline{\mathbf{B}}}_t=\mathrm{discretize}({\Delta }_t,\mathbf{A},{\mathbf{B}}_t)$$

(14)

Here $\mathbf{A}\in {\mathbb{R}}^{d_{state}\times d_{state}}$ is a structured state matrix (initialized with HiPPO), ${\Delta }_t$ is the discretization step size, and the discretization function converts continuous-time parameters to discrete-time parameters:

$${\overline{\mathbf{A}}}_t=\exp \left({\Delta }_t\mathbf{A}\right),{\overline{\mathbf{B}}}_t={\left({\Delta }_t\mathbf{A}\right)}^{-1}(\exp \left({\Delta }_t\mathbf{A}\right)-\mathbf{I})·{\Delta }_t{\mathbf{B}}_t$$

(15)

This input-dependent parameterization enables Mamba to dynamically adjust its state transitions based on context. When the input indicates important information, the model can increase ${\Delta }_t$ to better retain it in the hidden state; conversely, for less relevant information, smaller ${\Delta }_t$ values effectively filter it out. This selective mechanism allows Mamba to capture contextual information effectively, especially in long sequences, while maintaining linear computational complexity.

Recent theoretical work has rigorously analyzed the expressiveness and representational capacity of selective state-space models, providing foundations for understanding their advantages in sequence modeling [26]. These theoretical insights motivate exploring SSMs as efficient alternatives to attention mechanisms in sequential recommendation tasks.

As a linear-time sequence model, Mamba achieves Transformer-quality performance with better efficiency on long sequences. Recent theoretical work has established deep connections between Transformers and SSMs through structured state-space duality [27]. For comprehensive surveys on state-space models, we refer readers to [28,29]. Jamba [13] proposes a hybrid architecture that interleaves Mamba and attention layers for large language models, demonstrating improved throughput and efficiency while maintaining model quality. In the recommendation domain, MaTrRec [14] applies this hybrid paradigm by sequentially composing Mamba blocks for efficient sequence encoding followed by Transformer layers for representation refinement.

However, these approaches typically rely on sequential composition—processing inputs through Mamba layers first, then attention layers—which may not optimally integrate the two paradigms. Moreover, they lack explicit mechanisms for modeling collaborative relationships between items, which are fundamental to recommendation tasks. The attention mechanisms in these hybrid architectures are designed for general-purpose sequence modeling rather than specifically capturing item co-occurrence patterns, sequential transitions, and complementarity relationships that characterize collaborative filtering. Our work addresses these limitations by introducing a specialized collaborative awareness module with context-aware adaptive fusion tailored for sequential recommendation.

2.3. Theoretical Analysis and Motivation

2.3.1. Architecture of Mamba in Sequential Recommendation

To understand the potential and limitations of Mamba in sequential recommendation contexts, we examine its architectural components as implemented in Mamba4Rec. The Mamba block processes sequential information through a three-stage pipeline: local convolution, selective state-space modeling, and gated output projection.

The architecture begins with a 1D convolution of kernel size $d_{conv}$ applied to the input sequence, enabling each position to aggregate information from its neighboring positions for capturing short-range dependencies. Following this, Mamba applies the selective state-space mechanism where SSM parameters are input-dependent. The hidden state ${\mathbf{h}}_t\in {\mathbb{R}}^{d_{state}}$ evolves as follows:

$${\mathbf{h}}_t={\overline{\mathbf{A}}}_t{\mathbf{h}}_{t-1}+{\overline{\mathbf{B}}}_t{\mathbf{x}}_t,{\mathbf{y}}_t={\mathbf{C}}_t{\mathbf{h}}_t$$

(16)

where ${\overline{\mathbf{A}}}_t$ and ${\overline{\mathbf{B}}}_t$ are discretized state transition matrices, and ${\mathbf{C}}_t$ is the output projection. The state dimension $d_{state}$ determines the capacity for encoding historical information beyond the local window. Finally, the SSM output is modulated through gating before projection, producing representations encoding both local context and compressed sequential history.

2.3.2. Limitations of Mamba in Sequential Recommendation

Having established the architecture, we now examine how these design choices create challenges for recommendation tasks. While Mamba excels at general sequence modeling, we argue that its architecture presents inherent limitations for sequential recommendation, where capturing collaborative item relationships is essential. These arise not from implementation details but from fundamental architectural choices prioritizing efficiency over explicit pairwise modeling.

Consider first the spatial extent of direct interactions. The 1D convolution provides direct interaction primarily within a window of size $d_{conv}$. For item pairs beyond this window, information must propagate through recurrent state updates. One might consider increasing $d_{conv}$, but this introduces a fundamental trade-off: larger kernels increase parameters and computation while still providing only bounded coverage. Regardless of the value chosen, the proportion of directly interacting pairs remains $\mathcal{O}(d_{conv}/L)$, diminishing as sequences lengthen. This reveals a structural constraint rather than a tunable parameter issue.

Beyond the convolution window, information traverses recurrent updates ${\mathbf{h}}_t={\overline{\mathbf{A}}}_t{\mathbf{h}}_{t-1}+{\overline{\mathbf{B}}}_t{\mathbf{x}}_t$, encountering a second constraint: information compression. The state ${\mathbf{h}}_t\in {\mathbb{R}}^{d_{state}}$ must compress all history into a fixed-size vector. While parameters ${\overline{\mathbf{A}}}_t$ and ${\overline{\mathbf{B}}}_t$ enable selective retention, the compression itself limits fine-grained relationship information. Increasing $d_{state}$ faces two constraints: maintaining efficiency requires $d_{state}\ll L$ (otherwise eliminating Mamba’s linear complexity advantage), and recurrent propagation means early information traverses multiple transitions, potentially degrading compared to direct connections.

Perhaps most fundamentally, Mamba produces position-wise outputs without computing explicit item-pair relationships. Each position computation ${\mathbf{y}}_t={\mathbf{C}}_t\left({\overline{\mathbf{A}}}_t{\mathbf{h}}_{t-1}+{\overline{\mathbf{B}}}_t{\mathbf{x}}_t\right)$ depends on current input and compressed history, but never directly computes relationships between arbitrary positions i and j. In collaborative filtering, item co-occurrence patterns are central—items appearing together frequently exhibit collaborative affinity regardless of proximity. Attention mechanisms address this via explicit relationship matrix $\mathbf{A}\in {\mathbb{R}}^{L\times L}$, where $A_{ij}$ quantifies affinity between positions. Mamba lacks this architectural analog, a limitation that cannot be resolved through hyperparameter tuning but reflects fundamental differences in relationship modeling.

These characteristics are not inherent deficiencies across all domains—the compression and linear complexity enabling Mamba’s efficiency in language modeling become constraints specifically for recommendation, where explicit collaborative modeling is essential. Our collaborative awareness module thus complements Mamba’s sequential strengths while addressing these recommendation-specific limitations.

2.3.3. Motivation for Collaborative Awareness

This analysis motivates introducing a mechanism for explicit pairwise modeling while preserving Mamba’s efficient sequential processing. We augment Mamba’s recurrent compression with a lightweight attention mechanism providing direct pairwise connections:

$${\mathbf{A}}_c=\mathrm{softmax}\left({\displaystyle \frac{{\mathbf{Q}}_c{\mathbf{K}}_c^T}{\sqrt{d}}}\right),{\mathbf{H}}_g={\mathbf{A}}_c\mathbf{X}$$

(17)

where ${\mathbf{Q}}_c$ and ${\mathbf{K}}_c$ are query and key projections of input $\mathbf{X}$. The attention matrix ${\mathbf{A}}_c\in {\mathbb{R}}^{L\times L}$ explicitly represents all item-pair relationships, with each entry $A_{c,ij}$ quantifying collaborative affinity. Implementation details are provided in Section 3.3.2.

This design directly addresses the three constraints. Unlike convolution’s bounded window, attention enables any pair $(i,j)$ to interact directly regardless of distance, providing a global receptive field. Each position retains full d-dimensional representation, avoiding lossy compression. Most importantly, the attention matrix directly represents item-pair relationships, capturing collaborative patterns Mamba cannot explicitly model.

We employ a single attention layer rather than deep stacks typical of Transformers, reflecting our goal of complementing rather than replacing Mamba. The single layer provides sufficient capacity for pairwise relationship modeling while maintaining architectural efficiency.

We note that while the CA module employs attention with $\mathcal{O}\left(L^2\right)$ complexity per layer, this does not contradict our efficiency motivation. The cost concern with Transformer-based recommenders arises from stacking multiple attention layers (typically 2–6 in SASRec/BERT4Rec), each with multi-head computation and feed-forward sub-layers. Our CA module uses a single layer without multi-head decomposition, serving as a lightweight complement to the Mamba encoder. Empirical measurements confirm that MCARec’s total overhead remains moderate (see Table 1).

Table 1. Efficiency comparison on ML-1M.

Method	GPU Memory	Training Time	Inference Time
SASRec	3.68 GB	473.38 s	0.61 s
Mamba4Rec	1.23 GB	202.30 s	0.42 s
MaTrRec	1.98 GB	247.21 s	0.54 s
MCARec	1.76 GB	254.35 s	0.57 s

2.3.4. Adaptive Fusion Strategy

With both mechanisms in place, a natural question arises: how should we combine these complementary sources? Rather than fixed combination, we design adaptive fusion dynamically balancing sequential and collaborative information based on context:

$$\mathbf{g}=\sigma ({\mathbf{W}}_g[{\mathbf{H}}_m;{\mathbf{H}}_g]+{\mathbf{b}}_g)$$

(18)

$$\mathbf{H}=\mathbf{g}\odot {\mathbf{H}}_m+(1-\mathbf{g})\odot {\mathbf{H}}_g$$

(19)

where ${\mathbf{H}}_m$ captures sequential patterns, ${\mathbf{H}}_g$ captures pairwise relationships, and $\mathbf{g}\in {(0,1)}^{L\times d}$ represents input-dependent fusion weights computed from concatenated representations. This enables flexible adaptation: when temporal patterns dominate (e.g., strong recency effects), the model emphasizes Mamba’s output; when co-occurrence relationships are more informative (e.g., complementary item exploration), it emphasizes collaborative awareness. The mechanism supports smooth interpolation with optimal balance learned from data.

Based on this analysis, we hypothesize that the collaborative awareness mechanism provides greater benefit when important relationships span distances beyond the convolution window and sufficient interaction data exists to learn meaningful patterns. Conversely, with sparse interactions or predominantly local patterns, benefits may be limited. These hypotheses are empirically examined in Section 4, where we observe substantial performance variations across datasets with different interaction densities.

3. MCARec

This section describes the core architecture of MCARec. Based on our theoretical analysis, we build an end-to-end framework that combines selective state spaces with adaptive collaborative awareness. The framework has four key modules: (1) a semantic embedding layer for item representations; (2) a selective state-space encoder for capturing temporal patterns; (3) a collaborative awareness module for modeling item relationships; (4) a context-aware dynamic fusion mechanism for integrating sequential and collaborative information.

3.1. Framework Overview

The overall architecture of MCARec is shown in Figure 1, with three core components: the item embedding layer, model layer, and prediction layer.

Figure 1. Overall architecture of MCARec, showing the integration of the Mamba encoder, collaborative awareness module, and adaptive fusion mechanism.

3.2. Embedding Layer

Given item sequence $\mathcal{S}=\{v_1,v_2,\dots ,v_t\}$, each item $v_i$ is mapped to a d-dimensional vector via a learnable embedding table. The embedding sequence $\mathbf{E}\in {\mathbb{R}}^{t\times d}$ is processed through layer normalization [30] and dropout [31] to produce the regularized input ${\mathbf{E}}^{\prime }$:

$${\mathbf{E}}^{\prime }=\mathrm{Dropout}\left(\mathrm{LayerNorm}\left(\mathrm{Embed}\left(\mathcal{S}\right)\right)\right)$$

(20)

3.3. Model Layer

MCARec combines selective sequential modeling with collaborative awareness through three components. The selective state-space encoder captures temporal evolution patterns through Mamba’s selective mechanism, which adaptively retains and forgets historical information for efficient long sequence processing. The collaborative awareness module explicitly models global item-pair relationships through lightweight single-layer attention, capturing collaborative patterns that pure sequential models cannot access. The context-aware dynamic fusion mechanism learns joint representations of sequential and collaborative contexts, intelligently balancing these two information sources.

3.3.1. Mamba Encoder

The Mamba encoder applies the selective state-space model (Section 2.2.3) to process sequential item embeddings. For the normalized input sequence ${\mathbf{E}}^{\prime }$, the encoder processes each position through the input-dependent state transitions defined in Equations (10) and (11), where the generic input ${\mathbf{x}}_t$ is instantiated as the item embedding ${\mathbf{E}}_t^{\prime }\in {\mathbb{R}}^d$ at position t. The output sequence is collected and normalized to produce the sequential representations:

$${\mathbf{H}}_m=\mathrm{LayerNorm}\left({\mathbf{y}}_{1:L}\right)$$

(21)

where ${\mathbf{h}}_t\in {\mathbb{R}}^{d_{state}}$ is the hidden state carrying sequential information, and $\{{\overline{\mathbf{A}}}_t,{\overline{\mathbf{B}}}_t,{\mathbf{C}}_t\}$ are input-dependent parameters computed as:

$${\mathbf{B}}_t={\mathbf{W}}_B{\mathbf{E}}_t^{\prime },{\mathbf{C}}_t={\mathbf{W}}_C{\mathbf{E}}_t^{\prime }$$

(22)

$${\Delta }_t=\mathrm{Softplus}(\delta +{\mathbf{W}}_{\Delta }{\mathbf{E}}_t^{\prime })$$

(23)

$${\overline{\mathbf{A}}}_t,{\overline{\mathbf{B}}}_t=\mathrm{discretize}({\Delta }_t,\mathbf{A},{\mathbf{B}}_t)$$

(24)

Here ${\mathbf{W}}_B,{\mathbf{W}}_C,{\mathbf{W}}_{\Delta }$ are learnable projection matrices, $\delta$ is a learnable parameter, and $\mathbf{A}$ is the structured state matrix (HiPPO-initialized). The output sequence ${\mathbf{H}}_m=\{{\mathbf{y}}_1,{\mathbf{y}}_2,\dots ,{\mathbf{y}}_L\}$ undergoes layer normalization for training stability.

The input-dependent mechanism allows the model to selectively emphasize important items while filtering noise. For instance, when processing a highly relevant item, larger ${\Delta }_t$ values enable stronger retention in the hidden state ${\mathbf{h}}_t$; conversely, for less relevant items, smaller ${\Delta }_t$ values reduce their influence on subsequent predictions.

3.3.2. Collaborative Awareness Module

The collaborative awareness module implements the attention-based pairwise modeling described in Section 2.3.3. Operating on the normalized embeddings ${\mathbf{E}}^{\prime }$, the module computes collaborative relationships through three sequential phases.

First, it generates collaborative query and key representations:

$${\mathbf{Q}}_c={\mathbf{W}}_q{\mathbf{E}}^{\prime },{\mathbf{K}}_c={\mathbf{W}}_k{\mathbf{E}}^{\prime }$$

(25)

Second, it computes the normalized collaborative attention matrix:

$${\mathbf{A}}_c=\mathrm{softmax}\left({\displaystyle \frac{{\mathbf{Q}}_c{\mathbf{K}}_c^T}{\sqrt{d}}}\right)$$

(26)

Finally, it generates collaborative-aware representations:

$${\mathbf{H}}_g=\mathrm{LayerNorm}\left({\mathbf{A}}_c{\mathbf{E}}^{\prime }\right)$$

(27)

where ${\mathbf{W}}_q,{\mathbf{W}}_k\in {\mathbb{R}}^{d\times d}$ are learnable projection matrices. The layer normalization ensures training stability and output scale consistency with the Mamba encoder. The single-layer design keeps the additional computation manageable for typical recommendation sequence lengths, contrasting with Transformer architectures that stack multiple attention layers.

3.3.3. Adaptive Gating Fusion

To dynamically fuse Mamba features ${\mathbf{H}}_m$ and collaborative features ${\mathbf{H}}_g$, we design an adaptive gating mechanism. First, concatenate the two feature sets:

$$\mathbf{Z}=\mathrm{Concat}({\mathbf{H}}_m,{\mathbf{H}}_g)\in {\mathbb{R}}^{L\times 2d}$$

(28)

Then apply the gating mechanism with a residual connection:

$$g=\sigma ({\mathbf{W}}_g\mathbf{Z}+{\mathbf{b}}_g)$$

(29)

$${\mathbf{H}}_{\mathrm{fused}}=g\odot {\mathbf{H}}_m+(1-g)\odot {\mathbf{H}}_g$$

(30)

$$\mathbf{H}=\mathrm{LayerNorm}({\mathbf{X}}_{\mathrm{in}}+\mathrm{Dropout}\left({\mathbf{H}}_{\mathrm{fused}}\right))\left)\right)$$

(31)

Here ${\mathbf{W}}_g$ and ${\mathbf{b}}_g$ are learnable parameters, and the gating value g dynamically adjusts based on current context. The residual connection from the block input ${\mathbf{X}}_{\mathrm{in}}$ preserves representations from the preceding layer and stabilizes training. For a single-layer configuration, ${\mathbf{X}}_{\mathrm{in}}={\mathbf{E}}^{\prime }$; for stacked layers, ${\mathbf{X}}_{\mathrm{in}}$ is the output of the previous MCARec layer.

3.3.4. Feed-Forward Layer

The fused representation $\mathbf{H}$ is further refined through a standard position-wise feed-forward network with residual connection:

$${\mathbf{F}}_1=\mathrm{GELU}({\mathbf{W}}_1\mathbf{H}+{\mathbf{b}}_1)$$

(32)

$${\mathbf{F}}_2=\mathrm{Dropout}({\mathbf{W}}_2{\mathbf{F}}_1+{\mathbf{b}}_2)$$

(33)

$$\mathbf{Y}=\mathrm{LayerNorm}({\mathbf{F}}_2+\mathbf{H})$$

(34)

where ${\mathbf{W}}_1,{\mathbf{W}}_2$ and ${\mathbf{b}}_1,{\mathbf{b}}_2$ are learnable parameters. The intermediate layer expands the feature dimension by a factor of 4, and GELU [32] activation provides smooth non-linearity.

3.4. Prediction Layer

The prediction layer extracts the representation at the last valid position from the fused output sequence $\mathbf{Y}$ to compute prediction probabilities:

$$\widehat{\mathbf{y}}=\mathrm{Softmax}\left(\mathbf{h}{\mathbf{E}}_{\mathrm{item}}^T\right)\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$$

(35)

where $\mathbf{h}\in {\mathbb{R}}^d$ is the final representation encoding both sequential and collaborative information through the adaptive fusion mechanism, and ${\mathbf{E}}_{\mathrm{item}}$ is the full item embedding matrix. The result $\widehat{\mathbf{y}}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$ is a probability distribution over the entire item catalog $\mathcal{V}$, indicating the likelihood of each item being the user’s next selection.

4. Experiments

4.1. Experimental Setup

4.1.1. Datasets

We conducted experiments on three well-established real-world datasets. MovieLens-1M [33] is a benchmark dataset with approximately one million movie ratings from 6040 users on 3416 movies. This dataset is widely used in sequential recommendation research due to its rich temporal interaction patterns and dense user-item interactions. Amazon-Beauty and Amazon-Video-Games [34] are domain-specific datasets from the Amazon review corpus, containing product reviews and ratings from the “Beauty” and “Video Games” categories respectively. These datasets represent different user behavior patterns across product domains and are characterized by sparser interactions compared to MovieLens-1M.

For each user, we created chronological interaction sequences by sorting records by timestamps. Following standard protocols [8,9], we retained only users and items with at least 5 interactions to mitigate cold-start issues and ensure sufficient training signals. Table 2 shows the dataset statistics after preprocessing.

Table 2. Dataset statistics after preprocessing.

Dataset	#Users	#Items	#Interactions	Avg. Length	Sparsity
ML-1M	6040	3416	999,611	165.5	95.16%
Beauty	22,363	12,101	198,502	8.9	99.93%
Video-Games	24,303	10,672	231,780	9.5	99.91%

The datasets exhibit substantial differences in interaction density. MovieLens-1M has an average sequence length of 165.5 items per user, providing rich collaborative signals. In contrast, the Amazon datasets have average sequence lengths of only 8–9 items, resulting in much sparser collaborative information. This variation allows us to evaluate how MCARec performs across different data density regimes.

4.1.2. Baselines

We compared MCARec against several state-of-the-art sequential recommendation approaches. Traditional methods include BPR-MF [35], a classic matrix factorization approach with Bayesian personalized ranking optimization. RNN-based models include GRU4Rec [5] and NARM [36], which use recurrent neural networks to capture sequential dependencies. Transformer-based models include SASRec [8] and BERT4Rec [9], which employ self-attention mechanisms to model complex item relationships and bidirectional contexts. Linear Recurrent Unit-based models [37] include LRURec [38] for efficient sequential modeling. Mamba-based models include Mamba4Rec [11], which incorporates state-space models for capturing long-range dependencies, and MaTrRec [14], which combines Mamba and Transformer to balance performance on both long and short interaction sequences.

4.1.3. Evaluation Metrics

We used multiple ranking-based evaluation metrics with different cut-off thresholds. Hit Ratio (HR@K) measures the percentage of test cases where the ground truth item appears in the top-K recommendation list. Normalized Discounted Cumulative Gain (NDCG@K) evaluates ranking quality by assigning higher importance to correctly recommended items appearing earlier. Mean Reciprocal Rank (MRR@K) calculates the average of reciprocal ranks of ground truth items in recommendation lists. We report results with K = 10 and K = 20, yielding six metrics: HR@10, NDCG@10, MRR@10, HR@20, NDCG@20, and MRR@20.

4.1.4. Implementation Details

MCARec uses a single model layer with an Adam optimizer [39] and a 0.001 learning rate. We used a batch size of 512 for training and 1024 for evaluation, with an embedding dimension of 64 for all models. We applied dataset-specific dropout regularization: 0.4 for the Amazon datasets and 0.2 for MovieLens-1M. The maximum sequence length was 200 for MovieLens-1M and 50 for the Amazon datasets. The Mamba block parameters included an SSM state expansion factor of 32, a kernel size of 4 for 1D convolution, and a block expansion factor of 2. All experiments were run on NVIDIA GeForce GTX 1080 Ti GPUs following RecBole [40] framework conventions. For statistical validation, all experiments on ML-1M were repeated 5 times with random seeds {2020, 2021, 2022, 2023, 2024}. All hyperparameters were systematically tuned via grid search on validation sets, and the reported configurations represent optimal settings, ensuring fair comparison across all methods. For fair comparison, Mamba4Rec, MaTrRec, and MCARec all used a single model layer. SASRec, BERT4Rec, and LRURec followed their default configurations of two layers.

4.2. Overall Performance

Table 3 presents a comprehensive comparison of recommendation performance across three datasets with distinct interaction characteristics. The results reveal a clear correlation between interaction density and the effectiveness of collaborative awareness mechanisms.

Table 3. Recommendation performance comparison. The best results are in bold, and the second-best are underlined.

Dataset	Metric	BPR-MF	NARM	GRU4Rec	SASRec	BERT4Rec	LRURec	Mamba4Rec	MaTrRec	Ours	% vs. M4R	% vs. SAS
ML-1M	HR@10	0.1687	0.2500	0.2911	0.2958	0.2928	0.3042	0.3109	0.3146	0.3230	+3.89%	+9.19%
	NDCG@10	0.0882	0.1355	0.1688	0.1687	0.1674	0.1764	0.1792	0.1872	0.1891	+5.52%	+12.09%
	MRR@10	0.0641	0.1007	0.1314	0.1372	0.1275	0.1380	0.1391	0.1427	0.1488	+6.97%	+8.45%
	HR@20	0.2535	0.3594	0.3906	0.4033	0.4006	0.4058	0.4137	0.4193	0.4275	+3.34%	+6.00%
	NDCG@20	0.1109	0.1658	0.1939	0.1977	0.1937	0.1962	0.2056	0.2102	0.2156	+4.86%	+9.05%
	MRR@20	0.0683	0.1113	0.1383	0.1392	0.1374	0.1408	0.1525	0.1588	0.1592	+4.39%	+14.37%
Beauty	HR@10	0.0549	0.0634	0.0639	0.0842	0.0722	0.0713	0.0721	0.0840	0.0841	+16.64%	−0.12%
	NDCG@10	0.0289	0.0358	0.0360	0.0424	0.0391	0.0382	0.0424	0.0429	0.0434	+2.36%	+2.36%
	MRR@10	0.0226	0.0274	0.0275	0.0296	0.0282	0.0280	0.0334	0.0311	0.0310	−7.19%	+4.73%
	HR@20	0.0776	0.0904	0.0915	0.1180	0.1034	0.1025	0.1018	0.1187	0.1209	+18.76%	+2.46%
	NDCG@20	0.0345	0.0422	0.0429	0.0509	0.0497	0.0489	0.0499	0.0519	0.0525	+5.21%	+3.14%
	MRR@20	0.0241	0.0287	0.0293	0.0318	0.0308	0.0301	0.0355	0.0330	0.0336	−5.35%	+5.66%
Video-Games	HR@10	0.0645	0.1029	0.1017	0.1197	0.1053	0.1134	0.1153	0.1185	0.1294	+12.23%	+8.10%
	NDCG@10	0.0358	0.0540	0.0533	0.0573	0.0538	0.0560	0.0609	0.0581	0.0595	−2.30%	+3.84%
	MRR@10	0.0277	0.0393	0.0387	0.0383	0.0381	0.0386	0.0445	0.0405	0.0383	−13.93%	0.00%
	HR@20	0.0939	0.1558	0.1562	0.1827	0.1617	0.1685	0.1742	0.1819	0.1949	+11.88%	+6.68%
	NDCG@20	0.0432	0.0673	0.0669	0.0720	0.0699	0.0691	0.0757	0.0742	0.0757	0.00%	+5.14%
	MRR@20	0.0298	0.0429	0.0423	0.0411	0.0412	0.0420	0.0485	0.0439	0.0424	−12.58%	+3.16%

On MovieLens-1M, characterized by dense interactions with an average sequence length of 165.5 items, MCARec achieves substantial improvements over all baselines. Compared to Mamba4Rec, MCARec achieves 3.89% improvement in HR@10, 5.52% in NDCG@10, and 6.97% in MRR@10. These improvements are more pronounced relative to SASRec: 9.19% in HR@10, 12.09% in NDCG@10, and 8.45% in MRR@10. The larger gains in ranking-sensitive metrics (MRR) compared to hit-based metrics (HR) suggest that the collaborative awareness module particularly helps in distinguishing between relevant items and placing ground-truth items higher in rankings. These improvements can be attributed to the collaborative awareness module’s ability to capture rich co-occurrence patterns present in dense interaction data. With an average of 165 interactions per user, sufficient signal exists for learning meaningful item-pair relationships that complement the sequential patterns captured by Mamba. All improvements on ML-1M are statistically significant ($p<0.001$, paired t-test over 5 runs with seeds 2020–2024).

MCARec also outperforms MaTrRec, a recent 2024 hybrid architecture that combines Mamba and Transformer layers, across all six metrics on ML-1M. This suggests that our design—dedicated collaborative awareness with adaptive gating—may be more effective for recommendation tasks than general-purpose Mamba-Transformer composition, particularly when capturing item co-occurrence patterns is important.

On the Amazon-Beauty and Amazon-Video-Games datasets, characterized by sparse interactions (average sequence lengths of 8–9 items), MCARec shows mixed results. While hit-based metrics at larger cutoffs improve modestly (e.g., 2.46% in HR@20 on Beauty, 3.84% in NDCG@10 on Video-Games over SASRec), MRR metrics decline on both datasets (e.g., −7.19% MRR@10 on Beauty, −13.93% MRR@10 on Video-Games).

We attribute this divergent behavior to the interaction between the CA module and data sparsity. The CA module introduces $2d^2$ additional parameters for query and key projections that require sufficient co-occurrence observations to learn meaningful patterns. In sparse settings, attention weights may distribute nearly uniformly due to an insufficient discriminative signal, effectively adding noise. This noise particularly affects MRR, which is sensitive to the exact rank of the top-1 prediction, while HR at larger cutoffs is more tolerant since it only requires the ground-truth item to appear within the top-K list. The adaptive gating partially mitigates this—as evidenced by preserved HR improvements—but cannot fully compensate when collaborative signals are too sparse. This suggests that the CA module would benefit from regularization strategies (e.g., attention dropout or sparse attention constraints) in sparse scenarios, which we identify as a topic for future work.

4.3. Efficiency Analysis

Table 1 reports empirical efficiency on ML-1M, measured on a single NVIDIA GeForce GTX 1080 Ti GPU (11 GB). All models use identical training configuration to ensure fair comparison.

MCARec introduces moderate overhead compared to Mamba4Rec (1.43× in GPU memory, 1.26× in training time, and 1.36× in inference time), primarily due to the attention computation in the CA module. Notably, MCARec remains substantially more efficient than SASRec (1.86× faster training, 52.2% less GPU memory), and is comparable to MaTrRec while using less GPU memory (1.76 GB vs. 1.98 GB). These results confirm that the CA module adds bounded overhead in exchange for significant performance gains.

4.4. Ablation Study

To understand the contribution of individual components in MCARec, we conduct ablation studies on the ML-1M dataset. The default architecture uses a single model layer (L = 1). We introduce each variant and analyze its impact. The results are summarized in Table 4.

Table 4. Ablation analysis on ML-1M. Bold scores indicate better performance than the default version (L = 1).

Architecture	ML-1M
	HR@10	NDCG@10	MRR@10
L = 1 (Default)	0.3230	0.1891	0.1488
Remove CA	0.3103	0.1791	0.1392
Remove Gate	0.3182	0.1847	0.1441
Remove FFN	0.3117	0.1795	0.1400
L = 2	0.3320	0.1943	0.1528

• Remove CA: This removes the collaborative awareness module, reducing the model to a pure Mamba encoder with gating and FFN.
• Remove Gate: This replaces the adaptive gating mechanism with simple addition of Mamba and CA features.
• Remove FFN: This removes the feed-forward network after the fusion layer.
• L = 2: This stacks two MCARec layers to examine the effect of depth.

Both the CA module and FFN contribute substantially to overall performance. Removing CA causes 3.93% degradation in HR@10, 5.29% in NDCG@10, and 6.45% in MRR@10. The CA module is particularly impactful on ranking-sensitive metrics, confirming that explicit collaborative modeling improves the model’s ability to rank ground-truth items higher. Removing FFN yields comparable degradation (3.50% in HR@10, 5.08% in NDCG@10, 5.91% in MRR@10), indicating that non-linear feature refinement is equally important for the fused representations. The gating mechanism shows a smaller but consistent contribution (1.49% in HR@10, 2.33% in NDCG@10, 3.16% in MRR@10), confirming its role as an effective fusion strategy rather than a primary performance driver.

Stacking two MCARec layers (L = 2) yields further improvements over the single-layer default across all metrics (2.79% in HR@10, 2.75% in NDCG@10, 2.69% in MRR@10). This demonstrates that the collaborative awareness mechanism benefits from deeper feature extraction, though the single-layer configuration already achieves competitive performance.

A potential concern is whether MCARec’s improvements stem from the architectural design or simply from increased parameter count. MCARec introduces 307,776 parameters compared to Mamba4Rec’s 290,944, an increase of only 16,832 parameters (5.79%). To disentangle the effect of architecture from parameter count, we examine the ablation results: removing the CA module eliminates only $2d^2=8192$ parameters (2.66% of total) while retaining the gating and FFN modules with their substantially larger parameter budgets, yet causes 3.93% HR@10 and 6.45% MRR@10 degradation. Conversely, removing FFN eliminates far more parameters ($8d^2+5d$) but causes comparable degradation (3.50% and 5.91%). This asymmetry between parameter reduction and performance impact demonstrates that the CA module’s collaborative modeling function, not its parameter count, drives the improvement.

5. Conclusions

This paper proposes MCARec, which integrates selective state-space models with collaborative awareness mechanisms for sequential recommendation. We identify the expressiveness limitations of pure state-space models stemming from their Markovian information flow, and demonstrate how explicit collaborative modeling can address these limitations.

MCARec makes three key contributions. First, we provide a theoretical analysis explaining how the sequential state bottleneck in Mamba prevents direct modeling of item-pair relationships. Second, we design a collaborative awareness module with adaptive gating that effectively combines sequential and collaborative information sources. Third, we conduct extensive empirical validation revealing the relationship between interaction density and model effectiveness.

Extensive experiments validate our approach. On MovieLens-1M, MCARec achieves 3.89–6.97% improvements over Mamba4Rec and 8.45–12.09% over SASRec in all metrics. Ablation studies confirm that the collaborative awareness module and feed-forward network are the primary drivers of improvements. Importantly, performance gains correlate with data density: substantial improvements on dense datasets diminish on sparser Amazon datasets, confirming our hypothesis that collaborative awareness mechanisms require sufficient co-occurrence signals to be effective.

Based on our findings, MCARec is particularly well-suited for platforms with established user bases and dense interaction histories, such as video streaming services or mature e-commerce platforms. Conversely, for cold-start scenarios with limited interaction data or platforms with sparse user engagement, pure sequential models may be more appropriate as they avoid attempting to extract collaborative patterns from insufficient data.

MCARec has several limitations. First, it introduces additional computational cost through attention computation. Second, the collaborative awareness mechanism is less effective on sparse datasets where co-occurrence signals are insufficient. Third, like other collaborative methods, MCARec may exhibit popularity bias, favoring frequently occurring items.

Future work could explore sparse attention mechanisms to reduce computational cost for longer sequences, hybrid approaches that adaptively switch between collaborative and content-based signals based on data density, debiasing techniques to address popularity bias, and investigation of how collaborative awareness patterns learned by the model correlate with item semantics.