LLM-Driven News Recommendation via Lightweight Task-Adaptive Modules

Abstract

Large language models produce semantically rich embeddings, yet direct employment of generic LLM embeddings fails to satisfy news recommendation demands due to inherent semantic mismatches with task targets. Existing fine-tuning methods including LoRA can narrow such gaps but bring prohibitive computational overhead, restricting real-world deployment. This work proposes lightweight task-adaptive modules (TAMs). It keeps LLM parameters fixed and transforms offline embeddings into task-specialized representations without full-model backpropagation, drastically cutting training costs. Evaluated on MIND benchmarks across ten mainstream recommendation architectures, TAMs achieves comparable accuracy to LoRA, with computational cost reduced to 1/10 of LoRA’s level. TAMs outperform GloVe-based models by 1.3–12.4% in AUC and scale effectively to MINDlarge. Ablation experiments confirm that the nonlinear projection is pivotal to performance improvement, and statistical validation across three random seeds confirms result robustness. This paradigm provides an efficient low-cost solution for LLM-based news recommendation under resource constraints.

1. Introduction

The rapid growth of online news platforms has intensified the demand for personalized news recommendation systems that help users filter through massive volumes of daily content [1,2]. Neural news recommendation models have become the prevailing approach, typically relying on pre-trained word embeddings such as GloVe [3] combined with neural encoders to learn news and user representations [4,5]. These models have achieved promising performance, yet they suffer from one inherent limitation. Static word embeddings fail to capture the contextual semantics and linguistic nuance provided by large language models (LLMs).

LLMs such as BERT [6], GPT [7], and Qwen [8] produce contextualized embeddings that encode nuanced semantic relationships, offering representations qualitatively superior to static word vectors. Nevertheless, transferring these embeddings to downstream tasks remains challenging, given the semantic gap between general-purpose pre-training objectives and task-specific requirements. The standard remedy for this semantic gap is fine-tuning the LLM on the downstream task. Parameter-efficient methods like LoRA [9] reduce the cost relative to full fine-tuning, but still demand considerable computational resources. In our experiments, LoRA fine-tuning for news recommendation takes 10× longer than training the task-specific model alone, while the resulting performance gain is modest. This observation motivates a natural question. Can we bridge the semantic gap more efficiently, without backpropagating through the LLM at all?

This paper proposes task-adaptive modules (TAMs), lightweight end-to-end learnable projection layers that transform frozen LLM embeddings into task-specific representations. Rather than modifying the LLM’s internal parameters as fine-tuning does, TAMs keep the LLM entirely frozen and learn a compact transformation tailored to the recommendation objective. This design brings several practical benefits. TAMs introduce only 0.1–1% of additional parameters relative to the base model; training converges within the same time budget as the base model since no LLM backpropagation is involved; multiple TAMs for different tasks can share the same set of frozen LLM embeddings.

We evaluate TAMs on both MINDsmall and MINDlarge [10] across 10 neural news recommendation architectures spanning diverse design families, covering attention-based, temporal, topic-aware, multi-interest, hierarchical, and context-aware models. The main contributions of this paper are as follows:

• We propose TAMs, a lightweight and architecture-agnostic projection mechanism that bridges the semantic gap between frozen LLM embeddings and news recommendation objectives, without any LLM backpropagation. Unlike existing adapter layers or LoRA that require LLM backpropagation, TAMs operate entirely on pre-cached embeddings, decoupling the LLM from the training pipeline and enabling deployment on a single consumer GPU.
• We provide the first systematic comparison of TAMs versus direct LLM embeddings and LoRA fine-tuning across 10 architectures on both MINDsmall and MINDlarge, demonstrating that TAMs achieve performance comparable to fine-tuning at a fraction of the computational cost.
• We present ablation studies that identify the source of TAMs’ effectiveness and offer practical guidance for efficient LLM embedding transfer in recommendation systems.

The remainder of this paper is organized as follows. Section 2 reviews related work on neural news recommendation and LLM adaptation techniques. Section 3 describes the TAMs framework and experimental setup. Section 4 presents experimental results. Section 5 discusses implications and limitations. Section 6 concludes the paper.

2. Related Work

2.1. Neural News Recommendation

Neural news recommendation has progressed rapidly in recent years. Early work incorporated knowledge graphs to enrich news representations. DKN [5] introduced knowledge-aware CNNs that fuse entity embeddings from knowledge graphs with word-level features. Attention-based architectures soon became the mainstream approach. NRMS [1] uses multi-head self-attention for both news encoding and user modeling, relying entirely on attention mechanisms for representation learning. NAML [2] jointly models titles, bodies, and categories through parallel encoders with view-level attention. LSTUR [4] combines long-term user representations via user ID embeddings with short-term interests captured by GRU networks. TANR [11] augments news representations with auxiliary topic classification to capture topical preferences.

More recent architectures target complex user interest patterns. Multi-interest models like MINER [12] extract diverse user interests simultaneously through multiple context codes, moving beyond single-vector user representations. Hierarchical approaches such as HieRec [13] model interests at multiple granularities, from subtopic to topic to global levels. Context-aware models like CAUM [14] dynamically adjust user representations based on the candidate news article through candidate-aware attention. PP-Rec [15] conditions the recommendation model on user-specific signals via personalized prompts. FUM [16] concatenates clicked news into a long document and transforms user modeling into a document-level task with both intra-news and inter-news word interactions. BATM [17] introduces a bi-level attention mechanism that jointly models topic-level and word-level representations for explainable news encoding. More recently, MIPNR [18] extends multi-interest modeling by incorporating LLM-generated user preference descriptions alongside entity-level and category-level interest signals, constructing a local news entity graph to capture inter-article relationships for richer interest matching.

For all their architectural diversity, these models share a reliance on pre-trained word embeddings. This dependence on static embeddings constrains their capacity to represent contextual semantics, polysemy, and complex linguistic structures.

2.2. Pre-Trained Language Models for Recommendation

Integrating pre-trained language models (PLMs) into recommendation systems has received increasing attention. Wu et al. [19] first applied BERT to news recommendation and showed that PLM-based news encoders outperform traditional word embedding approaches. ONCE [20] uses both open-source and closed-source LLMs to generate enriched news representations, demonstrating that LLM-derived features can substantially improve recommendation quality. GENRE [21] takes a generative approach where LLMs directly produce news recommendations through prompting. More recently, PNR-LLM [22] proposes a hierarchical prompting pipeline that enriches news titles with deeper semantic context and entity information extracted by an LLM, then aggregates these features through attention for click prediction. HLLM [23] adopts a two-tier architecture where a first LLM extracts rich content features from item text and a second LLM models user interest over interaction history, demonstrating that hierarchical LLM deployment achieves strong scalability in sequential recommendation.

These methods, however, either fine-tune the language model or invoke it during online inference, both of which impose heavy computational costs. Fine-tuning a 7B-parameter LLM for recommendation demands GPU memory exceeding 40 GB and training times measured in days. Even parameter-efficient approaches like LoRA [9], which freeze most parameters and train low-rank adapters, still require backpropagation through the LLM, resulting in 10× slower training compared to task-specific models. ThinkRec [24] injects synthetic reasoning traces to activate chain-of-thought reasoning for recommendation, but requiring LLM inference at serving time. Subsequently the efficiency challenge is well recognized in the community. Wu et al. [25] survey efficient LLM4Rec solutions and identify the effectiveness–efficiency trade-off as the central open problem in the field.

A critical but underexplored issue is that LLMs may not attend to the most task-relevant parts of the input. Wang et al. [26] show that supervised fine-tuning induces context bias, causing LLMs to over-rely on superficial textual patterns rather than the semantics that drive user engagement. This finding reinforces the motivation for TAMs. Instead of fine-tuning the LLM and risking such bias, a lightweight adaptation layer can selectively amplify recommendation-relevant dimensions of frozen embeddings.

Another emerging research line follows an embed-then-recommend strategy, where LLM embeddings are pre-computed offline and treated as fixed input features for downstream models. Nevertheless, existing studies have not systematically explored how to bridge the distribution gap between generic LLM embeddings and recommendation task-specific demands.

2.3. Efficient Transfer Learning and Adaptation

Transfer learning from pre-trained models to downstream tasks has been studied extensively. Full fine-tuning updates all model parameters but is computationally expensive and susceptible to catastrophic forgetting. Parameter-efficient fine-tuning (PEFT) methods address this by updating only a small subset of parameters. LoRA [9] decomposes weight update matrices into low-rank factors, achieving competitive performance with fewer trainable parameters. Adapter layers [27] insert small bottleneck modules between transformer layers. Prompt tuning [28] prepends learnable tokens to the input sequence.

Although these methods reduce the number of trainable parameters, they still require gradient computation through the pre-trained model during training, which accounts for the bulk of the computational cost. Our proposed TAMs operate on frozen, pre-computed embeddings, eliminating LLM backpropagation entirely. This distinction matters in practice. TAMs can be trained on a single consumer GPU in minutes, whereas LoRA fine-tuning of a 7B model requires multiple high-end GPUs for hours.

Learned projections between embedding spaces have been studied in cross-lingual transfer [29] and vision-language alignment [30]. Our work extends this idea to recommendation, providing the first broad study of lightweight projection-based adaptation for LLM embeddings across diverse recommendation architectures.

TAMs differ from existing lightweight adaptation mechanisms along three dimensions. First, position: adapter layers and LoRA modify the internal transformer computation graph and therefore still require LLM backpropagation; TAMs operate entirely outside the LLM on pre-cached embeddings, so the LLM is never loaded during training. Second, granularity: projection heads in vision–language alignment [30] and cross-lingual transfer [29] are trained end-to-end with the backbone, whereas TAMs are trained solely with the downstream recommendation loss and no backbone gradient. Third, scope: prior embedding-alignment work targets a single architecture, while TAMs are designed to work across heterogeneous recommendation architectures spanning attention-based, temporal, topic-aware, multi-interest, hierarchical, and context-aware designs. These distinctions collectively position TAMs as a novel operating point in the efficiency–effectiveness trade-off for LLM-based recommendation.

3. Methodology

3.1. Problem Formulation

We formulate news recommendation as a click prediction task. Given a user’s historical clicked news sequence $\mathcal{H}=\{n_1,n_2,\dots ,n_k\}$ and a candidate news article $n_c$, the goal is to predict the probability $p\left(click\right|n_c,\mathcal{H})$ that the user will click on $n_c$. Each news article is represented by its textual content.

In traditional approaches, each word is mapped to a pre-trained embedding ${\mathbf{e}}_i$, which is typically GloVe. A neural encoder $f_{news}$ aggregates these word embeddings into a news representation ${\mathbf{r}}_i=f_{news}({\mathbf{e}}_1,\dots ,{\mathbf{e}}_m)$. A user encoder $f_{user}$ then aggregates historical news representations into a user representation $\mathbf{u}=f_{user}({\mathbf{r}}_1,\dots ,{\mathbf{r}}_k)$. The click probability is computed as $p\left(click\right|n_c,\mathcal{H})=\sigma \left({\mathbf{u}}^T{\mathbf{r}}_c\right)$, where $\sigma$ denotes the sigmoid function.

3.2. LLM Embeddings for News Representation

LLMs produce contextualized embeddings that encode richer semantics than static word vectors. Given a news article, we use Qwen2.5-7B-Instruct [8] as the pre-trained LLM to generate a news-level embedding ${\mathbf{z}}_n\in {\mathbb{R}}^d$, where $d_{LLM}=3584$. We extract the embedding from the last hidden layer using mean pooling over all tokens.

These embeddings are computed offline and cached, so no online LLM inference is needed during training or serving. Under this embed-then-recommend strategy, the resulting vectors can be reused across all subsequent experiments.

3.3. Task-Adaptive Modules

Figure 1 illustrates the overall TAMs framework, which consists of an offline embedding stage and an online training stage.

Figure 1. Overall framework of TAMs for news recommendation. Stage 1 (offline): a frozen LLM extracts embeddings for all news articles once and caches them to disk. Stage 2 (online): cached embeddings are loaded and transformed by TAMs before being fed into the recommendation model. Only TAMs and the recommendation model parameters are updated during training; the LLM remains frozen throughout. The dimensions and parameters can be modified according to different models.

3.3.1. The Semantic Gap as Motivation

Although LLM embeddings encode broad semantic information, they are not tailored to the specific demands of news recommendation. Our preliminary experiments reveal a counterintuitive finding: directly using frozen LLM embeddings ${\mathbf{z}}_n$ as news representations yields worse performance than GloVe embeddings on several architectures. NAML with direct Qwen embeddings achieves only 0.6274 AUC compared to 0.6747 with GloVe, a 3.6% drop despite the far greater capacity of the embedding model.

This performance degradation stems from the semantic gap between LLM pre-training objectives including masked language modeling and next-token prediction, and recommendation objectives that forecast user clicks via interest matching. LLM embeddings are optimized to capture linguistic traits like syntax, semantics and discourse structure, yet fail to prioritize factors driving user engagement including topic relevance, writing style and emotional tone.

3.3.2. TAMs Architecture

To bridge this semantic gap, we propose task-adaptive modules, lightweight neural networks that project LLM embeddings into a task-specific representation space. The detailed internal structure is shown in the right panel of Figure 1.

The TAMs architecture consists of four components.

$$\mathrm{TAMs}\left(\mathbf{z}\right)=\mathrm{Dropout}\left(\mathrm{ReLU}\right(\mathrm{LayerNorm}(\mathbf{W}\mathbf{z}+\mathbf{b})\left)\right)$$

(1)

where $\mathbf{W}\in {\mathbb{R}}^{d_{task}\times d_{llm}}$ is a learnable projection matrix, $\mathbf{b}\in {\mathbb{R}}^{d_{task}}$ is a bias vector, and $d_{task}$ is the target dimension, which ranges from 256 to 400 depending on the base architecture. Each component serves a specific role.

• Linear projection. The core learnable component that maps the 3584-dimensional LLM space to a lower-dimensional task space through a task-specific linear transformation.
• Layer normalization. Stabilizes the distribution of projected features across different news articles, preventing gradient instability during training.
• ReLU activation. Introduces nonlinearity so that TAMs can learn transformations beyond linear projections, which is important for capturing complex feature interactions.
• Dropout. Regularizes the model to prevent overfitting, which is particularly relevant given the high dimensionality of LLM embeddings relative to the size of recommendation datasets.

TAMs are integrated into the news encoder as the first layer, replacing the word embedding lookup.

$${\mathbf{r}}_n=f_{news}\left(\mathrm{TAMs}\left({\mathbf{z}}_n\right)\right)$$

(2)

The TAMs parameters are learned end-to-end with the recommendation model through standard backpropagation, while the LLM remains frozen. This design has three practical advantages.

1.

Computational efficiency. TAMs add only $d_{task}\times d_{llm}$ parameters, amounting to 0.01% of the 7B-parameter LLM. No LLM backpropagation is needed, reducing memory usage and training time to one-tenth of the original level.

2.

Modularity. Different TAMs can serve different tasks (news recommendation, sentiment analysis, topic classification) while sharing the same frozen LLM embeddings.

3.

Interpretability. The learned projection matrix $\mathbf{W}$ can be analyzed to identify which LLM dimensions are most relevant for the recommendation task.

The nonlinear activation is a deliberate design choice. A purely linear projection can only rotate and scale the LLM embedding space, leaving the relative structure of dimensions unchanged. The LayerNorm–ReLU block allows TAMs to selectively suppress dimensions that are irrelevant to recommendation and amplify those that correlate with user engagement signals, which is necessary given the distribution mismatch between LLM pre-training objectives and recommendation objectives. The performance gains of TAMs therefore arise from task-specific feature re-weighting, not merely from adding trainable projection parameters.

3.3.3. Architecture-Specific Integration

Although TAMs replace the word embedding lookup uniformly across all models, each base architecture requires tailored integration to preserve its original sequence-level design.

Attention-based models, including NRMS, NAML, MINER, HieRec, and CAUM. These models originally process sequences of word embeddings through CNNs or multi-head self-attention. After TAMs project the news-level LLM embedding ${\mathbf{z}}_n\in {\mathbb{R}}^{3584}$ to a single vector ${\widehat{\mathbf{z}}}_n\in {\mathbb{R}}^{d_{task}}$, this vector replaces the aggregated news representation that was previously produced by the word-level encoder, such as the CNN output or attention-pooled sequence. Sequence-level information is intentionally removed; the LLM embedding already integrates the full token sequence via mean pooling, so the position-wise aggregation originally performed by CNN or MHSA becomes redundant. Dimensional compatibility is ensured by setting $d_{task}$ to match the original news encoder output dimension: 256 for NRMS, 300 for NAML, and 256 for MINER, HieRec, and CAUM.

Temporal models, namely LSTUR and BATM. LSTUR employs a GRU over history and BATM uses bi-level attention; both rely on a sequence of news vectors rather than word vectors. TAMs produce one vector per news article, which feeds directly into the GRU hidden state update or the bi-level attention input without structural modification.

The topic-aware model TANR. TANR augments its news encoder with an auxiliary topic classification loss. TAMs replace the word embedding lookup; the projected vector is passed to both the news encoder and the topic classification head. No change to the auxiliary loss is required.

The personalized model PP-Rec. PP-Rec conditions predictions on real-time popularity signals in addition to textual content. TAMs supply the textual news representation; the popularity branch remains unchanged. Multi-view architectures such as NAML that separately encode title, abstract, and category are handled by feeding the single TAMs-projected vector to the joint view-attention layer, bypassing the separate view encoders while retaining the view-level attention weighting over the fused representation.

The document-level model FUM. FUM concatenates all clicked news into a single long document and models both intra-news and inter-news word interactions. TAMs replace the per-article word embedding lookup with a projected LLM embedding vector, which is then passed as the news-level representation into FUM’s document-level attention mechanism. The inter-news interaction structure is preserved unchanged.

3.3.4. Comparison with Fine-Tuning

To contextualize TAMs, we compare them with LoRA [9], a widely used parameter-efficient fine-tuning method. LoRA introduces low-rank decomposition of weight update matrices in the LLM.

$${\mathbf{W}}_{new}={\mathbf{W}}_{frozen}+\mathbf{B}\mathbf{A}$$

(3)

where $\mathbf{B}\in {\mathbb{R}}^{d\times r}$ and $\mathbf{A}\in {\mathbb{R}}^{r\times d}$ are trainable low-rank matrices with rank $r\ll d$. Although LoRA reduces the number of trainable parameters relative to full fine-tuning, it still requires forward passes through the entire LLM during training, gradient computation through the LLM even with most weights frozen, and storage of activations for all LLM layers during backpropagation.

TAMs sidestep all of these costs by operating on pre-computed, frozen embeddings, requiring neither LLM forward passes nor gradient storage through the LLM during training. This makes TAMs substantially more memory-efficient and faster to train than LoRA, while still benefiting from the rich semantic content encoded in LLM embeddings.

3.4. Experimental Setup

3.4.1. Dataset

We conduct experiments on the MIND dataset [10], a large-scale benchmark for news recommendation containing 1 million users, 161,013 news articles, and 15 million impression logs collected from Microsoft News. Each news article includes a title, abstract, category, and subcategory. The main experiments use the MINDsmall subset, which contains 50,000 users and 65,238 news articles. To assess scalability, we additionally evaluate all ten architectures on the full MINDlarge split, which covers all 161,013 news articles and approximately 1 million users. Both splits follow the official training/validation/test partition.

3.4.2. LLM Embedding Generation

We use Qwen2.5-7B-Instruct [8] to generate news embeddings. For each news article, we extract the embedding from the last hidden layer after feeding the title text through the model. The resulting embeddings have dimension 3584. All embeddings are pre-computed offline and cached to disk, requiring approximately 892 MB of storage for the entire MINDsmall dataset.

3.4.3. Baseline Architectures

To evaluate TAMs across a broad range of designs, we experiment with 10 neural news recommendation architectures:

• NRMS [1]: multi-head self-attention for news and user encoding.
• NAML [2]: attentive multi-view learning with parallel encoders.
• LSTUR [4]: long-term and short-term user representations via GRU.
• TANR [11]: topic-aware news representation with auxiliary classification.
• BATM [17]: bi-level attention based explainable news topic modeling.
• FUM [16]: fine-grained user modeling with hierarchical attention.
• CAUM [14]: context-aware user modeling with candidate-aware attention.
• PP-Rec [15]: news recommendation with personalized user interest and time-aware news popularity.
• HieRec [13]: hierarchical interest modeling at multiple granularities.
• MINER [12]: multi-interest extraction with context codes.

For each architecture, we implement three variants: the GloVe baseline using 300-dimensional GloVe embeddings, direct LLM using frozen Qwen embeddings without TAMs, and TAMs using Qwen embeddings with task-adaptive modules.

3.4.4. Training Details

All models are trained with the Adam optimizer, learning rate 1 × 10⁻⁴, batch size 64, and early stopping based on validation AUC. Negative sampling uses 4 negative samples per positive sample. Dropout rate is 0.2 for all models. TAMs projection dimensions are set to 256 for NRMS/LSTUR/TANR, 300 for NAML, and 400 for BATM/FUM/CAUM/PP-Rec/HieRec/MINER, matching the hidden dimensions of the original architectures. All experiments run on NVIDIA RTX A5000 GPUs with 24 GB memory.

3.4.5. Evaluation Metrics

We evaluate models using four standard metrics. AUC, which measures ranking quality as the area under the ROC curve; MRR, which is the mean reciprocal rank emphasizing top-ranked items; nDCG@5 and nDCG@10, which are normalized discounted cumulative gain at positions 5 and 10 respectively. All metrics are computed on the test set, and we report mean values across 3 random seeds with seeds 42, 123, and 456. Standard deviations across seeds are reported as italic second rows in Table 1 to assess result stability.

Table 1. Performance comparison of GloVe baselines and TAMs-equipped models on MINDsmall. Results are means over three random seeds; for each model the first row reports the mean and the second row (in italic) reports the standard deviation. Best TAMs values per architecture are in bold. ΔAUC is computed from the seed means.

Model	GloVe Baseline				TAMs				ΔAUC
	AUC	MRR	nDCG@5	nDCG@10	AUC	MRR	nDCG@5	nDCG@10
CAUM	0.6584	0.3093	0.3416	0.4059	0.6971	0.3416	0.3806	0.4426	+0.0387
	±0.0036	±0.0020	±0.0029	±0.0024	±0.0016	±0.0008	±0.0010	±0.0008
NRMS	0.6594	0.3122	0.3443	0.4075	0.6939	0.3386	0.3777	0.4400	+0.0345
	±0.0001	±0.0020	±0.0028	±0.0019	±0.0001	±0.0002	±0.0006	±0.0003
PP-Rec	0.6783	0.3260	0.3630	0.4248	0.6924	0.3324	0.3735	0.4353	+0.0141
	±0.0032	±0.0019	±0.0025	±0.0020	±0.0030	±0.0055	±0.0049	±0.0043
TANR	0.6121	0.2870	0.3142	0.3736	0.6882	0.3394	0.3768	0.4383	+0.0761
	±0.0044	±0.0052	±0.0055	±0.0048	±0.0008	±0.0021	±0.0026	±0.0021
MINER	0.6299	0.3018	0.3277	0.3900	0.6919	0.3392	0.3793	0.4399	+0.0620
	±0.0042	±0.0020	±0.0019	±0.0026	±0.0002	±0.0014	±0.0018	±0.0011
FUM	0.6430	0.2994	0.3286	0.3937	0.6880	0.3386	0.3764	0.4380	+0.0450
	±0.0011	±0.0037	±0.0023	±0.0032	±0.0051	±0.0035	±0.0051	±0.0042
NAML	0.6747	0.3217	0.3558	0.4186	0.6883	0.3358	0.3761	0.4370	+0.0136
	±0.0017	±0.0017	±0.0017	±0.0017	±0.0013	±0.0009	±0.0007	±0.0008
BATM	0.6618	0.3175	0.3484	0.4116	0.6817	0.3309	0.3662	0.4295	+0.0199
	±0.0031	±0.0016	±0.0028	±0.0017	±0.0024	±0.0021	±0.0023	±0.0023
HieRec	0.6671	0.3202	0.3537	0.4173	0.6758	0.3307	0.3661	0.4275	+0.0087
	±0.0030	±0.0027	±0.0038	±0.0032	±0.0008	±0.0003	±0.0005	±0.0004
LSTUR	0.6480	0.3012	0.3299	0.3940	0.6746	0.3197	0.3556	0.4189	+0.0266
	±0.0044	±0.0007	±0.0007	±0.0007	±0.0032	±0.0026	±0.0038	±0.0028

4. Experiments

4.1. Main Results with TAMs vs. GloVe Baselines

Table 1 compares GloVe-based models with their TAMs-equipped counterparts across all 10 architectures. TAMs consistently improve performance over GloVe baselines on every architecture and metric.

Several observations emerge from Table 1.

Consistent improvement across all architectures. Every TAMs-equipped model outperforms its GloVe counterpart, with AUC gains ranging from +0.0087 of HieRec to +0.0761 of TANR. This consistency across 10 architectures covering attention-based, temporal, topic-aware, multi-interest, hierarchical, and context-aware designs confirms that TAMs provide a broadly effective mechanism for transferring LLM embeddings.

Largest gains for models with weaker GloVe baselines. TANR achieves an AUC improvement of +0.0761 and a performance gain of 12.4 percent, while MINER obtains an AUC improvement of +0.0620 and a performance gain of 9.8 percent, making them the two models with the most prominent performance benefits. TANR yields a notably weak baseline result of 0.6121 AUC when adopting GloVe. Its auxiliary topic classification loss can only bring limited performance improvement under static word embeddings, and TAMs effectively supplement the semantic richness required by the topic-aware architecture. MINER adopts simple CNN encoders as well, which can receive substantial performance improvements from more abundant input representations. PP-Rec gains an AUC increase of +0.0141 and a performance lift of 2.1 percent, and BATM gains an AUC increase of +0.0199 and a performance lift of 3.0 percent. These two models show relatively modest performance gains because they already integrate powerful text modeling mechanisms.

CAUM achieves the highest overall performance. CAUM with TAMs reaches 0.6971 AUC. Its context-aware attention mechanism appears well-suited to exploit the semantic richness of LLM embeddings, yielding the highest MRR with 0.3416 and nDCG@5 with 0.3806 among all models.

Figure 2 visualizes the AUC comparison across models with available GloVe baselines, showing the consistent advantage of TAMs over both GloVe and direct LLM embeddings.

Figure 2. AUC comparison of GloVe, Direct LLM, and TAMs across representative architectures. TAMs consistently outperform both alternatives. Direct LLM embeddings underperform GloVe on NAML, underscoring the necessity of task adaptation.

Figure 3 ranks all 10 TAMs-equipped models, showing that all models achieve an AUC above 0.67 and eight of ten models exceed 0.68 across all architectures.

Figure 3. Performance ranking of all 10 TAMs-equipped models on the MIND dataset. CAUM achieves the highest AUC of 0.6971, while even the lowest-performing model LSTUR with 0.6746 AUC substantially outperforms GloVe baselines.

4.2. TAMs vs. Direct LLM Embeddings

A natural question is whether TAMs are truly necessary, or whether LLM embeddings can be fed directly into recommendation models. Table 2 compares three embedding strategies: GloVe, direct LLM embeddings without projection, and TAMs-projected LLM embeddings.

Table 2. Comparison of embedding strategies on NRMS and NAML. Direct Qwen uses PCA-reduced 300-dimensional Qwen2.5-7B-Instruct embeddings (matching the GloVe input dimension) without any learned projection; this controls for dimensionality so that the comparison isolates task adaptation rather than dimensionality effects. TAMs adds the supervised task-adaptive projection layer trained end-to-end with the recommendation objective.

Model	Embedding	AUC	MRR	nDCG@5	nDCG@10
NRMS	GloVe	0.6594	0.3122	0.3443	0.4075
	Direct Qwen	0.6663	0.3210	0.3535	0.4168
	TAMs-Qwen	0.6939	0.3386	0.3777	0.4400
NAML	GloVe	0.6747	0.3217	0.3558	0.4186
	Direct Qwen	0.6274	0.2898	0.3187	0.3825
	TAMs-Qwen	0.6883	0.3358	0.3761	0.4370

Note on the direct Qwen baseline. Feeding raw 3584-dimensional Qwen embeddings directly into models designed for 200–400-dimensional inputs creates a dimensionality mismatch that could independently depress performance, confounding the comparison with TAMs. To avoid this, the “Direct Qwen” entry in Table 2 uses PCA-reduced 300-dimensional embeddings to match the GloVe input dimension, not the raw 3584-dimensional vectors. This ensures that the comparison isolates the effect of task-supervised adaptation rather than dimensionality effects. The PCA projection is unsupervised and uses no recommendation labels; TAMs adds a supervised linear projection trained end-to-end with the recommendation objective. The gap between direct Qwen with PCA reduction and TAMs therefore reflects the benefit of task-supervised adaptation specifically.

The results reveal a clear pattern.

Direct LLM embeddings can bring performance gains on some architectures. For NRMS, direct Qwen embeddings substantially outperform the GloVe baseline across all metrics, with AUC rising from 0.6594 to 0.6663. However, direct embeddings may degrade performance on other architectures. For NAML, direct Qwen embeddings achieve only 0.6274 AUC, which is noticeably below the GloVe baseline of 0.6747, representing a 7.0% performance drop. The high dimensionality and general-purpose nature of LLM embeddings can overwhelm task-specific recommendation models, introducing representation components irrelevant to click prediction and making optimization harder.

TAMs are necessary, not optional. TAMs convert the same LLM embeddings from a liability into a clear advantage. NAML with TAMs achieves 0.6883 AUC, outperforming both direct Qwen with 0.6274 AUC and GloVe with 0.6747 AUC. The projection filters out task-irrelevant information while retaining and amplifying task-relevant semantic features. LLM embedding spaces contain considerable redundancy for recommendation tasks, and TAMs learn to extract a compact, task-relevant subspace.

4.3. Performance Analysis of TAMs and LoRA Fine-Tuning

Table 3 compares TAMs with LoRA fine-tuning on NRMS. Figure 4 visualizes the performance–cost trade-off, showing that TAMs sit at the optimal point on the Pareto frontier for resource-constrained settings.

Table 3. Comparison of TAMs and LoRA fine-tuning on NRMS. GloVe and TAMs values are means over 3 random seeds; LoRA values are single-run results. LoRA configuration: rank $r=8$, $\alpha =16$, dropout $=0.05$; “R10” targets $\{q,v\}$ for 10 rounds × 200 steps; “qkvo” additionally includes $\{k,o\}$; all runs on two RTX A5000 GPUs.

Method	AUC	MRR	nDCG@5	nDCG@10	Rel. Cost
GloVe baseline	0.6594	0.3122	0.3443	0.4075	1×
Direct Qwen	0.6663	0.3210	0.3535	0.4168	1×
TAMs-Qwen	0.6939	0.3386	0.3777	0.4400	1×
LoRA (R10, 200 steps)	0.6948	0.3406	0.3793	0.4416	∼10×
LoRA (qkvo, lr = 1 × 10−5)	0.7008	0.3420	0.3818	0.4445	∼15×

Figure 4. Performance vs. training cost trade-off. TAMs achieve performance comparable to standard LoRA, with a multi-seed mean AUC of 0.6939 against LoRA’s 0.6948, at 1/10th the cost. The aggressive LoRA variant with qkvo fine-tuning and lr = 1 × 10⁻⁵ reaches marginally higher performance with 0.7008 AUC but requires 15× the training cost.

The comparison yields three observations.

TAMs match standard LoRA at 10× lower cost. TAMs achieve 0.6939 AUC, broadly comparable to the standard LoRA configuration of 0.6948 AUC. The experimental setting adopts 200 steps per round for 10 rounds, and TAMs only require one tenth of the training time. TAMs training involves only the lightweight recommendation model, whereas LoRA requires forward and backward passes through the 7B-parameter LLM.

Aggressive LoRA can marginally surpass TAMs. The most expensive LoRA variant adopts qkvo targets and a learning rate of 1 × 10⁻⁵. It reaches 0.7008 AUC and exceeds the TAMs multi-seed mean of 0.6939 by 1.0%. However, this configuration requires approximately 15× the training cost and careful hyperparameter tuning. Whether the 1.0% AUC gain justifies a 15× cost increase depends on the deployment context.

Complete cost breakdown. The reported relative-cost figures above capture only online training time. Table 4 provides a full cost breakdown across all stages.

Table 4. Full cost comparison between TAMs and LoRA across all pipeline stages, evaluated on MINDsmall with a single RTX A5000 24 GB GPU.

Cost Component	TAMs	LoRA
Offline embedding generation	45 min, one-time	45 min, one-time
Online training time	8–12 min	90–120 min
GPU memory (training)	8 GB, single GPU	≥40 GB, multi-GPU
Embedding cache storage	892 MB, shared	892 MB, shared
Trainable model size	4.5–17 MB	60–80 MB for LoRA adapters
New-article embedding cost	≈0.1 s/article	≈0.1 s/article
Retraining on new articles	Not required	Not required

Offline embedding generation is a one-time cost: encoding all 65,238 MINDsmall news articles with Qwen2.5-7B-Instruct on a single RTX A5000 with 24 GB memory requires approximately 45 min with batch size 4. The resulting 892 MB cache is reused across all subsequent experiments and architectures.

Online training is substantially faster for TAMs: training completes in 8–12 min on a single GPU. LoRA fine-tuning with 10 rounds of 200 steps each requires approximately 90–120 min and a minimum of 40 GB GPU memory across multiple devices. The $10\times$ training-time advantage therefore holds after accounting for the shared embedding generation cost whenever more than one architecture or experiment reuses the same cache.

For incremental embedding of new articles, each new news article requires a single forward pass through the frozen LLM, taking approximately 0.1 s per article on an RTX A5000, whereas LoRA fine-tuning must re-run if new articles shift the distribution. TAMs require no retraining when new articles arrive; the frozen projection applies directly to the new cached embeddings.

4.4. Model Size Analysis

Table 5 compares TAMs and LoRA across four architectures, namely NRMS as an attention-based model, NAML as a CNN-based model, TANR as a topic-aware model, and MINER as a multi-interest model, to verify that the parity holds regardless of model design. Across NRMS, NAML, and TANR, LoRA leads TAMs by a small margin of 0.0009–0.0057 AUC at roughly 10× the training cost. On MINER, the direction reverses: TAMs reach 0.6919 AUC against LoRA’s 0.6898, suggesting that multi-interest architectures with weaker GloVe baselines derive stronger benefit from the task-specific supervision embedded in TAMs training.

Table 5. TAMs vs. LoRA across four architectures on MINDsmall. TAMs values are means over 3 random seeds; LoRA values are single-run results. LoRA uses rank $r=8$, $\alpha =16$, targets $\{q,v\}$, lr $=1\times {10}^{-5}$, 10 rounds × 200 steps.

Model	Method	AUC	MRR	nDCG@5	nDCG@10	Rel. Cost
NRMS	GloVe	0.6594	0.3122	0.3443	0.4075	1×
	TAMs	0.6939	0.3386	0.3777	0.4400	1×
	LoRA	0.6948	0.3406	0.3793	0.4416	∼10×
NAML	GloVe	0.6747	0.3217	0.3558	0.4186	1×
	TAMs	0.6883	0.3358	0.3761	0.4370	1×
	LoRA	0.6940	0.3381	0.3773	0.4392	∼10×
TANR	GloVe	0.6121	0.2870	0.3142	0.3736	1×
	TAMs	0.6882	0.3394	0.3768	0.4383	1×
	LoRA	0.6936	0.3410	0.3799	0.4420	∼10×
MINER	GloVe	0.6299	0.3018	0.3277	0.3900	1×
	TAMs	0.6919	0.3392	0.3793	0.4399	1×
	LoRA	0.6898	0.3386	0.3792	0.4394	∼10×

Table 6 compares the model sizes of GloVe-based and TAMs-equipped models. To avoid conflating different notions of size, we distinguish four components: trainable model size refers to the on-disk checkpoint of learnable parameters; deployment footprint is the memory required at serving time for model weights; embedding cache is the pre-computed LLM embedding matrix stored on disk and shared across all architectures; and total system cost is the sum of the above at inference time.

Table 6. Model size comparison between GloVe baselines and TAMs models. Compression ratio is computed as (GloVe size − TAMs size)/GloVe size. The 892 MB LLM embedding cache is shared across all TAMs models and counted once in total system cost.

Model	GloVe Size	TAMs Size	Compression
NRMS	20 MB	4.5 MB	77.5%
NAML	20 MB	4.4 MB	78.0%
PP-Rec	185 MB	17 MB	90.8%
CAUM	88 MB	5.5 MB	93.8%
MINER	86 MB	4.6 MB	94.7%
HieRec	87 MB	5.2 MB	94.0%

TAMs models are 78–95% smaller than their GloVe counterparts. This reduction has two sources. First, eliminating the large word embedding matrix, which is $\left|V\right|\times 300$ for GloVe where $\left|V\right|$ is the vocabulary size; second, the compact TAMs projection layer, which requires only $3584\times 256\approx 917$ K parameters. The pre-computed LLM embeddings are stored separately and shared across all models, amortizing their storage cost.

Regarding total system cost: the 892 MB embedding cache is a one-time storage cost shared across all TAMs-equipped models and all experiments. When deploying a single TAMs model, for example NRMS, the total system cost is 4.5 MB for the trainable checkpoint plus the shared 892 MB cache, yielding approximately 897 MB. By contrast, a GloVe-based NRMS model requires 20 MB for the model weights plus the 840 MB GloVe vocabulary, totaling approximately 860 MB. The per-model storage advantage therefore lies in the trainable checkpoint, which is 77–95% smaller, while the total system cost is broadly comparable once the shared embedding cache is included. This trade-off is favorable in practice because the embedding cache enables all ten TAMs models to be served simultaneously with no duplication of the 892 MB storage cost.

Figure 5 visualizes the model size reduction, with compression ratios ranging from 78% to 95%.

Figure 5. Model size comparison between GloVe baselines and TAMs-equipped models. TAMs achieve 78–95% compression while improving performance. The red percentages indicate compression ratios.

4.5. Ablation Studies

4.5.1. TAMs Component Analysis

To understand the contribution of each component, we conduct ablation studies on the NRMS architecture. Table 7 presents the results.

Table 7. Ablation study of TAMs components on NRMS. Each row removes one component from the full TAMs (Linear + LayerNorm + ReLU + Dropout).

Variant	AUC	MRR	nDCG@5	nDCG@10
Full TAMs	0.6939	0.3386	0.3777	0.4400
with Gating mechanism	0.6725	0.3328	0.3687	0.4291
with User profile fusion	0.6675	0.3264	0.3605	0.4222
No projection (direct)	0.6663	0.3210	0.3535	0.4168

The simple Linear + LayerNorm + ReLU + Dropout design outperforms more complex alternatives. Adding a gating mechanism with 0.6725 AUC or user profile fusion with 0.6675 AUC actually degrades performance relative to the standard TAMs with 0.6939 AUC. The effectiveness of TAMs comes from learning a clean, task-specific projection rather than from elaborate feature interactions. Simplicity here is a strength.

4.5.2. Projection Structure Analysis

We further investigate whether a two-stage bottleneck projection (3584 → intermediate dim → 256) outperforms the standard single-stage projection (3584 → 256). Table 8 reports results on NRMS for intermediate dimensions of 64, 128, and 512, compared against the single-stage baseline.

Table 8. Ablation study of TAMs projection structure on NRMS. Single-stage projects 3584 d directly to 256 d. Two-stage introduces an intermediate bottleneck before the final 256 d output.

Projection Structure	AUC	MRR	nDCG@5	nDCG@10
Single-stage (3584 → 256)	0.6939	0.3386	0.3777	0.4400
Two-stage (3584 → 64 → 256)	0.6885	0.3365	0.3746	0.4368
Two-stage (3584 → 128 → 256)	0.6910	0.3318	0.3715	0.4338
Two-stage (3584 → 512 → 256)	0.6932	0.3366	0.3758	0.4379

The single-stage projection consistently outperforms all two-stage variants. Introducing an intermediate bottleneck at 64 or 512 dimensions degrades AUC by 0.0054 and 0.0007, respectively, while the 128-dimensional bottleneck comes closest but still falls 0.0029 AUC short. The results suggest that forcing the projection through a narrower intermediate space discards task-relevant information before the final representation is formed. A direct linear mapping from the LLM embedding space to the recommendation space is both simpler and more effective.

4.5.3. Architecture-Specific Analysis of CAUM

We conduct additional ablation studies on CAUM, which achieves the best overall performance. Table 9 examines the contribution of CAUM’s context-aware components.

Table 9. Ablation study on CAUM-TAMs. “no_gate” removes the gating mechanism; “cand_only” uses only candidate news for attention. The Full TAMs value is the multi-seed mean; ablation variant values are single-run comparisons.

Variant	AUC	MRR	nDCG@5	nDCG@10	Best Epoch
CAUM-TAMs (full)	0.6971	0.3416	0.3806	0.4426	7
no_gate	0.6925	0.3392	0.3776	0.4401	7
cand_only	0.6906	0.3394	0.3789	0.4402	5

The full CAUM-TAMs model outperforms both ablation variants, confirming that the context-aware gating mechanism contributes meaningfully when combined with TAMs-projected LLM embeddings. The no_gate variant with 0.6925 AUC still substantially outperforms the GloVe baseline with 0.6584 AUC, indicating that TAMs account for the bulk of the improvement.

4.6. Improvement Magnitude Analysis

Table 10 shows the AUC improvement magnitude across architectures, revealing how different model families benefit from TAMs.

Table 10. AUC improvement of TAMs over GloVe baselines, ranked by magnitude. Relative improvement is computed as ΔAUC/GloVe AUC.

Model	GloVe	TAMs	Rel. Impr.	Architecture Type
TANR	0.6121	0.6882	+12.4%	Topic-aware
MINER	0.6299	0.6919	+9.8%	Multi-interest
FUM	0.6430	0.6880	+7.0%	Fine-grained
NRMS	0.6594	0.6939	+5.2%	Attention-based
CAUM	0.6584	0.6971	+5.9%	Context-aware
LSTUR	0.6480	0.6746	+4.1%	Long/short-term
BATM	0.6618	0.6817	+3.0%	Topic-attention
PP-Rec	0.6783	0.6924	+2.1%	Prompt-based
NAML	0.6747	0.6883	+2.0%	Multi-view
HieRec	0.6671	0.6758	+1.3%	Hierarchical

The improvement magnitude correlates inversely with the GloVe baseline strength. Models with weaker GloVe baselines benefit the most. TANR achieves a performance increase of 12.4%, as it suffers from limited topic signal in static embeddings. MINER gains 9.8% and FUM gains 7.0%; both rely on relatively simple encoders that gain substantially from richer input representations. Models with stronger GloVe baselines and more sophisticated text processing show smaller but consistent gains. NAML and HieRec show the smallest gains, +2.0% and +1.3%, respectively, as these models already incorporate multi-view or hierarchical mechanisms that partially capture the semantic richness TAMs provide.

4.7. Generalization to MINDlarge

To assess whether TAMs generalize beyond the MINDsmall benchmark, we evaluate all ten architectures on MINDlarge, which contains 161,013 news articles and approximately 1 million users, roughly 10× the scale of MINDsmall. Table 11 compares TAMs performance across both datasets, enabling a direct assessment of how each architecture scales with data volume. To substantiate the scalability claim, Table 12 additionally provides GloVe and direct Qwen baseline comparisons for two representative architectures, namely NRMS and NAML, on MINDlarge.

Table 11. TAMs performance on MINDsmall and MINDlarge. All models use Qwen2.5-7B-Instruct embeddings with task-adaptive projection. MINDsmall values are means over three random seeds; MINDlarge values are single-run results.

Model	MINDsmall				MINDlarge
	AUC	MRR	nDCG@5	nDCG@10	AUC	MRR	nDCG@5	nDCG@10
PP-Rec	0.6924	0.3324	0.3735	0.4353	0.7172	0.3559	0.3985	0.4603
CAUM	0.6971	0.3416	0.3806	0.4426	0.7080	0.3510	0.3929	0.4550
MINER	0.6919	0.3392	0.3793	0.4399	0.7069	0.3511	0.3921	0.4529
NRMS	0.6939	0.3386	0.3777	0.4400	0.7066	0.3528	0.3937	0.4558
NAML	0.6883	0.3358	0.3761	0.4370	0.7044	0.3494	0.3902	0.4509
TANR	0.6882	0.3394	0.3768	0.4383	0.7008	0.3458	0.3861	0.4477
BATM	0.6817	0.3309	0.3662	0.4295	0.6999	0.3443	0.3834	0.4455
FUM	0.6880	0.3386	0.3764	0.4380	0.6963	0.3437	0.3816	0.4446
HieRec	0.6758	0.3281	0.3636	0.4268	0.6908	0.3375	0.3764	0.4401
LSTUR	0.6746	0.3197	0.3556	0.4189	0.6802	0.3242	0.3603	0.4245

Table 12. Baseline comparison on MINDlarge for representative architectures. GloVe uses 300-dimensional pre-trained word embeddings; direct Qwen uses frozen Qwen2.5-7B embeddings (PCA-reduced to 300 d) without TAMs; TAMs uses the full task-adaptive projection. ΔAUC is computed relative to the GloVe baseline.

Model	Method	AUC	MRR	nDCG@5	nDCG@10	ΔAUC
NRMS	GloVe	0.6770	0.3243	0.3609	0.4246	–
	Direct Qwen	0.7013	0.3452	0.3857	0.4476	+0.0243
	TAMs	0.7066	0.3528	0.3937	0.4558	+0.0296
NAML	GloVe	0.6877	0.3325	0.3694	0.4321	–
	Direct Qwen	0.6563	0.3118	0.3453	0.4078	−0.0314
	TAMs	0.7044	0.3494	0.3902	0.4509	+0.0167

All ten models achieve strong performance on MINDlarge, with PP-Rec reaching 0.7172 AUC and five models exceeding 0.70 AUC. Compared to MINDsmall results, the absolute AUC values are consistently higher on MINDlarge, which is expected given the larger training set. The relative ordering across architectures is largely preserved: PP-Rec and CAUM remain the top performers, and MINER ranks third with 0.7069 AUC, consistent with its strong multi-interest modeling capability. Among the remaining architectures, TANR and BATM achieve competitive results at or above 0.69 AUC, while LSTUR shows the lowest performance at 0.6802 AUC, consistent with its MINDsmall ranking. These results confirm that TAMs scale effectively to larger datasets across all ten architecture families without any modification to the projection design or training procedure.

5. Discussion

5.1. TAMs Work for Understanding the Semantic Gap

LLM embeddings, produced by models with orders of magnitude more powerful than GloVe, can underperform GloVe when used directly in recommendation models. The following three factors offer a plausible explanation for why TAMs mitigate this gap; we note that these are interpretive hypotheses supported by the empirical results rather than directly demonstrated mechanistic claims.

Dimensionality mismatch. LLM embeddings, which are 3584-dimensional for Qwen2.5-7B-Instruct, far exceed the input dimensionality that recommendation models are designed to handle, which is typically 200–400 dimensions. When fed directly into architectures built for 300 d GloVe vectors, the high-dimensional input creates an over-parameterized space that the relatively small recommendation model struggles to learn from. TAMs address this by projecting into a manageable space, from 3584 d to 256–400 d, while preserving task-relevant information.

Optimization landscape. LLM embeddings reside on a manifold shaped by language modeling objectives. Recommendation requires navigating a different manifold where user-item relevance is the primary signal. Without adaptation, the recommendation model must simultaneously learn to interpret the LLM embedding space and model user preferences, a joint optimization problem harder than either task alone. TAMs decouple these two objectives: the projection learns embedding space interpretation, while the recommendation model focuses on preference modeling.

Feature relevance filtering. Not all dimensions of LLM embeddings carry equal relevance for recommendation. Some encode syntactic structure, discourse relations, or other linguistic properties that are irrelevant or even harmful for click prediction. The learned projection matrix $\mathbf{W}$ may act as a feature selector, potentially amplifying recommendation-relevant dimensions about topic, sentiment, and writing style while suppressing irrelevant ones. Directly demonstrating this selective amplification via probing or weight-analysis experiments is an interesting direction for future work.

5.2. When to Use TAMs or LoRA

TAMs and LoRA occupy different positions on the performance–cost Pareto frontier.

• TAMs can reach 0.69+ AUC with 1× cost. Best suited for resource-constrained settings, rapid prototyping, and production deployment where training efficiency is a priority. TAMs can be trained on a single consumer GPU in under an hour.
• Standard LoRA can reach 0.6948 AUC with 10× cost. In our experiments, standard LoRA offers no advantage over TAMs, delivering similar performance at higher cost. It may be preferred when the LLM itself needs to be adapted for multiple tasks simultaneously.
• Aggressive LoRA can reach 0.7008 AUC with 15× cost. Achieves the best absolute performance but demands careful hyperparameter tuning and substantial computational resources. Appropriate when marginal performance gains justify the cost.

The 1.0% AUC gap between TAMs and the best LoRA variant indicates a small but real benefit to adapting the LLM’s internal representations rather than projecting its outputs alone. Yet this gap is minor relative to the 15× cost difference, which is consistent with the hypothesis that a substantial portion of task-relevant information resides in the frozen LLM embeddings and can be surfaced through lightweight projection. We note, however, that our experiments do not directly measure the information content of LLM embeddings; this interpretation is supported by the empirical results but should be understood as a plausible explanation rather than a proven mechanistic claim.

5.3. Generalizability Across Architectures

A notable finding is that the same TAMs design consisting of Linear, LayerNorm, ReLU and Dropout performs well across all ten architectures without any architecture-specific tuning. This indicates that TAMs learn a task-specific transformation largely independent of the downstream model architecture, capturing a “recommendation-relevant subspace” of the LLM embedding space that is broadly useful for news click prediction.

The practical implication is straightforward: practitioners can adopt TAMs as a drop-in replacement for word embeddings in any neural news recommendation model and expect consistent improvements. No architecture-specific adaptation is required beyond setting the projection dimension to match the model’s hidden size.

5.4. Comparison with State-of-the-Art

To place our results in context, we compare TAMs-equipped models with recent approaches. ONCE [20] reports AUC values of 0.68–0.70 on MIND using LLM-generated features, comparable to our TAMs results with 0.68–0.70 AUC. GENRE [21] achieves competitive performance through generative recommendation but requires online LLM inference.

Our work differs in two respects: we provide the first systematic comparison of adaptation strategies across 10 architectures, and we show that a simple, architecture-agnostic TAMs design achieves competitive performance without LLM-specific model designs.

A recent directly comparable result comes from PNR-LLM [22], which reports an AUC of 68.88 on MINDsmall using a three-level hierarchical LLM prompting pipeline built on Gemini 1.0 Pro to enrich news representations offline, followed by a CNN-based encoder. TAMs-equipped NRMS achieves an AUC of 69.39 on the same benchmark, surpassing PNR-LLM by 0.51 points while relying on a smaller 7B-parameter open-source model and requiring no prompt engineering. On MINDlarge, PNR-LLM reports AUC of 69.52; TAMs-equipped models reach up to 71.72 AUC on the same split, with five of ten architectures exceeding 70 AUC. The key distinction is deployment cost: PNR-LLM invokes a closed-source commercial LLM for each new article to generate enriched descriptions, whereas TAMs perform a single offline forward pass through the frozen LLM and cache the embeddings indefinitely, eliminating recurring API calls and making the system self-contained.

5.5. Limitations and Future Work

Our work still has several limitations, which also suggest promising directions for future research.

First, the TAMs projection architecture is relatively simple, consisting of a single linear layer combined with normalization and activation. Future work could investigate more expressive adaptation structures, such as multi-layer projection modules, attention-based adaptation networks, and mixture-of-experts paradigms, to better balance recommendation performance and computational cost.

Second, the present evaluation focuses on news recommendation. The TAMs paradigm is not inherently domain-specific and could naturally extend to product, movie, and music recommendation, as well as other retrieval tasks. Future work should evaluate the embed-then-adapt strategy across these broader settings to more thoroughly assess generalization.

6. Conclusions

This paper introduces task-adaptive modules (TAMs), lightweight projection layers that transform frozen LLM embeddings into task-specific representations for news recommendation. Across 10 neural architectures on MIND, TAMs match LoRA fine-tuning at 10× lower cost, surpass GloVe-based models by 1.3–12.4% AUC with a mean over 3 seeds, and achieve consistent improvements confirmed by low standard deviations with ${\sigma }_{\mathrm{AUC}}\le 0.005$. On MINDlarge, five of ten architectures exceed 0.70 AUC, confirming that the approach scales without modification. The LoRA–TAMs parity is confirmed across four architecturally diverse models, namely NRMS, NAML, TANR, and MINER, and MINDlarge baseline comparisons demonstrate consistent TAMs advantages over GloVe and direct Qwen embeddings.

The key insight is that a single linear projection with layer normalization and dropout suffices to bridge the semantic gap between general-purpose LLM embeddings and recommendation objectives. This architecture-agnostic design transfers consistently across attention-based, temporal, topic-aware, multi-interest, hierarchical, and context-aware models. These results are consistent with the hypothesis that task-relevant structure is latent in frozen LLM embeddings and can be surfaced through a compact learned transformation, though directly probing the embedding space to confirm this interpretation remains an open direction for future investigation.

Future work includes extending TAMs to broader recommendation domains, investigating more expressive adaptation structures, and applying the embed-then-adapt strategy in federated settings where communication efficiency is critical.