MRID: Modeling Radiological Image Differences for Disease Progression Reasoning via Multi-Task Self-Supervision

Abstract

Automated radiology report generation has become a prominent research topic in medical multimodal learning. However, most existing approaches primarily focus on single-image interpretation and rarely address the task of tracking disease progression across longitudinal chest X-rays. This task presents two major challenges: accurately localizing pathological changes between temporally paired images, and effectively translating visual difference representations into clinically meaningful textual descriptions. To address these challenges, we propose MRID (Modeling Radiological Image Differences for Disease Progression Reasoning), a multi-task self-supervised framework that follows a pretraining–finetuning paradigm. MRID leverages multiple complementary self-supervised objectives to jointly achieve (1) intra-modal spatial alignment of organs and pathological regions across image pairs, and (2) cross-modal semantic alignment between visual difference representations and radiology report embeddings. Furthermore, we introduce a simple yet effective data augmentation strategy to alleviate the imbalance of disease progression categories. Extensive experiments conducted on the Longitudinal-MIMIC and MS-CXR-T datasets demonstrate that MRID effectively captures fine-grained disease progression patterns. In addition, the proposed framework achieves competitive performance on single-image radiology report generation, further highlighting its strong capability in modeling chest X-ray semantics.

1. Introduction

The automated interpretation of medical images has emerged as a key research topic in medical artificial intelligence [1]. Among these efforts, radiology report generation aims to translate visual findings in chest X-rays (CXRs) into structured and clinically meaningful textual descriptions, thereby improving diagnostic efficiency and ensuring reporting consistency [2]. However, most existing studies [3,4,5,6,7] primarily formulate this task as a single-image captioning problem, which limits their ability to model temporal disease progression—an essential factor for evaluating patient status and treatment response.

In real-world clinical practice, radiologists routinely compare longitudinal CXRs of the same patient acquired at different time points to assess subtle changes in lesions, anatomical structures, or opacity distributions [8,9]. Consequently, radiology reports often include expressions describing disease progression or stability, such as “pneumonia has improved” or “pleural effusion remains unchanged”. Despite its clinical importance, automating this process remains highly challenging due to the intrinsic complexity of temporal variations in CXRs and the limited availability of annotated longitudinal datasets for supervised learning [10].

To address these challenges, several prior studies [11,12,13] adopt explicit spatial alignment pipelines, in which external segmentation modules are employed to localize pathological regions within predefined anatomical structures. The resulting localized features are subsequently integrated with expert-defined priors to construct structured visual representations for downstream reasoning tasks.

While such approaches can incorporate spatial priors that guide the model’s attention toward clinically relevant regions, they also suffer from several fundamental limitations. First, the performance of current medical image detection and classification models remains suboptimal, particularly in complex pathological scenarios [14]. Inaccurate localization may propagate errors to subsequent modeling stages, thereby undermining the reliability of disease progression reasoning. Second, bounding-box-driven localization schemes tend to restrict the receptive field to isolated regions, neglecting global contextual information and the interdependence among multiple pathological areas—both of which are critical for comprehensive disease understanding.

To overcome the limitations of explicit localization strategies, recent studies have increasingly explored self-supervised learning paradigms that leverage large-scale unlabeled image–text data to induce implicit semantic alignment [15,16,17]. These approaches have demonstrated strong potential in capturing high-level correspondences between medical images and radiology reports. Building upon these insights, we decompose disease progression reasoning into two sub-tasks:

(1) Inter-image spatial alignment. This sub-task focuses on implicitly aligning pathological regions between the current image and its historical reference, and extracting disease progression features through a dedicated dual-image encoder. A key challenge lies in suppressing pseudo-changes introduced by imaging noise, acquisition variability, or patient positioning differences [10].

(2) Image–report semantic alignment. This sub-task aims to jointly model visual difference representations and their corresponding textual descriptions within a unified framework. By leveraging multiple self-supervised objectives, the model is encouraged to align unified visual representations with complete radiology reports that jointly describe static findings and disease progression.

Based on the above analysis, we propose a novel framework termed MRID, which aims to model disease progression from paired chest X-rays and generate clinically coherent radiology reports. Instead of relying on explicit region annotations, MRID formulates disease progression reasoning as a unified representation alignment problem, where pathological changes are implicitly captured through joint modeling of inter-image visual differences and report semantics. In addition, we design a data augmentation strategy to improve robustness under limited and imbalanced longitudinal supervision.

Our main contributions are summarized as follows:

(1) We propose a novel self-supervised framework MRID for longitudinal radiology report generation. The framework jointly aligns unified visual representations derived from paired chest X-rays with complete radiology reports, through five carefully designed self-supervised objectives that enforce fine-grained semantic alignment.

(2) We design a dual-image encoder for disease progression modeling. By extending conventional single-image representations to encode inter-image temporal differences, the proposed encoder enables the model to capture subtle pathological changes while preserving global contextual information.

(3) We propose a direction-aware data augmentation and curriculum learning strategy for longitudinal modeling. By synthesizing temporally reversed image–text pairs and adopting stage-wise training, the proposed framework achieves more stable optimization under limited and imbalanced data. Extensive experiments demonstrate that MRID achieves competitive performance on both longitudinal disease progression reasoning and single-image radiology report generation tasks.

2. Related Works

2.1. Medical Image Report Generation

Deep learning techniques have achieved remarkable success in medical image analysis [18]. However, acquiring high-quality medical annotations is both time-consuming and costly, making it challenging to construct sufficiently large datasets for fully supervised training [1,2]. To alleviate this limitation, vision–language pretraining (VLP) has emerged as a promising paradigm. Rather than relying on manually curated annotations, VLP models leverage paired yet weakly aligned image–report data and are optimized with self-supervised objectives to learn transferable multimodal representations. Once pretrained, these models can serve as generic feature extractors or be fine-tuned for downstream tasks such as radiology report generation and disease classification.

Representative VLP methods include MMBERT [4], which integrates ResNet-extracted visual features with textual embeddings and incorporates image information as contextual guidance for masked medical text modeling. GLoRIA [5] enhances cross-modal understanding by learning context-aware local representations, enabling fine-grained alignment between visual subregions and textual tokens through attention mechanisms. BioViL [6] introduces a domain-specific vocabulary and combines masked language modeling with a report–study matching objective to improve semantic alignment between paired images and reports. KIA [7] unifies three complementary implicit alignment objectives—masked modeling, global contrastive alignment, and image-to-report generation—within a single framework.

Despite their effectiveness, these approaches primarily focus on single-image report generation, treating each chest X-ray independently. As a result, temporal variations across longitudinal examinations are not explicitly modeled, which limits their applicability to disease progression analysis and longitudinal clinical reasoning.

2.2. Image Change Captioning in the General Domain

In the general domain, image change captioning aims to generate natural language descriptions that capture genuine semantic differences between two visually similar images. A central challenge of this task lies in distinguishing meaningful semantic variations from pseudo changes induced by illumination shifts, viewpoint differences, or other non-semantic visual perturbations.

To address these challenges, a variety of methods have been proposed. ReGAT [19] introduces a relation-aware graph attention network that models object interactions through structured graph reasoning to better capture relational changes. NCT [20] presents a neighborhood contrastive transformer that aggregates contextual information from neighboring features to enhance contextual representation learning. IFDC [21] extracts fine-grained visual, semantic, and positional object representations and performs multi-round feature fusion to accurately localize and describe changed regions. MCCFormer [22] further advances this task by learning dense cross-image associations, enabling the model to identify multiple changed areas and dynamically focus on relevant regions during description generation.

Compared with general image pairs, medical image change captioning introduces additional challenges. Variations in imaging conditions can lead to artifacts and grayscale inconsistencies, while normal anatomical variability and patient positioning differences hinder the establishment of a stable anatomical reference across images [15]. These factors make it substantially more difficult to distinguish true pathological changes from pseudo variations in medical imaging scenarios.

2.3. Disease Progression Reasoning

Early approaches to disease progression reasoning, primarily focused on single-modality visual change detection. Traditional computer vision techniques, as well as deep neural architectures such as convolutional neural networks (CNNs) [23] and recurrent models (e.g., LSTMs) [24] have been explored to identify visual differences in paired medical images. These studies laid the foundation for subsequent research on multimodal longitudinal modeling.

Unlike generic image captioning, disease progression captioning requires models to interpret subtle temporal variations in pathological findings, such as “slight improvement of consolidation.” This task demands accurate visual grounding, cross-temporal reasoning, and the ability to integrate multimodal cues into clinically coherent and context-aware textual descriptions.

Several recent methods have been proposed to address these challenges. CheXRelNet [11] jointly models local and global visual representations by leveraging a graph attention network to capture intra- and inter-image dependencies among regional features. RE-CAP [12] constructs a disease progression graph that explicitly encodes prior and current observations, pathological trends, and fine-grained attributes, and employs a dynamic reasoning mechanism to infer the evolution of individual clinical findings. BioViL-T [15] adopts a hybrid CNN–Transformer encoder for longitudinal image pairs together with a text encoder, and optimizes the framework using image-guided masked language modeling alongside both global and local contrastive objectives.

In contrast to these prior approaches, our model avoids explicit organ- or region-level extraction by introducing a dual-image encoder based on a cross-attention mechanism. The encoder jointly processes the current and historical images and fuses them into a unified, difference-aware visual representation that simultaneously captures static observations and temporal changes. In addition, we refine both the network architecture and training objectives to better model fine-grained visual differences, resulting in improved interpretability of chest X-ray changes and enhanced training efficiency.

3. Architecture of the MRID Framework

Given a current image $X_c$ and its historical reference image $X_p$, the objective of MRID is to learn a conditional generation model $P_{\theta }\left(R\right|X_c,X_p)$, where $R$ denotes the ground-truth radiology report corresponding to the current examination. During inference, the model produces a report $\widehat{R}$ that accurately describes the clinical observations in the current image while incorporating longitudinal comparisons with the prior study.

As illustrated in Figure 1, the overall architecture of MRID consists of several core components, which can be grouped into three functional categories:

• Uni-modal Encoders. This category includes a single-image encoder ${Enc}_i$, a dual-image encoder ${Enc}_d$, and a text encoder ${Enc}_t$. These modules extract semantic representations from their respective modalities, where ${Enc}_d$ further captures fine-grained spatial differences between the paired images.
• Cross-Modal Encoders. Two cross-modal encoders ${Cro}_t$ and ${Cro}_i$ are built upon cross-attention mechanisms to enable bidirectional fusion between visual and textual modalities.
• Text Decoder. The text decoder ${Dec}_t$ consumes the aligned multimodal features and autoregressively generates a coherent report that jointly reflects current clinical findings and longitudinal disease progression.

We next detail the design of each component.

3.1. Single-Image Encoder and Text Encoder

Accurate disease progression reasoning relies on robust uni-modal representations of both images and texts. We employ a pretrained Vision Transformer (ViT) [25] from KIA [7] as the frozen single-image encoder ${Enc}_i$ to extract stable and domain-specific visual semantics from individual chest X-rays. Given a current image $X_c$ and a historical reference image $X_p$, their visual representations are obtained as:

$$\begin{gathered} I_c={Enc}_i\left(X_c\right)=\left\{{\left[IMG\right]}_c,i_c^1,\dots ,i_c^N\right\} \in {\mathbb{R}}^{\left(N+1\right)\times D}, \\ {I}_p={Enc}_i\left(X_p\right)=\left\{{\left[IMG\right]}_p,i_p^1,\dots ,i_p^N\right\}\in {\mathbb{R}}^{\left(N+1\right)\times D}, \end{gathered}$$

(1)

where ${\left[IMG\right]}_{c,p}$ denote global image embeddings, $i_{c,p}^n$ are patch-level visual tokens, $N$ is the number of image patches, and $D$ is the embedding dimension. During training, ${Enc}_i$ is kept frozen to preserve reliable and stable single-image semantics learned from large-scale medical image–text data.

For the textual modality, MRID adopts BERT [26] as the text encoder ${Enc}_t$, initialized with weights pretrained in KIA [7]. Given a radiology report $R$, its textual representation is:

$$T={Enc}_t\left(R\right)=\left\{\left[TXT\right], t^1, \dots , t^M\right\}\in {\mathbb{R}}^{(M+1)\times D},$$

(2)

where $\left[TXT\right]$ denotes the global report embedding, $t^m$ are contextualized token representations, and $M$ is the report length. Unlike the image encoder, which is kept frozen during training, ${Enc}_t$ is fine-tuned within the MRID framework to enhance the modeling of temporal change expressions in radiology reports.

3.2. Dual-Image Encoder

To model longitudinal changes without relying on explicit lesion localization or region-level supervision, we introduce a dual-image encoder ${Enc}_d$ based on an asymmetric intra-visual cross-attention mechanism.

Following BioViL-T [15], given the visual features of the current image $I_c$ and the prior image $I_p$ extracted by ${Enc}_i$, spatial and temporal priors are injected to form the initial representations:

$$\begin{gathered} H_c^{\left(0\right)}=I_c+S+{1_{(N+1)}⨂t}_{curr}, \\ {H}_p=I_p+S+1_{(N+1)}⨂t_{prior}, \end{gathered}$$

(3)

where $S\in {\mathbb{R}}^{(N+1)\times D}$ denotes sinusoidal positional encodings, and $t_{curr}, t_{prior}\in {\mathbb{R}}^{1\times D}$ are learnable temporal embeddings indicating the chronological order of the image pair.

The dual-image encoder stacks $L$ cross-attention layers, where the current image features are updated using the prior image features as a fixed temporal reference. At the $l$-th layer, cross-attention is computed as:

$$CrossAttn\left(H_c^{\left(l-1\right)}, H_p\right)=softmax\left({\displaystyle \frac{H_c^{\left(l-1\right)}{W}_Q^{\left(l\right)}{\left(H_p{W}_K^{\left(l\right)}\right)}^{\top }}{\sqrt{D}}}\right)H_p{W}_V^{\left(l\right)},$$

(4)

followed by a Pre-LN Transformer update with residual connections:

$$\begin{array}{cc}{H}_c^{\left(l\right)}=& H_c^{\left(l-1\right)}+CrossAttn\left({LN(H}_c^{\left(l-1\right)}), H_p\right),\hfill \\ H_c^{\left(l\right)}=& H_c^{\left(l\right)}+FFN\left(LN\right(H_c^{\left(l\right)}\left)\right).\hfill \end{array}$$

(5)

This asymmetric design introduces a directional inductive bias that is consistent with the temporal semantics of disease progression, where historical observations provide contextual reference. After $L$ layers, the encoder outputs a unified, difference-aware visual representation:

$$I_d=H_c^{\left(L\right)}=\left\{{\left[IMG\right]}_d,i_d^1,\dots ,i_d^N\right\}\in {\mathbb{R}}^{\left(N+1\right)\times D}.$$

(6)

The resulting representation $I_d$ simultaneously encodes static visual observations of the current image and temporal differences relative to the prior image. By modeling inter-image dependencies across the global feature space, the encoder discourages spurious differences caused by acquisition variability (e.g., pose, illumination) from dominating the learned representation.

3.3. Cross-Modal Encoders

We employ two bidirectional cross-modal encoders ${Cro}_t$ and ${Cro}_i$ to enable fine-grained vision–language interaction. The text-side cross-modal encoder ${Cro}_t$ takes textual feature $T$ as the query (Q) and the visual representation $I_d$ as the key (K) and value (V). Through cross-attention, difference-aware visual cues are injected into the textual embedding space, allowing text tokens to attend to clinically relevant visual changes between the current and prior images. This design provides visual grounding for language modeling tasks related to disease progression. Conversely, the image-side cross-modal encoder ${Cro}_i$ adopts the opposite information flow, conditioning visual representations on linguistic semantics to suppress irrelevant visual noise.

Both ${Cro}_t$ and ${Cro}_i$ are implemented by inserting cross-attention layers into the text encoder and the dual-image encoder, respectively. Through this design, cross-modal interaction is selectively enabled by different training objectives to support image–text alignment.

3.4. Text Decoder

The text decoder ${Dec}_t$ is responsible for generating the final radiology report from the aligned multimodal representations. It shares a similar Transformer backbone with the text-side cross-modal encoder ${Cro}_t$, but replaces bidirectional self-attention with causal (masked) attention to support autoregressive text generation. During decoding, ${Dec}_t$ conditions on previously generated text tokens and attends to the difference-aware visual representation $I_d$ through cross-attention.

3.5. Parameter Sharing

Inspired by multimodal frameworks such as LLaVA [27], MRID adopts a cross-module parameter sharing strategy to improve parameter efficiency and promote synergistic optimization under multi-task self-supervised training. Specifically, parameter sharing is applied to the following components:

• The bidirectional self-attention layers in ${Enc}_d$ and ${Cro}_i$;
• The bidirectional self-attention layers in ${Enc}_t$ and ${Cro}_t$;
• The cross-attention layers in ${Cro}_t$ and ${Dec}_t$.

Taking the textual stream as an example, the computation at the $l$-th layer can be uniformly expressed as:

$$T^{\left(l\right)}=\left\{\begin{array}{c}{T}^{\left(l-1\right)} +BiSelfAttn\left({LN(T}^{\left(l-1\right)})\right), if in the encoding stage\\ T^{\left(l-1\right)} +CausalAttn\left({LN(T}^{\left(l-1\right)})\right), if in the decoding stage\end{array}\right..$$

(7)

When cross-modal interaction is required, visual features are incorporated through a shared cross-attention layer:

$$T^{\left(l\right)}=T^{\left(l\right)} +CrossAttn\left({LN(T}^{\left(l\right)}), { I}_d\right),$$

(8)

followed by a feed-forward network:

$$T^{\left(l\right)}=T^{\left(l\right)} +FFN\left({LN(T}^{\left(l\right)})\right).$$

(9)

Through this parameter sharing mechanism, MRID encourages consistent attention patterns across encoding and generation, ultimately improving the model’s generalization ability under limited supervision.

4. Multi-Task Self-Supervised Learning

To optimize MRID for fine-grained cross-modal reasoning, we adopt a multi-task self-supervised training strategy that jointly incorporates five complementary objectives: masked report modeling (MRM), masked image difference modeling (MIDM), image–report contrastive learning (IRC), image–report matching prediction (IRM), and report generation (RG). As illustrated in Figure 2, these objectives can be conceptually organized into three functional levels:

• Representation modeling objectives (MRM and MIDM), which strengthen the model’s understanding of textual semantic structure and fine-grained visual difference representations through masked modeling;
• Cross-modal alignment objectives (IRC and IRM), which enforce semantic consistency between visual and textual representations in a shared embedding space and enhance discriminative cross-modal reasoning;
• Generation objective (RG), which guides the model to synthesize coherent and clinically meaningful radiology reports by integrating learned multimodal information.

The detailed formulation and implementation of each training objective are described in the following subsections.

4.1. Domain-Aware Masked Report Modeling

Following standard VLP practice [28,29,30], we adopt masked report modeling (MRM) as an objective, where the model predicts masked report tokens conditioned on the unmasked textual context and the paired images. Given a report $R$ with masked tokens $w_m$, the objective is defined as:

$${\mathcal{L}}_{MRM}= {-\mathbb{E}}_{\left(X_c,X_p,R\right)~\mathcal{D}}\left[log P_{\theta }\left(w_m\right|R_{\backslash m},X_c,X_p)\right].$$

(10)

To strengthen the visual grounding effect of MRM, we adopt a domain-aware masking strategy with a prioritized two-stage design.

First, guided by the disease progression–related vocabulary listed in

Table A1. Common Terms Used in Medical Reports to Describe Progression Changes in Disease Condition.

Disease Progression Type	Tokens
Improving	better, cleared, decreased, decreasing, improved, improving, reduced, resolved, resolving, resolution, smaller, alleviated, alleviating, diminished, diminishing
Stable	constant, stable, unchanged, no change, maintained
Worsening	bigger, developing, developed, enlarged, enlarging, greater, growing, increased, increasing, larger, new, newly, progressing, progressive, worse, worsened, worsening, extended, extending, extension

, tokens explicitly describing disease changes are masked with a high probability (80%). This step ensures that clinically critical progression cues are preferentially removed, forcing the model to rely on visual evidence for accurate recovery.

Second, for the remaining tokens, TF–IDF scores computed over the MIMIC-CXR corpus are used to estimate token importance. These tokens are ranked by their TF–IDF values and masked with different probabilities according to four percentile bins: 80% for the top 10%, 50% for the 10–30% range, 20% for the 30–60% range, and 10% for the remaining 60–100% tokens. This graded masking scheme balances the suppression of informative words with the preservation of report readability.

This design encourages the model to attend to difference-aware visual representations, promoting fine-grained semantic alignment between textual descriptions and longitudinal image changes.

4.2. Masked Image Difference Modeling

Conventional masked image modeling (MIM) methods typically operate on single images and focus on reconstructing local structures or textures in the pixel space [31]. In contrast, MRID aims to model semantic differences between paired images, and the dual-image encoder ${Enc}_d$ is based on cross-attention in a high-level feature space, which is not suitable for precise pixel-level reconstruction. Based on these considerations, we introduce masked image difference modeling (MIDM) by shifting the masking objective from the pixel space to the difference representation space.

For each training triplet $\left(X_c,X_p,R\right)$, masking is applied only to the current image $X_c$, with a masking ratio of 60%, following SimMIM [32]. Through the image-side cross-modal encoder ${Cro}_i$, the masked current image features serve as queries and interact with the reference image and textual features to predict the difference-aware representation $\widehat{I_d}$.

During training, the unified visual representation $I_d$ obtained from the complete input $\left(X_c,X_p\right)$ is used as the reconstruction target. The reconstruction loss is computed only on the masked positions $\mathcal{M}$, while gradients are stopped on the target branch. The MIDM objective is defined as:

$${\mathcal{L}}_{MIDM}={\mathbb{E}}_{\left(X_c,X_p,R\right)~\mathcal{D}}{\displaystyle \frac{1}{\left|\mathcal{M}\right|}}{\sum }_{i\in \mathcal{M}}{‖{\widehat{I_d}}^{\left(i\right)}-stopgrad\left({I_d}^{\left(i\right)}\right)‖}_2^2,$$

(11)

By operating in the difference-aware feature space, MIDM provides an additional self-supervised signal that supports the learning of semantically meaningful inter-image representations and reduces sensitivity to spurious variations.

4.3. Image–Report Contrastive Learning

Contrastive learning has proven effective for cross-modal representation alignment in multimodal frameworks [33]. Following this paradigm, we introduce a momentum-based image–report contrastive learning (IRC) objective to align the global semantics of difference-aware visual representations with their corresponding radiology reports.

Specifically, two projection heads are applied to the visual CLS token ${\left[IMG\right]}_d$ from $I_d$ and the textual CLS token $\left[TXT\right]$ from $T$, respectively, mapping them into a shared embedding space for contrastive learning. We adopt a symmetric InfoNCE loss with a momentum-updated queue of negative samples. For the image-to-report (I2R) direction, the probability of matching an image representation $I_d$ with the $k$-th textual embedding $T_k^m$ in the momentum queue is defined as:

$$p_k^{I2R}\left(I_d\right)={\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(sim\right(I_d,T_k^m)/\tau )}{{\sum }_{j=1}^K\mathrm{e}\mathrm{x}\mathrm{p}\left(sim\right(I_d,T_j^m)/\tau )}},$$

(12)

where $sim\left(·\right)$ denotes cosine similarity of CLS tokens, $\tau$ is the temperature hyperparameter, and $K$ is the queue size. Similarly, for the report-to-image (R2I) direction:

$$p_k^{R2I}\left(T\right)={\displaystyle \frac{\exp \left(sim\left(I_{d,k}^m,T\right)/\tau \right)}{{\sum }_{j=1}^K\exp \left(sim\left(I_{d,j}^m,T\right)/\tau \right)}}.$$

(13)

The overall IRC objective is defined as the sum of cross-entropy losses in both directions:

$${\mathcal{L}}_{IRC}={\mathbb{E}}_{\left(X_c,X_p,R\right)~D}\left[{\mathcal{L}}_{CE}\left(y^{I2R},p^{I2R}\right)+ {\mathcal{L}}_{CE}\left(y^{R2I},p^{R2I}\right)\right],$$

(14)

4.4. Image–Report Matching Prediction

While IRC enforces global semantic alignment at the embedding level, IRM provides explicit instance-level supervision to refine visual grounding. IRM explicitly predicts whether a difference-aware image representation $I_d$ and a radiology report $T$ correspond to each other.

A special token $\left[MATCH\right]$ is prepended to the text sequence, and the augmented text is jointly processed with image features by the encoder ${Cro}_t$. After cross-modal interaction, the contextualized embedding of the $\left[MATCH\right]$ token, denoted as $h_{match}$, serves as a holistic representation of the global image–text alignment state. A lightweight binary classification head is applied to $h_{match}$ to predict the matching probability:

$$p^{IRM}\left(I_d, T\right)=\sigma \left(w^{\top }{h}_{match}\right),$$

(15)

where σ(⋅) denotes the sigmoid function and w is the learnable classifier weight. The IRM objective is optimized using binary cross-entropy loss:

$${\mathcal{L}}_{IRM}={\mathbb{E}}_{\left(X_c,X_p,R\right)~D}\left[{\mathcal{L}}_{BCE}\left(y^{IRM},p^{IRM}\left(I_d, T\right)\right)\right],$$

(16)

where $y^{IRM}\in \left\{\mathrm{0,1}\right\}$ indicates whether the image pair and report are matched.

4.5. Report Generation

Report generation (RG) serves as the primary downstream objective of MRID. During training, a special start token $\left[START\right]$ is prepended to the target report sequence, and the text decoder generates the report in an autoregressive manner, conditioning each token on all previously generated tokens and the difference-aware visual representation $I_d$. The generation objective is defined as the standard cross-entropy loss:

$${\mathcal{L}}_{RG}=-{\mathbb{E}}_{\left(X_c,X_p,R\right)~D}\left[{\sum }_{i=1}^{\left|R\right|}\mathrm{l}\mathrm{o}\mathrm{g}{P}_{\theta }\left(t_i\right|t_{<i}, I_d)\right],$$

(17)

where $\left|R\right|$ denotes the report length and $t_{<i}$ includes the $\left[START\right]$ token and all preceding tokens.

4.6. Curriculum Learning

MRID jointly optimizes five training objectives—MRM, MIDM, IRC, IRM, and RG—using a weighted multi-task loss:

$${{\mathcal{L}}_{total}={\lambda }_1\mathcal{L}}_{MRM}+{\lambda }_2{\mathcal{L}}_{MIDM}+{\lambda }_3{\mathcal{L}}_{IRC}+{\lambda }_4{\mathcal{L}}_{IRM}+{\lambda }_5{\mathcal{L}}_{RG},$$

(18)

where ${\lambda }_{1-5}$ denote task-specific loss weights.

Directly optimizing all objectives from scratch can lead to unstable training due to conflicting learning signals. Since these objectives differ in learning difficulty and influence on generation, MRID adopts a curriculum learning strategy that adjusts task weights across training stages. As shown in

Table 1. Task weights for each stage in the MRID curriculum learning framework.

Training Stage	Epoch Num	${\mathit{\lambda }}_1$ (MRM)	${\mathit{\lambda }}_2$ (MIDM)	${\mathit{\lambda }}_3$ (IRC)	${\mathit{\lambda }}_4$ (IDM)	${\mathit{\lambda }}_5$ (RG)
1	15	1	1	1	0	0
2	15	1	0.5	1	1	0.5
3	10	0.5	0.5	0.5	0.5	1
4	10	0	0	0	0	1

, the training process is divided into four stages:

• Stage 1: Representation and alignment warm-up. MRM, MIDM, and IRC are activated to learn difference-aware visual representations and initial cross-modal alignment. This stage establishes a stable multimodal foundation and prevents early overreliance on linguistic priors.
• Stage 2: Fine-grained discrimination enhancement. IRM is introduced to improve instance-level discrimination between matched and mismatched image–report pairs, while RG is incorporated with a small weight. This stage strengthens cross-modal reasoning and gradually adapts the model to generation without disrupting learned representations.
• Stage 3: Generation-oriented joint fine-tuning. The weight of RG is increased, while other objectives serve as auxiliary regularization.
• Stage 4: Task-specific fine-tuning for report generation. Only RG is retained for optimization using the pretrained encoders and difference modeling modules.

Overall, this stage-wise curriculum allows MRID to progressively transition from representation learning to cross-modal alignment and finally to generation modeling, ensuring stable optimization and robust longitudinal reasoning.

5. Dataset and Data Preprocessing

We train and evaluate the proposed MRID framework on multiple publicly available chest X-ray datasets.

5.1. Training and Evaluation Dataset

MIMIC-CXR [34] is one of the largest publicly available chest X-ray image–text datasets, containing 473,057 chest radiographs and 227,835 radiology reports from 63,478 patients. The dataset indexes studies by acquisition date and time, enabling chronological ordering of multiple examinations for the same patient and naturally supporting longitudinal modeling. Approximately 67% of patients have at least two examinations, covering disease progression across different stages [15]. MIMIC-CXR is primarily used for MRID pretraining and main experimental evaluation, following the official data splits.

Longitudinal-MIMIC [35] is a longitudinal subset constructed from MIMIC-CXR to support patient-level follow-up modeling and evaluation. It includes 26,625 patients with at least two visits, and forms longitudinal samples using adjacent visit pairs, which helps maintain temporal consistency while reducing cross-interval noise. The dataset contains 95,169 longitudinal samples and is mainly used for quantitative evaluation of longitudinal report generation.

MS-CXR-T [15] is a multimodal benchmark dataset designed for longitudinal chest X-ray analysis, aiming to assess vision–language models’ ability to understand disease evolution over time. It comprises two tasks: (1) a temporal image classification task, consisting of 1326 image pairs annotated with progression labels (improved, stable, or worsened) for five findings—Consolidation, Edema, Pleural Effusion, Pneumonia, and Pneumothorax; (2) a temporal text similarity task, containing 361 pairs of medical sentences for evaluating semantic consistency or contradiction in disease progression descriptions. This dataset is used to provide complementary evaluation of MRID’s longitudinal disease modeling capability.

5.2. Data Preprocessing

For the image modality, only frontal-view chest radiographs (PA/AP) are retained to ensure view consistency. For computational feasibility and fair comparison with prior methods, all images are resized to a unified resolution of 256 × 256 during training, followed by random rotation within ±10° and random cropping to 224 × 224 as the final input.

For text modality, radiology reports are structurally parsed using the rule-based tools provided in the official MIMIC codebase [34]. Only the Findings and Impression sections are retained. In addition, sentence shuffling is applied to increase linguistic diversity.

5.3. Temporal-Reversal Data Augmentation

Longitudinal chest X-ray datasets often exhibit a pronounced imbalance in temporal change patterns [10], where worsening-related findings are substantially more frequent than improvement-related changes. This skewed distribution limits the model to learn bidirectional disease progression trajectories, especially under data-scarce conditions. To enrich the diversity of temporal change patterns observed during pretraining and alleviate data scarcity, we propose a temporal-reversal text augmentation strategy that reconstructs clinically consistent reports for reversed image orders.

As illustrated in Figure 3, given a paired study consisting of a current image $X_c$ with report $R_c$ and a prior image $X_p$ with report $R_p$, the augmentation procedure comprises four steps:

(1) Static–Dynamic Report Decomposition. Using a curated vocabulary of disease progression cues from BioViL-T [15] (Table A1 in the Appendix A), each report is decomposed into static descriptions $R^{static}$, which characterize anatomical structures and lesion states, and dynamic descriptions $R^{dyn}$, which explicitly describe temporal disease changes. This decomposition isolates longitudinal semantics while preserving the fundamental image–text correspondence.

(2) Dynamic Description Inversion. Each sentence in $R_c^{dyn}$, where each sentence corresponds to a single atomic description of temporal change, is rewritten using a medical-domain large language model (Baichuan-M2 [36]) to express the opposite directional progression (e.g., worsened → improved), while stability-related statements remain unchanged. The resulting inverted dynamic descriptions are denoted as $R_{inv}^{dyn}$.

(3) Consistency Verification. To mitigate potential semantic drift introduced by LLM generation, each inverted dynamic sentence is independently evaluated through an LLM-based self-consistency verification process. A report-level all-or-nothing strategy is adopted: if any inverted dynamic sentence within a report fails the consistency check, the entire reversed sample is discarded and no temporal reversal is applied to that image–report pair.

(4) Reversed Triplet Construction. For accepted samples, a new report $R_{inv}$ is constructed by concatenating the static descriptions from the prior report with the inverted dynamic descriptions (Equation (19)). By reversing the image order, we obtain a temporally reversed triplet $\left(X_p,X_c,R_{inv}\right)$, which remains visually and semantically consistent under the reversed timeline.

$$R_{inv}=R_p^{static}\cup R_{inv}^{dyn}.$$

(19)

As symmetric semantic perturbations, these temporally reversed samples are incorporated only during the first two stages of the curriculum learning process. They are excluded from later training stages and downstream report generation, ensuring that the final model is optimized solely on clinically valid temporal sequences. Additional details regarding prompt design, failure cases, and bias analysis are provided in Appendix B.

6. Experiments

6.1. Implementation Details

Both the single-image encoder ${Enc}_i$ and the text encoder ${Enc}_t$ are initialized from components pretrained in the static radiology report generation framework KIA [7]. Specifically, ${Enc}_i$ adopts a 12-layer ViT-B/16 architecture and is kept frozen during all training stages, serving as a fixed visual feature extractor, whereas ${Enc}_t$ is implemented as a BERT-based text encoder and is further fine-tuned end-to-end together with the proposed MRID modules.

All experiments are conducted using two NVIDIA RTX 4090 GPUs with a total batch size of 32. The model is optimized with the AdamW optimizer, using an initial learning rate of 1 × 10⁻⁴. A linear warm-up strategy followed by cosine annealing is applied to schedule the learning rate throughout training. To enhance the stability of multi-task optimization, gradient clipping based on the L2 norm is employed, with the maximum gradient norm set to 1.0, effectively mitigating potential gradient explosion during training.

6.2. Evaluation Metrics

To comprehensively evaluate model performance on report generation tasks, we adopt both natural language generation (NLG) metrics and clinical efficacy (CE) metrics.

• Natural Language Generation Metrics

NLG metrics are used to measure the lexical and syntactic overlap between generated reports and reference reports. Specifically, we report BLEU [37], METEOR [38] and ROUGE-L [39], which are widely adopted in prior radiology report generation studies.

BLEU [37] evaluates local n-gram precision, reflecting word-level matching accuracy. METEOR [38] combines precision and recall with penalties for fragmentation and additionally accounts for linguistic variants such as stemming and synonymy, providing a more semantically informed assessment. ROUGE-L [39] measures sequence-level similarity based on the longest common subsequence, capturing global structural consistency.

All NLG metrics are computed over the entire test set and averaged to reduce variance caused by individual samples.

• Clinical Efficacy Metrics

To assess the medical validity of generated reports, we employ clinical efficacy (CE) metrics. Following prior work, we use CheXbert [40] as an automatic disease labeler to extract clinical findings from both generated and reference reports. The extracted labels are then compared at the disease level, and precision, recall, and F1-score are reported to quantify the consistency of clinical observations.

6.3. Report Generation Results

To evaluate the effectiveness of MRID in radiology report generation, we conduct experiments under two complementary settings that reflect both conventional and longitudinal clinical scenarios:

(1) Radiology Report Generation (RRG). In this setting, the input consists of a single chest X-ray from the MIMIC-CXR dataset, and only the current image representation $I_c$ is provided as visual evidence.

(2) Longitudinal Radiology Report Generation (LRRG). The input of this setting includes both the current chest X-ray and the temporally closest prior examination of the same patient from the Longitudinal-MIMIC dataset. The model is required to generate a report that not only accurately describes the current imaging findings but also correctly captures disease progression or stability relative to the reference study.

We compare MRID against a diverse set of representative baselines spanning both static and longitudinal radiology report generation paradigms. Static single-image models—R2Gen [41], KiUT [42], and KIA [7]—generate reports from individual CXRs without modeling temporal change, whereas longitudinal models—BioViL-T [15], HERGen [17], AoANet [43], PromptMRG [44], TiBiX [16], and RECAP [12]—explicitly incorporate paired CXRs to capture disease progression dynamics.

Table 2. Comparison of MRID with baseline methods on both the MIMIC-CXR and Longitudinal-MIMIC datasets. Input Resolution indicates the spatial resolution of the input images. The best results are in boldface and the second-best results are underlined.

Dataset	Method	Input	NLG Metrics				CE Metrics
		Solution	BLEU-1	BLEU-4	METEOR	ROUGE-L	Precision	Recall	F1
MIMIC-CXR (RRG Task)	R2Gen [41]	224 × 224	0.353	0.103	0.142	0.277	0.333	0.273	0.276
	KiUT [42]	256 × 256	0.393	0.113	0.16	0.285	0.371	0.318	0.321
	BioViL-T [15]	448 × 448	-	0.092	-	0.296	-	-	0.318
	HERGen [17]	384 × 384	0.395	0.122	0.156	0.285	0.415	0.301	0.317
	TiBiX [16]	512 × 512	0.324	0.157	0.162	0.331	0.3	0.224	0.25
	PromptMRG [44]	224 × 224	0.398	0.112	0.157	0.268	0.454	0.37	0.389
	RECAP [12]	224 × 224	0.429	0.125	0.168	0.288	0.389	0.443	0.393
	KIA [7]	224 × 224	0.401	0.138	0.167	0.307	0.504	0.425	0.461
	MRID (Ours)	224 × 224	0.403	0.154	0.163	0.323	0.525	0.486	0.498
Longitudinal-MIMIC (LRRG Task)	AoANet [43]	224 × 224	0.272	0.08	0.115	0.249	0.437	0.249	0.371
	R2Gen [41]	224 × 224	0.302	0.087	0.124	0.259	0.5	0.305	0.379
	HERGen [17]	384 × 384	0.389	0.117	0.155	0.282	0.421	0.289	0.295
	PromptMRG [44]	224 × 224	0.39	0.102	0.152	0.263	0.502	0.542	0.492
	MRID (Ours)	224 × 224	0.392	0.104	0.159	0.267	0.513	0.538	0.519

summarizes the results on both the MIMIC-CXR and Longitudinal-MIMIC datasets. MRID demonstrates consistent advantages across both RG and LRRG tasks, with particularly strong performance on CE metrics, which are more closely aligned with clinical correctness. Compared with the second-best baseline, MRID achieves an average relative improvement of 7.3% on CE metrics in the RG setting and 2.3% in the LRRG setting, indicating a more reliable preservation of disease-level semantics in generated reports.

Given that CE metrics evaluate consistency at the level of clinical findings rather than surface text overlap, these improvements suggest that leveraging longitudinal supervision signals enables more effective fine-grained alignment between visual representations and clinically meaningful textual descriptions.

In the static RRG task, MRID replaces the unified difference-aware representation $I_d$ with the single-image representation $I_c$, which introduces a representation mismatch and partly accounts for the smaller gains on surface-level NLG metrics. In contrast, in the LRRG setting—where $I_d$ is fully utilized—MRID achieves clear and consistent improvements on NLG metrics, reflecting more accurate modeling of longitudinal changes.

In addition, some competing methods, such as BioViL-T [15], adopt higher input image resolutions (e.g., 448 × 448), which provide an inherent advantage in capturing fine-grained lesion details. Despite operating on lower-resolution inputs (224 × 224), MRID achieves higher CE performance, while the differences in NLG metrics compared with the best-performing baseline remain within ±0.03 points. This suggests that MRID’s performance improvements are primarily driven by effective fine-grained modeling of longitudinal visual differences, rather than increased input resolution or model capacity.

Taken together, these results demonstrate that MRID not only maintains strong performance in conventional single-image report generation but also offers more substantial and clinically meaningful gains in longitudinal settings, validating the effectiveness of difference-aware representation learning and longitudinal supervision for radiology report generation.

6.4. Disease Progression Classification Results

The disease progression classification task aims to determine the evolution trend of specific clinical findings by comparing imaging differences between two examinations. Consistent with the evaluation protocol of BioViL-T [15], we formulate this task under an autoregressive text generation paradigm.

Table 3. Macro-accuracy comparison of MRID and baseline methods for disease progression classification on the MS-CXR-T dataset. Results are reported for zero-/few-shot (10%) and fully supervised learning settings. The best results are in boldface.

Setting	Methods	Consolidation	Pl. Effusion	Pneumonia	Pneumothorax	Edema	Average
Zero-shot	BioViL-T [15]	53.6	59.7	58.0	34.9	64.2	54.1
	MRID (Ours)	56.9	59.6	61.9	32.6	65.6	55.3
Few-shot	BioViL-T [15]	59.7	62.4	60.1	35.3	62.6	56.0
	MRID (Ours)	60.0	63.1	64.7	37.8	66.1	58.3
Supervised	CheXRelNet [11]	47.0	47.0	47.0	36.0	49.0	45.2
	BioViL-T [15]	61.1	67.0	61.9	42.6	68.5	60.2
	CoCa-CXR [45]	69.6	68.1	56.4	59.3	70.8	64.8
	MRID (Ours)	67.3	67.8	63.9	62.3	67.7	65.8

reports the disease progression classification results of MRID and representative baseline methods on the MS-CXR-T dataset, in terms of macro-accuracy averaged across the three progression states for each disease category. Overall, MRID outperforms the baseline BioViL-T [15] on most disease categories. Under zero-shot and few-shot settings, MRID achieves moderate improvements of 2.3% and 4.1% in average accuracy, respectively. The advantage in these low-supervision scenarios is present but not yet fully pronounced. It can be attributed to the training objective of MRID, which aligns a unified, difference-aware visual representation with complete radiology reports, embedding both static findings and longitudinal change semantics. When progression classification is performed based on a single textual prompt in zero- or few-shot inference, the semantic granularity of the prompt may not fully match the rich multimodal representation learned during training, leading to performance fluctuations.

In contrast, under the fully supervised setting, MRID exhibits a substantially larger improvement of 9.3% in macro-accuracy compared with BioViL-T [15]. With access to labeled training samples, the model can rapidly adapt the cross-modal embedding space to the target classification task, allowing its longitudinal difference modeling capability to be more effectively exploited. These results indicate that MRID benefits more significantly from explicit supervision, where the alignment between visual differences and progression labels can be directly reinforced.

6.5. Ablation Study

• Effect of Difference-Aware Representation and Dual-Image Encoder Design

To analyze the role of dual-image encoder design in longitudinal difference modeling, we compare four representative configurations under a unified experimental setting, which differ only in whether and how longitudinal image differences are modeled. Specifically, we evaluate:

(a) KIA [7], a static radiology report generation model without explicit longitudinal modeling. KIA can be regarded as the architectural predecessor of MRID, as the single-image encoder in MRID is directly inherited from the KIA pre-trained model.

(b) MRID (Asymmetric), which incorporates difference-aware representation modeling using the proposed asymmetric dual-image encoder.

(c) MRID (Bidirectional Cross-Attention), where current and prior image features attend to each other in both directions, with fully shared parameters.

(d) MRID (Symmetric Fusion), where current and prior image tokens are concatenated and jointly processed by self-attention layers.

For all MRID variants, only the updated current-image tokens are used as the final visual representation, ensuring identical output dimensionality across settings. Since KIA is designed for single-time-point report generation, it is evaluated only on the static RRG task, whereas the MRID variants (b–d) are evaluated on the LRRG task and disease progression classification task. Quantitative results are summarized in

Table 4. Ablation results on difference-aware image modeling and dual-image encoder design. Classification average accuracy results are reported on the full evaluation set and the MC Subset, where MC Subset denotes the mixed-change subset containing samples with heterogeneous temporal progression states across disease categories. The best results are in boldface.

Task	Setting	NLG Metrics				CE Metrics			Classification AVG. Acc.
		BL-1	BL-4	MTR	RL	Pre.	Rec.	F1	Full Set	MC Subset
RRG	(a) KIA	0.401	0.138	0.167	0.307	0.504	0.425	0.461	-	-
	(b) MRID (Asym.)	0.403	0.154	0.163	0.323	0.525	0.486	0.498	-	-
LRRG	(b) MRID (Asym.)	0.392	0.104	0.159	0.267	0.513	0.538	0.519	65.8	57.3
	(c) MRID (Bi-CA)	0.390	0.099	0.157	0.270	0.501	0.515	0.507	63.2	49.2
	(d) MRID (Sym.)	0.394	0.104	0.156	0.268	0.509	0.529	0.514	65	54.4

.

Comparing the static baseline KIA (a) with MRID (b) highlights the effect of introducing explicit longitudinal difference modeling. As shown in Table 4, on the RRG task, MRID achieves a substantial improvement over KIA on clinical efficacy (CE) metrics, with an average gain of 8.8%. This improvement indicates that incorporating temporal difference information during training enables the model to better capture disease progression patterns, leading to more accurate disease-level descriptions even in static report generation.

The comparison among MRID variants (b–d) further examines how longitudinal differences should be modeled within the dual-image encoder. As reported in Table 4, the asymmetric encoder (b) consistently outperforms both bidirectional cross-attention (c) and symmetric fusion (d) on progression-sensitive metrics, including progression classification accuracy and CE metrics, while maintaining comparable performance on static anatomical descriptions.

To further analyze model behavior under non-monotonic progression, we evaluate performance on a subset of mixed-change samples, where different pathological findings exhibit heterogeneous temporal trends (e.g., improvement in one disease accompanied by worsening findings in another). As shown in Table 4, performance on the mixed-change subset is consistently lower than that on the full evaluation set across all models, indicating the difficulty of modeling heterogeneous progression patterns. Within this challenging setting, the asymmetric design outperforms the symmetric variants (c/d) in classification performance.

A plausible explanation is that in symmetric designs, the prior image is continuously updated through bidirectional interaction, weakening its role as a stable reference for comparison. When both current and prior representations evolve together, it becomes harder to clearly attribute observed differences to temporal progression. This effect is more pronounced in mixed-change cases, where heterogeneous regional trends may be averaged out. In contrast, the asymmetric design keeps the prior image as a fixed reference and updates only the current representation, allowing localized change cues to be retained.

Overall, these results indicate that the performance gains of MRID are not solely attributable to the introduction of difference-aware modeling, but also depend on enforcing a directional information flow that aligns with the semantics of disease progression reasoning.

• Effect of Auxiliary Tasks and Curriculum Learning

To systematically investigate the contribution of individual auxiliary learning objectives and the curriculum learning strategy, we design a series of ablation experiments with different training configurations.

Specifically, Setting (a) corresponds to training the model solely on the RG task without any auxiliary supervision. Settings (b–h) introduce different auxiliary tasks or their combinations during the pretraining stage, followed by fine-tuning exclusively on the RG task. Among them, Setting (h) jointly incorporates all five learning objectives during pretraining without any curriculum scheduling. Setting (i) further decomposes pretraining into two stages, first jointly optimizing MRM, MIDM, and IRC, and then introducing all five objectives. The full MRID framework extends Setting (i) by adopting a more fine-grained curriculum learning strategy with four training stages (Section 4.6), enabling progressively structured optimization.

From the results of

Table 5. Ablation results of multi-task learning and curriculum learning. AVG. Δ denotes the average percentage improvement of each metric relative to Setting (a). Num of Stages indicates the total number of training stages in the curriculum learning strategy. The best results are in boldface.

Setting	Num of Stages	Learning Objective					RRG NLG Metrics					RRG CE Metrics
		MRM	MIDM	IRC	IRM	RG	BL-1	BL-4	MTR	RL	AVG.Δ	Pre.	Rec.	F1	AVG.Δ
(a)	1					√	0.318	0.116	0.119	0.270	-	0.412	0.305	0.379	-
(b)	2	√				√	0.356	0.127	0.132	0.284	9.5%	0.442	0.315	0.399	5.3%
(c)			√			√	0.329	0.128	0.127	0.272	5.4%	0.485	0.330	0.418	12.0%
(d)		√	√			√	0.359	0.124	0.141	0.281	10.6%	0.493	0.347	0.417	14.5%
(e)				√		√	0.371	0.121	0.140	0.260	8.6%	0.487	0.425	0.434	24.1%
(f)					√	√	0.347	0.134	0.148	0.278	12.8%	0.464	0.354	0.420	13.3%
(g)				√	√	√	0.386	0.143	0.155	0.289	20.4%	0.482	0.436	0.439	25.3%
(h)		√	√	√	√	√	0.306	0.103	0.108	0.248	−8.1%	0.416	0.282	0.385	−1.7%
(i)	3	√	√	√	√	√	0.400	0.153	0.162	0.319	27.9%	0.523	0.480	0.494	38.3%
MRID	4	√	√	√	√	√	0.403	0.154	0.163	0.323	29.0%	0.525	0.486	0.498	39.4%

, it can be observed that all settings incorporating auxiliary tasks (b–g) consistently outperform the baseline Setting (a). Among them, introducing image–report contrastive learning (IRC) yields the most significant improvement, increasing NLG and CE metrics by 8.6% and 24.1%, respectively. This highlights the critical role of IRC in establishing robust global semantic alignment between visual representations and radiology reports.

In contrast, masked report modeling (MRM) mainly benefits surface-level language quality, leading to an average 9.5% improvement on NLG metrics, which underscores its effectiveness in enhancing textual modeling capability. Meanwhile, masked image difference modeling (MIDM) and image–report matching prediction (IRM) primarily improve clinical efficacy, with gains of 12.0% and 13.3%, respectively, suggesting that these objectives are particularly effective in strengthening fine-grained semantic understanding of disease progression.

A further analysis reveals that directly introducing all five objectives simultaneously during pretraining without curriculum learning (Setting h) leads to degraded performance, with decreases of 8.1% and 1.7% on NLG and CE, respectively. This phenomenon indicates that, in the absence of proper training scheduling, strong gradient competition among multiple objectives can hinder effective representation learning and may even prevent stable convergence.

Finally, comparing Setting (i) with the full MRID training strategy demonstrates that a more structured and fine-grained curriculum learning design yields additional performance gains. In particular, introducing a generation-oriented fine-tuning stage—where RG serves as the primary objective while auxiliary tasks are retained as weak regularizers—proves to be essential.

• Effect of Temporal-Reversal Data Augmentation

To evaluate the effectiveness of the temporal-reversal data augmentation strategy proposed in Section 5.3, we conduct a set of ablation experiments under four different configurations. Specifically, setting (a) corresponds to training MRID without any data augmentation, while Settings (b–d) apply the temporal-reversal augmentation at different stages of the curriculum learning process.

The models are evaluated on both the longitudinal radiology report generation (LRRG) task and the disease progression classification task, with results summarized in

Table 6. Ablation study of the temporal-reversal data augmentation strategy. Used in Training Stage indicates the curriculum learning stages during which the augmentation is applied. AVG. Δ denotes the average percentage improvement of each metric relative to Setting (a). The best results are in boldface.

Setting	Used in Training Stage			LRRG NLG Metrics					LRRG CE Metrics				Classification
	1	2	3	BL-1	BL-4	MTR	RL	AVG.D	Pre.	Rec.	F1	AVG.D	AVG.Acc.
(a)				0.393	0.106	0.151	0.265	-	0.501	0.506	0.503	-	64.1
(b)	√			0.389	0.106	0.156	0.270	1.0%	0.507	0.523	0.511	2.0%	64.5
(c)	√	√		0.392	0.104	0.159	0.267	0.9%	0.513	0.538	0.519	3.9%	65.8
(d)	√	√	√	0.387	0.102	0.152	0.265	−1.1%	0.511	0.535	0.520	3.7%	65.8

. Compared with the baseline Setting (a), all augmented settings (b–d) consistently achieve improvements in CE metrics on the LRRG task as well as in classification accuracy for disease progression prediction. These gains indicate that the proposed augmentation strategy effectively encourages the model to attend to longitudinal changes and inter-image difference features, thereby enhancing its ability to capture disease evolution patterns.

However, as the augmented samples are introduced across a larger number of pretraining epochs, increased fluctuations are observed in surface-level NLG metrics. This phenomenon suggests that reports reconstructed by large language models may still exhibit distributional discrepancies compared with real reports. Excessive reliance on such reconstructed samples can therefore introduce noise into language modeling, particularly for metrics sensitive to lexical or syntactic variations.

Taking both performance gains and training stability into account, we adopt Setting (c) as the final configuration for MRID, as it provides the best trade-off between improved longitudinal supervision and stable language modeling.

6.6. Qualitative Result

(1) 

Case Study of Generated Reports

To qualitatively assess the effectiveness of MRID in modeling longitudinal disease progression, we present two representative case studies in Figure 4, comparing the generated reports of MRID with those produced by baseline BioViL-T [15], alongside the corresponding ground-truth reports.

In the case (a), BioViL-T captures worsening lung consolidations but incorrectly reports the absence of pleural effusion and describes cardiomegaly as mildly enlarged, thereby distorting key longitudinal semantics. In contrast, MRID correctly identifies unchanged cardiomegaly, acknowledges the presence of a right pleural effusion, and reflects the worsening trend of pulmonary edema, resulting in a more clinically coherent progression description.

The case (b) focuses on interval improvement and stability across multiple findings. While BioViL-T recognizes the presence of several medical devices and describes them as being in satisfactory position, MRID further captures their unchanged positioning across examinations. More importantly, MRID correctly identifies mild interstitial pulmonary edema as slightly improved, whereas BioViL-T characterizes it as stable, indicating that MRID exhibits stronger sensitivity to temporal directionality in disease progression assessment. Both models fail to explicitly identify the persistent pneumothorax in this case, revealing a shared limitation in detecting subtle or loculated abnormalities.

(2) 

Heatmap Visualization of Image Representations

Figure 4 visualizes the attention heatmaps of the single-image encoder and the dual-image encoder in MRID, revealing a clear progression in representation focus between the two modules. The single-image encoder primarily attends to anatomically salient regions, such as the lung fields and the cardiac silhouette, reflecting its ability to capture global structural and pathological cues from an individual chest X-ray. The resulting attention patterns remain largely structure-oriented and do not explicitly differentiate temporally evolving abnormalities from static findings.

In contrast, after incorporating the historical reference image, the dual-image encoder exhibits a distinct shift in attention behavior. The attention distribution becomes more localized and asymmetric, concentrating on difference-sensitive regions that correspond to meaningful inter-image variations, such as areas associated with mildly improved interstitial pulmonary edema. This change indicates that the dual-image encoder is capable of selectively highlighting regions relevant to disease progression rather than static anatomical structures.

Moreover, when the temporal order of the image pair is reversed, noticeable changes emerge in the attention patterns of the dual-image encoder. This observation suggests that the learned representations are sensitive to temporal conditioning, capturing not only the presence of inter-image differences but also their chronological direction. Such direction-aware attention enables MRID to reason about disease progression trajectories rather than treating longitudinal image pairs as unordered sets.

To further substantiate the clinical interpretability and robustness of the learned difference-aware representations, we provide a more controlled qualitative analysis in Appendix C where cross-attention patterns are examined across representative cases under temporal reversal, artificial occlusion, and geometric/intensity perturbations.

7. Discussion

7.1. Discussion of Experimental Findings

This work investigates longitudinal radiology report generation from the perspective of difference-aware representation learning. By integrating asymmetric dual-image encoding, multi-task self-supervision, and temporal-reversal data augmentation, MRID provides a unified approach for progression-aware reasoning. Compared with prior methods such as KIA [7] and BioViL-T [15], MRID explicitly conditions visual representations on historical reference images, enabling finer alignment between paired images and radiology reports. Both clinical efficacy gains and attention heatmap analyses indicate a shift in model focus from global anatomical structures toward temporally discriminative regions. Although designed for longitudinal settings, MRID also improves single-image report generation, suggesting that longitudinal supervision acts as an effective regularizer.

7.2. Extending Beyond Pairwise Longitudinal Modeling

In its current formulation, MRID operates in a pairwise setting, comparing the current image with a single historical reference, which aligns with common clinical comparison practice. Accordingly, MRID focuses on modeling the temporal ordering between paired images, and does not explicitly encode or reason about the absolute time interval separating the two studies.

Extending MRID to multi-timepoint longitudinal scenarios introduces additional challenges beyond pairwise comparison. Recent work [46] on spatiotemporal representation learning for medical time series has highlighted the importance of incorporating temporal distance as an explicit conditioning signal when modeling long-range disease trajectories. Motivated by these findings, a principled future direction is to learn an adaptive historical representation that aggregates multiple prior examinations while incorporating temporal distance through dedicated time embeddings. We leave the explicit modeling of absolute temporal intervals as future work.

7.3. Toward More Faithful Progression Evaluation

MRID is evaluated using complementary metrics, including standard NLG metrics, clinical efficacy scores, and temporal classification accuracy. These metrics capture textual similarity, disease-level recognition consistency, and coarse-grained progression states, respectively. However, they do not explicitly assess the sentence-level correctness of disease progression descriptions. As a result, favorable scores may still coexist with partially inaccurate or clinically implausible progression statements. Developing evaluation protocols that directly measure fine-grained temporal correctness and longitudinal consistency remains an open challenge. Addressing this limitation is important for advancing reliable longitudinal report generation.

7.4. Considerations on Modality Generalization

In the current implementation, MRID adopts the image encoder from the static chest X-ray report generation model KIA as a frozen module and initializes the text encoder from the same pretrained framework, thereby inheriting CXR-specific priors. Beyond this initialization, the asymmetric dual-image encoding, difference-aware objectives, and curriculum-based training strategy are not inherently modality-specific. These components operate on latent representations and model relative inter-image changes, making them applicable to other 2D longitudinal medical imaging modalities. For volumetric modalities such as CT or MRI, architectural adaptations such as slice aggregation or 3D encoders would be required. Such adaptations primarily affect the visual encoding stage and do not alter the core difference-aware modeling principle of MRID.

8. Conclusions

In this work, we propose MRID, a multi-task self-supervised framework that explicitly models radiological image differences to support disease progression reasoning. The proposed approach is built upon three key contributions.

First, we design an asymmetric dual-image encoder that conditions the representation of the current image on a historical reference, enabling implicit spatial alignment and fine-grained modeling of inter-image differences without relying on explicit lesion localization.

Second, we develop a set of complementary self-supervised objectives and a structured joint training strategy that aligns unified difference-aware visual representations with complete radiology reports, effectively bridging static findings and temporal progression within a shared semantic space.

Third, we introduce a novel data augmentation strategy that synthesizes direction-aware training samples, improving robustness under data imbalance while being carefully integrated into early-stage curriculum learning.

Extensive experiments demonstrate that MRID consistently improves clinical efficacy across RRG and LRRG tasks. These results confirm that explicitly modeling radiological image differences provides a principled and effective foundation for longitudinal vision–language learning and clinically meaningful disease progression analysis.