LLaMA-XR: A Novel Framework for Radiology Report Generation Using LLaMA and QLoRA Fine Tuning

Abstract

Background: The goal of automated radiology report generation is to help radiologists in their task of creating descriptive reports from chest radiographs. However, the process of creating coherent and contextually accurate reports has been challenging, mainly due to the intricacies of medical language and the need to correlate visual data with textual descriptions. Methods: This study presents LLaMA-XR, a novel framework that integrates Meta LLaMA 3.1 Large Language Model with DenseNet-121-based image embeddings and Quantized Low-Rank Adaptation (QLoRA) fine-tuning. Results: The experiment conducted on the IU X-ray dataset demonstrates that LLaMA-XR outperforms a range of state-of-the-art methods. It achieves an ROUGE-L score of 0.433 and a METEOR score of 0.336, establishing new performance benchmarks in the domain. Conclusions: These results underscore LLaMA-XR’s potential as an effective artificial intelligence system for automated radiology reporting, offering enhanced performance.

1. Introduction

Medical imaging plays a crucial role in diagnosing diseases by providing detailed visual representations of internal structures [1]. Among various radiological imaging modalities, chest X-ray (CXR) is one of the most commonly used techniques for diagnosing a wide range of pulmonary and cardiovascular conditions [2]. Given its widespread use in clinical practice, CXR reports are essential for accurately interpreting specific abnormalities and guiding treatment decisions [3]. However, generating these reports is a labor-intensive process that demands meticulous analysis of radiological images, making it both time-consuming and susceptible to errors [4].

With the advancement of deep neural networks, automatic radiology report generation has emerged as a promising tool to assist radiologists [5]. This approach requires models to generate accurate, coherent, and medically precise reports based on radiological images, significantly reducing workload and improving efficiency. Early studies [6,7] in automated radiology report generation were largely inspired by recurrent neural network (RNN)-based image captioning architectures. While these methods achieved initial success, they struggled with handling long and complex sentences [8,9], which are crucial for generating detailed and clinically relevant reports.

To overcome these challenges, recent advances have focused on the use of large language models (LLMs), such as GPT [10] and LLaMA [11], which have benefited from extensive pre-training on diverse datasets and demonstrate strong performance in generating radiology reports [12,13,14,15,16,17,18,19]. In the radiology of chest X-rays, the latest multimodal foundation models aim to combine visual understanding and clinical language generation, covering abnormal assessment and report generation [20,21]. LLM-based approaches have shown significant promise in capturing intricate medical contexts and producing high-quality reports. However, earlier versions of these models, such as GPT-2 (up to 1.5 B parameters) and BERT-Large (340 M parameters), often lacked the capacity to effectively handle the complexity of medical data due to their relatively smaller scale. This limitation underscores the need for a more advanced framework to better align with the demands of medical report generation.

In this study, we propose a novel framework for automated chest X-ray report generation that addresses the challenge of effectively handling the complexity of medical data. Our approach takes advantage of the LLaMA 3.1 8 B parameter model [22], which offers superior performance in generating coherent, contextually rich, and medically precise reports. Compared to its predecessors, LLaMA 3.1 benefits from enhanced training data and a significantly larger parameter scale, enabling it to better capture the nuances of medical language [23,24]. To extract visual features from CXR images, several techniques can be employed, including convolutional neural networks (CNNs) such as DenseNet, ResNet, and Vision Transformers. In this study, we utilize DenseNet-121 [25], a CNN architecture known for its ability to reuse features across layers and provide detailed image embeddings [23,26]. However, this choice is flexible, and other architectures, such as ResNet, EfficientNet, or Vision Transformers, can be incorporated depending on specific requirements. By combining DenseNet-121 with LLaMA 3.1, our framework effectively bridges the gap between visual data and textual report generation.

Furthermore, we employ fine-tuning techniques such as QLoRA [27], which enables efficient adaptation of large language models by significantly reducing memory requirements through low-rank adaptation, allowing us to fine-tune LLaMA 3.1 on medical datasets without the high computational cost of full fine-tuning. We use Supervised Fine-Tuning (SFT) to adapt LLaMA 3.1 to medical datasets, ensuring that the generated reports are accurate and clinically relevant. For this study, we used the IU X-ray dataset [28], a widely used benchmark dataset that contains chest X-ray images paired with the corresponding radiology reports. This dataset enables effective model training and evaluation by providing diverse medical cases with detailed annotations. This fine-tuning process not only improves the quality of report generation, but also reduces the time and computational resources required for training. Our framework is designed to be scalable and efficient, making it suitable for real-world applications where accuracy and interpretability are paramount. The key contributions of this study are as follows:

• We propose a novel framework for automated chest X-ray report generation that leverages LLaMA 3 as the core language model, enhanced through parameter-efficient QLoRA and supervised fine-tuning (SFT).
• We perform an extensive comparison with prior works, demonstrating superior performance in the ROUGE-L and METEOR metrics.
• We further assess the clinical quality of the generated reports, demonstrating their strong factual accuracy, semantic alignment, and clinical relevance.

2. Related Work

Automated radiology report generation has garnered significant attention in recent years, with a wide array of deep learning architectures being proposed to enhance diagnostic accuracy, linguistic coherence, and computational efficiency. This section reviews the evolution of this field, focusing on methods applicable to generate chest X-ray reports. We categorize recent work into four main classes: graph-based models, reward-based learning approaches, transformer-based architectures, and large language models (LLMs).

2.1. CNN-RNN Based Methods

The roots of radiology report generation lie in image captioning, a foundational computer vision task designed to produce descriptive sentences for visual input. Early models typically utilized CNN-RNN frameworks, where Convolutional Neural Networks (CNNs) extracted spatial features and Recurrent Neural Networks (RNNs) generated the corresponding captions [29,30,31]. However, these approaches often struggled with long-range dependencies and lacked contextual depth. One of the earliest tailored approaches for radiology reports is Tm-HRNN [32], which introduced a topic modeling-based hierarchical RNN framework. It used DenseNet for image feature extraction, followed by a CNN-RNN-based decoder that generates reports in a sentence-by-sentence fashion, guided by topic vectors. This hierarchical structure mimicked how radiologists organize findings into coherent paragraphs, laying an early foundation for structure-aware generation. Despite its innovative structure, the model relied on heuristic topic assignments and lacked explicit medical knowledge modeling.

To address these limitations, later enhancements like anchor-caption mechanisms [33], human-like control techniques [34], and self-attention networks [35] were introduced. While effective in general contexts, their reliance on object annotations made them less suitable for medical imaging tasks, where such annotations are scarce or unavailable. This led to the development of medical imaging captioning models that employed image embeddings combined with large language models (LLMs) to extract richer semantic features without extensive supervision. The transition from captioning to report generation marked a shift from surface-level description to clinical reasoning and structured reporting, requiring models to understand complex visual cues and their clinical implications.

2.2. Graph- and Reward-Based Methods

To better capture the interdependencies between anatomical entities and clinical findings, researchers have turned to graph neural networks (GNNs). These models encode the relationships between observed pathologies and anatomical locations to improve report coherence. For example, Zhang et al. [36] introduced SentSAT+KG, which combines DenseNet-121 features with a knowledge graph to guide the generation process. Similarly, Li et al. [37] proposed DCL, a ViT-based approach that leverages dynamic graphs to represent complex entity relationships for improved interpretability and performance. However, these models require the construction and maintenance of complex medical knowledge graphs, which can be labor-intensive and often depend on domain expertise. Moreover, their performance is highly sensitive to the quality and completeness of the graph, making them less robust in diverse clinical scenarios.

To optimize report generation beyond traditional supervised objectives, reinforcement learning (RL) has been explored. These methods employ task-specific reward functions that promote diagnostic relevance and linguistic quality. Notably, Li et al. [38] proposed the H-Agent model, leveraging DenseNet-based image features with a reward-based training strategy. Building on this, Jing et al. [39] introduced CMAS-RL, which uses ResNet-50 as the backbone and incorporates domain-specific rewards to refine output quality. Further enhancing this line of research, Qin and Song [40] presented CMM+RL, which applies a more powerful ResNet-101 encoder and a reinforcement learning framework to improve factual correctness and report coherence. Nonetheless, reinforcement learning-based models often face challenges such as unstable training dynamics, sensitivity to reward function design, and high computational overhead, which hinder their practical deployment in real-world clinical systems.

2.3. Transformer and Large Language Models Based Methods

The transformative success of self-attention mechanisms in NLP has inspired numerous transformer-based models for radiology. These models can better contextualize spatial and semantic features across the image–report pipeline, with the original Transformer model [41] laying the foundation for this architecture class. R2Gen [42], PPKED [43], and R2GenCMN [44] adopted custom transformer designs on top of ResNet-based image encoders to generate structured reports. Models such as SGF [45], PGT [46], JPG [47], and ITA [48] further refine this paradigm by integrating domain-specific priors, knowledge embeddings, or training schemes such as curriculum learning, as used in CMCL [49]. Variational approaches like VTI [50] and vision–language alignment methods such as UAR [51] demonstrate the versatility of transformer-based architectures in balancing generation fidelity and clinical accuracy. However, these models often struggle with generating detailed and accurate descriptions, especially when handling complex sentence structures and mitigating data biases, largely because they are not pre-trained on large-scale, diverse datasets to effectively capture the nuanced relationships between visual features and textual descriptions.

The application of LLMs, such as GPT and LLaMA, has revolutionized medical report generation by providing powerful pre-trained language priors. These models can generate coherent, factually accurate reports using multimodal cues. Nicolson et al. [23] introduced CvT-DistilGPT2, combining CvT-21 image features with a lightweight DistilGPT2 language model to enhance generation efficiency while preserving quality. This model exemplifies how compact transformer variants can be used effectively in resource-constrained environments. Another notable approach is MCSAM by Tao et al. [24], which uses MedCLIP for visual encoding and integrates it with BERT for report generation. This memory-compressed alignment model leverages semantic consistency across image–text pairs, highlighting the value of aligning cross-modal embeddings. However, despite its efficiency, the reduced model size limits its capacity to capture complex medical terminologies and relationships, potentially affecting the depth and accuracy of the generated reports.

In summary, existing research on the generation of chest radiographs has mainly relied on earlier transformer models, resulting in notable performance improvements. However, these models still encounter limitations in generating highly relevant and medically precise reports. A key gap in the literature is the underutilization of larger LLMs, such as LLaMA-3, which have the potential to enhance both the accuracy and contextual understanding necessary for clinical applications. We address this gap by leveraging the LLaMA-3.1 8B model, which offers improved performance and better contextual comprehension to generate more accurate and medically relevant reports.

3. Methodology

3.1. LLaMA-XR Framework

Our framework involves two primary components: one for visual information extraction from CXRs and the other for generating reports using LLMs. For visual information extraction, we utilized DenseNet-121 and, to generate reports, we employed LLaMA 3. Figure 1 provides an illustration of this framework.

Figure 1. Overview of the proposed framework for automated radiology report generation. (a) anteroposterior (AP) and lateral (LAT) X-ray images are processed through DenseNet-121 to produce a combined 36-dimensional confidence vector. (b) The vector, together with the corresponding radiology report, is converted into Alpaca-style prompts, and the resulting prompts along with QLoRA-generated outputs are passed to LLaMA 3.1 8B for radiology report generation, where the adapter weights guide and enhance the model’s generation process. (c) QLoRA enables low-rank adaptation of a quantized base model by introducing a small set of trainable adapter parameters, updating only a minimal subset of parameters during fine-tuning.

The “DenseNet121-res224-all” model [25] was pre-trained on a broad collection of public CXR datasets that includes the IU X-ray dataset. Although the visual extractor is kept completely frozen (no gradients are back-propagated) and is used only to produce fixed 36-dimensional embeddings, we acknowledge that pre-training on IU X-ray may introduce a potential source of bias. In particular, the model may generate higher-quality embeddings for images drawn from a familiar distribution, which could slightly inflate performance compared to entirely unseen datasets. This limitation is consistent with prior work using large-scale pre-trained CXR models and should be considered when interpreting the results. In the model name, DenseNet-121 refers to the architecture, res224 indicates the resolution of $224\times 224$ pixels, and ‘all’ signifies that the model was trained on all available chest X-ray datasets. DenseNet-121 is a convolutional neural network known for its dense connectivity pattern, which enables feature reuse across layers and promotes the learning of compact and informative image representations [23,26]. This model extracts detailed visual features from chest radiographs and outputs confidence scores for 18 different medical conditions: Atelectasis, Cardiomegaly, Consolidation, Edema, Effusion, Emphysema, Enlarged Cardiomediastinum, Fibrosis, Fracture, Hernia, Infiltration, Lung Lesion, Lung Opacity, Mass, Nodule, Pleural Thickening, Pneumonia, and Pneumothorax. These scores serve as semantically rich embeddings for the report generation module, guiding the language model in producing clinically meaningful narratives. We chose to use a small 36-dimensional vector of 18-condition confidence scores instead of larger high-dimensional vectors. This allows us to obtain clinically interpretable priors, reduce the dimensionality of the input for the LLM, and align with radiology terminology.

This approach has the benefits of reducing computational costs without compromising on strong semantic performance. We also have plans for an ablation study involving a comparison against late-layer CNN/ViT embeddings, and the modularity makes this very accessible. Although DenseNet-121 was selected due to its strong performance and efficient parameter usage in medical imaging tasks, our framework is modular and flexible, allowing alternative architectures such as ResNet, EfficientNet, or Vision Transformers to be substituted based on specific needs or experimental goals. Figure 2 illustrates the output classes of the multi-label classification, as determined by the pre-trained DenseNet classifier. Figure 3 then adds a heatmap to the chest X-ray image, indicating the parts of the image that were considered most relevant during the classification process. These heatmaps are based on the DenseNet feature extractor and should not be considered as an explanation of the report generation process of the LLaMA component. These visualizations enhance interpretability by linking model decisions to specific image regions, thereby supporting transparent and informed clinical use.

Figure 2. Illustration of the “DenseNet121-res224-all” output classes.

Figure 3. The heatmap highlights the critical regions in a chest X-ray image that influenced the Densenet-121 model’s classification decision. Warmer colors indicate higher importance in the model’s feature extraction process. The probability distribution illustrates the confidence score of each abnormality, and 18 colors represent 18 abnormalities.

LLaMA 3’s architecture is a part of the LLaMA series, which are transformer-based models. The LLaMA series models are trained on vast amounts of textual data to predict the next token in a sequence, making them highly effective for tasks such as text generation, translation, and summarization. Specifically, the “LLaMA 3.1 8B” [22] model has 8 billion parameters, which makes it capable of understanding and generating complex and contextually accurate text. The architecture of LLaMA 3 is illustrated in Figure 1, consisting of multiple layers of transformer blocks, each containing self-attention mechanisms and feedforward neural networks. These layers enable the model to capture long-range dependencies in the text, making it adept at understanding context and generating coherent and relevant text based on the given input. The model uses a tokenization process to convert input text into tokens, which are then processed through the transformer layers. The output is a sequence of tokens which are converted back into readable text, resulting in detailed and contextually accurate reports.

For generating reports, we used the “LLaMA 3.1 8B 4-bit quantized” model [22], implemented via the Unsloth (https://unsloth.ai/, accessed on 5 December 2025) framework alongside two fine-tuning strategies: SFT and QLoRA [27]. The LLaMA 3.1 8B model was selected for its strong performance in language understanding and generation, offering a robust balance between model size and capability. The 8 B parameter scale is large enough to capture complex medical semantics while remaining computationally feasible. The 4-bit quantization significantly reduces memory use and accelerates inference, but does so with little impact to model performance. As such, it is a good candidate for practical use, especially in resource-constrained settings. Unsloth was used because of its highly optimized implementation for quantized models, enabling faster and more memory-efficient training. SFT improves the ability of the model to generalize in domain-specific tasks by training it on labeled radiology report data. Meanwhile, QLoRA enables low-rank adaptation of a quantized base model, updating only a small, learnable subset of parameters. This approach improves efficiency, reduces overfitting, and reduces overall training cost. Together, this configuration allows us to fine-tune the model up to two times faster and with approximately 60% less memory usage [27], making the entire system scalable and accessible for wider real-world applications in the generation of medical reports.

3.2. QLoRA

The QLoRA framework [27] is a significant advancement over the Low-Rank Adaptation (LoRA) [52] technique, specifically designed to make fine-tuning LLMs more efficient in terms of memory usage and computational cost. QLoRA builds upon the core concept of LoRA, which freezes the pre-trained weight matrix, $W_0\in {\mathbb{R}}^{d\times k}$, during fine-tuning to prevent computationally expensive updates. Instead, LoRA introduces two trainable low-rank matrices, $A\in {\mathbb{R}}^{d\times r}$ and $B\in {\mathbb{R}}^{r\times k}$, where r is the rank hyperparameter, representing a trade-off between efficiency and model expressiveness. The weight updates in QLoRA are similarly represented as in Equation (1), offering a low-rank approximation of the necessary adjustments to the original weights.

$$\Delta W=AB,$$

(1)

where $A\in {\mathbb{R}}^{d\times r}$ and $B\in {\mathbb{R}}^{r\times k}$ represent the low-rank matrices, and $\Delta W$ is the update to the original weights.

At inference, the combined weight matrix W is computed by merging the frozen pre-trained weights and the learned updates, as shown in Equation (2).

$$W=W_0+\Delta W,$$

(2)

where $W_0$ is the frozen pre-trained weight matrix, and $\Delta W$ is the learned update from the low-rank matrices.

The forward pass in QLoRA is expressed as:

$$h=W_{0}x+ABx,$$

(3)

where x is the input data, and h is the hidden representation. This decomposition enables the model to retain the general knowledge encoded in $W_0$ while incorporating task-specific information through A and B. The QLoRA model was configured with the following hyperparameters: rank r = 16, alpha scaling factor $\alpha$ = 32, and dropout rate = 0.05. LoRA adapters were applied to the q_proj, k_proj, v_proj, and o_proj modules of the transformer. The adapters were implemented using the Unsloth framework with 4-bit quantization.

QLoRA [27] further improves upon LoRA [52] by introducing 4-bit quantization for the pre-trained weight matrix $W_0$. Quantization is a technique that reduces the precision of numerical representations, enabling computation with lower memory and faster execution speeds. In QLoRA, the pre-trained weights are quantized to 4 bits, significantly reducing the memory footprint of the model by approximately 60%. Despite the reduction in precision, this quantization minimally affects model accuracy because $W_0$ remains frozen and is complemented by the precise updates provided by the full-precision low-rank matrices A and B. This hybrid approach ensures that task-specific adaptations are handled with high fidelity while general pre-trained knowledge is retained in a compressed form.

Another critical component is the rank hyperparameter (r) in the low-rank matrices. By adjusting r, practitioners can balance the trade-off between model performance and computational efficiency. A lower rank results in fewer parameters to update, which reduces memory and accelerates training, while a higher rank increases the model’s capacity to learn complex task-specific patterns.

These innovations in QLoRA lead to substantial performance gains. By restricting updates to the smaller matrices A and B and leveraging quantized pre-trained weights, QLoRA achieves up to twice the training speed of traditional fine-tuning methods. This requirement is reduced considerably, and it is possible to fine-tune large-scale models such as LLaMA 65B on a single 48 GB GPU [27]. This efficiency does not compromise the model’s ability to produce high-quality, task-specific outputs, making QLoRA a practical solution for deploying LLMs in resource-constrained environments.

3.3. Dataset Description

We evaluated our approach using the IU-Xray [28] dataset, which consists of 7470 chest X-ray images and 3955 corresponding radiology reports. Each report is paired with both a frontal and a lateral X-ray image or contains one image only. In the IU X-ray dataset, approximately 12.4% of cases contain only a single view. These single-view cases were proportionally distributed across the patient-level 70% training, 10% validation, and 20% test splits, ensuring a consistent data distribution across all subsets. For such cases, the 18 confidence scores from the available view are duplicated to maintain the required 36-dimensional input representation. This approach preserves architectural consistency without introducing additional preprocessing complexity. Given the relatively small proportion of single-view cases, their overall impact on model performance is expected to be limited, and no significant deviation in training dynamics or evaluation metrics was observed. Table 1 presents two patients’ X-ray images from the dataset along with their associated reports.

Table 1. Examples from the IU-Xray dataset, showing chest X-ray images (frontal and lateral views) paired with their corresponding radiology reports.

Radiology Image	Radiology Report
	There are diffuse bilateral interstitial and alveolar opacities consistent with chronic obstructive lung disease and bullous emphysema. There are irregular opacities in the left lung apex that could represent a cavitary lesion in the left lung apex. There are streaky opacities in the right upper lobe, XXXX scarring. The cardiomediastinal silhouette is normal in size and contour. There is no pneumothorax or large pleural effusion.
	The cardiomediastinal silhouette and pulmonary vasculature are within normal limits. There is no pneumothorax or pleural effusion. There are no focal areas of consolidation. Cholecystectomy clips are present. Small T-spine osteophytes. There is biapical pleural thickening, unchanged from the prior. Mildly hyperexpanded lungs.

3.4. Dataset Preparation

Our dataset preparation process is divided into two key steps. The first step focuses on preparing the data for the visual information extractor, which extracts useful features from the X-ray images. The second step involves using these extracted features to fine-tune the LLM for medical report generation.

3.4.1. Image Preparation

In the first step, we used DenseNet121-res224-all [25] as the visual information extractor. To prepare the images for this model, we first resized all X-ray images to a consistent dimension of $224\times 224$ pixels. This resizing ensures that the input images meet the required size for the DenseNet-121 model. After resizing, we normalized the images to scale the pixel values, making them consistent and aiding the model’s performance. Since X-ray images are typically grayscale, we also converted the images to a single color channel to ensure that only the relevant intensity information is used during feature extraction. Once processed, the images were fed into the DenseNet-121 network, which outputs a 36-dimensional vector representing classification scores for 18 different medical conditions across two views: frontal and lateral, as illustrated in Figure 1. These vectors are critical for the subsequent steps in the report generation process.

3.4.2. Prompt Generation

In the second step, we used the extracted visual information to construct a dataset for fine-tuning the LLM. For this purpose, we employed Unsloth: https://unsloth.ai/(acessed on 5 December 2025) and QLoRA [27] techniques, along with a structured Alpaca prompt system to format the input data appropriately. We use the Alpaca prompt format for instruction tuning because it is simple, has a consistent task-response structure, and is compatible with QLoRA fine-tuning, which is important for the inclusion of radiology-specific instruction prompts. The Alpaca prompt format, as shown in Figure 4, begins with a general instruction that describes the task at hand. This is followed by a specific instruction that provides detailed context about the input data, which in this case consists of a 36-dimensional vector, 18 values from the frontal view and 18 values from the lateral view. The input part of the prompt contains these classification scores, while the corresponding response contains the relevant findings based on the medical conditions associated with the scores.

Figure 4. Example prompt for fine-tuning.

During the testing phase, the response portion of the prompt remains empty. This allows the model to generate the medical report based on the input data, as it will be expected to fill in the response with the appropriate findings for the given classification scores.

3.5. Fine-Tuning LLaMA

For fine-tuning the LLaMA 3.1 model, we used Google Colab equipped with an A100 GPU, which provided the computational resources necessary to efficiently handle large-scale fine-tuning tasks. The training process was implemented using the SFTTrainer from the trl library. To achieve an effective balance between computational efficiency and gradient stability, the batch size was set to 8, with gradient accumulation steps of 4, effectively simulating a batch size of 32. The model was trained for three complete epochs, which was sufficient for convergence demonstrated in Figure 5, given the size of the training dataset and the complexity of the task. A learning rate of $2\times {10}^{-6}$ was used, optimized using the AdamW 8-bit optimizer (optim = “adamw_8bit”), which significantly reduces memory overhead by quantizing the optimizer states without compromising performance. Mixed precision training (fp16) was enabled to take advantage of the A100 GPU’s capabilities, accelerating training while reducing memory usage. The fine-tuning involved a total of 41,943,040 trainable parameters, enabling the model to adapt to the target domain while preserving its pre-trained knowledge.

Figure 5. Training and validation loss curves showing consistent convergence and stable generalization across training iterations.

A linear learning rate scheduler (lr_scheduler_type = “linear”) was employed to ensure smooth and controlled adjustments to the learning rate during training, while warmup steps (warmup_steps = 30) facilitated a gradual ramp-up to the optimal learning rate, mitigating potential instabilities during the initial training phase. A weight decay of 0.01 was applied to regularize the model, preventing overfitting by penalizing large weight magnitudes. To further enhance training robustness, early stopping was implemented, thereby avoiding unnecessary computations and overfitting. Model checkpoints were periodically saved, with the best-performing model automatically restored at the end of training (load_best_model_at_end = True). Evaluations were conducted at regular intervals to track performance metrics and ensure the model was progressing as expected.

3.6. Evaluation Metrics

BiLingual Evaluation Understudy (BLEU) [53] is a metric that compares human-generated output with machine-generated output based on n-gram. It evaluates how accurately the machine has generated the report by comparing it with the reference report. The BLEU score is defined in Equation (4), where $BP$ is the brevity penalty as shown in Equation (5), $p_n$ is the precision of n-grams given in Equation (6), and $w_n$ is the weights for n-grams as shown in Equation (7).

$$\mathrm{BLEU}=BP·exp\left(\sum _{n=1}^N{w}_{n}log{p}_n\right)$$

(4)

$$BP=\left\{\begin{array}{cc}1\hfill & \mathrm{if}c>r,\hfill \\ e^{(1-{\displaystyle \frac{r}{c}})}\hfill & \mathrm{if}c\le r\hfill \end{array}\right.$$

(5)

$$p_n={\displaystyle \frac{\mathrm{Number}\mathrm{of}\mathrm{matching}n\text{-}\mathrm{grams}}{\mathrm{Total}\mathrm{number}\mathrm{of}n\text{-}\mathrm{grams}\mathrm{in}\mathrm{candidate}}}$$

(6)

$$w_n={\displaystyle \frac{1}{N}}$$

(7)

The ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation—Longest Common Subsequence) [54] score is calculated based on the longest common subsequence (LCS) between a reference text and a candidate text. The ROUGE-L score considers precision, recall, and an F-measure based on the LCS and is defined in Equation (8). The precision and recall are computed using Equations (9) and Equation (10), respectively. Here, LCS is the length of the longest common subsequence between the candidate and reference sequences, $\beta$ is a weighting factor (commonly set to 1 to equally weight precision and recall), and $F_1$ is the harmonic mean of precision and recall.

$$\mathrm{ROUGE-L}=F_1={\displaystyle \frac{(1+{\beta }^2)·\mathrm{Precision}·\mathrm{Recall}}{{\beta }^2·\mathrm{Precision}+\mathrm{Recall}}}$$

(8)

$$\mathrm{Precision}={\displaystyle \frac{\mathrm{LCS}}{\mathrm{Length}\mathrm{of}\mathrm{the}\mathrm{candidate}\mathrm{sequence}}}$$

(9)

$$\mathrm{Recall}={\displaystyle \frac{\mathrm{LCS}}{\mathrm{Length}\mathrm{of}\mathrm{the}\mathrm{reference}\mathrm{sequence}}}$$

(10)

The Metric for the Evaluation of Translation with Explicit ORdering (METEOR) [55] score evaluates the similarity between a candidate sentence and a reference sentence using, and $F_{\mathrm{mean}}$ (a harmonic mean of precision and recall) and penalty P, as defined in Equation (11). Precision and recall are calculated using Equation (12) and Equation (13), respectively. $F_{\mathrm{mean}}$ is the harmonic mean of precision and recall controlled by the parameter $\alpha$, as shown in Equation (14). Furthermore, the penalty function to handle excessive fragmentation is defined in Equation (15).

$$\mathrm{METEOR}=F_{\mathrm{mean}}·(1-P)$$

(11)

$$\mathrm{Precision}={\displaystyle \frac{\mathrm{Number}\mathrm{of}\mathrm{matched}\mathrm{unigrams}}{\mathrm{Total}\mathrm{unigrams}\mathrm{in}\mathrm{the}\mathrm{candidate}\mathrm{sentence}}}$$

(12)

$$\mathrm{Recall}={\displaystyle \frac{\mathrm{Number}\mathrm{of}\mathrm{matched}\mathrm{unigrams}}{\mathrm{Total}\mathrm{unigrams}\mathrm{in}\mathrm{the}\mathrm{reference}\mathrm{sentence}}}$$

(13)

$$F_{\mathrm{mean}}={\displaystyle \frac{\mathrm{Precision}·\mathrm{Recall}}{\alpha ·\mathrm{Precision}+(1-\alpha )·\mathrm{Recall}}}$$

(14)

$$P=\gamma ·{\left({\displaystyle \frac{\mathrm{Chunks}}{\mathrm{Matched}\mathrm{unigrams}}}\right)}^{\beta }$$

(15)

To fine-tune the METEOR metric, $\alpha$ is used, enhancing the correlation between human and machine translation evaluation. $\gamma$ and $\beta$ are parameters that control the severity of the penalty (typically $\gamma =0.5$ and $\beta =3$), Chunks refers to contiguous matched segments in the candidate sentence, and Matched unigrams is the number of unigram matches between the candidate and reference sentences. The METEOR score penalizes fragmented matches to improve alignment quality and ensures a balanced combination of precision and recall.

Although formal confidence intervals are not explicitly reported, the evaluation was conducted over the full test set, which reduces variance associated with small sample evaluations. The observed improvements in ROUGE-L and METEOR are consistent across qualitative and quantitative analyses, suggesting that the gains are not driven by isolated examples but reflect a general trend in performance.

4. Results

Table 2 presents a comparative evaluation of our proposed method, LLaMA-XR, against existing methods previously benchmarked on the IU X-ray dataset. LLaMA-XR demonstrates leading performance in terms of ROUGE-L [54] and METEOR [55] on the IU X-ray dataset, two widely used metrics that emphasize semantic fidelity and syntactic fluency. In particular, LLaMA-XR achieves a 4.34% improvement in ROUGE-L over the result of SGF method [45]. This significant gain is primarily attributed to the incorporation of a larger pre-trained language model (LLM) within our architecture, which enhances the model’s ability to generate radiology reports that are both contextually relevant and clinically coherent. Although formal confidence intervals are not explicitly reported, the evaluation was conducted over the full test set, which reduces variance associated with small sample evaluations. The observed improvements in ROUGE-L and METEOR are consistent across qualitative and quantitative analyses, suggesting that the gains are not driven by isolated examples but reflect a general trend in performance. Moreover, LLaMA-XR surpasses the VTI [50] and UAR [51] methods in the METEOR metric by an impressive 54.13%, underscoring its strength in capturing synonym variability and maintaining semantic alignment with the reference reports. Whereas LLaMA-XR performs slightly below some recent methods in terms of BLEU-4 scores, it achieves competitive BLEU-4 performance while substantially outperforming all baselines on ROUGE-L and METEOR, suggesting its outputs are more semantically rich but less lexically precise than template-heavy approaches. In the context of radiology reporting, where semantic accuracy and clinical relevance are paramount, metrics like ROUGE-L and METEOR provide a more meaningful assessment of model quality. In the test environment with a single A100 GPU, it takes 1.86 s on average to generate a chest X-ray report.

Previous research has demonstrated that METEOR is more effective than BLEU in capturing semantic accuracy and linguistic fluency, particularly in specialized domains where precise terminology and structured language are critical, such as medical reporting [56]. In a similar vein, ROUGE-L has been widely recognized as a more suitable metric for assessing textual coherence and semantic fidelity, especially in tasks involving summarization and natural language generation within healthcare contexts [54]. Given that medical reports demand factual correctness, structured completeness, and linguistic coherence, the use of METEOR and ROUGE-L provides a more clinically meaningful and domain-appropriate evaluation of model performance than surface-level n-gram matching metrics.

To rigorously assess the performance of our model and highlight that, despite a marginally lower BLEU-4 score, our ROUGE-L and METEOR scores reflect better linguistic coherence and clinical relevance, we conducted a comparative analysis against CvT-DistilGPT2 [23], a model incorporating a pretrained GPT architecture with a substantial parameter count. The results of this qualitative performance comparison between our proposed LLaMA-XR framework and CvT-DistilGPT2 are presented in Table 3. Although LLaMA-XR yields a slightly lower BLEU-4 score, it achieves notable improvements in both ROUGE-L and METEOR. This distinction is particularly important in the domain of medical report generation, where lexical overlap alone, as measured by BLEU-4, is often inadequate for evaluating the semantic integrity and clinical accuracy of generated text [57]. These findings underscore the relevance of using metrics that prioritize meaningful content alignment over surface-level word matching when assessing models in the clinical NLP domain.

Table 2. Comparison with existing studies. All results are taken from the respective published works. Boldface denotes the highest score, and underlined values indicate the second-highest score for each evaluation metric. The last column reports the performance difference ($\Delta B$, $\Delta R$, and $\Delta M$) of each method compared to the second-best performing baseline.

Method	Year	BLEU-4	ROUGE-L	METEOR	Performance Difference
					$\mathbf{\Delta }\mathit{B}$ (%)	$\mathbf{\Delta }\mathit{R}$ (%)	$\mathbf{\Delta }\mathit{M}$ (%)
Transformer [41]	2017	0.137	0.335	0.172	$-31.84$	$-19.28$	$-21.10$
H-Agent [38]	2018	0.151	0.322	–	$-24.88$	$-22.41$	–
CMAS-RL [39]	2019	0.154	0.362	–	$-23.38$	$-12.77$	–
Tm-HRNN [32]	2019	0.154	0.344	0.175	$-23.38$	$-17.11$	$-19.72$
SentSAT+KG [36]	2020	0.147	0.367	–	$-26.87$	$-11.57$	–
R2Gen [42]	2020	0.165	0.371	0.187	$-17.91$	$-10.60$	$-14.22$
PPKED [43]	2021	0.168	0.376	0.190	$-16.42$	$-9.40$	$-12.84$
VTI [50]	2021	0.154	0.375	0.218	$-23.38$	$-9.64$	0.0
SGF [45]	2022	0.215	0.415	0.201	+6.97	0.0	$-7.80$
CMCL [49]	2022	0.162	0.378	0.186	$-19.40$	$-8.92$	$-14.68$
R2GenCMN [44]	2022	0.168	0.370	0.198	$-16.42$	$-10.84$	$-9.17$
CMM+RL [40]	2022	0.181	0.384	0.201	$-9.95$	$-7.47$	$-7.80$
JPG [47]	2022	0.174	0.377	0.193	$-13.43$	$-9.16$	$-11.47$
PGT [46]	2022	0.181	0.381	0.203	$-9.95$	$-8.19$	$-6.88$
ITA [48]	2022	0.188	0.382	0.208	$-6.47$	$-7.95$	$-4.59$
CvT-DistilGPT2 [23]	2023	0.177	0.377	0.193	$-11.94$	$-9.16$	$-11.47$
DCL [37]	2023	0.163	0.383	0.193	$-18.91$	$-7.71$	$-11.47$
UAR [51]	2023	0.200	0.405	0.218	$-0.50$	$-2.41$	0.0
R2GenGPT [58]	2023	0.173	0.377	0.211	$-13.93$	$-9.16$	$-3.21$
METransformer [59]	2023	0.172	0.380	0.192	$-14.43$	$-8.43$	$-11.93$
Med-LLM [60]	2024	0.168	0.381	0.209	$-16.42$	$-8.19$	$-4.13$
MAN [61]	2024	0.170	0.386	0.213	$-15.42$	$-6.99$	$-2.29$
MCSAM [24]	2024	0.184	0.394	0.210	$-8.46$	$-5.06$	$-3.67$
ASGMD [62]	2024	0.173	0.397	0.206	$-13.93$	$-4.34$	$-5.50$
MAMRG [61]	2024	0.170	0.386	0.213	$-15.42$	$-6.99$	$-2.29$
MiniGPT-4-RRG [63]	2024	0.184	0.390	0.208	$-8.46$	$-6.02$	$-4.59$
MDAKF [64]	2024	0.174	0.389	0.194	$-13.43$	$-6.27$	$-11.01$
SSM [65]	2025	0.201	0.411	0.211	0.0	$-0.96$	$-3.21$
VLCI [66]	2025	0.189	0.397	0.204	$-5.97$	$-4.34$	$-6.42$
CAT [67]	2025	0.176	0.383	0.195	$-12.44$	$-7.71$	$-10.55$
LACCOL [68]	2025	0.172	0.392	0.198	$-14.43$	$-5.54$	$-9.17$
R2gen-mamba [69]	2025	0.176	0.382	0.208	$-12.44$	$-7.95$	$-4.59$
MWCL [70]	2025	0.196	0.409	0.213	$-2.49$	$-1.45$	$-2.29$
LLaMA-XR (this study)	-	0.180	0.433	0.336	$-10.45$	+4.34	+54.13

Table 2. Comparison with existing studies. All results are taken from the respective published works. Boldface denotes the highest score, and underlined values indicate the second-highest score for each evaluation metric. The last column reports the performance difference ($\Delta B$, $\Delta R$, and $\Delta M$) of each method compared to the second-best performing baseline.

Method	Year	BLEU-4	ROUGE-L	METEOR	Performance Difference
					$\mathbf{\Delta }\mathit{B}$ (%)	$\mathbf{\Delta }\mathit{R}$ (%)	$\mathbf{\Delta }\mathit{M}$ (%)
Transformer [41]	2017	0.137	0.335	0.172	$-31.84$	$-19.28$	$-21.10$
H-Agent [38]	2018	0.151	0.322	–	$-24.88$	$-22.41$	–
CMAS-RL [39]	2019	0.154	0.362	–	$-23.38$	$-12.77$	–
Tm-HRNN [32]	2019	0.154	0.344	0.175	$-23.38$	$-17.11$	$-19.72$
SentSAT+KG [36]	2020	0.147	0.367	–	$-26.87$	$-11.57$	–
R2Gen [42]	2020	0.165	0.371	0.187	$-17.91$	$-10.60$	$-14.22$
PPKED [43]	2021	0.168	0.376	0.190	$-16.42$	$-9.40$	$-12.84$
VTI [50]	2021	0.154	0.375	0.218	$-23.38$	$-9.64$	0.0
SGF [45]	2022	0.215	0.415	0.201	+6.97	0.0	$-7.80$
CMCL [49]	2022	0.162	0.378	0.186	$-19.40$	$-8.92$	$-14.68$
R2GenCMN [44]	2022	0.168	0.370	0.198	$-16.42$	$-10.84$	$-9.17$
CMM+RL [40]	2022	0.181	0.384	0.201	$-9.95$	$-7.47$	$-7.80$
JPG [47]	2022	0.174	0.377	0.193	$-13.43$	$-9.16$	$-11.47$
PGT [46]	2022	0.181	0.381	0.203	$-9.95$	$-8.19$	$-6.88$
ITA [48]	2022	0.188	0.382	0.208	$-6.47$	$-7.95$	$-4.59$
CvT-DistilGPT2 [23]	2023	0.177	0.377	0.193	$-11.94$	$-9.16$	$-11.47$
DCL [37]	2023	0.163	0.383	0.193	$-18.91$	$-7.71$	$-11.47$
UAR [51]	2023	0.200	0.405	0.218	$-0.50$	$-2.41$	0.0
R2GenGPT [58]	2023	0.173	0.377	0.211	$-13.93$	$-9.16$	$-3.21$
METransformer [59]	2023	0.172	0.380	0.192	$-14.43$	$-8.43$	$-11.93$
Med-LLM [60]	2024	0.168	0.381	0.209	$-16.42$	$-8.19$	$-4.13$
MAN [61]	2024	0.170	0.386	0.213	$-15.42$	$-6.99$	$-2.29$
MCSAM [24]	2024	0.184	0.394	0.210	$-8.46$	$-5.06$	$-3.67$
ASGMD [62]	2024	0.173	0.397	0.206	$-13.93$	$-4.34$	$-5.50$
MAMRG [61]	2024	0.170	0.386	0.213	$-15.42$	$-6.99$	$-2.29$
MiniGPT-4-RRG [63]	2024	0.184	0.390	0.208	$-8.46$	$-6.02$	$-4.59$
MDAKF [64]	2024	0.174	0.389	0.194	$-13.43$	$-6.27$	$-11.01$
SSM [65]	2025	0.201	0.411	0.211	0.0	$-0.96$	$-3.21$
VLCI [66]	2025	0.189	0.397	0.204	$-5.97$	$-4.34$	$-6.42$
CAT [67]	2025	0.176	0.383	0.195	$-12.44$	$-7.71$	$-10.55$
LACCOL [68]	2025	0.172	0.392	0.198	$-14.43$	$-5.54$	$-9.17$
R2gen-mamba [69]	2025	0.176	0.382	0.208	$-12.44$	$-7.95$	$-4.59$
MWCL [70]	2025	0.196	0.409	0.213	$-2.49$	$-1.45$	$-2.29$
LLaMA-XR (this study)	-	0.180	0.433	0.336	$-10.45$	+4.34	+54.13

Table 3. Qualitative and metric-level comparison of reports generated by LLaMA-XR and CvT-DistilGPT2.

Input X-Ray Image with Attention Map
Reference report	The heart and mediastinum are unremarkable. The lungs are clear without infiltrate. There is no effusion or pneumothorax.
CvT-DistilGPT2	The lungs are clear bilaterally. Specifically no evidence of focal consolidation pneumothorax or pleural effusion. Cardiomediastinal silhouette is unremarkable. Visualized osseous structures of the thorax are without acute abnormality.
LLaMA-XR	The heart size and mediastinal contours appear within normal limits. The lungs are clear. There is no focal airspace opacity. No pleural effusion or pneumothorax is seen.
Category	Reference report	CvT-DistilGPT2 [23]	LLaMA-XR	Remark
Performance	-	BL4  RL  MRT	BL4  RL  MRT	-
		0.1  0.25  0.29	0.07  0.57  0.5
Heart	The heart and mediastinum are unremarkable.	The cardiomediastinal silhouette is unremarkable.	The heart size and mediastinal contours appear within normal limits.	LLaMA-XR appears to capture more detailed anatomical descriptions in this case.
Lungs	The lungs are clear without infiltrate.	The lungs are clear bilaterally.	The lungs are clear.	LLaMA-XR tends to generate coherent outputs, whereas CvT-DistilGPT2 introduces redundancy.
Focal Airspace Opacity	Not explicitly mentioned.	No focal consolidation.	There is no focal airspace opacity.	Both models identified a clinically significant feature that was not explicitly mentioned in the reference report.
Pleural Effusion	There is no effusion.	No pleural effusion.	No pleural effusion is seen.	Both models are similar, but LLaMA-XR maintains structured phrasing.
Pneumothorax	There is no pneumothorax.	No pneumothorax.	No pneumothorax is seen.	Both models are similar, but LLaMA-XR is more structured.

In order to understand the performance of the modest BLEU-4 score of 0.180, we analyzed the reports both qualitatively and quantitatively. Based on quantitative analysis on the test set, LLaMA-XR generates reports with a mean length of $44.14\pm 12.79$ tokens ($\mathrm{mean}\pm \mathrm{standard}\mathrm{deviation}$), where the mean represents the average report length and the standard deviation reflects variability across samples. This enhances the variety of lexicons through clinically precise synonyms and paraphrased sentences, such as “cardiomediasinal silhouette is unremarkable” becoming “heart size and mediastinal contours appear within normal limits.” This decreases n-gram overlap, hence reducing BLEU-4, even though semantic content and fidelity are enhanced, as reflected in the improved ROUGE-L and METEOR scores. LLaMA-XR’s BLEU-4 remains greater than 22/31, staying competitive with other recent LLM-based approaches such as R2GenGPT (0.173) and Med-LLM (0.168). Although BLEU-4 is still an important metric, clinical report generation focuses on semantic and factual content.

Our LLaMA-XR framework consistently outperforms CvT-DistilGPT2 in terms of both METEOR and ROUGE-L, underscoring its superior capability in generating semantically accurate and coherent medical texts. The METEOR metric, which accounts for synonymy, stemming, and semantic matching, is particularly well-suited for the medical domain, where multiple terminologies may describe the same condition or finding. In contrast, ROUGE-L evaluates the longest common subsequence (LCS), making it highly effective for assessing the preservation of structured medical content, such as causal relationships and anatomical consistency [54]. Although CvT-DistilGPT2 achieves a higher BLEU-4 score, its outputs tend to prioritize lexical overlap over true semantic alignment, often producing more generic reports that lack the specificity and clinical precision required in radiological interpretation. The 54% relative METEOR improvement over the previous best baseline (0.218 → 0.336) is due to the enormous pre-trained knowledge base of LLaMA 3.1 (8 B parameters) and its ability to handle synonyms and semantic paraphrasing better than the smaller transformer-based approaches in previous works. These results highlight the limitations of relying solely on BLEU-based metrics and emphasize the importance of clinically informed evaluation frameworks in medical text generation.

5. Discussion

5.1. Qualitative Analysis

Table 4 presents a qualitative comparison between reports generated by CvT-DistilGPT2 and our proposed LLaMA-XR framework. The results reveal that LLaMA-XR consistently produces reports with enhanced linguistic coherence and clinical relevance. In contrast, CvT-DistilGPT2 [23] frequently introduces extraneous content not present in the reference report, which may compromise the reliability of its outputs in clinical settings due to the risk of hallucinated or inaccurate findings. However, it is worth noting that when such additional observations are clinically accurate, they could potentially offer supplementary value to clinicians by highlighting findings that may have been overlooked in the original interpretation. Nevertheless, distinguishing between helpful augmentation and unreliable hallucination remains a critical challenge for safe and trustworthy AI deployment in medical contexts. To quantify this, we report a term-level analysis on qualitative samples, yielding a 15% hallucination rate when comparing generated pathology mentions against reference reports.

Table 4. Comparison of reference and generated reports from CvT-DistilGPT2 and LLaMA-XR.

Image	Reference	CvT-DistilGPT2 [23]	LLaMA-XR (This Study)
	The lungs and pleural spaces show no acute abnormality. Heart size and pulmonary vascularity within normal limits.	The cardiomediastinal silhouette and pulmonary vasculature are within normal limits in size. The lungs are clear of focal airspace disease pneumothorax or pleural effusion. There are no acute bony findings.	The lungs are clear. The heart is normal in size and contour. The pulmonary vascularity is within normal limits in appearance.
	Heart size normal. Lungs are clear. XXXX are normal. No pneumonia, effusions, edema, pneumothorax, adenopathy, nodules or masses.	The lungs are clear bilaterally. Specifically no evidence of focal consolidation pneumothorax or pleural effusion. Cardiomediastinal silhouette is unremarkable. Visualized osseous structures of the thorax are without acute abnormality.	Heart size normal. Lungs are clear. There is no evidence of pneumonia, pleural effusion, pulmonary edema, pneumothorax, lymphadenopathy, nodules, or masses.

LLaMA-XR appears to retain key clinical details such as normal heart size and pulmonary vascularity in the generated report. Additionally, it appears to include some findings that are not present in the CvT-DistilGPT2 output, such as the absence of pneumonia, pleural effusion, and pulmonary edema. These observations suggest that LLaMA-XR may have a tendency to preserve more relevant clinical details in the generated report. However, formal validation by radiologists would be necessary to draw conclusive comparisons regarding clinical accuracy and reliability.

Table 5 presents a qualitative comparison between LLaMA-XR and the PGT model. In the observed outputs, LLaMA-XR appears to generate reports with relatively higher fluency and completeness. While PGT produces some correct descriptions, it tends to omit certain findings that are present in the reference reports. For example, in the first case, PGT does not mention focal consolidation, which appears in the LLaMA-XR output. In the second case, LLaMA-XR also includes additional observations such as bronchovascular crowding. These differences suggest that LLaMA-XR may tend to generate more complete outputs in these examples, though formal radiologist validation would be needed to confirm clinical alignment and diagnostic reliability.

Table 5. Comparison of reference and generated reports from PGT and LLaMA-XR.

Image	Reference	PGT [46]	LLaMA-XR (This Study)
	Lungs are clear without focal consolidation, effusion, or pneumothorax. Normal heart size. Bony thorax and soft tissue is unremarkable.	The lungs are clear. There is no pleural effusion or pneumothorax. The heart and mediastinum are normal.	The lungs are clear. There is no focal consolidation effusion, or pneumothorax. Bony thorax unremarkable.
	Low lung volumes bilaterally, with lungs otherwise grossly clear. No focal consolidation, pneumothorax, or large pleural effusion. The cardiomediastinal silhouette is unremarkable. No acute osseous abnormalities identified.	Lung volumes are low. Specifically no evidence of focal consolidation pneumothorax or pleural effusion. Cardiomediastinal silhouette is unremarkable. Visualized osseous structures of the thorax are without acute abnormality.	Low lung volumes bilaterally with central bronchovascular crowding without focal consolidation, pleural effusion, or pneumothorax. Cardiomediastinal silhouette is unremarkable. No acute osseous abnormality.

5.2. Findings and Observations

In this study, several key findings and observations were made regarding the use of various models for medical report generation. After fine-tuning of LLaMA 3.1 [22], a large autoregressive decoder-only transformer with 8 billion parameters, it demonstrated strong performance on the IU X-ray benchmark in generating medical reports. This model benefits from being trained on a substantial amount of diverse data, allowing it to effectively capture the patterns required for generating clinically relevant text. Another important observation is that classification models such as DenseNet-121 [25], when used as an embedding, can achieve excellent performance in generating medical reports when provided with clear and meaningful instructions. In this study, we used a pre-trained version of DenseNet-121, fine-tuned on the same domain of medical imaging data, which likely contributed to its strong performance. This pre-training allows the model to better understand the specific nuances of medical images, enhancing its ability to interpret and generate relevant medical text. In addition, the use of Unsloth, which supports QLoRA [27] 4-bit quantization, has significantly improved the efficiency of the report generation process in terms of both speed and memory consumption. Unsloth’s ability to perform quantization reduces the model’s memory requirements, enabling faster processing without compromising the quality of the generated reports.

5.3. Limitations and Future Work

While our study utilized IU-Xray [28], one of the two widely recognized benchmark datasets, we selected it due to its high relevance and suitability for the research objectives. Its well-curated structure and established use in prior work make it a robust foundation for evaluating our approach. Nonetheless, future work could benefit from extending the analysis to the MIMIC-CXR [71] dataset to further validate and generalize the findings.

Among the various pre-trained models available for image embedding, we employed DenseNet-121 due to its demonstrated strong performance and efficient parameter utilization in medical imaging tasks. The use of a DenseNet-121 model pre-trained on datasets including IU X-ray may introduce bias, as familiarity with the data distribution could slightly inflate performance despite the extractor being frozen. Nevertheless, the choice reflects a balance between computational efficiency and representational power, while also acknowledging potential dataset-specific advantages inherent in pre-training. Importantly, our framework is designed to be modular and architecture-agnostic, allowing for seamless integration of alternative backbone models such as ResNet, EfficientNet, or Vision Transformers. Future studies may explore these and other architectures within our framework to assess their comparative effectiveness, mitigate potential pre-training biases, and adapt the model to diverse application requirements.

Additionally, we focused exclusively on QLoRA as our fine-tuning method, chosen for its efficiency and effectiveness in resource-constrained settings. Future work may investigate other fine-tuning strategies, including LoRA, Prefix Tuning, or Full Parameter Fine-Tuning, to assess their relative impact and broaden the applicability of our approach across various model sizes and domains.

Ablation studies were not conducted in this work due to the computational cost associated with retraining the 8B-parameter LLaMA 3.1 model with the Colab A100 environment. Nevertheless, the LLAMA-XR architecture’s modularity means that the following ablation studies can be conducted in future work: (a) single-view (frontal only) versus dual-view 36-dimensional embeddings, (b) 18-condition confidence scores versus higher-dimensional late-layer CNN or ViT embeddings, (c) full fine-tuning versus QLoRA, and (d) smaller versus 8B-parameter LLaMA models.

From a clinical radiology perspective, the integration of LLaMA-XR into radiologist workflows requires further exploration, particularly in terms of real-world deployment and validation [72]. While the framework demonstrates improved semantic and linguistic performance, its clinical utility hinges on how it supports or augments human interpretation [73]. For instance, LLaMA-XR could be deployed to draft preliminary reports for radiologist review in settings with limited specialist availability, such as rural or resource-constrained environments [74]. The model’s attention heatmaps may support transparency and clinician trust by visually linking predictions to image regions of interest [75], though their practical interpretability in diagnostic workflows still needs to be assessed. Since LLaMA-XR generates narrative reports, future development should explore alignment with structured reporting frameworks such as RSNA templates or BI-RADS to enhance consistency, completeness, and data interoperability [76]. Additionally, broader validation using clinical datasets, including cases with incidental findings, medical devices, or less common pathologies, is essential for robust performance [77]. Ongoing radiologist feedback and real-world evaluation will be critical to ensure safety, relevance, and integration into clinical practice [73].

Building on these system-level considerations, further validation should focus on the clinical content and safety of the generated reports. The generated reports were evaluated on IU X-ray dataset using BLEU, ROUGE-L, and METEOR, following standard practice and allowing comparison with previous studies; while these metrics capture semantic and syntactic quality, they do not fully assess whether key clinical findings are correctly identified or whether critical abnormalities are omitted or hallucinated [13,77]. Clinical metrics, including RadGraph F1 and CheXbert label-based scores, assess the accuracy of entities and relations, and we aim to incorporate these clinically grounded metrics in future evaluations. A small-scale qualitative review by radiologists, such as case-based annotations comparing generated reports to reference standards, could offer valuable insight into clinical accuracy, safety, and interpretability [74]. This type of human evaluation would help determine whether the model supports diagnostic decision-making or introduces risks through omission or irrelevant information [73].

Furthermore, it is important to acknowledge that the comparison between LLaMA-XR and most prior methods is architecturally asymmetric. LLaMA-XR utilizes a large language model with 8 billion parameters, whereas the majority of baseline models employ sub-1 B parameter architectures. A portion of the performance gains observed in ROUGE-L and METEOR may therefore reflect differences in model scale, in addition to the proposed architectural contributions. For instance, LLaMA-XR achieves a METEOR score of 0.336, representing a 54% improvement over the next best-performing baseline (0.218 for VTI/UAR). However, as the baseline studies do not report parameter counts, a precise quantitative comparison of model scale is not feasible, and the relative differences in model capacity may vary. Future comparisons with models of comparable parameter sizes would help to isolate the specific contributions of the proposed framework.

In addition, the model’s ability to recognise uncommon or incidental findings requires further exploration. Chest radiographs frequently contain incidental nodules, bony lesions, or devices such as central lines and pacemakers, which may be underrepresented in training datasets. These elements are clinically important but can be overlooked by automated systems that rely on frequent or highly visible patterns [77]. Understanding how LLaMA-XR handles such low-prevalence features is critical for deployment in real-world practice. Future studies should include cases with atypical pathologies and radiological devices to assess whether the model can accurately identify, interpret, or appropriately omit these findings based on clinical context [73,76]. Confidence intervals and statistical significance testing (e.g., via bootstrap resampling) are not included in the current study, and these analyses can be considered in future work.

6. Conclusions

In this study, we proposed LLaMA-XR, a novel framework for automated radiology report generation that combines DenseNet-121 for visual feature extraction with LLaMA 3.1 for text generation and structured text completion. By leveraging multi-view X-ray embeddings and prompt-based conditioning, our model bridges the semantic gap between visual and textual modalities more effectively than previous approaches. LLaMA-XR achieved top-tier performance on the IU X-ray dataset, attaining a ROUGE-L score of 0.433 and a METEOR score of 0.336, indicating its ability to generate semantically accurate and clinically coherent reports. Although BLEU scores were slightly lower than some baselines, they remained competitive, while ROUGE-L and METEOR indicate stronger semantic richness and clinical coherence. The use of structured prompts and optimized fine-tuning strategies contributed to enhanced report quality and computational efficiency. This underscores the effectiveness of our framework in producing reliable medical narratives with reduced memory and processing overhead. Overall, this work demonstrates the promise of combining advanced vision and language models for AI-assisted radiology, contributing to the advancement of AI-assisted systems for supporting clinical decision-making.