Effective Machine Learning Techniques for Non-English Radiology Report Classification: A Danish Case Study

Abstract

Background: Machine learning methods for clinical assistance require a large number of annotations from trained experts to achieve optimal performance. Previous work in natural language processing has shown that it is possible to automatically extract annotations from the free-text reports associated with chest X-rays. Methods: This study investigated techniques to extract 49 labels in a hierarchical tree structure from chest X-ray reports written in Danish. The labels were extracted from approximately 550,000 reports by performing multi-class, multi-label classification using a method based on pattern-matching rules, a classic approach in the literature for solving this task. The performance of this method was compared to that of open-source large language models that were pre-trained on Danish data and fine-tuned for classification. Results: Methods developed for English were also applicable to Danish and achieved similar performance (a weighted F1 score of 0.778 on 49 findings). A small set of expert annotations was sufficient to achieve competitive results, even with an unbalanced dataset. Conclusions: Natural language processing techniques provide a promising alternative to human expert annotation when annotations of chest X-ray reports are needed. Large language models can outperform traditional pattern-matching methods.

1. Introduction

Artificial intelligence research has long been working towards the automation of medical image interpretation with high accuracy and efficiency. Supervised machine learning algorithms show promising results in terms of diagnostic accuracy, while taking little time to identify a pre-defined set of abnormalities [1,2,3]. These algorithms are data-hungry, meaning that they need to be fed thousands of images to perform in the best way possible. Each case needs an annotation, or set of labels, that indicates whether abnormalities are present or not. These labels are expensive to obtain, as only human experts can become annotators. Concerns arise when a clinician’s workload shifts from diagnostics to annotation, creating a new problem rather than solving it [4,5]. However, some information is already available when images are obtained from clinics, as radiologists describe their findings in a detailed free-text report.

Natural language processing (NLP) techniques have proven to be effective in automatically detecting the findings from a radiology report [6,7]. The use of these methods can aid us in acquiring the annotations required for the development of computer vision systems. Examples of possible techniques include string-matching rules which identify pre-defined patterns to output annotations, as well as state-of-the-art deep learning methods.

Rule-based systems use regular expressions (RegExs), a set of textual matching rules, to match patterns in phrases which indicate the presence or absence of a clinical finding, in addition to the type of presence, i.e., a positive or negative mention [8,9,10]. Rule-based systems require carefully crafted RegEx rules that are tailored to the target domain and language. Consider the sentence “We exclude the presence of pneumothorax”, which contains a negatively mentioned finding. A simple RegEx like (s/pneumothorax) would match the word “pneumothorax” and wrongly interpret it as a positive mention. Modeling negations and spelling errors that naturally occur in free-text language is the main challenge of using RegEx annotation methods.

NegBio [11] is a RegEx method that detects negations and uncertainty using special language patterns. It inspired the methods used to extract labels from reports of chest X-ray images in English [12,13] and other languages such as Brazilian Portuguese [14], Vietnamese [15], and German [16]. Large language models (LLMs) have shown strong results in previous work on this task: CheXbert [17] is a BERT-based [18] language model trained on labels extracted using another rule-based method and human expert annotations, achieving better performance than the previous state-of-the-art rule-based methods.

Most of the available automatic annotation tools work only on radiology reports written in English. When working in other languages, fewer technical resources are available, which necessitates the development of task- and language-specific methods. The severity of this challenge can be quantified with reference to the European Language Equality Programme, which defines the technological DLE factor to measure the level of technological support for a language in terms of the available data and tools [19]. Although Danish is not considered a low-resource language, its technological DLE factor (according to the European Language Grid, release 3) is only 10,439, in contrast to 78,335 for English. It can also be compared to German and Spanish, for which automatic annotation tools have been developed, which have DLE factors of 39,929 and 36,751, respectively. The lack of resources for languages such as Danish makes the development of an annotation tool for radiology reports more difficult.

This study explored techniques for the efficient and cost-effective automatic annotation of chest X-rays reports written in Danish and proposes a machine learning model training strategy that matches the performance of similar English methods. With this in mind, the aim of this work is the following:

(1)

Develop a string-matching algorithm for the automatic identification of positive and negative mentions of abnormalities in Danish chest X-ray reports.

(2)

Compare the performance of the above method against various BERT-based machine learning models, including models trained on multi-lingual datasets versus models fine-tuned on the target language.

(3)

Present an overview of the experimental results on the annotation effort needed to achieve the desired performance.

2. Materials and Methods

Ethical approval was obtained on 11 May 2022 from the Regional Council for Region Hovedstaden (R-22017450). Approval for data retrieval and storage was obtained on 19 May 2022 from the Knowledge Center on Data Protection Compliance (P-2022-231).

2.1. Materials: Data Collection

The data for this study were a subset of a larger dataset extracted from 2010–2019 from the PACS system of a major network of hospitals and private clinics in Denmark ($n=11$). The dataset contained 1,452,561 unique cases and 878,305 associated reports, representing 808,662 studies with 357,205 unique patients. From this, 547,758 unique text reports were selected for annotation, of which a random subset of 2475 entries were selected for human expert labeling by medical personnel. See Figure 1 for an overview of the annotation process. The site with the most reports contributed ca. 139,000 examples (25% of the total). The site with the fewest reports contributed ca. 10,000 examples (2% of the total).

The label hierarchy used in our dataset was developed by Danish radiologists to match their diagnostic terminology [20,21,22]. It includes parent labels and increasingly specific sub-labels, with definitions to ensure consistent application. The hierarchy was designed to cover all findings comprehensively and to minimize the application of a catch-all Other label. The hierarchy was iteratively refined with an external medical software company through testing and comparisons with public datasets. Radiologists made all final label decisions, drawing on related studies [12,23]. Listing  A1 in Appendix A presents the full hierarchy of labels.

In the report labeling process, each abnormality (finding) could be assigned to one of the following classes: a “positively mentioned finding” (a finding is present), “negatively mentioned finding” (no evidence of a finding is present), or “the finding not mentioned in the report” (the models were trained to identify this class but not evaluated against it). Positive mentions were sometimes found with contextual modifiers in the reports, such as annotators noting uncertainty about the classification or using adjectives or comments to grade the severity of the finding. Since these mentions were still positive in the sense of describing the presence of a finding, they were coded as positive.

2.2. Methods: Regular Expressions ($RE$)

Inspired by the work by Irvin et al. [12], 574,758 reports in our dataset were labeled using a simple pattern-matching method-based NegEx [24], which we refer to as $RE$ (Figure 2). Following this protocol, a total of 360 RegEx rules were created, which unfolded on 1,548,505 unique raw string patterns.

2.3. Methods: Pre-Trained Language Models

Large language models (LLMs) are the current state of the art in many NLP tasks [25]. The training strategy developed by Smit et al. [17] to classify chest X-ray reports in English using BERT-like models was followed to classify Danish reports. Five LLMs pre-trained on Danish text were selected. The order of the presentation of the models’ results is based on the amount of Danish text used in the model’s pre-training, from the least to the most (

Table 1. Summary of the size (as in the number of parameters) and data used to pre-train the selected BERT-like models. indicates multi-lingual data, stands for Danish, and for Danish medical data. The models are presented from the least amount of Danish data used in their pre-training (mBERT) to the most (DanskBERT).

Pre-Training Data	Model Name	Parameters	Tokens/Words
	mBERT	110 M	≈200 M words 1
	BotXO	110 M	1.6 B words
	MeDa-BERT	110 M	+123 M tokens
	XML-RoBERTa	125 M	7.8 B tokens
	DanskBERT	125 M	+1 B words

¹ This number is a rough estimate, as the specific number was not reported by [18].

). Box 1 summarizes terminology related to LLMs used in this paper.

Box 1. Large language models (LLMs) terminology

Pre-training	Models are trained with unstructured, unlabeled data by predicting missing parts of the input as a next word prediction task.
Fine-tuning	A pre-trained LLM is further trained to specialize it for a specific task, e.g., radiology report classification.
Token	Fundamental unit of text in LLMs, representing words or subwords.
Tokenizer	A function that converts the input text into tokens.
Context	Amount of text, in tokens, that the model can process at any one time.

Since the primary focus of this study was to explore the feasibility of translating research techniques developed for English to a non-English language (Danish), LLMs with billions of parameters were excluded. This practical approach ensured that the findings in this paper are relevant and applicable for organizations with similar data, hardware, human resources, and privacy limitations. More details about pre-trained language models and the RegEx-based method are available in Appendix B.

2.4. Experimental Protocol

Using $RE$ rules, labels were extracted from 547,758 text reports and collected in the rule-based $RB$ dataset. This set was divided with a 98/2 split for validation during training.

The human expert labeled ($HL$) dataset with 2475 samples was divided into train ($n=1600$), dev ($n=125$), and test ($n=750$) sets. The splits were generated using stratification for multi-label data as described by Sechidis et al. [26] and implemented using the iterative-stratification Python library (version 0.1.7). $RB$ labels were generated in $H{L}_{test}$ to compare $RE$ against machine learning models. The datasets presented a huge imbalance between the findings and assigned classes (Figure 3 and Figure A1).

All language model weights were initialized in the respective pre-trained architecture, which produced a CLS token. This token was the input to a multi-label, multi-class linear prediction layer. Following the labeling protocol, this translated into a layer of 49 linear heads that could predict three different classes: a positive mention, negative mention, and no mention. The input text was tokenized, and only the first 512 tokens were used as the context for making predictions. In the dataset, a very small number of reports exceeded this limit, but the exact number depended on the model’s tokenizer. All models were trained with unfrozen weights, using the cross-entropy loss and AdamW optimization with an initial learning rate of 1 × 10⁻⁵. The loss was computed by summing the individual losses of the linear heads. All models were trained using one NVIDIA RTX 4090 GPU, with a batch size of 28 for RoBERTa base models and 30 for BERT base models.

2.4.1. Rule-Based Labels ($RB$)

A given machine learning model was trained on $R{B}_{train}$. The models’ performance was tracked on $R{B}_{dev}$. All models were trained for 8 epochs, with an average wall clock training time of 6 h.

2.4.2. Human Expert Labels ($HL$)

A given machine learning model was trained on a subset of the data that was labeled by human experts ($HL$). $H{L}_{train}$ was much smaller than $RB$; therefore, the models were trained for 50 epochs to improve the convergence. $H{L}_{dev}$ was used for validation, on which the models showed performance degradation when training for more epochs.

2.4.3. Transfer Learning ($RB\to HL$)

Starting with models trained on the $RB$ set, the models were further trained on the human expert-annotated data in $HL$, following the same hyper-parameters as the models trained exclusively on $HL$. This experiment is indicated in the results as $RB\to HL$. This technique is also called transfer learning (Figure 4), as in when a model trained on one task is then adapted to the target task by training further on a limited dataset from the desired domain. The main results report the average performance over 5 repeated training runs with different random seeds to address training variation.

2.5. Resource-Driven Experiments

This study examined training strategies that could enhance performance in scenarios with limited manual labeling or data resources. Additionally, the robustness of the methods was evaluated through the use of model ensembles.

2.5.1. Definition of Most Frequent Findings

A subset of the initial findings was selected so that it corresponded to the top ten most frequent annotations for the positive and negative classes in the entire dataset, as annotated by human expert annotators and using the RE method. This subset resulted in 14 findings and was defined only for evaluations, meaning that no model was trained exclusively on this subset. Comparing the performance of only the most frequent labels allowed us to compare the Danish models to English methods that use smaller label sets.

2.5.2. Definition of Model Ensemble

Model ensembles are a well-established machine learning method for increasing the accuracy of predictions by training different models and then averaging their predictions [27]. They have been shown to outperform single classifiers within the ensemble, as they tend to decrease the generalization error without increasing the model variance [28]. In the second stage of the language model training, a model ensemble was trained using 5-fold cross-validation. $H{L}_{train}$ was divided into 5 folds of 320 examples each: 4 folds were used to train one model, and the left-out fold was used for validation. This process was repeated to train 5 models on their respective training–validation folds. For testing, the predictions on $H{L}_{test}$ from each model were averaged using majority voting.

2.5.3. Definition of Data Ablation

Knowing how many expert annotations are needed to develop a good annotation tool is crucial. The $RB\to HL$ experiment was repeated by varying the proportion of the total human-labeled data used to train the model. These subsets were again randomly sampled with data stratification. Predictions on $RB$ are reported as $H{L}_{train}^{0\%}$, while $H{L}_{train}^{100\%}$ refers to $RB\to HL$. This ablation experiment was repeated 5 times to address training variation.

3. Results

3.1. Evaluation Metrics

Each model was evaluated for its ability to automatically extract findings from the reports, specifically by computing metrics on the negative and positive mentions of the findings. The performance was assessed by computing the F1 score for the positive and negative mention classes across 49 findings. These scores were then averaged to obtain the macro F1 score. The F1 score represents both precision and recall in one metric. It can be defined as

$$\mathrm{F}1={\displaystyle \frac{2·\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}={\displaystyle \frac{2TP}{2TP+FP+FN}}$$

(1)

To address the class imbalance, the weighted F1 score is also reported, which weights the macro F1 score by the class support.

3.2. RegEx and Transfer Learning

When comparing the distribution of the F1 across 49 findings in Figure 5, most models performed better on the positive mention class. However, BotXO and DanskBERT showed similar distributions for both negative and positive findings.

Table 2. Macro F1 and weighted F1 scores for positive and negative mentions for the 49 findings in the $HL$ test set, for $RB\to HL$ models. The scores are the average score and one standard deviation interval over 5 repeated $RB\to HL$ training runs: for each run, the same $RB$ checkpoint was used for fine-tuning on $HL$ labels, but with a different seed. Best score per metric is highlighted in bold.

	Positive F1	Negative F1	Weighted F1
			All Findings	Most Frequent
RE	0.721	0.478	0.667	0.846
mBERT	0.742 ± 0.003	0.477 ± 0.008	0.732 ± 0.003	0.869 ± 0.001
BotXO	0.745 ± 0.007	0.509 ± 0.012	0.737 ± 0.004	0.876 ± 0.002
MeDa-BERT	0.739 ± 0.005	0.480 ± 0.006	0.742 ± 0.003	0.873 ± 0.002
XLM	0.738 ± 0.007	0.498 ± 0.004	0.736 ± 0.004	0.884 ± 0.001
DanskBERT	0.738 ± 0.011	0.524 ± 0.008	0.744 ± 0.007	0.882 ± 0.003

shows that every machine learning model surpassed $RE$ in terms of the macro F1 score, with the exception of mBERT on the negative mentions. The latter barely reached the performance of the $RE$ method, but still showed an improvement in the other classes. Interestingly, despite being trained on the target domain data, MeDa-BERT did not perform as well as BotXO on both positive and negative mentions. The larger RoBERTa models XLM and DanskBERT performed slightly worse on positive mentions compared to BERT models; however, they achieved the biggest improvement over $RE$ for negative mentions ($\Delta =+0.046$) and the weighted F1 score for all findings ($\Delta =+0.077$). Looking at the standard deviation across every experiment, there was very little variance between the multiple training runs with different seeds, meaning that the selected training schedule was consistent.

3.3. Most Frequent Findings Subanalysis

By restricting the number of tested findings to only the most frequent ones, there was a important improvement in the weighted F1 scores, as shown in Table 2. For example, the F1 score for frequent findings increased ($\Delta =+0.138$) for DanskBERT compared to that for all findings. This means that even if, on average, the model’s average performance was lowered by the inclusion of underrepresented labels, its performance on the most frequent findings was still enhanced compared to $RE$. For the macro F1 scores of the positive and negative mentions for the most frequent findings, see

Table A3. Macro F1 score for the 49 findings on the $HL$ test set for $RB\to HL$ models, but reporting only the averages over the 14 most frequent findings. Best score per metric is highlighted in bold.

	Positive F1	Negative F1	Weighted F1
RE	0.848 (0.80–0.89)	0.635 (0.46–0.81)	0.844 (0.80–0.89)
mBERT	0.857 (0.80–0.91)	0.682 (0.50–0.87)	0.874 (0.83–0.92)
BotXO	0.860 (0.80–0.92)	0.721 (0.57–0.87)	0.881 (0.84–0.93)
MeDa-BERT	0.877 (0.83–0.92)	0.663 (0.46–0.87)	0.883 (0.84–0.93)
XLM	0.868 (0.81–0.92)	0.653 (0.42–0.88)	0.879 (0.82–0.94)
DanskBERT	0.867 (0.82–0.92)	0.679 (0.47–0.88)	0.883 (0.84–0.93)

in Appendix C.

Figure 6 illustrates the performance of models on the individual most frequent findings. Most models tended to cluster around the same score when the class support was high enough, and the language models performed better than the RE. Exceptions arose with findings that were underrepresented, such as Consolidation in the positive mention class ($n=54$) and ChronicLungChanges in the negative mention class ($n=2$).

3.4. Model Ensemble Subanalysis

Table 3. Macro F1 scores of the 49 findings for the $HL$ test set for the $RB\to HL$ model ensembles, compared to $RE$. Best score per metric is highlighted in bold.

	Positive F1	Negative F1	Weighted F1
RE	0.721	0.478	0.667
mBERT	0.743	0.650	0.766
BotXO	0.726	0.693	0.763
MeDa-BERT	0.747	0.397	0.727
XLM	0.756	0.516	0.748
DanskBERT	0.740	0.717	0.778

presents the results of the model ensemble experiments. In comparison to the single-model performance in Table 2, all of the ensembled models performed slightly worse on positive mention classification but improved on negative mention prediction. The only exception was MeDa-BERT, which consistently struggled with this class. Overall, the performance of the ensembled DanskBERT model improved the most compared to its unensembled counterpart for negative mention prediction ($\Delta =+0.193$). We noted that the ensembled DanskBERT model did not perform as well as XLM on positive mention classification ($\Delta =-0.016$), but it outperformed all models on negative mention classification.

Figure 7 shows the macro F1 scores of both the negative and positive mentions for each k-fold, compared against the fold average and the final result obtained through majority voting. The majority voting score for all models was higher than the average score across individual folds, indicating a substantial enhancement in performance through ensembling. The results for XLM and MeDa-Bert were clustered together, while the rest of the models showed more variation. Every model, except for MeDa-Bert, improved (became closer to the top-right corner) with majority voting compared to just the fold average, especially for negative mentions, demonstrating that model ensembles can improve the robustness of predictions.

3.5. Data Ablation Subanalysis

Figure 8 illustrates the impact of increasing the amount of human-labeled training data to better demonstrate how much expert-labeled data is necessary to achieve these improvements over the rule-based system. For all findings (Figure 8a), larger training sets slightly improved the accuracy of negative mentions, the most challenging class to learn. However, this improvement was accompanied by a slight decrease in performance for positive mentions. The $H{L}_{train}^{50\%}$ training set demonstrated a marked drop in performance across all five runs. We can find no explanation to support this unexplained sudden drop in performance, which persisted over every model and on different training runs. Only BotXO and mBERT’s performance gradually recovered, but with more variation. In the right part of the figure (Figure 8b), the evaluations were limited to the most frequent findings, which resulted in little to no variance among training runs, but they showed a slight decrease in performance on $H{L}_{train}^{50\%}$. More detailed results on data ablation are reported in

Table A5. Results of the $RB\to HL$ models trained on only a percentage of $H{L}_{train}$, averaged over 5 training runs.

		10%	20%	30%	50%	80%
mBERT	Negative F1	0.478	0.478	0.521	0.372	0.493
	Positive F1	0.699	0.695	0.699	0.684	0.664
	Weighted F1	0.690	0.692	0.710	0.685	0.696
BotXO	Negative F1	0.488	0.495	0.543	0.408	0.518
	Positive F1	0.728	0.709	0.666	0.639	0.623
	Weighted F1	0.711	0.708	0.695	0.666	0.677
MeDa-BERT	Negative F1	0.472	0.478	0.504	0.372	0.418
	Positive F1	0.723	0.729	0.665	0.652	0.628
	Weighted F1	0.712	0.715	0.702	0.676	0.673
XLM	Negative F1	0.486	0.503	0.602	0.377	0.391
	Positive F1	0.736	0.722	0.687	0.724	0.695
	Weighted F1	0.722	0.720	0.722	0.711	0.700
DanskBERT	Negative F1	0.466	0.468	0.536	0.372	0.385
	Positive F1	0.726	0.716	0.660	0.657	0.628
	Weighted F1	0.703	0.705	0.699	0.675	0.665

(Appendix C).

Lastly,

Table 4. Weighted F1 score of the 49 findings for the $HL$ test set, summarizing the results obtained using different training sets and strategies. Extended results are available in Table A1, Table A2, Table A3, Table A4 in Appendix C.

	$\mathit{RB}$	$\mathit{HL}$	$\mathit{RB}\to \mathit{HL}$	Ensemble
RE	0.667	-	-	-
mBERT	0.702	0.566	0.732	0.766
BotXO	0.700	0.511	0.737	0.763
MeDa-BERT	0.704	0.540	0.742	0.727
XLM	0.701	0.532	0.736	0.748
DanskBERT	0.707	0.535	0.744	0.778

summarizes the results presented, including those of experiments on models trained exclusively on $RB$ and $HL$. If no rule-based annotations were available, the performance degraded drastically, to the point that $RE$ performed better than machine learning models trained on only 1600 samples.

4. Discussion

The aim of this study was to classify Danish chest X-ray text reports by the abnormalities mentioned by radiologists. An ensemble of predictions from the larger model trained on more Danish data achieved the best overall classification performance, particularly in negative mention detection. Machine learning models trained on a combination of automatically and manually extracted labels outperformed classic rule-based methods. Models pre-trained on Danish slightly outperformed multi-language models. Surprisingly, the model trained on more high-quality Danish medical data did not benefit from continued pre-training on the target domain data. Taken together, these results are encouraging for languages that lack good, open-weight pre-trained models, suggesting that multi-lingual models may be used instead to achieve a similar performance. However, it may still be necessary to adapt such a multi-lingual model to each language because it is not straightforward to train a single classification model for many languages [29].

For comparison to English methods, Smit et al. [17] reported a weighted F1 score of 0.798 for 14 findings on reports written in English and a radiologist benchmark of 0.805. While it cannot be directly compared, the proposed Danish method exceeded the expectation of matching previous work results, as it achieved an F1 score of 0.882 on a similar label set, although without an evaluation of the uncertainty extraction.

The labeling protocol plays a crucial role when evaluating performance against human annotators. Chen et al. claim that implementing a hierarchical taxonomy for label extraction improves the labeling accuracy and reduces missing annotations [30]. However, classification problems get harder when the set of classes increases [31]. All the tested machine learning models, when limited to the most frequent findings, showed an improvement in the macro F1 score, demonstrating that including less frequent abnormalities does not imply a drastic drop in performance on the more frequent findings. However, to ensure a better result for uncommon findings, more samples are required. When considering a large and diverse set of findings, model ensembles showed an increase in the robustness and generalization of the predictions. The expert-annotated data ablation study further emphasized the importance of the quality and distribution of training dataset labels over the dataset size. This result suggests that there might be less need for large sets of annotated samples than expected if previous training on low-quality data is performed. A curated distribution of findings may be more beneficial than increasing the dataset’s size.

Some challenges presented by this task were addressed by implementing and testing a strategy for the creation of annotation tools for the classification of radiology reports; creating or collecting different NLP methods for Danish and medical-domain Danish; and addressing the variation in the performance of tools that were created using domain-specific data. The limitations of this study include the difficult access to medical datasets due to restricted access to radiology reports and the corresponding annotations. Since the focus of this study was on a single dataset, it potentially limits the generalization of the reported findings. Additionally, while this dataset did not necessitate the use of larger language model context windows for analysis, reports from different clinical settings may be longer and place critical information towards the end. No class-balancing techniques were deployed in this study as the dataset was large enough to reflect the final target distribution, but these may be advised when the distribution of classes and labels is highly irregular and not representative.

In summary, the most effective model was the 125 million-parameter DanskBERT model pre-trained on general-purpose text in our target language, with predictions determined through majority voting from an ensemble of five such models. Methods developed for English can also be applied to another language, such as Danish, but models benefit from a bigger target language pre-training dataset. The combination of good-quality RegEx rules and large language models proved essential to solve this task, and accessible, open-weight, multi-language models are an adequate solution in the absence of language-specific models.

5. Conclusions

This study compared regular expression rules and machine learning models for detecting medical findings within Danish radiology reports. The task was challenging as the available methods have been developed mainly for English, and no such tools are available for Danish. In general, large language models outperformed RegEx rules, particularly when models were pre-trained on more Danish text, especially in capturing negative mentions. Ensemble methods improved the model generalization. In this task, a detailed labeling protocol did not hurt the performance of machine learning models on the most frequent abnormalities. However, to achieve optimal performance on underrepresented findings, the dataset size and its diversity must be prioritized. While this study is limited to a single European language, future work might verify if the techniques developed for this study can be applied to other Germanic languages or other Indo-European languages. Exploring the integration of text-based findings with image classifiers could offer valuable insights into potential performance improvements for the development of AI radiology solutions.