An Open-Source Clinical Case Dataset for Medical Image Classification and Multimodal AI Applications

Abstract

High-quality, openly accessible clinical datasets remain a significant bottleneck in advancing both research and clinical applications within medical artificial intelligence. Case reports, often rich in multimodal clinical data, represent an underutilized resource for developing medical AI applications. We present an enhanced version of MultiCaRe, a dataset derived from open-access case reports on PubMed Central. This new version addresses the limitations identified in the previous release and incorporates newly added clinical cases and images (totaling 93,816 and 130,791, respectively), along with a refined hierarchical taxonomy featuring over 140 categories. Image labels have been meticulously curated using a combination of manual and machine learning-based label generation and validation, ensuring a higher quality for image classification tasks and the fine-tuning of multimodal models. To facilitate its use, we also provide a Python package for dataset manipulation, pretrained models for medical image classification, and two dedicated websites. The updated MultiCaRe dataset expands the resources available for multimodal AI research in medicine. Its scale, quality, and accessibility make it a valuable tool for developing medical AI systems, as well as for educational purposes in clinical and computational fields.

Dataset: https://doi.org/10.5281/zenodo.10079369.

Dataset License: CC-BY-NC-SA

1. Summary

Acquiring high-quality, open-access datasets is a persistent challenge in the clinical domain. In medical image classification, datasets are typically limited to a single type of medical image, such as chest x-rays [1,2], gastrointestinal endoscopies [3,4], skin photographs [5,6,7], or histopathological images [8,9,10,11]. Only exceptionally do medical image classification datasets include a broader variety of image types [12]. When it comes to multimodal datasets that include both visual and textual data, they typically feature a broader range of image types [13,14,15,16,17,18,19,20,21,22,23,24]. However, the text provided in these datasets usually focuses on the image content, and only a few exceptions refer to the image context (the actual clinical case) [17,21,23].

This paper introduces a new version of MultiCaRe, a dataset consisting of clinical cases, labeled images, and captions, extracted from 72,581 open-access case reports from PubMed Central (see Figure 1 for an example) [25,26]. This version addresses the limitations and errors identified in the original release, and includes an updated collection of clinical cases (93,816 in total) and images (130,791), and a richer taxonomy (145 categories, with a hierarchical structure). The dataset is optimized for image classification, and can also be used for other tasks, such as fine-tuning large multimodal models, training image captioning or image generation models, and for data annotation (for example, for Natural Language Processing or image segmentation tasks). Differences between this updated version and the initial release are detailed in Appendix A. Citation data for each article used in MultiCaRe is available in the metadata.parquet file. When compared to other datasets sourced from PubMed Central articles, MultiCaRe offers two main advantages: its large number of image classes and the inclusion of clinical cases associated with the images (see

Table 1. Comparison of datasets derived from PubMed Central data.

	MultiCaRe [25]	PMC-OA [19]	ROCO [22]	LLaVA-Med [29]	PMC-15M [30]	PMC-VQA [31]	PMC-Patients [32]	MedICaT [33]
Images	130K	1.6M	79K	600K	15M	149K	–	217K
Image classes	140+	–	20+	5	30+	–	–	–
Hierarchical taxonomy	✓	–	–	–	–	–	–	–
Captions	✓	✓	✓	✓	✓	✓	–	✓
Textual references *	✓	–	–	✓	–	–	–	✓
Clinical cases	93K	–	–	–	–	–	167K	–
Q&A	–	–	–	✓	–	✓	–	–

* Textual references consist of mentions of specific figures within the article text (e.g., “The patient was found to have a brain tumor (Figure 2)”).

). Apart from the dataset, this release includes: (a) a Python package containing all the code needed to create and use the dataset; (b) medical image classification models; (c) the corresponding taxonomy with documentation; and (d) two websites with clinical cases intended for healthcare professionals and medical students (all links are available in Section 4). The project’s code is available in the GitHub repository [27].

Before describing the dataset and its creation, it is important to address some key points regarding case report articles. Case reports are a type of medical publication that describe and discuss the clinical presentation of one or more patients. Compared to other forms of clinical research, this article type has both advantages and limitations: on the one hand, it enables the detection of clinical novelties, the generation of hypotheses, and the study of rare diseases; on the other hand, it does not allow for generalizations or causal inferences [34]. Case reports are a relatively underutilized resource for model training and AI application development, despite being a rich resource of multimodal clinical data. Key differences between case report datasets and data extracted from medical records are detailed in Appendix B.

2. Data Description

This section outlines the contents and structure of the entire dataset, which is available for download on Zenodo [25]. However, for specific use cases (such as training image classification models for CT scans), the recommended approach is to create customizable subsets using the MedicalDatasetCreator class from Python’s multiversity library, rather than accessing the data directly from the repository (see Section 4) [35].

These are the files included in the dataset:

• data_dictionary.csv: This file describes the dataset’s contents, providing definitions for all data fields in each file.
• metadata.parquet: This file contains metadata for case report articles, including authors, year, journal, keywords (extracted from the article’s content), MeSH terms, DOI, PMCID, license, and link.
• abstracts.parquet: This file stores the abstracts of the articles.
• cases.parquet: This file contains the text of the clinical cases, along with the age of the patient (in years), and the gender (Female, Male, Transgender or Unknown).
• case_images.parquet: This file contains metadata from the raw images, such as the file name and the tag present in the EuropePMC API, captions, and text references (substrings from the clinical case text that refer to the images).
• captions_and_labels.csv: This file contains the metadata of all the images of the dataset. It includes two types of columns:

  –

  file information columns: they include details such as the image name, the source image (original name in EuropePMC’s API), license, size, image caption, and substrings from the case text referencing the image.

  –

  label columns:

  ∗

  image_type: multiclass classification column with seven possible classes (‘radiology’, ‘pathology’, ‘chart’, ‘endoscopy’, ‘medical_photograph’, ‘ophthalmic_imaging’ and ‘electrography’).

  ∗

  image_subtype: multiclass classification column with 40 possible classes (including ‘x_ray’, ‘ct’, ‘mri’, ‘ekg’, ‘immunostaining’, etc).

  ∗

  radiology_region: multiclass classification column, including nine possible anatomical classes that are applicable only for ‘radiology’ images (‘thorax’, ‘abdomen’, ‘head’, ‘lower_limb’, etc).

  ∗

  radiology_region_granular: multiclass classification column, including 19 possible anatomical classes that are applicable only for ‘radiology’ images (the same classes as in radiology_region, but ‘upper_limb’ and ‘lower_limb’ are replaced with more detailed classes, such as ‘hip’ and ‘knee’).

  ∗

  radiology_view: multiclass classification column, including 12 possible classes that are only applicable for ‘radiology’ images (e.g., ‘axial’, ‘frontal’, ‘transabdominal’, etc).

  ∗

  ml_labels_for_supervised_classification: multilabel classification column (there can be more than one label per image). This column includes all the labels assigned to images by machine learning (ML) models, including the ones present in the multiclass columns.

  ∗

  gt_labels_for_semisupervised_classification: multilabel classification column (there can be more than one label per image). Labels were primarily assigned based on the image captions, except when they conflicted with the ML-generated labels. In such cases, the caption-based label was removed—unless the ML label had a confidence score above 0.95, in which case it replaced the caption-based label. Compared to the labels from the ml_labels_for_supervised_classification, the ones from gt_labels_for_semisupervised_classification have a higher quality and a wider taxonomy, but they are not complete (for example, not all the CT scans have a ‘ct’ label, because some of their captions do not mention words that are specific to that class). Coincidences between both columns mean that the ML models predicted the same labels as extracted using the caption-based approach. This column is well-suited for semi-supervised learning, as it includes a mix of labeled and unlabeled data. Additionally, it can be used to sample images from specific classes when higher label quality is required—even if the total number of labeled examples is lower than in ml_labels_for_supervised_classification.

• Image files: This dataset contains 130,791 images in .webp format, with an average resolution of 430 × 365 pixels (this average varies depending on the image class). The image files are located in 90 folders (from ‘\PMC1\PMC10\’ to ‘\PMC9\PMC99\’), based on the first digits of the PMCID of the article they were sourced from. The file names encode relevant information about each image. For example, in PMC10000728_fmed-09-985235-g001_A_1_3.webp, ‘PMC10000728’ is the PMCID of the corresponding case report article, ‘fmed-09-985235-g001’ is the name of the file present in EuropePMC’s API, ‘A’ is the reference used in the caption to refer to this part of the original file, and ‘1’ and ‘3’ mean that the file is the first component from a compound image with three components. The names of images that were not split during post-processing end with ‘_undivided_1_1.webp’.

3. Methods

3.1. Dataset Creation

This section provides an explanation of the dataset creation process, which involved multiple steps: downloading data from PubMed, preprocessing texts and images, developing a hierarchical taxonomy for image labels, and assigning labels to images—initially based on their captions, and later refined using machine learning models. The entire process can be replicated by running the cell from Figure 2 (for more details on the code, please refer to the multiversity folder from the GitHub repository).

3.2. Data Download

The IDs of the relevant case report articles were retrieved from PubMed using Biopython, a bioinformatics library that provides access to the NCBI Entrez API [36]. The query string was designed with the following criteria: (a) the “Publication Type” field had to be “case report,” or the “Title/Abstract” field needed to include specific terms such as “clinical case,” “case report,” or “case series"; (b) only full-text articles freely available were retrieved using the “ffrft” filter; (c) articles categorized as “animal” were excluded; and (d) only articles published after 1990—and before the search date (19 February 2025)—were considered. Although language was not explicitly filtered during the query process, non-English articles were excluded later due to the reliance on English-specific regex patterns for text processing.

Article metadata and abstracts were retrieved from PubMed’s website using the requests [37] and BeautifulSoup [38] packages. The BioC API for PMC was used to extract the remaining textual contents of the articles, including image captions and file names [39]. Figures were downloaded via the Europe PMC RESTful API [40]. The articles used to create this dataset are licensed under CC BY, CC BY-NC, CC BY-NC-SA, or CC0. Other licenses included in PMC Open Access Subset, such as no-derivatives licenses, were filtered out.

3.3. Data Preprocessing

Case report articles include information beyond the actual case presentations (such as introduction or conclusion sections), and may describe more than one patient. Therefore, it was necessary to identify the boundaries of each case. To achieve this, HTML tags, section headers, and mentions of patient ages—commonly found at the beginning of clinical cases—were considered. Additionally, text processing tasks included extracting and normalizing mentions of age and gender to populate demographic metadata fields, identifying figure mentions in text to map image files to their corresponding textual references, and splitting compound image captions into individual captions (e.g., converting ‘Chest X-rays (A) AP view and (B) lateral view’ into ‘Chest X-rays AP view’ and ‘Chest X-rays lateral view’). Demographic factors other than age and gender, such as ethnicity or socioeconomic status were not extracted because they are typically not explicitly mentioned in clinical cases.

Articles often include compound images, which are single files containing multiple image components. Using OpenCV [41], these figures were split into individual components, as illustrated in the example in Figure 3. This process involved the following steps: (1) edge detection, achieved by blurring the image with a Gaussian filter followed by applying a Canny edge detector; (2) limit detection, based on identifying rows and columns of pixels with a low density of edges; (3) image slicing, using the detected limits; and (4) order sorting, where image components were arranged in the original file’s layout from left to right and top to bottom (ensuring, for example, that the second image is the one directly to the right of the first). Borders present in the input images were removed throughout this procedure. The results from the image preprocessing were cross-checked with those from caption splitting (i.e., only the images that were split into the same number of pieces as their corresponding captions were included in the dataset).

3.4. Programmatic Image Labeling

The image labeling process comprised three main steps: taxonomy development, caption-to-label conversion, and machine learning-based annotation.

3.4.1. Taxonomy Development

The taxonomy of a classification dataset consists of its classes, their hierarchical relationships, and the logical constraints among them (such as mutual exclusivity). In this version of the dataset, the taxonomy was developed using the previous version as a foundation, considering the terms present in image captions, and referencing different domain ontologies accessed via BioPortal: SNOMED CT, MeSH, the Foundational Model of Anatomy (FMA) and RadLex [43,44,45,46,47]. The structure of the taxonomy follows the principles of the Multiplex Classification Framework, a methodology that enables complex classification problems to be addressed as a set of simpler binary or multiclass tasks, organized either in parallel or sequentially [48]. The full taxonomy was published in Bioportal as MCR_TX, and includes 163 classes (including the root_class, 17 negative classes such as no_doppler, and 145 positive classes) [49]. A reduced version with 81 classes (80 positive ones and the root_class) was developed by excluding the least important ones in order to simplify the training of machine learning (ML) models (see the ML_MCR_TX taxonomy in BioPortal) [50]. Figure 4 displays multiple classes from the MCR_TX taxonomy, each accompanied by an example image. In the MultiCaRe dataset, an image can have multiple associated labels, provided that the logical constraints among them are respected (for more details, refer to the taxonomy documentation available in the GitHub repository).

3.4.2. Caption-to-Label Conversion

At this step of the process, each image was assigned labels based on the content of its caption. To do this, first a list was created including all the unique sequences of words or tokens (n-grams) present in captions. Sequences longer than five tokens were excluded (no important class was lost in the process because relevant long n-grams generally consist of a combination of multiple shorter n-grams). The list was manually annotated using a spreadsheet, assigning corresponding labels to each n-gram based on the dataset taxonomy. The annotated spreadsheet was then used to assign labels to each caption based on the n-grams it contained (e.g., the label ‘ct’ was assigned to captions containing the n-gram ‘computed tomography’). When captions included overlapping n-grams (e.g., ‘right’ and ‘right lobe’), only the longest n-gram was considered. Polysemic n-grams, such as ‘left’ (which could indicate laterality or the past tense of ‘to leave’), were annotated disregarding such polysemy, and then the n-grams associated with irrelevant meanings (e.g., ‘nothing left’) were tagged with a dummy label, which was removed during post-processing to prevent the assignment of incorrect labels. Finally, any pairs of incompatible labels assigned to the same caption (e.g., ‘mri’ and ‘medical_photograph’) were also removed during post-processing. The outcome labels are included in the gt_labels_for_semisupervised_classification column of captions_and_labels.csv, except for those incompatible with machine learning predictions, which were filtered out.

This type of annotation is effective for image captions because they rarely include negated or hypothetical terms (unlike other medical texts), and therefore there is no need for additional resources like negation detection or assertion status models. The main advantage of this approach is its efficiency, as it requires fewer working hours to annotate larger datasets. However, it also has limitations. First of all, it demands annotators with significant expertise both in data annotation and medical text handling. Second of all, for negative classes (such as ‘no_doppler’), manual annotations are generally required because captions do not typically mention absent facts. Finally, this approach tends to produce a high number of false negatives because sometimes captions do not explicitly reference the image classes (for example, not all CT scan images contain n-grams related to the ‘ct’ class in their captions). This limitation can be addressed using ML-based annotations (as detailed in the following section).

3.4.3. Machine Learning-Based Annotation

After converting captions into image labels, ML was used to improve label quality for the two most important sets of classes: (a) main image types, such as ‘radiology’ or ‘endoscopy’; and (b) radiology subclasses, such as ‘ultrasound’ or ‘mri’. This process consisted of three steps: (1) image classification models were trained on pre-labeled images; (2) cases with the highest loss values were manually reviewed and analyzed to establish a quality threshold; (3) labels with loss values exceeding this threshold were replaced with the model’s predictions. Following this initial quality improvement, thirteen ML models were trained on the entire image dataset for all classes within the reduced taxonomy to predict missing labels and complete the dataset (details of model training and metrics of each model can be found in MultiCaReClassifier folder from the GitHub repository). Finally, the quality improvement process was reapplied, yielding more robust labels and the final set of thirteen image classification models. The models, along with the code for model ensembling and data post-processing, have been published on HuggingFace [96]. The predictions of these models have been used to populate the supervised learning columns from captions_and_labels.csv (ml_labels_for_supervised_classification and all the multiclass columns, such as image_type). The distribution of image types, along with the subtypes of the main class (’radiology’), is shown in Figure 5. Additional information—such as the number of images per class, average image height, average image width, and average file size—is provided in the file image_data_per_class.csv available in the GitHub repository.

All the models mentioned in this work used the ResNet50 architecture, with the implementation provided by the Torchvision library [97,98]. An intensive data augmentation pipeline was applied to prevent overfitting, consisting of the HorizontalFlip, VerticalFlip, Rotate, Sharpen, ColorJitter, RGBShift, GaussianBlur, GaussNoise and RandomSizedCrop methods from the Albumentations library [99]. Oversampling was also used, meaning that the probability of sampling each image depends on the relative frequency of its class (images from underrepresented classes are more likely to be sampled). Models were trained using an image size of 224, an 80/20 split for training and validation data, a batch size of 16, and a learning rate of 0.0003. The cross entropy loss function with label smoothing was employed with the Adam optimizer over 30 epochs. The training process utilized the ‘fit_one_cycle’ method from the FastAI Python library [100]. The performance of the models was evaluated using accuracy and F1 score as metrics. The confusion matrices of each model in the ensemble can be found in the MultiCaReClassifier folder from the GitHub repository. However, it’s important to note that these are not traditional confusion matrices, as ML-generated labels are compared to caption-based labels (which are incomplete and somewhat inaccurate), instead of manually annotated ground truth labels.

3.5. Data Validation and Quality Control

Data quality assessment was conducted at three levels: text quality, image quality, and label quality. The first two aspects were evaluated in the previous version of the dataset, whereas the assessment of image label quality was introduced in this version [26].

For textual data, 100 case reports were reviewed, with 84% found to have no problems. The identified problems were: (a) incorrect article type (3%); (b) errors in demographic extraction (4%); (c) extra splits (3%), which means that the text of one patient was split into multiple cases; (d) missing splits (3%), which means that multiple patients were combined into a single case; and (e) incorrect case content (3%), for example if part of the introduction of an article was included in the content of a case.

Regarding image quality, 153 images (corresponding to 100 article figures) were examined, and 86% were verified as correct. The problems found at this point were: (a) missing splits in compound images (4%), generally related to errors in the image caption; (b) incorrect image order (2%); (c) removal of relevant parts of the image (2%); and (e) incorrect patient assignment (1%). Although errors related to image splitting are relatively infrequent, if complete avoidance is necessary, users can filter out split images by selecting only those that contain the term ‘undivided’ in their names.

Finally, regarding label quality, two types of tests were performed by reviewing labels manually:

• Random subset review: Six random samples of 50 labels each were reviewed for quality assessment—three from the gt_labels_for_semisupervised_classification column and three from the ml_labels_for_supervised_classification column. In the first group, 96%, 90%, and 96% of the labels were correct, corresponding to a mean error rate of 6% with a standard deviation of 3.46%. In the second group, 70%, 78%, and 76% of the labels were correct, resulting in a mean error rate of 25.3% and a standard deviation of 4.16%.
• Stratified manual review: To account for class imbalance, another manual review was conducted on 200 caption-based labels and 200 ML-based labels. This time, the sampling ensured a minimum number of examples per class (at least one per class for caption-based labels and two per class for ML-based labels). The review found that 90% of the caption-based labels and 70% of the ML-based labels were correct. Notably, nearly half of the errors in the ML-based labels stemmed from incorrect parent class assignments—for instance, when a skin image is misclassified as ‘immunostaining’, the root cause is often an upstream misclassification under the broader ‘pathology’ class instead of ‘medical_photograph’.

Given this, we recommend using caption-based labels when higher label quality is needed, ideally with review by a clinical data annotator. ML-generated labels are preferable when a larger dataset is required, provided that the labels are also reviewed to ensure quality.

4. Usage Notes

4.1. Creation of Customized Subsets

As mentioned before, rather than accessing the whole dataset through Zenodo, we recommend creating subsets based on the specific use case using the MedicalDatasetCreator class from Python’s multiversity library. This can be accomplished through a few simple steps. First, the library needs to be installed in order to import and instantiate the MedicalDatasetCreator class (see Figure 6). At this step, which may take around five minutes, the MultiCaRe Dataset is imported from Zenodo into a folder called ‘whole_multicare_dataset’ in the main directory (‘/sample/directory/’ in this example).

The part of the whole dataset that will be included in the customized subset depends on the list of filters that are used. Some filters work at an article level (e.g., ‘min_year’ filters by the year of the case report article), others work at a patient level (e.g., ‘gender’) and some others work at an image level (e.g., ‘caption’). The contents of the subset also depend on the selected subset type (text, image, multimodal, and case series). For additional examples and details, please refer to the Demos folder from the GitHub repository. In all cases, the subsets include these three files:

• readme.txt: a short description of the subset, including its name, type, creation date, and the filters that were used during its creation.
• case_report_citations.json: file including all the metadata necessary to cite the case report articles, such as title, authors, and year.
• article_metadata.json: file including the rest of the metadata (license, keywords and mesh terms).

4.1.1. Text Subsets

This type of subsets contains textual data without annotations, which can be used for text labeling, language modeling or text mining. All the clinical cases are included in the cases.csv file, along with the patient’s age and gender. In the example from Figure 7, the created text subset includes cases after 2015 involving patients older than 18 years diagnosed that mention either ‘lung cancer’ or ‘lung carcinoma’. First the corresponding filters are defined, and then the subset is created by using the .create_dataset() method. In this case, the dataset_type parameter is set to ‘text’.

4.1.2. Image Classification Subsets

This subset type is intended for training image classification models. The data are organized into folders: one folder per image class (containing image files and the image_metadata.json file) and an additional folder with the image_classification.csv file.

• image_metadata.json: it includes the patient ID, file path, license, caption, and labels for each image.
• image_classification.csv: it contains one column with the file path, and another one with the corresponding class.

It is possible to create a subset of the captions_and_labels.csv file by using the .create_dataset() method with the dataset_type parameter set to ‘image’ (this works the same way as for ‘text’ subsets). However, users may sometimes need to create multiple customized subsets to train models that classify images into one of these subsets. For example, a user might want to create two subsets of head MRIs showing metastases—one for female patients and one for male patients—in order to train a gender classifier. This can be achieved using the .create_image_classification_dataset() method, which allows the application of a general filter list for all the subsets, as well as specific filter lists for each of them. The output is a CSV file with a ‘class’ column for the classes included in the dataset_dict (‘female’ and ‘male’ in the example below), along with the same label columns as captions_and_labels.csv if the keep_label_columns parameter is set to True. The filtered images will have exactly one label in the ‘class’ column (instances with multiple labels are excluded). In the example from Figure 8, the created subset is named ‘gender_classifier’, it includes head MRI images, and it has two possible classes based on the patient’s gender: male and female.

The .create_image_classification_dataset() method allows to create datasets for specific use cases based on the contents of image captions, which can be useful in cases where specific image classes are not present in the dataset taxonomy. If negative classes are needed, ‘none’ can be used as a logical operator to include in the negative subset any image whose caption does not contain relevant terms associated with the positive class. In the example from Figure 9, a thorax X-ray dataset is created with the classes ‘fracture’ and ‘no_fracture’ using this approach. It is important to note that any dataset containing negative classes defined by the absence of terms in captions should be manually reviewed. This helps prevent the negative subset from including samples of the positive class—which can occur if the image class is not explicitly mentioned in the caption.

4.1.3. Multimodal Subsets

This subset type is intended for the development of multimodal AI applications. It contains image files organized in folders, the same image_metadata.json file as in image classification subsets, and the same case.csv file as in text subsets, with one key difference: image references (e.g., “Figure 1”) in the clinical case text are replaced by either image tokens (e.g., <image_token>) or the file path to the corresponding image (see example in Figure 1).

In the example from Figure 10, the created multimodal subset includes patients diagnosed with lung cancer and with a head MRI showing the presence of metastases. First the corresponding filters are defined, and then the subset is created by using the .create_dataset() method, with ‘multimodal’ as the dataset_type.

4.2. Use of Image Classification Models

The model ensemble for medical image classification is available on HuggingFace [96]. These models can be used by following the steps from Figure 11.

4.3. Resources for Healthcare Professionals

All the resources mentioned so far are intended specially for data scientists and related roles. However, this release also includes resources specially intended for healthcare professionals and students.

• Case series subsets: If ‘case_series’ is used as the dataset_type for the .create_dataset() method, then a subset with multimodal patient data is created, with the following distinctions: (a) the image organization differs from other subset types, with each patient having their own folder, and (b) the cases.csv file is very similar to the one from text subsets, except for some additional columns (including link to the case report article, number of images per patient, and path to the corresponding image folder).
• Clinical Case Hub [101]: This specialized platform is designed for medical students and healthcare professionals to explore the dataset’s contents, including clinical case texts and medical images. The hub’s user-friendly interface enables users to apply filters without requiring programming knowledge.
• Digital Twin Retriever [102]: This AI-powered tool with a chatbot interface can be used to search clinical cases and answer questions about them, helping healthcare professionals make more informed decisions. The current version operates only on textual data.

4.4. Dataset Limitations

The following limitations should be considered when using this dataset:

• Differences from real-life data: As noted in Appendix B, data from case reports differs from real clinical data in several ways. The text is shorter, rare cases tend to be overrepresented, images are generally smaller and lower in resolution, and they may contain burned-in annotations such as arrows or asterisks.
• Label quality: This is a silver standard dataset, meaning that image labels were generated semi-automatically and may contain a relatively high number of errors. As a result, label quality is lower than in datasets annotated manually by domain experts.
• Data imbalance: There is substantial variation in both the number and resolution of images across classes. This imbalance reflects the nature of the original data sources.
• Data loss: Not all content from PMC case reports was included. Two major causes of data loss are: (a) exclusion of articles with non-derivative licenses, and (b) removal of images when the number of cropped subimages did not match the number of caption parts, as explained in Section 3.3.