Prior Knowledge Shapes Success When Large Language Models Are Fine-Tuned for Biomedical Term Normalization

Abstract

Large language models (LLMs) often fail to correctly associate biomedical terms with their standardized ontology identifiers, posing challenges for downstream applications that rely on accurate, machine-readable codes. These linking failures can compromise the integrity of data used in precision medicine, clinical decision support, and population health. Fine-tuning can partially remedy these issues, but the degree of improvement varies across terms and terminologies. Focusing on the Human Phenotype Ontology (HPO), we show that a model’s prior knowledge of term–identifier pairs, acquired during pre-training, strongly predicts whether fine-tuning will enhance its linking accuracy. We evaluate prior knowledge in three complementary ways: (1) latent probabilistic knowledge, revealed through stochastic prompting, captures hidden associations not evident in deterministic output; (2) partial subtoken knowledge, reflected in incomplete but non-random generation of identifier components; and (3) term familiarity, inferred from annotation frequencies in the biomedical literature, which serve as a proxy for training exposure. We then assess how these forms of prior knowledge influence the accuracy of deterministic identifier linking. Fine-tuning performance varies most for terms in what we call the reactive middle zone of the ontology—terms with intermediate levels of prior knowledge that are neither absent nor fully consolidated. Fine-tuning was most successful when prior knowledge as measured by partial subtoken knowledge, was ‘weak’ or ‘medium’ or when prior knowledge as measured by latent probabilistic knowledge was ‘unknown’ or ‘weak’ ($p<0.001$). These terms from the ‘reactive middle’ exhibited the largest gains or losses in accuracy during fine-tuning, suggesting that the success of knowledge injection critically depends on the level of term–identifier pair knowledge in the LLM before fine-tuning.

1. Introduction

Biomedical term normalization—the process of mapping natural language expressions to standardized ontology concepts with machine-readable identifiers—is a cornerstone of precision medicine and biomedical research. Accurate normalization enables clinical and scientific text to be aligned with structured knowledge resources, thereby supporting reproducible analyses and computational reasoning [1,2,3,4,5,6,7]. Beyond biomedicine, there is growing interest in leveraging ontologies, knowledge graphs, and large language models for knowledge management across a wide range of domains, including ergonomics, construction safety, and information systems [8,9,10,11,12,13,14,15].

Two widely adopted biomedical ontologies include SNOMED CT, which is maintained by SNOMED CT International and has more than 360,000 concepts used to document patient care, and the Gene Ontology (GO), which supports functional annotation in gene and protein research with more than 39,000 concepts [16,17]. The Human Phenotype Ontology (HPO) is a structured vocabulary for describing phenotypic abnormalities associated with human disease [7,18,19]. It plays an important role in elucidating genetic disease mechanisms with more than 18,000 terms used to create 268,000 annotations that link a phenotypic feature to a specific disease, enabling large-scale biomedical research into the genetic basis of human disease [20].

Successful use of the HPO for disease annotation relies on the accurate linking of each phenotype term to its corresponding ontology identifier [7]. For example, Charcot–Marie–Tooth disease type 2B2 is annotated with phenotypic features such as Distal muscle weakness (HP:0002460), Areflexia (HP:0001284), and Distal sensory impairment (HP:0002936). Each HPO term is expressed in title case and uniquely identified by a seven-digit code prefixed with HP. Precise alignment of the term–identifier pair is essential for ensuring semantic accuracy and interoperability, enabling downstream applications such as diagnostic decision support, cohort discovery, and automated biomedical data analysis [7,20].

Large language models (LLMs) are increasingly used in biomedical natural language processing (NLP) tasks such as entity recognition, relation extraction, and term normalization [21]. Despite their broad success, LLMs often struggle with the seemingly straightforward task of biomedical term normalization [22]. Even state-of-the-art models frequently fail to link Human Phenotype Ontology (HPO) terms to their correct identifiers. A core limitation lies in the pre-training paradigm: autoregressive LLMs such as GPT-4 and LLaMA 3.1 are optimized for next-token prediction, not explicit fact memorization. While these models are exposed to vast biomedical corpora during pre-training, many rare biomedical concepts remain sparsely represented. As a result, model performance drops sharply across the long tail of biomedical vocabularies, where domain-specific terms are underexposed [23]. For instance, when GPT-4 was queried for identifiers corresponding to 18,880 HPO terms, it returned the correct identifier for only 8% [22].

Fine-tuning offers a potential remedy. Targeted adaptation using parameter-efficient methods such as LoRA has shown promise for injecting missing knowledge into large language models (LLMs) [24,25,26,27,28]. However, recent evaluations caution that these gains are often inconsistent: smaller models struggle to generalize reliably, and improvements in newly injected content may come at the cost of degrading existing knowledge—particularly when that knowledge is fragile or only weakly anchored [24,29,30,31,32,33,34].

A growing body of work suggests that while fine-tuning enables models to memorize new term–identifier mappings, it instills a limited ability to generalize from these mappings to unseen expressions [32,33]. Rather than promoting conceptual integration, fine-tuning may act as a form of rote injection, reinforcing isolated facts without building robust representations. Consequently, the success of fine-tuning appears to depend not only on the added data but also on how well the target concept is already embedded in the model’s pre-training knowledge [33,35]. The jury is still out on whether certain biomedical ontologies, by virtue of their innate structure or prevalence of their concepts in training data, may support broader conceptual generalization during fine-tuning than other less robust ontologies.

This raises a core question: Which biomedical terms are most likely to benefit from fine-tuning, and which are most vulnerable to degradation? We hypothesize that both improvement and degradation are systematically shaped by the model’s prior knowledge of each term–identifier pair.

To test this hypothesis, we evaluate the predictive value of three dimensions of prior knowledge, defined as the model’s ability to produce or approximate correct identifier mappings before fine-tuning:

• Latent probabilistic knowledge. Hidden or partially accessible knowledge revealed through probabilistic querying of the model, even when deterministic (greedy) decoding fails [36]. For example, if an LLM is queried 100 times and returns the correct ontology ID only 5% of the time, this indicates latent probabilistic knowledge, distinct from not known at all.
• Partial subtoken knowledge. Incomplete but non-random knowledge of the subtoken sequences comprising ontology identifiers, reflected in deterministic outputs that are close to, but not exactly, correct. For example, an LLM that predicts HP:0001259 for Ataxia instead of the correct HP:0001251 demonstrates partial subtoken knowledge, even though greedy decoding produces an incorrect response.
• Term familiarity. The likely exposure of the model to specific term–identifier pairs during pre-training, estimated using external proxies such as annotation frequency in OMIM and Orphanet [37,38], and identifier frequency in the PubMed Central (PMC) corpus [39]. For example, the LLM is more likely to be familiar with the term Hypotonia (decreased tone), which has 1783 disease annotations in HPO, than with Mydriasis (small pupils), which has only 25 annotations.

Across these three dimensions, latent probabilistic knowledge, partial subtoken knowledge, and term familiarity, we uncover a striking pattern: terms with intermediate levels of prior knowledge, neither fully consolidated nor entirely absent [36], are the most responsive to fine-tuning. These reactive middle terms exhibit both the largest improvements and the greatest degradations, whereas terms at the extremes remain more stable and less affected by fine-tuning. This dual effect suggests that fine-tuning is both most effective and most disruptive in regions of moderate prior knowledge. Our findings extend prior work by Pletenev et al. [33] and Gekhman et al. [35,36] and underscore the importance of considering term susceptibility to fine-tuning when attempting knowledge injection.

This raises a core question: Which biomedical terms are most likely to benefit from fine-tuning, and which are most vulnerable to degradation? We hypothesize that both improvement and degradation are systematically shaped by the model’s prior knowledge of each term–identifier pair.

The remainder of this paper is organized as follows: Section 2 describes the materials and methods, including the experimental setup, probabilistic querying protocol, subtoken analysis, and familiarity scoring. Section 3 presents the results of our evaluation of fine-tuning performance and the predictive value of the three dimensions of prior knowledge. Section 4 discusses the implications of these findings, including how latent knowledge and term amenability shape fine-tuning outcomes, and outlines directions for future research. Section 5 concludes with practical recommendations for designing effective knowledge injection strategies for biomedical term normalization.

2. Materials and Methods

2.1. Workflow

Figure 1 summarizes the overall workflow. A total of 1618 de-identified clinical notes were mapped to the Human Phenotype Ontology (HPO, 18,988 terms), from which we curated a test set of 799 ground-truth term–identifier pairs. From this set, we derived five measures of prior knowledge: (1) annotation counts from curated HPO disease annotations, (2) PMC identifier counts obtained via the PubMed Central API, (3) latent probabilistic knowledge estimated from LLaMA 3.1 8B (Base model, Temp = 1.0, top-k = 50), (4) partial subtoken knowledge from deterministic queries (Temp = 0.0, top-k = 1), and (5) fine-tuning outcome classes (Correct, Incorrect, Gainer, Loser) from the fine-tuned LLaMA 3.1 8B model. These measures were then integrated into unified statistical analyses (ANOVA, t-tests, chi-square) and compared across groups.

2.2. HPO Dataset

We obtained the Human Phenotype Ontology (HPO) in CSV and OBO formats from BioPortal (https://bioportal.bioontology.org/ontologies/HP, 1 August 2025) and the OBO Foundry (http://purl.obolibrary.org/obo/hp.obo, 1 August 2025), respectively. As of 5 May 2025, the ontology comprised 23,065 classes (18,988 were active phenotype terms used for disease annotations). Each entry includes the standardized term label, unique HPO identifier, hierarchical parent–child relationships, synonyms, and definitions.

Phenotype–disease associations were obtained from the HPO annotation file (https://hpo.jax.org/data/annotations, 1 August 2025), containing 272,061 annotations that span 12,691 diseases. Most annotations are derived from OMIM and Orphanet. On average, each disease has 21 phenotype annotations.

2.3. Test Terms

To evaluate model performance on samples of realistic clinical language, we curated a test set of 799 HPO term–identifier pairs extracted from 1618 de-identified neurology physician notes from the University of Illinois electronic health record (EHR) system [40]. Use of these notes was approved by the Institutional Review Board (IRB) of the University of Illinois at Chicago. The clinical term extraction process used a natural language processing pipeline described in [40], which employed GPT-4, prompting strategies to identify 2718 candidate phrases that corresponded to HPO concepts. The prompts were designed to capture both explicit phenotype mentions and implicit clinical descriptions. After filtering and mapping, 799 unique HPO term–identifier pairs formed the test set for model evaluation.

2.4. Large Language Models and Fine-Tuning

We used LLaMA 3.1 8B as our base model, a transformer-based autoregressive language model with 8 billion parameters, pre-trained on a diverse mixture of web data, books, code, and scientific text. Comprehensive details of the model architecture and pre-training are available in [41], and the foundational transformer architecture is described in [42]. All experiments employed consistent software versions and configuration parameters to ensure reproducibility. Fine-tuning was performed locally using Hugging Face Transformers with Unsloth and Low-Rank Adaptation (LoRA) [27,43], a parameter-efficient method that inserts rank-decomposed matrices into attention layers without altering base weights. While LoRA reduces the number of trainable parameters and can preserve prior capabilities, catastrophic forgetting can still occur [44,45].

Training was conducted on a multi-GPU workstation equipped with three NVIDIA Quadro RTX 8000 GPUs (48 GB VRAM each), running CUDA 12.0 with mixed precision (fp16) for optimized memory usage. Each fine-tuning run required more than 12 hours of compute time, reflecting the scale of the HPO vocabulary. The model was fine-tuned on the full HPO vocabulary (18,988 terms) using five prompt variations per term–identifier pair. Following Unsloth’s recommendations, fine-tuning was limited to three epochs to mitigate overfitting.

2.5. Top-1 Accuracy (Deterministic Inference)

To evaluate baseline model performance, we measured top-1 accuracy using deterministic inference. For each HPO term, we prompted the model to return the corresponding HPO identifier in a strict format:

What is the HPO ID for {term}?

Return only the code in format HP:1234567

A response was scored as Correct if the predicted identifier exactly matched the ground-truth HPO ID. All other responses were considered Incorrect. This approach corresponds to the model’s most confident prediction, with no sampling or temperature variation. We refer to this strict evaluation metric as top-1 accuracy, in contrast to accuracy under stochastic inference (e.g., temperature-based sampling), which we report separately.

2.6. Outcome Classification for Fine-Tuning

We hypothesized that a term’s prior knowledge state would systematically influence its response to fine-tuning. To test this, we classified each of the 799 test terms based on deterministic accuracy before and after fine-tuning:

Correct: Correct both before and after fine-tuning;

Gainer: Incorrect before but correct after fine-tuning;

Loser: Correct before but incorrect after fine-tuning;

Incorrect: Incorrect both before and after fine-tuning.

This classification scheme provided the four outcome groups (Correct, Incorrect, Gainer, and Loser) used in subsequent analyses.

2.7. Latent Probabilistic Knowledge

Latent probabilistic knowledge was quantified following the method of Gekhman et al. [36]. The base model was prompted 50 times for each term at a temperature of 1.0, enabling diverse probabilistic outputs. Probabilistic accuracy (proportion of correct responses) was computed for each term. Model latent probabilistic knowledge of each term was categorized as follows:

Unknown: probabilistic accuracy = $0.0$;

Weak: $0.0<$ accuracy $<0.2$;

Medium: $0.2\le$ accuracy $<0.5$;

Strong: accuracy $\ge 0.5$.

Figure 2 shows the distribution of probabilistic accuracy (log-scaled y-axis).

2.8. Partial Subtoken Knowledge

Partial subtoken knowledge was the model’s knowledge of each of the three HPO identifier subtokens during deterministic sampling. The LLaMA 3.1 tokenizer extracts three subtokens from the seven digits of the HPO identifier (e.g., HP:1234567), which are tokenized into three numeric subtokens in the format 123-456-7. Deterministic model outputs were scored from 0–3 based on the number of correctly predicted numeric subtokens. The four categories of partial subtoken knowledge were as follows:

None: 0 of 3 correct;

Weak: 1 of 3 correct;

Medium: 2 of 3 correct;

Complete: 3 of 3 correct.

Figure 3 shows the counts of terms in each of the partial subtoken knowledge categories.

2.9. Term Familiarity

We hypothesized that a model’s familiarity with term–identifier pairs is related to its exposure during pre-training. We therefore used two direct surrogates for term familiarity. The first was the number of times each HPO identifier appeared in the PubMed Central (PMC) full-text database, as accessed via the PMC API. The second was the number of times each HPO term was used to annotate the phenotype of a disease in the curated dataset available at https://hpo.jax.org/data/annotations, 1 August 2025. Histograms of annotation counts and PMC identifier counts are shown in Figure 4. Both distributions follow a long-tail pattern, with many terms having very few annotations or PMC identifiers. The 799-term test set is shifted to the right relative to the full HPO, indicating that the test set is enriched for more commonly used terms. These familiarity measures were then used to compare outcome groups (Correct, Incorrect, Gainers, and Losers) to test the hypothesis that fine-tuning effects are concentrated in a reactive middle zone.

2.10. Statistical Analysis

We compared deterministic accuracy before and after fine-tuning within each latent probabilistic knowledge category (Unknown, Weak, Medium, and Strong) using paired t-tests to assess significance and Cohen’s d to measure effect sizes. Bar plots with standard errors visualized group differences, and chi-square tests measured the proportion of terms corrected by fine-tuning relative to baseline. The same approach was applied to the four partial subtoken knowledge groups (None, Weak, Medium, and Complete). The significance of the differences in gainers and losers across latent probabilistic knowledge and partial subtoken knowledge categories was also assessed with chi-square tests. Finally, to evaluate how term familiarity is related to fine-tuning outcomes, we compared annotation counts and PMC identifier counts across the four outcome groups (Correct, Incorrect, Gainers, and Losers). Because both predictors were long-tailed, counts were log-transformed prior to analysis. Group differences were evaluated using one-way ANOVA with Tukey’s HSD post hoc comparisons. As a robustness check against non-normality, we also conducted Kruskal–Wallis tests on the raw values. Summary statistics (mean, standard deviation, median, and interquartile range) were reported for each group.

3. Results

3.1. Baseline and Fine-Tuned Performance of LLaMA 3.1 8B on HPO Term Normalization

The baseline LLaMA 3.1 8B model demonstrated significantly higher accuracy on the curated test set of 799 clinically relevant HPO terms compared to the full HPO vocabulary. When evaluated on all 18,988 HPO terms, the model correctly linked only 96 terms (0.5%), whereas on the curated test set, it correctly linked 32 terms (4.0%). A chi-square test confirmed that this difference was significant (${\chi }^2=140.7,p<1\times {10}^{-30}$). This suggests that the curated test terms are more common clinical terms that the model was more likely to have encountered during pre-training, making them easier to normalize than the broader and rarer long-tail terms in the complete HPO.

Fine-tuning improved model performance on the curated set of 799 HPO terms. The baseline model correctly linked 32 terms (4.0%), whereas the fine-tuned model correctly linked 118 terms (14.8%). This represents a more than threefold improvement in deterministic accuracy. A chi-square test confirmed that the increase in correct mappings after fine-tuning was significant (${\chi }^2=58.9,p<1\times {10}^{-14}$).

3.2. Latent Probabilistic Knowledge Predicts Fine-Tuning Success

Latent probabilistic knowledge, as adapted from Gekhman et al. [36], was quantified by computing the proportion of correct responses across 50 probabilistic queries (temperature = 1.0). Each term was classified into one of four categories: Unknown, Weak, Medium, or Strong (Figure 2).

Figure 5 shows the mean deterministic accuracy of both the base and fine-tuned models for each latent probabilistic knowledge category, with standard error bars and significance markers from two-sample t-tests. Fine-tuning significantly improved accuracy for terms with Weak latent probabilistic knowledge ($p<0.001$) and modestly for those classified as Unknown ($p<0.001$), though absolute accuracy for Unknown terms remained low. For terms with Medium latent probabilistic knowledge, accuracy improvements did not reach statistical significance, and for Strong latent probabilistic knowledge, both models already performed near ceiling levels with no measurable gain.

These results suggest that fine-tuning is most beneficial for terms where the model possesses partial but incomplete latent probabilistic knowledge, the hypothesized reactive middle. Gains are limited for completely unknown terms and negligible for strongly represented terms.

3.3. Partial Subtoken Knowledge Predicts Fine-Tuning Success

The LLaMA 3.1 tokenizer divides each 7-digit HPO identifier into three numeric subtokens in the format “123–456–7.” We assessed partial knowledge by comparing the model’s predicted identifier against the ground-truth identifier at the subtoken level. Based on the number of correctly predicted numeric subtokens (0–3), each term was assigned to one of four partial subtoken knowledge categories: None, Weak, Medium, or Complete. Fine-tuning significantly improved deterministic accuracy in the Weak and Medium categories ($p<0.001$) but showed no improvement in the None category, where both models performed poorly. In the Complete category, the base model slightly outperformed the fine-tuned model ($p<0.05$), indicating mild degradation of already well-consolidated knowledge. These results suggest that fine-tuning is most effective when partial knowledge is present but not yet fully established (Figure 6).

3.4. Term Familiarity

We used annotation counts (uses of phenotype terms for disease annotation) and PubMed Central (PMC) identifier counts as proxies for prior model exposure to HPO term–identifier pairs during pre-training. Annotations and PMC ID counts were highest for Correct terms (terms correctly linked to their identifiers before and after fine-tuning) and lowest for Incorrect terms (terms consistently mislinked before and after fine-tuning). Gainer terms (incorrect before but corrected after fine-tuning) and Loser terms (correct before but incorrect after fine-tuning) occupied intermediate positions, consistent with a ‘reactive middle’ (Figure 7).

Statistical analysis confirmed these patterns. For annotation counts, ANOVA on the log-transformed scale showed a highly significant effect of outcome class ($F(3,686)=88.1$, $p<{10}^{-47}$), with Tukey’s HSD indicating that both Gainers and Losers had significantly higher counts than Incorrect terms but did not differ significantly from Correct terms. A non-parametric Kruskal–Wallis test also supported group differences ($H=186.8$, $p<{10}^{-39}$). For PMC identifier counts, results were similar ($F(3,686)=87.6$, $p<{10}^{-47}$; Kruskal–Wallis $H=150.2$, $p<{10}^{-31}$). Tukey’s HSD indicated that Correct terms had significantly higher identifier counts than both Gainers and Incorrect terms, while Losers again showed intermediate values not significantly different from either extreme.

3.5. Positive and Negative Knowledge Flows During Fine-Tuning

Pletenev et al. [33] emphasized that fine-tuning can introduce new knowledge while also degrading previously consolidated knowledge. To characterize these flows, we classified terms as Gainers (incorrect before but correct after fine-tuning) or Losers (correct before but incorrect after). Terms that remained consistently correct or incorrect are included in the figures for context but are not the focus of this analysis.

Figure 8 shows outcomes stratified by latent probabilistic knowledge. Gains were most common among terms in the Unknown and Weak bins, with progressively fewer gains in the Medium and Strong bins. Losses were less frequent overall but appeared across bins, including in the Strong category, indicating that even well-represented terms may degrade under fine-tuning. The association between latent probabilistic knowledge and outcome class was highly significant (${\chi }^2\left(9\right)=667.1$, $p<{10}^{-137}$).

Figure 9 shows outcomes stratified by partial subtoken knowledge. Here, gains were concentrated in the Weak and Medium bins, while losses occurred most often in the Complete bin. This suggests that while partial structural knowledge of identifiers makes terms more likely to improve with fine-tuning, even fully correct identifier encodings are susceptible to disruption. The association between partial subtoken knowledge and outcome class was also highly significant (${\chi }^2\left(9\right)=868.9$, $p<{10}^{-180}$).

Together, these results support a reactive middle interpretation: fine-tuning has its largest positive impact on terms with intermediate prior knowledge, while both very weak and very strong prior knowledge states exhibit resistance to change. However, the presence of Losers in the Complete and Strong categories underscores the risk of negative knowledge transfer during fine-tuning.

4. Discussion

Our findings demonstrate that fine-tuning improves term-to-identifier linking in biomedical ontologies in a systematic rather than random manner, with outcomes strongly influenced by the model’s prior knowledge of each term–identifier pair. We evaluated three dimensions of prior knowledge—latent probabilistic knowledge, partial subtoken knowledge, and term familiarity—and found each to be predictive of fine-tuning outcomes. Latent probabilistic knowledge refers to hidden knowledge accessible under stochastic sampling; partial subtoken knowledge reflects partial mastery of an identifier’s numeric structure; and term familiarity captures prior exposure in pre-training, estimated from annotation counts and PubMed Central identifier frequencies.

A consistent pattern emerged: terms with intermediate levels of prior knowledge were the most responsive to fine-tuning. This reactive middle showed both the largest gains and the most losses in deterministic accuracy, while terms that were either completely unknown or highly consolidated changed little. Specifically, gains clustered in intermediate categories of latent probabilistic and subtoken knowledge, as well as terms with moderate annotation and identifier counts. In contrast, terms with high familiarity tended to remain stable, and terms with no prior signal showed limited improvement.

These observations align with the frameworks proposed by Gekhman et al. [36] and Pletenev et al. [33], which emphasize that fine-tuning can reinforce weakly consolidated representations while also destabilizing existing ones. Our results extend this theory by quantifying how gains and losses are distributed across distinct forms of prior knowledge, showing that both are concentrated where prior knowledge is partial.

We also find partial alignment with Wang et al. [46], who reported near-perfect knowledge injection when fine-tuning LLaMA 2 (7B) for 99 epochs using LoRA. In contrast, our shorter three-epoch training of LLaMA 3.1 (8B) produced modest gains alongside degradations of previously known terms. This contrast highlights how training depth and capacity influence whether fine-tuning consolidates knowledge or risks destabilization.

4.1. Limitations

This study has several limitations. We evaluated only one model architecture (LLaMA 3.1 8B) on a single ontology (HPO), and our test set of 799 terms represents only a subset of the 18,988-term vocabulary. Findings may not generalize to larger models, domain-specific pre-training, or other ontologies such as GO, SNOMED CT, or RxNorm. In addition, because all terms were included in training but only a subset was evaluated, training–test mismatch may have influenced results. Finally, our fine-tuning setup was restricted to three epochs with five prompt variants per term. Alternative designs—including broader prompt diversity, longer training, or larger context windows—may yield different outcomes. Ablation studies could further clarify which factors most strongly influence performance.

We also did not address whether parameter-efficient fine-tuning (PEFT) methods such as LoRA enable genuine generalization or primarily reinforce memorized mappings [31]. Our findings suggest the latter: fine-tuning largely strengthens partially known associations but does not necessarily promote deeper semantic abstraction beyond the training data.

4.2. Future Work

A key next step is to test whether these patterns of fine-tuning responsiveness generalize to other biomedical ontologies, larger models, and alternative training regimes. Systematic comparisons across fine-tuning methods—including QLoRA [47], Direct Preference Optimization (DPO), and full-model fine-tuning—are needed to assess their differential effects on knowledge injection and retention.

Future studies should also examine how temperature settings affect estimates of latent probabilistic knowledge, potentially refining predictive models of fine-tuning responsiveness. A particularly promising direction is targeted fine-tuning focused on the reactive middle—terms with partial prior knowledge—where gains are most likely and risks of loss are present. Finally, investigating the velocity of fine-tuning, i.e., how rapidly different terms adapt during training, may reveal predictors of responsiveness and inform more efficient training strategies.

5. Conclusions

Fine-tuning effectiveness for term-to-identifier linking in biomedical ontologies is governed by a model’s prior knowledge rather than being uniform across terms. Across three dimensions of prior knowledge—latent probabilistic knowledge, partial subtoken knowledge, and term familiarity—we found consistent evidence for a reactive middle: terms with intermediate prior knowledge undergo the most substantial changes, showing both the largest gains and the greatest degradations. In contrast, well-consolidated and completely unknown terms remain relatively unaffected.

These results suggest practical opportunities to improve how large language models are deployed for biomedical term normalization, and they may also provide insight into how such models internalize mappings between terms and identifiers.

More broadly, our findings challenge the assumption that fine-tuning is universally beneficial. By explicitly modeling prior knowledge, we can anticipate which terms are most susceptible to gains and losses, enabling fine-tuning strategies that maximize benefits while minimizing degradation of previously accurate knowledge.