Evaluating ChatGPT’s Effectiveness in Generating Pharmacogenomic Prescribing Guidelines

Abstract

Background/Objectives: In precision medicine, patient genetic data is increasingly being used to inform physicians’ prescribing of medication, yet current resources to access gene-drug interaction data are often difficult and unintuitive to use. This study compared the efficacy of ChatGPT and PharmGKB^® to extract and generate drug pharmacogenomic prescribing guidelines based on an individual patient’s genetic profile and medication use. Methods: Drug and gene data for 24 patients were input into ChatGPT with a standardized prompt. The recommendations generated were compared to guidelines generated from PharmGKB^®, a curated pharmacogenomic knowledge resource that integrates CPIC^®, DPWG, and FDA guidance. Evaluation metrics included time to obtain guidelines, the accuracy of the recommendations, and the types of errors made by ChatGPT. Results: ChatGPT output was concordant with PharmGKB^® for 148/204 medications (73%), but 124 of these agreements were cases in which neither source identified a guideline. Where a PharmGKB^® guideline existed, ChatGPT reproduced it for only 24/53 medications (45%). Errors included omitted medications (n = 41), unsupported recommendations (n = 9), incorrect dosing (n = 4), and missed guidelines (n = 2); ChatGPT also hallucinated 28 medications not on patients’ lists. PharmGKB^® averaged 6 min 20.8 s for each patient (SD = 3.5 min), versus 2 min 24.7 s (SD = 32 s) for ChatGPT (p < 0.001). Conclusions: While ChatGPT offers speed and ease of use, its clinical reliability remains limited due to frequent and potentially serious errors. Practical and effective utility of pharmacogenomics is imperative to advancing the use of pharmacogenomics in precision medicine, and more research must be done to elucidate the best tools available for physicians.

1. Introduction

Pharmacogenomics enables individualized prescribing of medications based on a patient’s unique genetic profile, with the goal of optimizing drug efficacy and reducing adverse drug reactions (ADRs). Individual variation in hepatic cytochrome P450 metabolic activity can influence the pharmacokinetics and pharmacodynamics of many commonly prescribed medications [1,2].

Despite numerous evidence-based guidelines for the use of personalized pharmacogenomic information for medication prescribing, the ability of practicing physicians to synthesize and utilize the information remains difficult. The complexity of genotype interpretation, the need to synthesize multiple gene–drug interactions, and the time required to access and apply guideline recommendations all represent barriers for practicing clinicians.

Tools designed to improve the clinical usability of pharmacogenomic data, including PharmGKB^® and PharmCAT, are available but remain time-consuming and cumbersome. PharmGKB^® provides an intuitive interface that incorporates Clinical Pharmacogenetics Implementation Consortium (CPIC^®) guidance, allowing clinicians to enter genomic data and review potential drug–gene interactions for many medications. Although this approach is more accessible than consulting individual CPIC^® guidelines, the usability remains complex, time-consuming, and insufficiently streamlined for routine practitioner use [3].

Artificial intelligence (AI) has the ability to support clinical decision-making [4,5] with models, like random forest and [6] deep learning, to predict drug efficacy and prevent ADR’s [7]. ChatGPT is a widely used deep learning model for both clinicians and the general public. However, concerns remain regarding its efficacy, accuracy, reliability, and tendency to generate incorrect or fabricated outputs (“hallucinations”) [8].

To date, no studies have directly evaluated the ability of ChatGPT to extrapolate pharmacogenetic guideline information from those derived from PharmGKB^®. This database integrates CPIC^®, Dutch Pharmacogenetics Working Group (DPWG), and U.S. Food and Drug Administration (FDA) guidance and is widely regarded as a widely used, expert-curated resource for clinical pharmacogenomics. This study aimed to compare ChatGPT with PharmGKB^® in their ability and efficiency to generate pharmacogenomic recommendations for a sample of geriatric emergency department (ED) patients who had presented with a fall injury.

2. Materials and Methods

2.1. Study Design

We conducted a prospective observational study evaluating the performance of ChatGPT using demographic, clinical, and pharmacogenomic data from a convenience sample of older adults presenting to the ED with a fall-related injury. The study protocol was approved by the Institutional Review Board of Florida Atlantic University.

2.2. Participants and Setting

Participants were drawn from an ongoing prospective study of pharmacogenomics in older adults with falls (the GREAT PHARM study); the first 24 participants with genotype results available at the time of analysis were included in the current study. The study was conducted at a Level 1 trauma center in southeast Florida with an annual ED census of approximately 50,000 visits. Between July 2025 and July 2026, a convenience sample of patients was screened by trained research assistants. Inclusion criteria were: (1) age ≥ 65 years; (2) ED presentation for a fall-related injury; (3) ability to provide written informed consent; and (4) willingness to participate in the study. Exclusion criteria were: (1) refusal to provide a buccal sample, (2) prior enrollment, (3) critical illness preventing consent, and (4) hospice care.

2.3. Data Collection

Clinical data were collected at the patient’s initial ED presentation. Demographic information and medication history were obtained through structured chart review and patient interview.

A trained research assistant collected a single buccal swab from each participant in the ED, using the DNALabs™ collection kit according to the manufacturer’s instructions. Samples were labeled only with a study identifier, stored at room temperature, and shipped within 7 days to DNALabs™ (Toronto, ON, Canada), an ISO 17025-accredited laboratory. The DNALabs™ panel genotypes 26 pharmacogenes involved in the metabolism of more than 130 commonly used medications. For each gene, results were reported as a star-allele diplotype with the laboratory’s predicted phenotype. Median turnaround was 30 days.

Of the 26 genes on the DNALabs™ panel, 11 are among the 21 pharmacogenes supported by PharmGKB^®’s clinical guideline annotation tool: CYP2B6, CYP2C19, CYP2C9, CYP2D6, CYP3A4, CYP3A5, DPYD, NUDT15, SLCO1B1, TPMT, and VKORC1. The remaining 15 DNALabs™ genes have no guideline-level annotation in PharmGKB^®. The DNALabs™ panel does not report the other 10 PharmGKB^® pharmacogenes. Genotypes for all 26 genes were supplied to ChatGPT.

2.4. Pharmacogenomic Guideline Extraction

Following receipt of genotype data, pharmacogenomic prescribing recommendations were independently generated using PharmGKB^® and ChatGPT-4o.

2.4.1. PharmGKB^®

PharmGKB^® was used as the reference standard because it is an expert-curated pharmacogenomic knowledge resource that integrates and annotates CPIC^® and DPWG guidelines and FDA drug-label pharmacogenomic information, providing genotype-specific prescribing recommendations. Its freely available, interactive interface also reflects a resource a clinician could realistically use at the point of care.

After completion of data collection, PharmGKB^® merged with two other pharmacogenomic resources to become ClinPGX. These three entities were separate websites that merged into one that continues to provide PharmGKB^® recommendations with additional available tools. Each patient’s diplotypes for the 11 overlapping genes were entered into the PharmGKB^® interface using the dropdown menu for each gene. PharmGKB^®-generated pharmacogenomic recommendations were recorded for all relevant medications identified for each patient. Only recommendations explicitly available within PharmGKB^® at the time of analysis were considered valid reference guidelines.

2.4.2. ChatGPT

ChatGPT (GPT-4o) was used to generate pharmacogenomic prescribing recommendations using a standardized prompt (see Table 1).

Table 1. The standardized prompt given to ChatGPT to elicit guidelines.

The uploaded Excel file contains a single patient’s pharmacogenomic profile in the following format: Row 1: Patient identifier (e.g., “Patient 3”) Row 2: Gene names (e.g., CYP2C19, CYP2D6) Row 3: Patient’s genotype for each gene (e.g., *1/*2, *41/*5) Row 4: A comma-separated list of medications the patient is currently taking (all within one cell)

Task: Using PharmCAT, DPWG, FDA, and CPIC® guidelines, create a structured sum-mary table with the following columns: Drug Name Gene(s) Affecting the Drug Patient’s Genotype(s) for Each Gene Full CPIC®/PharmCAT/DPWG/FDA/PharmGKB® Recommendation Text for Each Genotype

Requirements: Include the full CPIC®/PharmCAT/DPWG/FDA/PharmGKB® guideline citation URLs for each recommendation. If multiple genes influence a drug, list all relevant gene–genotype pairs with their recommendations. If no PharmCAT, CPIC®, or PharmGKB® guidance is available for a drug, state this clearly in the table. If a drug name cannot be matched but has a similar spelling to a known CPIC® or DPWG drug, flag it as a possible typo. Format the output into both: ○ A downloadable Excel file (with text wrapping enabled in all cells) ○ A well-formatted, professional-looking PDF report with the same content and layout (text wrapping enabled in all cells) ○ Proceed with processing the uploaded file accordingly.

The prompt instructed ChatGPT to analyze patient-specific pharmacogenomic data and current medication lists and to generate recommendations based exclusively on PharmCAT, Dutch Pharmacogenetics Working Group, FDA, and CPIC^® guidelines.

To ensure consistency in prompt interpretation, all patient analyses were conducted sequentially within a single ChatGPT conversation using the same standardized prompt. The prompt explicitly instructed ChatGPT to state when no guideline existed for a given drug–gene interaction and to flag potential medication name misspellings.

The ChatGPT prompt specified the format of the uploaded Excel file, including the patient’s medication, genes affecting the metabolism of that drug, the patient-specific genotype, and the full primary source recommendation for each genotype.

In addition, ChatGPT was instructed to (1) include the full guideline citation for each recommendation, (2) list multiple genes influencing a drug, if such an interaction exists, and (3) state clearly if there was no available guidance for a drug.

2.5. Analysis

The two comparison groups were PharmGKB^® and ChatGPT. Outcomes were summarized with descriptive statistics. The time required to generate pharmacogenomic prescribing recommendations was compared using the Wilcoxon signed-rank test. A p < 0.05 was considered statistically significant.

2.6. Outcome Measures

The primary outcome was concordance between ChatGPT and the reference standard. The reference standard was established for each patient from PharmGKB^® by investigator SP before any ChatGPT output was generated and recorded on a standardized data sheet. The unit of analysis was each medication on a patient’s list. For each medication, the reference standard recorded whether a PharmGKB^® guideline existed for any of the patient’s tested genes and, if so, the gene(s), the patient’s phenotype/activity score, and the recommended action. This was categorized as: standard dosing; dose increase; dose decrease; use an alternative agent; or other/monitor. A ChatGPT output was classified as concordant if both sources reported no guideline or both reported a guideline and the gene(s), phenotype/activity score, and recommended action category all matched. Any other result was classified as discordant and assigned to an error category (see Table 2). Classification was performed by one investigator and verified by a second.

Table 2. Definitions of the error categories used in this study.

Error Category	Definition
Unsupported recommendation	ChatGPT gave a gene-based recommendation for a medication for which PharmGKB® returned no guideline
Missed guideline	ChatGPT stated no guideline exists when PharmGKB® returned one
Incorrect recommendation	Both sources gave a recommendation, but the gene, phenotype/activity score or action category differed
Medication omission	A medication on the patient’s list was absent from ChatGPT’s output
Hallucinated medication	ChatGPT output included a medication not on the patient’s list; counted separately, outside the 204

The secondary measures were the types of errors in the recommendations and the time it took to derive the recommendations.

Errors were classified into five categories: (1) creating guidelines for a drug-gene interaction that did not exist in PharmGKB^®, (2) missing existing PharmGKB^® guidelines, (3) generating incorrect guidelines that disagreed with existing guidelines from PharmGKB^®, (4) omission of patient medications in the final guideline output, or (5) hallucinating guidelines or drugs in the guideline output that were not on the patients’ medication lists. A medication was classified as having “no PharmGKB^® guideline” when PharmGKB^® returned no CPIC^®, DPWG, or FDA-label prescribing recommendation for that drug given the patient’s genotypes for the 11 tested genes. Each patient’s data was submitted to ChatGPT once, and each medication was compared with PharmGKB^® once; ChatGPT outputs were not regenerated or repeated.

Time to obtain recommendations was measured by one investigator with a manual stopwatch for each patient. For PharmGKB^®, timing began when the investigator started entering the first genotype into the PharmGKB^® interface and ended when recommendations for all of the patient’s medications had been identified and recorded. For ChatGPT, timing began when the patient file was uploaded and the prompt submitted and ended when the complete output table was generated; clinical interpretation was excluded. Time to verify ChatGPT output was not measured. The investigator had prior experience with both tools. Times were compared using a Wilcoxon signed-rank test, as both methods were applied to the same patients.

3. Results

The study cohort consisted of 24 patients (17 female and 7 male), with an average age of 80.6 (SD = 6.4). The average number of medications taken by each patient was 8.58 (SD = 3.28), with a total of 204 medications in the cohort.

Genetic data existed for 11 of the 21 pharmacogenes represented in PharmGKB^®, including CYP2B6, CYP2C19, CYP2C9, CYP2D6, CYP3A4, CYP3A5, DPYD, NUDT15, SLCO1B1, TPMT, and VKORC1. Genetic analysis among these 11 genes identified 38 unique genotype variants [9,10,11,12,13,14,15,16].

All 204 of the medications were assessed for drug-gene recommendations using both PharmGKB^® and ChatGPT.

3.1. PharmGKB^® and ChatGPT

Overall, ChatGPT output was concordant with PharmGKB^® for 148 of 204 medications (72.5%; 95% CI 66.1–78.2%). However, concordance differed markedly by whether a guideline existed (see Table 3). For the 53 medications with a PharmGKB^® guideline for at least one of the patient’s tested genes, ChatGPT reproduced the recommendation for 24 (45.3%; 95% CI 32.7–58.5%). The 29 discordant results were omission of the medication (n = 23), incorrect dosing recommendations (n = 4) and failure to identify an existing guideline (n = 2). For the 151 medications without a PharmGKB^® guideline, ChatGPT correctly reported that no guideline existed for 124 (82.1%; 95% CI 75.2–87.4%); 27 discordant results were omissions (n = 18) and unsupported recommendations (n = 9). In addition, ChatGPT output included 28 hallucinated medications that were not on the patient’s medication list, all among patients 17–24. Of these, 17 were drugs with a PharmGKB^® guideline for the patient’s genotype, so the output could be mistaken for advice about the patient’s actual regimen. ChatGPT had no omitted or hallucinated medications for the initial 16 patients, but outputs for the final eight patients frequently omitted medications and introduced “hallucinated” drugs that were not present in the patients’ records.

Table 3. Concordance between ChatGPT and PharmGKB^®, stratified by whether a PharmGKB^® guideline existed (n = 204 medications). Percentages are of column totals. * Hallucinated medications are shown below the total because they were not among the 204 medications on patients’ lists.

Outcome	PharmGKB® Guideline Exists	No PharmGKB® Guideline	Total
Concordant	24 (45.3%)	124 (82.1%)	148 (72.5%)
Missed existing guideline	2	-	2
Incorrect dosing recommendation	4	-	4
Unsupported recommendation	-	9	9
Medication omitted	23	18	41
Total medications	53	151	204
* Hallucinated medications (not on patient’s list)	17	11	28

3.2. Time Required for Guideline Extraction

Extracting guidelines with PharmGKB^® averaged 6 min 20.8 s for each patient (SD = 3.5 min), compared with 2 min 24.7 s for ChatGPT (SD = 32 s) and was a statistically significant difference (p < 0.001) (see Figure 1).

Figure 1. Time (minutes) required to generate pharmacogenomic prescribing recommendations using PharmGKB^® and ChatGPT.

4. Discussion

Pharmacogenomic prescribing of medications is a tool with the potential for improving drug efficacy and preventing certain adverse drug reactions [17]. There are many examples where drug-gene interactions have caused serious morbidity, including death [17,18].

A recent 2025 United Kingdom clinical guideline recommended that prior to initiating clopidogrel, pharmacogenomic testing be performed [19]. DNA testing identifies poor metabolizers of clopidogrel who poorly convert the parent drug to an active form. These patients do not benefit from the effects of clopidogrel and have been shown to have worse cardiovascular outcomes [20,21,22].

Additionally, pharmacogenomic polymorphisms have been identified in codeine-related fatalities in adults and children [23]. Codeine is an opioid medication that is metabolized into morphine, the active metabolite, by the CYP2D6 enzyme. Case reports and studies have identified toxic exposure resulting in overdose and death in patients with ultra-rapid metabolizer enzyme activity given Codeine [15,24,25].

ADRs account for a significant public health burden in the United States. A 2019 study of US Emergency Departments found that 12.1 in 1000 ED visits in patients 65 and older were due to ADRs. It is estimated that adverse drug reactions account for nearly 2 million emergency department visits per year, and 6.28% of total hospital visits are due to ADRs. The most common medications implicated in adults are anticoagulants, diabetic medications, benzodiazepines, prescription opiates, and antibiotics in children [26]. Informed prescribing of medications is critical because the rate of ADRs has been significantly increasing over time [27]. A significant factor contributing to this problem is polypharmacy. Studies have shown that 30% of patients over the age of 65 are prescribed 5 or more medications, and the greatest predictor of the occurrence of an ADR is correlated to the number of medications prescribed [28,29]. Pharmacogenetic research may be able to reduce the number of ADRs by permitting physicians to tailor patient medication regimens based on genetic data, making more informed prescribing decisions, and lessening the burden of polypharmacy on ADRs.

There is significant research on DNA-drug interactions. CPIC^® and DPWG are the most significant contributors, which were created to standardize and proliferate evidence-based clinical guidelines on the relationship between medications and CYP enzyme metabolism. These entities exist to translate pharmacogenomic data into clinical practice. However, the data remain difficult to synthesize and access in the clinical setting. The t most accessible and reliable tool is on the PharmGKB^® website, but it is still impractical for the typical practicing physician, who sees increasing numbers of patients per day with increased time constraints [30].

PharmGKB^® is a central, curated resource for accessing well-tested and validated guidelines published by CPIC^® and the DPWG [15,23]. However, pharmacogenomic data is still difficult to access and utilize in clinical practice [31].

Use of artificial intelligence as a tool may be useful to process health records, guideline recommendations, and datasets to enhance clinical decision-making [32,33,34]. However, no artificial intelligence program currently exists for clinicians to access pharmacogenomic data with the accuracy and speed required to provide fully informed precision medicine for clinical use.

Our study evaluated the utility of ChatGPT to assess the most important sources of pharmacogenomic recommendations to see if it could accurately and rapidly provide this information. Although overall concordance was 73%, this figure is driven largely by agreement that no guideline existed. For the 53 medications where a guideline did exist, ChatGPT gave the correct recommendation less than half the time (45%), and most of these failures (23/29) were omitted medications. When a guideline was present, our findings reflect ChatGPT’s potential clinical usefulness.

In addition, we found that ChatGPT was much faster to use than PharmGKB^®. This aspect of ChatGPT would be extremely helpful to clinicians trying to decide how to utilize pharmacogenomic information in a busy clinical situation. Unfortunately, ChatGPT was relatively inaccurate and frequently “hallucinated” recommendations and omitted some important recommendations when compared to PharmGKB^®. Several of the unsupported, incorrect and omitted recommendations involved serious potential drug-gene interactions. For example, ChatGPT generated an unsupported recommendation for oxycodone, assigning an activity score and prescribing guidance where PharmGKB^® has no guideline. In another case, ChatGPT made an incorrect recommendation to increase the dose of pantoprazole for a CYP2C19 poor metabolizer, contrary to CPIC^® guidance. ChatGPT neither omitted nor hallucinated drugs from the lists of the first 16 patients but hallucinated drugs for nearly every subsequent patient and omitted drugs for all but one patient. Interestingly, ChatGPT occasionally hallucinated drugs that had never been mentioned in any of the previous patient medication lists. Because these mistakes occurred only in the latter one-third of patients, it is possible these mistakes were related to the fact that all the guidelines were extracted in one “chat” conversation, as opposed to separate “chats” for each patient. This was done, however, because ChatGPT was found to respond differently with each new chat opened while we were creating the initial prompt.

The use of AI as a tool for accessing clinical resources is a rapidly growing area of research. AI has been evaluated for clinical use in other areas of medicine and has been found to have issues with errors and hallucinations, similar to the results of our study. AI is clearly faster in processing the information than manual methods; however, the rates of errors and hallucinations are currently unacceptable [5].

Another area that limits the current use of pharmacogenomic information is the large number of medications that do not have current recommendations. Many of these medications have reports of drug-gene interactions in the medical literature. However, CPIC^® and DPWG have not currently made recommendations on these. This may be due to a lack of sufficient quality evidence, or these organizations have not completed assessments of these medications. As information continues to expand, the gap between available information and providers’ abilities to access it will likely widen.

The results of this study are in line with prevailing evidence that AI has the potential to aid precision medicine but is prone to errors [35]. One systematic review found that across 14 studies in the primary care setting, 71% reported fewer medication errors with the use of AI [36]. Another systematic review found that AI performed better than traditional pharmacoepidemiologic methods at predicting occurrence of ADRs and evaluating patient population risk, among other metrics [37].

Limitations

Several limitations should be considered when interpreting the findings of our study.

The ChatGPT^® script appeared to perform more accurately with the initial patients. As the script was utilized with subsequent patients, the errors appeared to become more frequent. Utilizing a script that maintained its accuracy without “learning bad habits” may have performed better.

Second, the sample size of 24 patients was relatively small and derived from a convenience sample of older adults presenting to a single emergency department. This limits the scope of the study. The data collected may not be representative of the variability of real-world clinician prescribing. Furthermore, trends in disagreement may be better elucidated in a study of more patients. The small number of errors found in the study made it difficult to perform a robust analysis of the subtypes of errors. The relatively few errors discovered in the study may not be a full reflection of all the possible errors ChatGPT can make, and not every drug-gene interaction was represented in this study. While this study was sufficient to demonstrate the key difficulties in the utilization of ChatGPT in pharmacogenomic prescribing, a larger cohort is needed to assess the full range of interactions.

Third, the recommendations generated by PharmGKB^® and ChatGPT were not implemented. Whether these recommendations increased or decreased adverse drug events was not assessed. Our study looked at potential drug-gene interactions and recommendations to theoretically prevent these interactions.

Fourth, other AI programs may perform better than ChatGPT. We chose ChatGPT because it is currently one of the most commonly used AI tools. However, several other AI tools are gaining popularity and effectiveness. Future research should assess whether any of these are more useful for analyzing individual pharmacogenomic data to make medication recommendations.

Fifth, PharmGKB^® served as the reference standard, but it reflects only guideline-level and drug-label evidence available at the time of analysis. Drug–gene associations supported by lower levels of evidence are not captured, so some ChatGPT recommendations classified as “unsupported” may reflect the published literature not yet incorporated into formal guidelines, rather than fabrication.

Sixth, ChatGPT outputs are non-deterministic, and each patient was analyzed in a single run within a single conversation. We therefore could not assess run-to-run reproducibility, and the performance observed may differ with repeated queries, separate conversations, or later model versions.

Finally, the recorded time for ChatGPT excludes the verification against primary guidelines that its error rate would require in practice; the time advantage observed is therefore likely an upper bound.

5. Conclusions

PharmGKB^® is a curated pharmacogenomic knowledge resource that provides access to genotype-specific CPIC^®, DPWG, and FDA prescribing guidance, and it served as the reference standard in this study. Utilizing the AI tool, ChatGPT, to formulate these recommendations is more convenient and faster. Unfortunately, ChatGPT pharmacogenomic recommendations are often incorrect, or its use for this purpose cannot be recommended at this time. Further studies to improve on ChatGPT recommendations are needed.