Validity of Administrative Data for Identifying Birth-Related Outcomes with the End Date of Pregnancy in a Japanese University Hospital

This study aimed to develop and validate claims-based algorithms for identifying live birth, fetal death, and cesarean section by utilizing administrative data from a university hospital in Japan. We included women who visited the Department of Obstetrics at a university hospital in 2018. The diagnosis, medical procedures, and medication data were used to identify potential cases of live birth, fetal death, and cesarean section. By reviewing electronic medical records, we evaluated the positive predictive values (PPVs) and the accuracy of the end date of pregnancy for each claims datum. “Selected algorithm 1” based on PPVs and “selected algorithm 2” based on both the PPVs and the accuracy of the end date of pregnancy were developed. A total of 1757 women were included, and the mean age was 32.8 years. The PPVs of “selected algorithm 1” and “selected algorithm 2” were both 98.1% for live birth, 99.0% and 98.9% for fetal death, and 99.7% and 100.0% for cesarean section, respectively. These findings suggest that the developed algorithms are useful for future studies for evaluating live birth, fetal death, and cesarean section with an accurate end date of pregnancy.


Introduction
The age of women giving birth has been rising in Japan [1]. Due to advanced maternal age, more women are likely to have chronic diseases, which require medication before or during pregnancy. For instance, it is now clear that cancer treatments, including anticancer medication, may cause organ damage, such as in the cardiovascular and endocrine systems [2][3][4][5][6][7][8][9]. Thus, evaluating birth outcomes for women with or after cancer medications is clinically important in real-world settings.
Utilizing the healthcare administrative database is one of the practical approaches to evaluating birth outcomes in real-world clinical settings and the information on rare treatment exposures [10][11][12]. Various administrative databases are currently available in Japan [13]; however, since administrative data are not collected for research purposes, the recorded information can sometimes result in incomplete and inaccurate data with misclassified outcomes. To provide a certain level of credibility for administrative data, evaluating the validity of the outcomes is crucial [14][15][16][17]. In addition, the regulatory agency in Japan, the United States, and Europe prepared the document to point out the importance of the outcome validation for the research use of administrative data [18][19][20]. Since the structure and contents of administrative claims and clinical practice differ among countries, a validation study conducted in Japan is necessary. Except for congenital malformation [21], the validity of birth-related outcomes has not been evaluated in Japan. Regarding the accuracy of the date of claim, a previous study in Japan reported the algorithms to determine the onset of pregnancy and delivery date [22]. However, the study population only included women who had given birth in the hospital, and the end date of pregnancy by specific birth-related outcomes was not assessed. Therefore, previous validation studies in Japan have been limited to addressing congenital malformation and the onset of pregnancy and delivery date without evaluating specific birth-related outcomes.
The current study bridges this research gap. It aims to develop and validate algorithms to identify birth-related outcomes (live birth, fetal death, and cesarean section) and the accuracy of the end date of pregnancy by using healthcare claims data from the Tohoku University Hospital in Japan.

Data Source and Study Population
We used electronic claims data at Tohoku University Hospital, containing diagnoses according to the International Classification of Diseases, 10th revision (ICD-10) codes, medical procedures (surgery and internal treatment), and medication. The study population included all women who visited the Department of Obstetrics at Tohoku University Hospital between January 2018 and December 2018. There were no exclusion criteria.

Definition and Identification of Birth-Related Outcomes
We evaluated live birth, fetal death, and cesarean section. In this study, we classified fetal death as miscarriage (defined as spontaneous abortion ≤ 21 weeks gestation), stillbirth (defined as spontaneous abortion > 21 weeks gestation), and induced abortion. To identify potential cases in each outcome, we used the obstetric diagnosis without the "suspected" flag, medication, and medical procedure listed in Table S1.

Review of the Potential Cases
We used electronic medical records (EMRs) as the gold standard to determine the accuracy of potential cases in each outcome. If a potential case had pregnancies several times in the study period based on EMR information, the accuracy was determined separately considering the date of the claim for each outcome. At Tohoku University Hospital, the Department of Obstetrics has developed and maintained a listing of EMR information regarding birth outcomes. We utilized the listing to review potential cases. However, since the listing only includes the birth outcomes of women who terminated their pregnancy after 12 weeks of gestation at Tohoku University, women who terminated their pregnancy before 12 weeks or those who delivered in other hospitals were not identified. Thus, if outcomes of potential cases were not available in the listing, the details of other unstructured data from EMRs were manually reviewed by three pharmacists. In the manual EMR review, two out of three pharmacists mutually reviewed the outcomes and the end date of pregnancy for each potential case. Any disagreements were resolved through discussion to obtain a final judgment by the third pharmacist.

Data Analysis
The age of the study population was calculated based on the first date of the claim entry and their birth date. However, if the women who visited the department of obstetrics at Tohoku University Hospital did not have any claims, their age at the time of their visit could not be identified directly. Therefore, we calculated the maternal age on 30 June 2018 because 30 June lies midway on the calendar.
To evaluate how accurately each claim datum identifies birth-related outcomes, we descriptively summarized the cases identified by each claim data. If the final outcomes could not be judged even after the EMR review due to a hospital transfer, these cases were categorized as "unevaluable".
Additionally, we aimed to evaluate the accuracy of the date of the claim for birthrelated outcomes. Regarding the end date of pregnancy in each birth-related outcome, we calculated the difference between the date of the claim data and that of the EMR. If one woman had multiple entries in the same claims data, we evaluated both the earliest date and the latest date and compared the results.
For the subsequent analysis, we took "selected algorithm 1" and "selected algorithm 2" to evaluate the validity of birth-related outcomes and the accuracy of the end date of pregnancy; "selected algorithm 1" was developed by combining the claims data (based on an "or" condition), of which positive predictive values (PPVs) were 80% or higher. PPVs were defined as the proportion of true-positive cases divided by the claims-positive cases. As a conservative approach for developing "selected algorithm 1", we regarded "unevaluable" as a false positive instead of removing it from the dataset to avoid overestimating the accuracy. For exploratory analysis, we assessed the additional algorithms by subtracting those who had the claims codes of "selected algorithm 1" for fetal death from those who had inaccurate claims codes (with the PPVs under 80%) for live birth to evaluate whether the accuracy for live birth would improve.
From the "selected algorithm 1", we developed a "selected algorithm 2" by strictly selecting the claims data with an accurate end date of pregnancy. We considered the date of the claim to be accurate if the percentage that met the criteria (the difference between the date of claim and the true end date of pregnancy was within ±7 days) was 80% or higher. For "selected algorithm 1" and "selected algorithm 2", PPVs and corresponding 95% confidence intervals (CI) were also estimated.
Each claim entry and its name, the number of subjects, and the difference between the date of administrative data and EMRs for birth-related outcomes are summarized in Table 1 for live birth, Table 2 for fetal death, and Table 3 for cesarean section. All data were identified as true positives with PPVs of 80% or higher and were included in the selected algorithms in each outcome and subsequent analyses. Exceptions were some data for live birth in Table 1 (O711, rupture of the uterus during labor; O720, third-stage hemorrhage; O723, postpartum coagulation defects; J8001, uterine dilation and labor induction; methylergometrine maleate; gemeprost suppository) and fetal death in Table 2 (O011, complete and partial hydatidiform mole; O028, other specified abnormal products of conception). Regarding the large number of codes with PPVs of 80% or higher, oxytocin injection (n = 731), cesarean section (complexity fee addition) (K89800, n = 262), and cesarean section (elective cesarean section) (K89802, n = 175) were often observed for live birth (Table 1). Additionally, the surgical management of miscarriage (until 11 weeks) (K90901, n = 35), missed abortion (O021, n = 29), and dilatation and curettage (missed abortion) (K90920, n = 27) were common in fetal death (Table 2), while cesarean section (complexity fee addition) (K89800, n = 262), cesarean section (elective cesarean section) (K89802, n = 175), and cesarean section (emergency cesarean section) (K89801, n = 166) were common in cesarean sections (Table 3). It is worth noting that the number of codes (O601, preterm spontaneous labour with preterm delivery) was limited (n = 1).
The date was generally accurate regarding all medical procedures with a PPV of 80% or higher for live birth, fetal death, and cesarean section. Considering the date difference between claims data and the end date of pregnancy in EMRs, the percentages that fell within the range of ±7 days were almost 100.0%, although K90920 (dilatation and curettage (missed abortion)) was 96.3%. However, regarding some claims of diagnosis, the percentages that fell within the range of ±7 days were under 80.0% for live birth (O680, labor and delivery complicated by fetal heart rate anomaly; O757, vaginal delivery following previous cesarean section; O820, delivery by elective cesarean section) in Table 1, fetal death (O009, ectopic pregnancy, unspecified; O021, missed abortion; O081, delayed or excessive haemorrhage following abortion and ectopic and molar pregnancy) in Table 2, and cesarean section (O820) in Table 3. Considering the date of diagnosis claims, the latest date was more accurate than the earliest date.
For fetal death outcomes, only two cases resulted in stillbirth; thus, the table focusing on stillbirth has not been displayed. Miscarriage and induced abortion are summarized in Tables S2 and S3. Overall, although the potential cases in most claims data for identifying miscarriage and induced abortion were true-positive cases, K90901 (surgical management of miscarriage (until 11 weeks)) and K90902 (surgical management of miscarriage (from 12 weeks to 21 weeks)) were used for miscarriage instead of induced abortion; K90920 was used for induced abortion instead of miscarriage. The PPVs of "selected algorithms 1" and "selected algorithms 2" were 100.0% (95% CI 93.4-100.0%) and 100.0% (95% CI 92.9-100.0%) for miscarriage, and both were 89.2% (95% CI 74.6-97.0%) for induced abortion (Table S4). Table 1. Accuracy of claims data to identify live birth and the difference between the date of claim data and the end date of pregnancy in EMRs.       Abbreviations: EMRs, electronic medical records; TP, true positive; FP, false positive. a Regarding the relative difference, cases that were both true positive and available for "the end date of pregnancy in EMRs" are shown. Therefore, some numbers could have discrepancies between the number of true-positive cases and the numbers in the rows of the relative difference if some cases missed the end date of pregnancy in EMRs, even though the EMR outcome itself was identified. b Existing multiple entries in the same claim codes for the same women; both the earliest and the latest dates were analyzed. Table 2. Accuracy of claims data to identify fetal death a and the difference between the date of claim data and the end date of pregnancy in EMRs.      Tables S2 and S3. b. Regarding the relative difference, cases that were both true positive and available for "the end date of pregnancy in EMRs" are shown. Therefore, some numbers could have discrepancies between the number of true-positive cases and the numbers in the rows of the relative difference if some cases missed the end date of pregnancy in EMRs, even though the EMR outcome itself was identified. c Existing multiple entries in the same claims code for the same women; both the earliest and the latest dates were analyzed. Table 3. Accuracy of claims data to identify the Cesarean section and the difference between the date of claim data and the end date of pregnancy in EMRs. Abbreviations: EMRs, electronic medical records; TP, true positive; FP, false positive. a Regarding the relative difference, cases that were both true positive and available for "the end date of pregnancy in EMRs" are shown. Therefore, some numbers could have discrepancies between the number of true-positive cases and the numbers in the rows of the relative difference if some cases missed the end date of pregnancy in EMRs, even though the EMR outcome itself was identified. b Existing multiple entries in the same claims code for the same women; both the earliest and the latest dates were analyzed.  Abbreviations: CI, confidence interval; EMRs, electronic medical records; PPV, positive predictive value; TP, true positive. a Regarding the relative difference, cases that were both true positive and available for "the end date of pregnancy in EMRs" are shown. Therefore, some numbers could have discrepancies between the number of true-positive cases and the numbers in the rows of the relative difference if some cases missed the end date of pregnancy in EMRs, even though the EMR outcome itself was identified. b Existing multiple entries of claims codes in the algorithm for the same women; both the earliest and the latest dates were analyzed. c Codes in selected algorithm 1 for live birth were as follows: O601, O624, O654, O655, O669, O680, O683, O690, O700, O717, O721, O757, O800, O820, K89300, K89601, K89603, K89800, K89801, K89802, K90400, oxytocin injection, and dinoprost injection. d Codes in selected algorithm 2 for live birth were as follows: O601, O624, O654, O655, O669, O683, O690, O700, O717, O721, O800, K89300, K89601, K89603, K89800, K89801, K89802, K90400, oxytocin injection, and dinoprost injection. e Codes in selected algorithm 1 for fetal death were as follows: O008, O009, O010, O021, O034, O039, O049, O080, O081, O364, P95, K90901, K90902, K90920, and K91100. f Codes in selected algorithm 2 for fetal death were as follows: O008, O010, O034, O039, O049, O080, O364, P95, K90901, K90902, K90920, and K91100. g Included codes in selected algorithm 1 for cesarean section were as follows: O820, K89800, K89801, and K89802. h Included codes in selected algorithm 2 for cesarean section were as follows: K89800, K89801, and K89802.

Discussion
To the best of our knowledge, this is the first study to develop and evaluate the validity of algorithms to identify live birth, fetal death, including miscarriage and induced abortion, and cesarean section using administrative data in Japan.
The value of the current study lies in providing information on the accuracy of multiple birth-related outcomes based on claims information in Japan. In Western countries, various validation studies identifying live birth, miscarriage, stillbirth, induced abortion, and cesarean section have been reported and have demonstrated accuracy [23][24][25][26][27][28][29][30][31]. However, it is essential to conduct validation studies in Japan because some claims data in Japan are unique, and the procedures and treatment options might differ across countries. From a public health perspective, adverse obstetric outcomes affected by medication before or during pregnancy are of interest. Especially in the field of cancer, the improvement of patient survival outcomes through treatment progress has enabled cancer survivors to attempt childbearing [5]. In Taiwan, a study utilizing an administrative database revealed that cancer survivors had a higher risk of overall adverse birth outcomes and cesarean section [32]. It is expected that our findings will accelerate future studies in Japan to evaluate birth-related outcomes in the context of cancer treatment by using a claims-based administrative database.
In the current study, we developed an accurate algorithm for each birth-related outcome as "selected algorithm 1" by selecting the claims data with PPVs of 80% or higher. Combining the claims data could capture a larger number of potential outcomes than using the single claims data alone. Most of our investigated claims data related to fetal death and cesarean section represented accurate outcomes. However, some data (O711, O720, O723, J8001, methylergometrine maleate, and gemeprost suppository) resulted in inaccuracy for live birth and were not incorporated into "selected algorithm 1". Considering this result, we assessed additional algorithms in the exploratory analysis by subtracting those who had the claims data of "selected algorithm 1" for fetal death from those who had these inaccurate claims data for live birth. Overall, although the PPV of methylergometrine maleate tablets was over 80% in this exploratory analysis, there was no remarkable improvement in the accuracy of live birth. Considering these exploratory results and the complexity of the algorithms, these claims codes might not be utilized for future studies to identify live births. Notably, the number of codes (O601, preterm spontaneous labour with preterm delivery) was very limited. Although preterm birth is an important adverse birth outcome, it might be difficult to identify it using the ICD-10 code in Japan. To identify preterm birth, the use of algorithms for the onset of pregnancy [22], as well as our algorithms of live birth, should be considered. To provide detailed information regarding fetal death, we also evaluated the fetal outcome as miscarriage and induced abortion separately. In general, although our planned claims codes were reasonable for identifying both outcomes, our results suggested that K90901 and K90902 were not used for induced abortion, and K90920 was not used for miscarriage.
Following the assessment of "selected algorithm 1", we also assessed the accuracy of the end date of pregnancy; "selected algorithm 2" was identified as an accurate algorithm for both the outcome and its date. Considering the accuracy of the date, the medical procedure codes and the claims data by using the latest date were accurate in general, which was consistent with previous reports [22]. We recommend utilizing "selected algorithm 2" for each birth-related outcome for future studies that require an accurate outcome and its date. Considering the methodological point of view, we followed a step-by-step approach through the evaluation of (1) "single claims data", (2) "selected algorithm 1 (combining the claims data with PPVs of 80% or higher)", and (3) "selected algorithm 2 (combining both the claims data with PPVs of 80% or higher and the claims data with the accuracy of the end date of pregnancy (within ±7 days))". Although claims data in Japan are unique and clinical or billing procedures might differ across countries, our approach to developing outcome algorithms would help in other countries.
Our study had some limitations. First, since the current study was conducted in a single university hospital in Japan, our results may not be generalizable to other medical institutions, including general clinics. The population in this study likely included older women, who may be at risk during pregnancy and delivery. Furthermore, the number of fetal deaths and cesarean sections might be larger in university hospitals than in general clinics. Although the system for medical fee processing was standardized and the government provided the master files under the healthcare system in Japan, further studies may be required to determine whether our results could be extrapolated to other hospitals. Second, we were unable to review the negative cases of the algorithm due to the study's resources. Thus, the sensitivity, specificity, and negative predictive value were not assessed. We believe that our results provide useful information, especially for future comparative studies, to evaluate the relative risk of birth-related outcomes between exposure and control. Third, regarding birth-related outcomes, we assessed outcomes for women who had transferred to another hospital as "unevaluable". However, as a conservative approach, we defined the selected algorithms in the current study by regarding "unevaluable" as false positives instead of removing them from the dataset to avoid overestimating the accuracy. Lastly, we could not evaluate some claims data related to birth-related outcomes because some specific codes were not observed in the administrative claims during our study period. For example, our study did not cover some of the ICD-10 codes of "O00-O99 Pregnancy, childbirth, and the puerperium". Therefore, the accuracy of non-evaluated claims data was uncertain. Furthermore, given that some claims data were based on only a few potential cases in this study, results with these uncommon claims data should be carefully interpreted. Therefore, future studies with these non-evaluated and uncommon claims data in our study are suggested.

Conclusions
We developed reliable combination algorithms to identify live birth, fetal death, and cesarean section based on the evaluation of the accuracy of each claim data, as well as the accuracy of the date of claim. Since validated claims-based algorithms with information on the PPVs and the accuracy of the dates are required to provide a certain level of credibility for the research use of claims data, we recommend utilizing our validated algorithms for each birth-related outcome. Our findings will benefit future studies that attempt to evaluate birth-related outcomes in real-world clinical settings.
Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/ijerph19084864/s1, Table S1: Planned claims codes to identify birth-related outcomes; Table S2: Accuracy of claims data to identify miscarriage and the difference between the date of claim data and the end date of pregnancy in EMRs; Table S3: Accuracy of claims data to identify induced abortion and the difference between the date of claim data and the end date of pregnancy in EMRs; Table S4: Positive predictive values and the date difference between claims data and EMRs of algorithms for miscarriage and induced abortion.

Informed Consent Statement:
The requirement of informed consent was waived by the Institutional Review Board for the purpose of this study.