Development of a Model for Quantitative Assessment of Newborn Screening in Japan Using the Analytic Hierarchy Process

Whether or not conditions should be included in publicly funded newborn screening (NBS) programs should be discussed according to objective and transparent criteria. Certain criteria have been developed for the introduction of NBS programs in the context of individual countries; however, there are no standard selection criteria for NBS programs in Japan. This study aimed to develop a quantitative scoring model to assess newborn screening that incorporates the views of a variety of stakeholders in Japan. The five recommended eligibility criteria for NBS were stratified based on previous studies and expert opinions, using the analytic hierarchy process. We conducted a cross-sectional, web-based questionnaire targeting a wide range of people involved in NBS to investigate pairwise comparisons of the evaluation items between February and April of 2022. There were 143 respondents. Most of our respondents (44.1%) were physicians. Fifty-eight respondents (40.6%) had been engaged in NBS-related research or work for more than 10 years. The distribution of allocation points was the highest for ‘intervention’, ‘screening test’, ‘follow-up setting’, ’economic evaluation’, and ’disease/condition’, in that order. The algorithm in this study will guide decision makers in collecting and evaluating objective data, thus enabling transparent discussions to occur.


Introduction
Newborn screening (NBS) is a public health program aimed at screening newborns for conditions that can lead to death or severe disability if left undetected [1]. Early detection of such conditions allows for prompt intervention to reduce or avoid the undesirable condition. The implementation of NBS varies widely depending on national government policies, ranging from countries that screen for many conditions to countries that do not implement NBS [2].
The NBS program in Japan started in 1977 as a public health program [3]. The introduction of tandem mass spectrometry in 2013 has increased the number of conditions screened for. As of 2022, the NBS program covers 20 conditions in all municipalities. The targeted conditions are congenital disorders of endocrinology and metabolism for which diet and relatively inexpensive drug therapy are effective. In recent years, the development of innovative treatment methods and improvements in testing techniques have led to discussions on how to further expand the current NBS program. Adding new conditions Int. J. Neonatal Screen. 2023, 9, 39 2 of 13 to the NBS requires careful and transparent discussions on the benefits and harms of a screening program among the relevant stakeholders.
As principles for evaluating conditions for screening, Wilson and Jungner's principles of screening are still an important perspective today [4]. On this basis, many principles have been developed for the introduction of NBS within the context of each country [5][6][7][8][9]. The Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC) in the United States has described the process of adding a condition to the Recommended Uniform Screening Panel [8]. The process requires that the candidate condition is nominated according to uniform evaluation criteria and after multiple steps of the review process, the evaluation results are organized by a decision matrix, and a final decision is made. The UK National Screening Committee (UK NSC) has established criteria for objective evaluation based on evidence from screening programs [9]. However, in Japan, there are no standard criteria for selecting conditions for NBS programs. This study aimed to develop a quantitative scoring model to assess newborn screening that incorporates the views of a variety of stakeholders in Japan.

Materials and Methods
The study was approved by the Institutional Review Board of the National Institute of Public Health, Japan (#706).

Analytic Hierarchy Process
We applied the analytic hierarchy process (AHP) in the development of our scoring algorithm. The AHP, one of the decision support tools [10,11], is used not only in healthcare research but also in many other fields [12,13]. The AHP decomposes complex decision-making issues into a hierarchical structure and calculates quantitative weights via pairwise comparisons of each component of the hierarchical structure. The advantage of pairwise comparisons is that they facilitate comparisons of complex problems and deal with subjective perspectives. We used the AHP in three steps. The first step was to identify the evaluation components of the issue and stratify them into a hierarchical structure. The second step was to perform a pairwise comparison of each component of the hierarchical structure. In the third step, the weights of the evaluation components were calculated based on the values obtained in the second step.

Structuring the Assessment Items
We conducted a literature review to select items to be assessed when considering the addition of a condition to the NBS. Twenty-two clinical and NBS experts discussed these items, after which they selected and tailored the structure of the assessment items to the Japanese context. A pilot study of the pairwise comparisons was then conducted within the study group, and the structure of the assessment items was reconsidered ( Table 1). The items were finally classified into five categories: disease/condition, screening test, intervention, follow-up setting, and economic evaluation. Table 1. Structure of the assessment items.
2. Onset of serious symptoms within 96 h of birth * (No onset, ≥1% but <30%, ≥30% but <70%, ≥70% or unknown) The incidence of serious symptoms observed before obtaining screening test results should be clear in order to obtain optimal outcomes for early detection and treatment via screening. After the patient has been diagnosed, there should be a core hospital that has a specialist on the disease/condition within the accessible range. A system to coordinate cooperation between the local hospital that the patient visits routinely or on an emergency basis and the hospital with a specialist should be well established.
15. Availability of post-screening consultation (available; partially available; not available) A system that can sufficiently explain the disease to the patients and family members who tested positive and the patient's family (e.g., genetic counseling, brochures to explain the disease, and contacts for inquiries) should be well established. Furthermore, this system should be standardized nationwide to allow providing information fairly to all individuals identified through screening.
Economic evaluation 16. Economic evaluation (scientific evidence is available; some scientific evidence is available; other) An economic evaluation of the screening program should include appropriate resources used and health outcomes simultaneously reflecting the national context.
The structure of the evaluation items to add conditions to the NBS is divided into three levels: 'Categories' are the largest classification, describing the five representative aspects of the evaluation items; 'Subcategories' are intermediate classifications, describing specific evaluation items; and 'Criteria' are classifications that define the criteria for the subcategories. * Dried blood specimens for NBS are usually collected on the 4th day after birth.

Participants and the Questionnaire
We conducted a cross-sectional, questionnaire-based survey to anonymously investigate pairwise comparisons of the evaluation items between February and April of 2022. The subjects included members of the board of directors of societies related to NBS, employees of facilities that perform NBS, employees of local government departments related to NBS, representatives of patient organizations, medical students, and healthcare workers whose work is not related to NBS (hereinafter referred to as 'general healthcare workers'). Invitations to participate in the survey were sent via e-mail, together with the survey's URL link, except for those to general healthcare workers. Six general healthcare workers were selected from the survey panel provider. The extracted features were as follows: type of job (doctor, pharmacist, or nurse), sex, with/without children. General healthcare workers were invited to take part in individual online interviews so that they could ask questions about NBS expertise.
The questionnaire was prepared on a dedicated website in Japanese. It consisted of a description of the assessment items, a pairwise comparison questionnaire, and an open-ended question about the survey. There were 88 questions (10 in categories, 22 in sub-categories, and 56 in criteria) for pairwise comparisons, and the response options were equal, moderate importance, strong importance, and very strong importance (Table S1). In the pairwise comparison section, participants were presented with two evaluation items for each question and asked to compare the relative importance of the evaluation items in selecting conditions for NBS. If respondents had difficulties with technical content about the NBS, they were allowed to skip the 56 questions in criteria that required the most technical knowledge.

Determination of Allocated Points
Weights were calculated using the geometric mean of the AHP. The consistency index (CI) was used to assess the consistency of responses. The CI for a completely consistent response was 0, and a CI greater than 0.15 was considered inconsistent. The calculated weights were multiplied by 1000 and rounded off to the nearest whole numbers to determine the allocation of points for each evaluation item. Data analysis was performed using SAS 9.4 (SAS Institute Inc., Cary, NC, USA) and Microsoft Excel 2019.

Validation of the Scoring Algorithm
We applied scoring algorithms for phenylketonuria, medium-chain acyl-CoA dehydrogenase (MCAD) deficiency, and congenital hypothyroidism (CH). These diseases are expected to score high because they are regarded as the most suitable diseases for NBS and have already been included in the NBS program in Japan. A literature review was conducted to examine the existing evidence, and disease experts in the study group evaluated this evidence.

Participants
A total of 156 respondents participated in the study. The analysis included 143 respondents who completely answered the categories and the subcategories (95 respondents answered all questions). Males accounted for 57.3% of the respondents, and many (31.5%) of the respondents were in their 50s (Table 2). Physicians were the most represented professionals among the respondents (44.1%), followed by laboratory technicians (14.0%). Fifty-eight respondents (40.6%) had been engaged in NBS-related research or had been working for more than 10 years.

Calculated Weights and Scores
The calculated weights were 0.292 for the 'intervention', 0.198 for the 'screening test', 0.198 for the 'follow-up setting', 0.178 for the 'economic evaluation', and 0.133 for the 'disease/condition ' (Figure 1). CIs were below 0.15 for all items. The final allocation of scores is shown in Table 3. The highest possible score was 620 points, and the lowest possible score was 114 points. The results of the evaluation of the current NBS target diseases, phenylketonuria, MCAD deficiency, and CH were 609, 605, and 540 points, respectively (Tables 4-6). 0.198 for the 'follow-up setting', 0.178 for the 'economic evaluation', and 0.133 for the 'disease/condition ' (Figure 1). CIs were below 0.15 for all items. The final allocation of scores is shown in Table 3. The highest possible score was 620 points, and the lowest possible score was 114 points. The results of the evaluation of the current NBS target diseases, phenylketonuria, MCAD deficiency, and CH were 609, 605, and 540 points, respectively (Tables 4-6). The pie chart shows the weight distribution assigned to each evaluation item as a result of a pairwise comparison by the subject. The larger the weight assigned, the larger the representative area of the figure, which means that it is more important to the respondents. The inner circle indicates the weight distribution assigned to each of the five categories. The numbers indicate the percentage of weight assigned and the total value is 1 (although, due to rounding issues, the numbers in the figure do not add up to 1). The outer circles show the relative weight percentage assigned to the subcategories.   The others 18 This table shows the final distribution of the scores assigned as a result of the survey. Higher scores indicate items that were rated more important by the respondents. Criteria in the subcategories are sorted in descending order of score. The maximum score is 620 and the minimum score is 114 in this scoring model. Total score 609 620 The results of applying the developed scoring model to phenylketonuria (PKU) are shown. The rating score represents the score assigned to the selected 'Criteria' for the PKU. Deviations from the full marks indicate scores that PKU did not obtain. Total score 605 620 The results of applying the developed scoring model to medium-chain acyl CoA dehydrogenase (MCAD) deficiency are shown. The rating score represents the score assigned to the selected criteria for the MCAD deficiency. Deviations from the full marks indicate scores that MCAD deficiency did not obtain. Total score 540 620 The results of applying the developed scoring model to congenital hypothyroidism (CH) are shown. The rating score represents the score assigned to the selected criteria for CH. Deviations from the full marks indicate scores that CH did not obtain.
The pie chart shows the weight distribution assigned to each evaluation item as a result of a pairwise comparison by the subject. The larger the weight assigned, the larger the representative area of the figure, which means that it is more important to the respondents. The inner circle indicates the weight distribution assigned to each of the five categories. The numbers indicate the percentage of weight assigned and the total value is 1 (although, due to rounding issues, the numbers in the figure do not add up to 1). The outer circles show the relative weight percentage assigned to the subcategories.

Discussion
The algorithm developed in this study helps decision makers to gather objective evidence and consider the priorities of target conditions for NBS in Japan. The score assigned to a condition also helps to ensure the transparency of the assessment. We have developed this algorithm primarily focusing on clinical factors. Organizations such as the ACHDNC and UK NSC, which are public bodies involved in evaluating NBS, incorporate elements related to the ethical aspects of the screening program and the feasibility of implementation in public health systems. However, we did not include these elements because we found it challenging to incorporate them into the hierarchical structure. These must be discussed separately from the objective evidence collected using this algorithm, if necessary.
'Intervention' was the most highly rated item. In this item, 'scientific evidence of the benefits of early intervention' was ascribed the highest score. This item was considered important because the goal of NBS is to improve patient outcomes rather than detect disease early. The fewest points were allocated to 'disease/condition'. Respondents seemed to consider screening performance and the intervention process for patients more important than a better understanding of conditions. 'Economic evaluation' was ranked second from the bottom. Local government employees tended to respond that cost-effectiveness was an important aspect of the NBS.
The three current NBS conditions were evaluated using our algorithm and scored as highly as expected; however, the 'economic evaluation' item for CH was deducted 68 points. This is because CH is screened independently during NBS, and has been included in NBS even before more attention was paid to cost-effectiveness; thus, sufficient information was not available for Japanese context. CH is known to have a high incidence and inexpensive treatment can prevent serious clinical consequences. Therefore, screening for CH is explicitly considered to be cost-effective, although no formal analysis has been conducted.
The algorithm does not set a threshold for selecting target conditions because we consider that the threshold should be determined by decision makers according to their context. There is no independent and dedicated organization in Japan that discusses NBS comprehensively and continuously; however, each local municipality makes decisions on its own terms. Therefore, the decision-making process and thresholds were not set as they depend on each municipality's public health policy. However, if we were to suggest a guide to thresholds, scores obtained for the current NBS-targeted diseases would be helpful.
We cannot rule out the possibility of selection bias for the data used in the development of this algorithm. As the NBS is a public health project, a broadly representative view was required in the development of the algorithm. However, it was difficult to seek in-depth understanding and opinions from the general population because the field of NBS is a highly specialized one. Therefore, we collected the views of individuals from relevant fields of NBS and those with medical backgrounds. Although we encouraged the subjects to join the survey several times, the response rate is considered to be about 30%. Some of the subjects may not have responded to the questions because they thought they did not have the expertise to answer the questions properly or they did not think it was relevant to them. It is also possible that the subjects were busy and may have given up on answering the questions due to the large number of questions. Because 66% of the respondents have NBS-related work or research experience, the score reflects more input from subjects who are more interested in NBS.

Conclusions
The number of candidate conditions for NBS is increasing due to recent developments in genomic technologies and improvements in screening techniques. The selection of such NBS candidate conditions in Japan requires multi-criterion decision making. The algorithm in this study will guide decision makers to collect and evaluate objective data, which will facilitate transparent discussions. It is also expected to provide a reference for data to be investigated by those who wish to expand the NBS. Future revisions will be necessary because it is possible that further advances in medical care and social changes will lead to changes in the assessment items and the distribution of scores.
Supplementary Materials: The following supporting information can be downloaded at: https:// www.mdpi.com/article/10.3390/ijns9030039/s1, Table S1: Pairwise comparisons in the questionnaire. Institutional Review Board Statement: The study was approved by the Institutional Review Board of the National Institute of Public Health, Japan (#706).