3. Results
The response rates were 100% for the FGD with Y-FETP technical staff, 94% for IDIs with program directors, 100% for senior policymakers, and 50% for organizations that employed graduates. The response rate for the online survey of graduates was 30/43 graduates (70%).
Table 1 shows the demographic characteristics of the online survey respondents. Seventy percent were males, 47% were 40–45 years of age, around 50% had a master’s degree, and 30% were physicians working onsite. The graduates were nominated from 11 governorates, 50% from Amanat Al Asimah (Sana’a city), and about 66% being currently employed by the MoPHP.
All FGDs and the majority of IDI participants indicated that Y-FETP helped to identify public heath challenges through performing many epidemiologic activities, and they provided useful recommendations to the programs as well as MoPHP decision-makers to overcome such challenges. Most participants agreed that Y-FETP residents and graduates have good skills and are superior to peers in other training programs, e.g., the Master of Public Health and Community Medicine Diploma. They stressed that such skills are more needed during the current siege and war situation that has made it difficult to bring external consultants to Yemen.
“We benefited from the accumulated experiences of the CDC and other FETPs to build the epidemiological capacity of our residents. Thus, the curriculum ensures the best mix between theoretical and practical components that a filed epidemiologist requires”. Y-FETP staff.
“Y-FETP residents supported different MoPHP programs through performing a wide-range of epidemiological activities such as analyzing and evaluating surveillance data; and conducting filed investigations. Thro ugh such activities not only epidemiological capacities of the residents were strengthened but also the capacities of the programs and the staff”. Senior MoPHP Policymaker
“During the 2005 polio outbreak in Yemen, there was a shortage of local epidemiologist, and investigation was not done in a proper and timely manner … the reports were also weak and incomplete; therefore the response was inadequate and external expertise was needed … But after we had Y-FETP when the cholera outbreak occurred in 2017, we noticed marked improvement in the quality of the investigations reports and we were able to convey scientific and evidence-based messages regarding the situation, both to national and the international community …. Moreover, the response was mainly led by local staff”. Organization staff employee.
“Y-FETP graduates had robust scientific background and good communication skills … This came from the curriculum which focuses on theoretical and practical approach. So, the residents and graduates were well versed with field and practice compared to many other teaching programs”. Program director.
“In the past, international WHO consultants routinely came to evaluate the Acute Flaccid Paralysis (AFP) program thrice a year …. these evaluations were interrupted with the war and blockade in the country as there was no possibility to recruit international consultant to do the evaluation …. Thankfully, the Y-FETP residents played a major role towards this end and conducted and published two evaluations for the AFP program and helped us identify gaps and correct deviations on a timely manner”. AFP Program Director.
Such qualitative findings support the quantitative results obtained from the graduates’ online survey.
Figure 1 shows that 60–80% of the graduates reported that Y-FETP greatly helped them in performing epidemiological field activities such as outbreak investigation and response, surveillance analysis/evaluation, managing surveillance systems/projects, public health communication (reports/presentation), and applying basic statistical methods.
In addition, 70% to 90% of graduates rated their skills as good in performing epidemiological field activities such as outbreak investigations, surveillance analysis/evaluation, managing surveillance systems/projects, public health communication (reports/presentation), using computers for specific applications relevant to public health practice, using epidemiological methods to design/conduct studies or field investigations, managing field projects/programs and conducting basic statistical analysis (
Figure 2).
The desk review results showed that Y-FETP residents and graduates conducted >171 outbreak investigations, 138 surveillance system analyses/evaluations, and 53 planned studies. According to IDI with one of the policy makers, these numbers are remarkable.
“Before Y-FETP, the MoPHP usually conducted 2–3 outbreak investigations on ad hoc bases and according to availability of funds from the Ministry of Finance … In many situations there was no funds for outbreak investigation … the untrained governorate surveillance officers investigated some of these outbreaks …. furthermore, no surveillance systems were analyzed nor evaluated before starting the Y-FETP…Only WHO did one external evaluation for acute flaccid surveillance system ……………. Also, the General Directorate for Diseases Control and Surveillance was not able to conduct planned studies to investigate health problem”. Senior MoPHP policymaker.
Qualitative and quantitative findings showed that the Y-FETP helped deliver evidence-based data that supported decision making using analysis and evaluation of surveillance programs, filing investigations of the priority health problems, and timely outbreak investigation and control, thus leading to a faster, more efficient response, reducing both morbidity and mortality and helping save lives. FGDs with Y-FETP staff and IDIs with senior policymakers, program directors, and employer organization staff were recorded to recount examples.
“One of the 2nd cohort residents who evaluated the electronic integrated Diseases Early Weaning System (eIDEWS) program showed that one of the reasons for weak reporting was due to using none SMART phones that cannot support some of the reporting functions. Therefore, she recommended that the program provide the field coordinators with SMART phones …. The eIDEWS program was able to implement the recommendation …. After that reporting improved considerably that improved timely alerts and response to outbreaks”. Y-FETP staff.
“The residents and graduates’ analyses and filed investigations contributed remarkably to making better decisions, solving problems and making policies”. Senior MoPHP policymaker.
“The Y-FETP residents analyzed surveillance data and found that the incidence of some diseases increased in some areas … They highlighted the problem and presented it to the decision-makers in the MoPHP and stakeholders such as WHO …. The MoPHP with support from WHO in turn found solutions and overcome the problem”. Organization staff employee.
“Y-FETP team went to investigate a measles outbreak …. The team found that 95% of the affected children were unvaccinated therefore, the team recommend to conduct an immunization campaign…. Thus UNICEF agreed to conduct such campaign …. We noticed marked reduction of measles cases after that”. Y-FETP staff.
“In Expand Program for Immunization, the Y-FETP contributed remarkably in providing data on the spread of vaccine preventable diseases and outbreaks and identified areas of low vaccine coverage, this helped us in getting funds to conduct vaccination campaign and outreach activities” EPI Program Director.
Findings from the graduates’ online survey showed that 97% of the Y-FETP graduates participated in investigation and control of important outbreaks such as COVID-19, cholera, and diphtheria.
Furthermore, through engagement with wide range of epidemiological activities such as outbreak investigation and analyzing and evaluating surveillance systems/programs, these activities resulted in a better understanding of the current status of the health system and the challenges it is facing, especially under the difficult circumstances in Yemen; this contributed to evidence-based decisions being made (
Figure 3).
Finally, all participants of the FGDs and IDIs indicated that Y-FETP had a significant role in increasing the awareness of health issues through publishing in peer-reviewed journals as well as through presenting at national, regional, and international conferences.
“Y-FETP had a significant role in documenting health problems through publishing in scientific journals and participation in national, regional and international conferences”. Y-FETP Staff.
“The Y-FETP residents and graduates analyzed programs’ surveillance data such as acute flaccid paralysis, measles, and other programs. The outcome of the analyses was shared in international conferences and published in peer-reviewed journals”. Senior MoPHP policymaker.
“A team from our program participated with Y-FETP on a scientific research on HIV/AIDS situation in Yemen that was presented in a conference in Thailand”. Program Director.
In fact, the desk review showed that Y-FETP published >50 scientific articles in peer-reviewed journals and presented >155 abstracts at national, regional, and international conferences. According to the IDIs, publication rarely occurred before the program was implemented:
“Before Y-FETP, the general directorate for diseases control and surveillance published only three papers in collaboration with WHO and CDC”. Senior MoPHP policymaker.
Findings from the graduates’ online survey showed that around two-thirds of Y-FETP graduates published one or more articles in scientific peer-reviewed journals, and all had submitted accepted abstracts in national, regional, and international conferences, e.g., TEPHENT global conferences, EMPHNET regional conferences and Epidemic Intelligence Service (EIS) conferences.
Despite the many remarkable achievements of the program, it is threatened by several factors. One of the Y-FETP staff who participated in the FGD indicated that one of these challenges is its dependence on international funding (TEPHNIET), making it less likely to be sustainable.
“The program mainly depends on TEPHINET financial support e.g., for outbreak investigation, staff salaries, residents’ stipend, running cost etc”. Y-FETP staff.
The desk review identified several challenges. The first is that the program only covers 18 (82%) out of the 22 governorates in Yemen, and half of the trainees who joined the program came from the capital city Sana’a, i.e., Amanat Al Asimah. This was also raised during the Y-FETP FGD.
“We, as a national program, always invite and encourage all Yemeni governorates to nominate their eligible candidates to enroll in the program… some governorates did not respond for different reasons e.g., no eligible or interested candidates, or their candidates did not pass the screening course”. Y-FETP Staff
The second challenge is low retention of graduates by the National Health System (NHS). The desk review showed that out of the 43 graduates, 19 (44%) were currently working with donor organizations, e.g., WHO, UNICEF, MSF, Save the Children. A policymaker raised this challenge in one of the IDIs:
“Due to the current war and blockade on Yemen, many of the graduates prefer to join donor organization for better salary and living condition”. Senior MoPHP policymaker.
Finally, most of senior policymakers and program directors pointed out that Y-FETP residents and graduates do not contribute directly to policies and strategies and have weak management skills.
“… despite strengths, residents and graduates do not play a strong role in supporting the ministry and programs in developing policies and strategies and many are weak in management skills”. Senior MoPHP Policymaker
Graduates raised similar limitations in their responses to the online survey: only 40% reported that Y-FETP helped them in policy development (
Figure 1) and 47% rated their skills as good (
Figure 2). Furthermore, less than 10% of graduates reported that they often engaged in policy development (
Figure 3) and only 30% reported being regularly engaged in program management.
4. Discussion
An important indicator for achieving a training program’s—such as the Y-FETP—intended outcomes and impact the dependence on the graduate’s ability to apply those learned skills at work to support and improve the health system, where the Y-FETP plays a major role in an enriched NHS, as indicated by several evidence-based data. For example, a timely response to an outbreak involves determining the cause of the outbreak, and the population at risk, to allow for effective implementation of control measures, and thus reduce morbidity and mortality. Both the qualitative and quantitative findings showed that the Y-FETP program helped build national epidemiological capacities to respond to outbreaks consistent with the International Health Regulation and GHSA [
17]. Furthermore, the results showed that Y-FETP played a critical role in building national epidemiological capacities through providing residents with theoretical and practical skills. This finding is similar to results from the Central America Field Epidemiology Training Program’s evaluation, which showed that the FETP helped in the development of high-caliber field epidemiologists at various levels of the public health system [
18]. Similarly, evaluation of the United Kingdom’s FETP found a direct impact on capacity building through training fellows and an indirect impact through developing other staff within local teams and bringing insights from other areas [
15].
Our results confirmed the findings from the Indian Ocean FETP evaluation that specified the need for competency-based field training programs to build capacity to support the health system [
6]. This was accomplished through the many epidemiological activities the residents performed, including analyzing and evaluating surveillance system data, outbreak investigation and field studies.
War and the blockade on Yemen disrupted the NHS with the severe shortage of resources and difficulty in bringing international expertise, especially from International agencies. A study in the Eastern Mediterranean region (EMR) showed that the FETPs played a vital role in the region as many countries experienced unrest as a result of wars, political conflicts, massive forced displacement and disease outbreaks [
1]. Our results from the quantitative data supported the EMR study and showed that the Y-FETP built the capacities of its graduates in key field epidemiological activities such as outbreak investigations, surveillance analysis/evaluation, public health communication (reports/presentation), and basic statistical analysis. These important skills were instrumental in responding to the challenges facing NHS such as cholera, diphtheria, and COVID-19 [
1].
Tacker and Buffington defined applied epidemiology as “transforming findings to policy and action” [
19]. The cornerstone in establishing the Y-FETP was to transfer the results of the various activities to evidence-based actions. This evaluation clearly demonstrated that Y-FETP met its objective through utilizing the results of the many epidemiological field activities for evidence-based decisions. There was consensus among all participants that the program played a crucial role in helping MoPHP decision-makers, program directors, and stakeholders to make informed decisions and overcome shortcomings. Such findings are consistent with similar evaluations of FETPs [
1,
16,
17].
Sharing findings from scientific research and field investigation is an important tool for increasing awareness about the health problems and challenges facing the NHS. Being part of the EMPHNET Regional Field Epidemiology Network and the TEPHINET International Field Epidemiology Network, Y-FETP shared knowledge and increased awareness regarding the most challenging global public health problems and threats. The desk review and qualitative and quantitative findings showed that Y-FETP achieved this using published scientific articles in peer-reviewed journals and abstracts that were presented in national, regional, and international conferences. Y-FETP contributed to a better understanding and awareness of Yemen NHS problems and challenges and exposed emerging and reemerging disease threats in Yemen such as COVID-19, diphtheria, and cholera and their containment measures. [
11,
12,
20,
21,
22,
23,
24,
25].
This evaluation identified some gaps that need attention. First, this evaluation revealed that the Y-FETP depends solely on donor support, which is a major challenge for the survival of the program [
26]. Evaluation of the Ghana Field Epidemiology and Laboratory Training Program showed that over the years, the donors, stakeholders, and residents funded the advanced FETP program while the WHO fully funded the frontline FETP program [
26]. A global evaluation found that, of the 19 programs established during 1980–2000 with CDC support, 17 continue to function, graduate trainees, and provide service [
17]. For sustainability purposes and long-term contribution of the FETPs toward strengthening public health, the studies suggested that FETP programs should be attached to the ministries of health or other public health institutions with gradual replacement of the financial and technical support received from external donors and partners [
27]. Therefore, sustainability of Y-FETP requires proper attention from the national health system, exploring different stakeholders and potentially support from graduates. Gradual replacement of donor funds using governmental funds may be difficult in the current situation of war and blockade in Yemen; however, the Y-FETP should consider piloting some innovative approaches such as partnership, introduction of self-sponsorship, and service provision/consultancies.
Furthermore, the Y-FETP needs to widen its national coverage and target all governorates using more coordinating and sensitizing visits to the non-represented governorates to discuss obstacles and overcome possible challenges.
Migration of health professionals from the public health system is a major concern that hinders development in Yemen [
28]. The Y-FETP recommends that the MoPHP employs Y-FETP graduates at the national level of MoPHP and its governorate branches as field epidemiologists or managers of national disease prevention and control programs, and in leadership positions requiring a high degree of analytic capability. Currently, more than 40% of Y-FETP graduates leave the NHS to work with international organizations. Such a departure is usually for better salary or living conditions. Other factors include better working conditions, professional supervision and management, and greater access to education and training opportunities. Therefore, improving public health salaries and working conditions would be useful measures to improve retention.
Since only a small percentage of Y-FETP graduates became engaged in policy development and program management, similar to published results from global and EMR FETP evaluations [
2,
9], the Y-FETP should revisit its curriculum and include these topics.
Finally, this evaluation has strengths. First, it focused on level 3 and 4 of the Kirkpatrick model [
12], i.e., “the degree of applying what was learned” and “the degree to which outcomes occur as a result of the training”, which intend to measure impact. Other strengths include collecting information from different sources (i.e., Y-FETP staff, senior MoPHP policymakers, program directors, and organization staff employees and graduates), and using different tools (i.e., FGD, IDIs, and an online survey) that allowed for triangulation and provided robust findings. Furthermore, because the graduate survey was anonymous and online, this likely decreased social desirability bias.
This evaluation also had some limitations. First, one of the Y-FETP residents conducted this internal evaluation; therefore, subjectivity could be a problem. Second, as the graduates’ capacities were self-assessed using an online survey, there may be an overrating. Thirdly, the evaluation elicited views and opinions from teams already involved in the program and from the program’s stakeholders that may be biased towards more favorable views, but there is no choice because the Y-FETP chooses the programs that it is more involved with, but the Y-FETP needs to coordinate with programs outside of the MoPH, such as climatic, environmental, cultural, and others, to achieve a unified health approach. Finally, this evaluation did not state longer-term outcomes and impacts such as changes, e.g., in morbidity and mortality, brought to the health system. These features will require a longer-term perspective, maybe in the situation of an external evaluation.