Given the significant amount of time children spend at school, the school environment has an important influence on children’s energy balance-related behaviors (EBRBs), i.e., sedentary behavior, physical activity (PA), and nutrition behavior. The World Health Organization advocates school health promotion [1
]. However, not all school health promotion initiatives are successful. Systematic reviews and meta-analyses on the effectiveness of school-based physical activity (PA) and nutrition interventions on children’s energy balance-related behaviors and body mass index (BMI) found mainly mixed or inconclusive results [2
To understand why interventions succeeded or failed, insight into what really happens during implementation is indispensable [12
]. The number of studies regarding the implementation of school-based PA-promoting interventions is currently limited as compared with the number of effectiveness studies [13
]. When school-based health-promoting intervention studies investigate implementation, it is often studied conventionally by assessing fidelity to the standardized intervention components [14
]. This type of evaluation is appropriate for “one size fits all” evidence-based programs.
However, “one size fits all” evidence-based interventions do not take into account contextual differences between settings. In contrast, interventions which allow local adaptation to ensure contextual fit do take these differences in context into account and are considered to be more appropriate, implementable, effective, and ultimately sustainable [15
]. It is recommended that school health-promoting interventions should be sufficiently flexible to fit a specific context [14
], and thus allow local adaptation. Mutual adaptation interventions are interventions in which adaptation of top-down principles and bottom-up development and implementation take place concurrently [16
]. These interventions lead to different outputs and are implemented differently in different settings. To study the implementation of such an intervention and factors influencing implementation, a flexible evaluation approach and sensitivity regarding contextual influences and changes are required [14
In this paper, we evaluated a mutual adaptation physical activity and nutrition intervention that was implemented in primary schools in the Netherlands [22
], i.e., the KEIGAAF intervention. KEIGAAF is a Dutch acronym for “Chances in Eindhoven for a family-based approach by Fontys” (in Dutch, Kansen in EIndhoven voor GezinsAAnpak met Fontys) and refers to a local term for “super cool” [22
]. We studied how KEIGAAF was implemented in primary schools and which contextual factors influenced implementation.
2. Materials and Methods
2.1. Study Design
To study the implementation of KEIGAAF in the intervention schools, a qualitative, multiple-case study was conducted [23
]. This process evaluation was part of a larger study, which also evaluated the effectiveness of KEIGAAF on children’s BMI z-score, physical activity, and nutrition behavior [22
]. The process evaluation was conducted prior to the effectiveness study.
Eight intervention schools were recruited in April and May 2016 in Eindhoven, a city in the south of the Netherlands. These schools were located in low socioeconomic neighborhoods. Eligibility criteria and recruitment strategies are described in detail in the study protocol [22
2.2. The KEIGAAF Intervention
The KEIGAAF intervention was not a prepackaged program, but an approach that consisted of an interplay between top-down and bottom-up influences reinforcing each other in order to optimize the implementation of school-based PA and nutrition activities by ensuring contextual fit [22
]. The overall aim was to create a school environment that stimulates children to be active and have healthy eating behaviors.
The top-down part of the KEIGAAF approach consisted of a steering committee of health behavioral experts and representatives of local organizations, who provided the basic principles of the intervention (see Figure 1
) and supported the bottom-up part, for example, with scientific advice or financial resources. The bottom-up part consisted of local working groups that defined local intervention needs with respect to PA engagement and healthy nutrition and were responsible for the implementation of the intervention.
The working groups were encouraged to follow the steps proposed in the model by Van Kann et al. [24
] which consisted of the following: (1) Compose a working group, (2) define local needs, (3) develop an activity plan, (4) apply for resources (additional ones), (5) implement PA and healthy nutrition-promoting activities, and (6) guarantee sustainability. Although these steps suggest a linear process, in reality it is a dynamic process with multiple feedback loops. The local working groups were supported by health-promoting (HP) advisors who advised the working groups on actions and effective activities. The HP advisors exchanged best practices and served as a link between the steering committee and the working groups.
The working groups were advised on implementing a comprehensive and integrated set of PA and healthy nutrition-promoting activities. School health promotion is considered comprehensive and integrated when children’s health behaviors are promoted through health education, by the school’s physical and social environment, and beyond the school gates (thus also before and after school time) by engaging families and the wider community [1
The KEIGAAF intervention started in April 2016 and lasted until June 2019 [22
]. The intervention period consisted of a preparation period of about one year (April 2016 to April 2017) and an implementation period of two years (May 2017 to June 2018 and September 2018 to June 2019). When referring to year one, year two, and year three, we are referring to the preparation year (2016/2017), and the first (2017/2018) and second (2018/2019) year of implementation, respectively.
2.3. Study Setting and Study Population
The context of the intervention schools served as the study setting. By context, we mean “the set of circumstances or unique factors that surround a particular implementation effort” [26
] (p. 52). To study contextual factors, we used the Consolidated Framework for Implementation Research (CFIR) [26
], which is a tool to study factors that can influence intervention implementation [26
]. It consists of five general domains which interact in complex ways to influence implementation effectiveness. These five domains are characteristics of the outer setting (the economic, political, and social context to which the organization belongs), the inner setting (attributes of structural, political, and cultural context), the individuals involved in the intervention (characteristics of the implementers), the intervention (divided into unadapted and adapted intervention), and the implementation process [26
In this study, the outer setting was considered the external context of the school that could influence implementation, for example, national and local policies or collaborations with external organizations. The inner setting was considered the school environment, for example, involvement of school staff and the principal. The individuals involved in the implementation of the intervention were the working group members, i.e., schoolteachers, external professionals, parents, and the HP advisor.
In addition to the individuals involved in implementation, the study population consisted of all actors in the eight intervention schools, for example, schoolchildren, teachers and parents who were not involved in the working group but were part of the intervention setting. Observations focused on the entire context (i.e., outer and inner setting, and the entire study population). Interviews were conducted with a selection of the study population and members of the steering committee (described below).
2.4. Data Collection
Data were collected in and around the schools. Data collection started in September 2016 and ended in June 2019 (Figure 2
). For this, a flexible data collection approach was applied, i.e., data collection tools were added or removed during implementation to gain the best insights into the implementation process and the contextual influence. Ultimately, multiple qualitative evaluation methods were used to study implementation. The main researcher (SV-J) was involved in the implementation process as HP advisor. This engagement in practice enabled her to gain insight into the implementation process and sense the interplay between top-down and bottom-up influences, while supporting the implementation process [16
]. Intuitive findings of the researcher concerning implementation were confirmed by the use of multiple qualitative measurement tools across multiple stakeholders. The Medical Ethics Committee of the Maastricht University Medical Centre approved the study (METC163027, national number: NL58554.068.16).
2.4.1. Minutes of Working Group Meetings and Participatory Observations
From the start of the intervention until the end of the intervention, minutes were collected of each working group meeting (N = 113) and of the meetings of the steering committee (N = 8). Minutes of the working group meetings were mainly prepared by the HP advisor, and sometimes by another member of the working group (e.g., a teacher or a health professional). The minutes of these meetings provided in-depth information about the implementation plan ranging from informing the principals to developing and implementing plans and the (supporting) role of the steering committee in implementation.
After each working group meeting, the HP advisor made notes about their observations of the meeting (N = 89). Notes could concern the process (e.g., problems encountered by the members during implementation), interactions (e.g., communication between a working group member and a new partner), or other contextual influences (e.g., changes at the municipal level), but also included “soft” measurements, such as the atmosphere during meetings. These participatory observation notes were not shared with the working groups. In addition, a researcher regularly visited the participating schools to observe activities implemented there. Special attention was paid to contextual fit. These observations were recorded as notes. The participatory observation notes mainly served as secondary data to verify results found in the other data sources. They were used to give meaning and explanation to identified processes.
2.4.2. School Scan
At the start of the intervention, the starting situation of the schools concerning the promotion of PA and healthy nutrition was assessed. At the beginning of the preparation year (school year 2016 to 2017), the school principal or a teacher filled in a school scan, which was an online questionnaire. The questionnaire was based on theoretical frameworks concerning the comprehensiveness of primary schools’ efforts towards PA and healthy nutrition promotion [1
]. The school scan assessed the school’s physical education (PE), nutrition education, PA, and healthy nutrition-promoting policies; whether and how the physical school environment stimulated PA and healthy nutrition behavior; the involvement and support of staff; and parental involvement in the school’s PA and healthy nutrition promotion. At the beginning of each school year, the school scan was filled in online by the principal or the chair of the working group (i.e., a teacher).
2.4.3. Timeline Sessions
Initially, the working group members filled in a team climate checklist at the end of the preparation period to assess the climate for innovativeness within the working groups [22
]. The team climate and changes within the team climate were expected to affect the implementation process. However, it did not appear feasible to measure follow-up due to a high participant turnover. Thus, this measure was considered inappropriate for this context. Instead, the narrative timeline technique was used at the end of the first and second year of implementation to evaluate activities implemented in that particular school year and to discuss perceived highlights and perceived failures during implementation with the working groups [29
]. Participants concluded with a discussion about “what to do next”. The data were used to analyze the implementation process and factors influencing implementation. The HP advisor who assisted the working group acted as a participant in the timeline session. Another HP advisor moderated the session. The timeline sessions were recorded, and participants provided oral consent before the start of each session. In total, 16 timeline sessions were conducted, and 60 working group members participated in these sessions. The timeline sessions were conducted at school. Session duration ranged from 39 to 70 min.
2.4.4. Semi-Structured Interviews
In year three, a selective sample of principals (N = 5) and working group chairs (N = 4) was asked to participate in individual interviews to gain more insight into implementation at different levels (operational level and management level) and factors influencing implementation. This sample was chosen based on the diversity in implementation of the KEIGAAF approach. Additionally, members of the steering committee (N = 5) were asked to participate in individual interviews. All participants agreed to participate. The interviews were scheduled at the convenience of the participant and took place at the participant’s place of work. Before conducting the interview, a semi-structured interview guide was developed using the CFIR [26
] (Supplemental File S1
). The interviews were recorded, and participants provided oral consent prior to the interview. The interview duration ranged from 28 to 57 min.
2.5. Data Coding and Analysis
Recorded data were transcribed verbatim. Inductive content analysis was performed when coding the data [30
]. Two researchers (SV-J and AV) coded 10% of the transcribed interviews independently (one semi-structured interview and two timeline evaluations). These interviews were chosen at random. The two researchers discussed emerging themes and concepts, as well as constructed a preliminary codebook. After agreeing on the first version of the codebook, the first author (SV-J) continued the coding of the remaining interviews and adapted the codebook accordingly. Adaptations to the codebook were discussed with the second researcher. Subsequently, the codes were linked to the five domains of the CFIR [26
]. Codes represented information concerning the implementation process (research Question 1), intervention factors, or the contextual factors, i.e., outer setting, inner setting, and characteristics of the individuals (research Question 2). Examples of codes linked to the process are “internal communication”, “modus operandi working group”, and “collaborations”. Examples of codes linked to the intervention are “KEIGAAF research” and “added value KEIGAAF’. Examples of codes linked to the context are “modus operandi school board” (outer setting), “school staff support” (inner setting), and “characteristics working group member” (individuals). Throughout the analysis, an iterative process was applied. The interpretation of the results was compared with the verbatim data. Data coding and analysis were supported by the use of NVivo 12.
We studied how a mutual adaptation intervention aimed at promoting children’s physical activity and nutrition behavior was implemented in eight primary schools located in low socioeconomic neighborhoods. Although the eight schools were located in the same municipality, the schools differed greatly in the implementation of PA and healthy nutrition-promoting activities. This was caused by the high level of bottom-up design of the intervention and differences in the contexts of the schools. Secondly, we studied which contextual factors influenced implementation. Schools had differing starting situations concerning PA and healthy nutrition promotion andthey differed in received parental support, staff and principal support, and in employee turnover. Moreover, differences in characteristics related to the working group members, i.e., interpretation of the approach, the degree of practice-oriented thinking, the presence of one or more champions, and the dynamics within the working group, but also the differing degrees of influence of factors within the outer setting (i.e., support from the school board, national health-promoting trends, and the presence of and the capacity to collaborate with potential partners) resulted in different implementation processes. Other studies also found these contextual factors to facilitate and hinder implementation of school health-promoting interventions [33
The implementation of the KEIGAAF intervention was the result of an interplay between top-down influences and bottom-up development and implementation which led to adaptation to the local context [16
]. This mutual adaptation was a key element of KEIGAAF, with the feedback loops and the HP advisors playing a crucial role in this adaptation. For example, feedback on the schools’ physical and social health-promoting environment was used to define local needs and behavioral outcome measures to catalyze implementation processes. However, the latter appeared inappropriate for the schools. Reporting short-term, null effects of the first interventions on behavioral outcomes was considered demotivating by the working groups. Instead, children’s enjoyment was a preferred indicator for implementation success by the working groups. Therefore, it would have been better to evaluate and report back on children’s enjoyment and preferences [42
]. To support working groups in implementing context-appropriate activities, the advisor had to find the right balance between top-down and bottom-up influences for each school. This meant that for the one working group, the HP advisor had to take a more active role in implementation, while for the other working group merely informative support was appropriate. For example, a working group that experienced difficulties with the implementation of the intervention outside of school hours and the school premises due to a lack of partners or inability to find potential partners required support from the HP advisor to find, inform, and connect partners to the working group. Whereas another working group already had a strong external network and required a different type of support, for example, assistance in obtaining parental support. In that case, the HP advisor supported finding the right strategies to involve parents. Thus, the HP advisor also played an important role in overcoming barriers experienced during implementation. However, some barriers were difficult to overcome, for example, working groups that continued to misinterpret the KEIGAAF approach. In such a case, the HP advisor eventually decided to decrease support because it was not the right time for the working group and school to implement the intervention. Since this balance between top-down influences and bottom-up design differed per school and throughout the process, the advisor had to have adaptive management skills and be context sensitive. Being context sensitive meant having all senses open and observing physical structures and organizational dynamics [43
] prior to and during intervention implementation. To achieve this, engagement in practice was essential [16
Interestingly, the need for mutual adaptation between top-down intervention principles and bottom-up changes in school-based health promotion had already been acknowledged in 1976 [44
]. However, applying an adaptive intervention instead of implementing a predefined set of activities is still not common practice in school-based interventions. This can be explained by a certain degree of “lack of control” as well as fear of “cherry picking” (e.g., implementing convenient intervention elements) affecting the key components of an intervention. However, intervention adaptation (i.e., local tailoring) is not a threat to intervention effectiveness when intervention functionality is maintained [17
]. Adaptation is even considered necessary to maximize the effects [17
There is currently no guidance on how to adapt evidence-based interventions for new contexts [45
], but we know that the ability to mutually adapt top-down effective principles and bottom-up changes “is a special niche and it’s not easy to do” [16
] (p. 179). It takes skills, perseverance, and time. Informing the schools about an intervention with an unknown output was a challenging process which had to be repeated multiple times throughout implementation. The HP advisor had to familiarize themself with the school and had to gain the trust of the working group [33
]. Both investments were essential, but time-consuming. This is outweighed by the high potential of sustainability of the output of the intervention because of a high perceived ownership by the schools due to contextual fit [16
]. Important lessons we have learned from implementing the KEIGAAF intervention are listed in Figure 5
Strengths and Limitations of the Study
This is one of the few studies examining the implementation of school-based PA and nutrition-promoting activities with a focus on context [46
]. The combination of the use of different evaluation tools, such as the timeline sessions and the school scan, and our continuous presence in practice provided us with a deep understanding of this context. The engagement of the researcher in practice was a strength of this study but can also be seen as a limitation because of a loss of objectivity. To increase the objectivity of the interpretation of the results, a second researcher was involved in the data coding and analysis. The flexibility in the research approach was another strength of the study [19
]. For example, initially the team climate in the working groups (e.g., a safe environment to share ideas) was measured using validated questionnaires at the end of the preparation year with the intention of repeating this measurement at the end of years two and three, because this outcome was expected to impact the process. However, follow-up was impossible due to a high participant turnover. Thus, this measure was not repeated. Additionally, more insight into the process was needed and the minutes of the meetings and observations did not provide enough detail, thus the timeline sessions were used.
There are also some limitations to this study that should be mentioned. First, it was not possible to fully assess the implementation process (e.g., full insight into the integration of the activities in school or the spill-over effects of activities from school to home) or to study every contextual factor (i.e., contextual factors broader than the school context) due to time and resource limitations. In addition, we aimed to minimize participant burden [19
]. Second, the results regarding the process and contextual factors are specific for this intervention region and are not generalizable to other regions. This is an inherent aspect of the KEIGAAF intervention. When implementing the KEIGAAF intervention in other regions, it is up to the researchers and HP advisors to gain insight into the contextual factors of that region that potentially influence implementation. A final limitation of the study was that the duration was relatively short to be able to measure sustainability. The KEIGAAF approach was intended to be actively developed and implemented in schools with the support of the HP advisor in years one and two. In year three, the support of the HP advisor was intended to diminish. That last year would then give us a first insight into the potential sustainability. However, this was not realistic because implementation was a continuous process of trial-and-error, and integration and adjustment of plans, activities, and actions required more time than the three years of evaluation. Only preliminary insights into the sustainability of the output of the approach could be acquired. Further research is needed concerning the sustainability of the output of the KEIGAAF intervention, especially when a context requires a more top-down approach.