Effects of a Dutch Family Literacy Program: The Role of Implementation

Abstract

It is hypothesized that variability found in the effects of family literacy programs results from differences in implementation by parents. In this study, the implementation and effects of a Dutch program were examined in a sample of 207 kindergarteners (mean age at pre-test: 64 months). No main intervention effects on children’s literacy development were found. The quality of implementation proved to be higher for high-SES and native Dutch (speaking) parents than for low-SES, ethnic-minority parents with other home languages. Parent SES, ethnic-minority status, and home language did not moderate the program effects on child language scores and the program failed to impact targeted parental attributes, namely, the home literacy environment and parent self-efficacy. Finally, children’s development proved unrelated to implementation variables. Our results stress the importance of delivery for adequate implementation.

1. Introduction

Recognizing the strong influence of parents as first educators of their children, Family Literacy Programs (FLPs) aim to promote children’s literacy development by stimulating their Home Literacy Environments (HLEs) [1,2]. Hannon [3] defines FLPs as ‘programmes to teach literacy that acknowledge and make use of learner’s family relationships and engagement in family literacy practices’ (p. 100). Although this definition encompasses different sorts of programs [4,5] many interventions encourage parents to engage in joint literacy activities with their child. There appears to be substantial variability in FLP effects on children’s literacy skills. Since it is hypothesized that this variability is partly due to differences in parental implementation, this study tests the role that different aspects of parental implementation play in program effects.

1.1. Variability in Effects of FLPs

Over the past decades, various FLPs have been developed and many have been the subject of effect studies. These effect studies were summarized in a number of meta-analyses, showing that FLPs are generally effective, although there is great variability in effect sizes. Meta-analyses showed significant but small effects of FLPs on children’s literacy outcomes [6,7]. Comparing the impact of different types of FLPs, Sénéchal and Young [8] found tutoring programs, in which parents teach literacy skills such as letter knowledge, to yield large effects on reading acquisition, whereas shared reading programs generally had trivial effects. Mol, Bus, de Jong, and Smeets [9] summarized the effects of Dialogic Reading programs on vocabulary development, examining the added value of this approach—which requires the child’s active participation—above and beyond typical shared reading. They found a medium mean effect in favor of Dialogic Reading.

The characteristics of the target population appear to be one source of variability in program outcomes. Two of the afore-mentioned meta-analyses provided evidence for differential effects of shared reading programs for different subgroups of children. Mol et al. [9] found that for children who were at risk of language and literacy impairments (based on family income or maternal education), the effects of Dialogic Reading on vocabulary skills were trivial compared to the effects for non-at risk children (d = 0.13 vs. d = 0.53). Manz et al. [6] reported a significant difference in effect sizes between Caucasian and ethnic-minority families (d = 0.64 versus d = 0.16), as well as in effect sizes between middle- or high-income and low-income families (d = 0.39 versus d = 0.14). An overview of different meta-analyses [10] suggests that this raises doubts about whether low-SES and ethnic-minority families are capable of executing FLPs optimally and, consequently, that studies analyzing the effects of FLPs in such groups should take program implementation into account.

The quality of program implementation by parents may vary for different reasons. Many shared reading programs, for instance, target parental strategies such as scaffolding, which require parents to be sensitive and responsive to their children’s input [11]. Previous research has shown that low-SES parents demonstrate less of this behavior compared to high-SES parents [12,13]. Another possible reason is that ethnic-minority families are hampered in conducting program activities by limited language proficiency. Often, FLPs are delivered in the majority language, which might lead to the suboptimal implementation of these programs in families with other home languages [14]. A shortcoming in many intervention studies on FLPs so far is that they rarely include measures of program implementation in effect analyses [6,10,15].

1.2. Defining Implementation

Implementation is assumed to play an important role in the effectiveness of any intervention program [16,17,18]. In their landmark review, Durlak and DuPre [16] analyzed over 500 studies on (mental) health prevention and promotion programs for children and adolescents and found strong support for the importance of implementation in determining program effects. Summarizing the outcomes of five meta-analyses, the authors concluded that good implementation generally results in effect sizes two to three times larger than when implementation is poor. They therefore state that ‘the assessment of implementation is an absolute necessity in program evaluations. Evaluations that lack carefully collected information on implementation are flawed and incomplete’ [16] (p. 340).

In the current study, we build on a framework proposed by Powell and Carey [19] to systematically analyze the implementation of FLPs. This framework consists of three main components, two of which focus on parental behaviors: receipt and enactment. Both components contain a quality and a quantity dimension. Receipt refers to parent engagement in training and program activities. Attendance at training sessions is an example of a measure of receipt quantity, whereas quality can be assessed by parents’ use of targeted program strategies, understanding of program content, and their engagement during program activities with their child. Enactment pertains to the degree to which participants use the gained knowledge and skills in their day-to-day life. Are parents able to transfer the learned program strategies to activities outside of the intervention? Enactment quality refers to the quality of parent–child interaction during reading or other targeted activities outside program time or after the intervention has ended. It also includes parents’ intentions to change their behavior as a result of the intervention and changes in their sense of self-efficacy in supporting their child’s learning. The quantity of enactment pertains to the frequency of reading or other targeted activities outside program time or after the intervention has ended. For sustained program effects, it is important that parents are able to maintain their use of newly learned skills in order to reach more long-term goals such as improving children’s literacy skills.

Although both variables pertain to parental behavior, they can be argued to take on different roles in program effectiveness. Receipt pertains to parental behavior during program activities and can thus only be assessed in families taking part in the intervention. Variability in receipt is then assumed to predict variability in child outcomes in participating families [20]. Enactment can, in essence, be seen as a mediator of intervention effects on child development: FLPs are hypothesized to induce changes in parental behavior outside program time and such changes are assumed to contribute to children’s literacy development [21,22]. The implication of this view is that enactment variables should be assessed both in families that participate in the intervention and control families that do not; only then can the hypothesized mediation effect be tested.

Delivery is the third component in Powell and Carey’s [19] framework and refers to the transfer of main program contents from trainers to parents. The quantity dimension of delivery involves the dosage of parent training (e.g., number and duration of training sessions), whereas the quality dimension reflects the way program contents are communicated to parents. Because the current study focuses on parental implementation, delivery is not included in our analyses, although we do assess it as an indicator of treatment fidelity (see Method).

Recently, de la Rie et al. [15] reviewed the available research on implementation of FLPs and its relation to program effects. The authors analyzed 46 studies and found that information on implementation varied in breadth and quality: Almost all studies provided information on parents’ quantitative engagement in programs (i.e., receipt quantity), but fewer studies reported about quality of engagement (i.e., receipt quality), and transfer to daily life (i.e., enactment). The relationships between implementation and FLP effects remained largely unexplored. Moreover, studies that did analyze this relationship reported inconsistent findings. Some studies found relationships between implementation and effects [20,22], whereas others did not [23,24]. None of the included studies examined the quality and quantity dimensions of both receipt and enactment, as well as relationships among implementation variables and program effects. In conclusion, a comprehensive approach in measuring implementation seems to be lacking, even though this can provide crucial information on how to improve program effectiveness [16].

1.3. The Current Study

In the current study, we examined the effectiveness of ‘Early Education at Home’ (EEH) [25], an FLP that is conducted in Dutch primary schools and that serves a diverse population of families in terms of SES, ethnic background, and home language. In light of the hypothesized implementation issues in low SES and ethnic minority families, we examined whether parental SES, ethnic-minority status, and home language are related to quality and quantity of receipt and enactment of EEH. Because implementation issues in these groups of families are assumed to hamper program effects, we first tested whether program effects were moderated by parent background variables (SES, ethnic-minority status, home language). Subsequently, we tested whether the quality and quantity of both receipt and enactment were associated with program outcomes. As argued in the introduction, receipt variables are included in our analyses as predictors of experimental children’s growth in language and literacy skills. Enactment was measured in both conditions and treated as a mediator of intervention effects.

1.4. Research Questions

The following research questions are addressed:

• Does EEH positively affect children’s language and literacy skills?
• What are the relationships among parental SES, ethnic-minority status, and home language, and implementation of EEH?
• Are effects of EEH moderated by parental SES, ethnic-minority status, and home language?
• Do receipt variables (quantity and quality of parental engagement in the intervention) predict EEH children’s growth in language and literacy skills?
• Do enactment variables (HLE, parents’ sense of self-efficacy, and the quality of parents’ behavior and language) mediate the effects of EEH on children’s language and literacy development?

1.5. Hypotheses

Hypothesis 1.

EEH positively affects children’s language and literacy skills.

Hypothesis 2.

Parents of lower SES, ethnic-minority parents and parents with a home language other than the majority language show lower implementation quality. We base this hypothesis on the notion that FLPs are often not well tailored to the needs of at-risk families [6].

Hypothesis 3.

Parents of lower SES, ethnic-minority parents and parents with a home language other than the majority language show lower implementation quality. We base this hypothesis on the notion that FLPs are often not well tailored to the needs of at-risk families [6].

Hypothesis 4.

As can be expected from the wider implementation quality literature [16,19], we hypothesize that receipt variables will be positively associated with children’s growth in language and literacy skills.

Hypothesis 5.

In line with the literature on our selected enactment variables [26,27], as well as with program theory [25], we expect program effects to be mediated by the HLE, parents’ sense of self-efficacy, and the quality of parents’ behavior and language while they engage in literacy related activities with their child.

2. Materials and Methods

2.1. Sample

Primary schools in the Western part of The Netherlands were invited to participate by letters and subsequent telephone calls, as well as by posting a call in a digital community for kindergarten teachers. Although teachers self-selected participation in the experimental condition, the comparability of experimental and control conditions was maximized by including both an experimental and a control class from the same school, in order to minimize school effects. The study involved a total of 7 schools and 27 kindergarten teachers. On average the participating teachers were 49 years of age, ranging from 23–59 years. All 18 participating classes were second year kindergarten classes, with pupils aged between 4 and 6 years old. In six of the schools, two classes took part, whereas one larger school participated with six classes (3 experimental and 3 control classes). See

Table 1. Overview of participants.

	Control Group	Intervention Group a	Total
Schools	7	7	7
Classes	9	9	18
Teachers	13	14	27
Kindergartners and their parents	98	119	217

Note: ^a 7 out of the 14 teachers that were teaching in intervention group delivered EEH to parents and children.

for an overview of participants (number of schools, classes, teachers, children and their parents). Teachers delivered the intervention to the parents of their pupils. In the larger school, one teacher delivered the intervention to three experimental classes. None of the teachers had prior experience working with EEH.

All parents were informed about the study by the school through a letter and they could indicate if they did not wish to participate. None of the parents refused participation. All children in the selected classes participated in the study, with three exceptions: two children who had a twin in the other condition (to prevent bias from a control group child being exposed to the intervention), and one child with Down’s syndrome. In total, parents of 217 children from 18 classes agreed to participate in the study; 119 children participated in the intervention (9 classes) and 98 were in the control condition (9 classes). Children in both the intervention group and the control had a mean age of 64 months at pre-test (range intervention group: 57–76 months; range control group: 57–75 months). In the intervention group, 51% of the sample consisted of girls, whereas for the control group, this was the case for 49% of the participants.

We checked for significant differences between the experimental and control conditions on children’s pre-test language and literacy scores, using independent samples t-tests, and none were found. Regarding relevant background characteristics of children (gender, age) and parents (SES, migration background, home language), and richness of the HLE and parent self-efficacy (PSE), we again found no significant differences between the experimental and control group participants at pre-test, suggesting that the two conditions were comparable on important characteristics.

We asked the parents who considered themselves to be most involved in the child’s upbringing to fill in a parent questionnaire (79% mothers, 20% fathers, 1% foster parents or extended family members). All but three parents completed this questionnaire. For parent–child observations, we asked parents who were most involved in conducting EEH with the child (experimental group) or in the upbringing of the child (control group) to participate. Parents’ characteristics are presented in

Table 2. Parent characteristics for final sample (n = 207).

Parent Characteristics	% of Control Group	% of Intervention Group	% of Total Sample
Education
No education	2.2	2.7	2.5
Primary school	3.4	0.9	2.0
Secondary education (12–15 years of age)	2.2	4.4	3.5
Secondary education (15–18 years of age)	12.4	15.0	13.9
Senior secondary vocational education	38.2	37.2	37.6
College/university degree	41.6	39.8	40.6 a
Ethnic-minority status	38.9	36.8	37.7
Home language
More proficient in Dutch	64.8	62.7	63.6
Equally proficient in Dutch and other language	11.5	14.5	13.1
More proficient in other language	23.9	22.7	23.2

Note: ^a although EEH is in principle meant for children from low-SES families, the sample is heterogeneous. This reflects the program’s whole-class approach, as mentioned previously.

.

During the school year, ten pupils left the study as a result of their families moving out of the area, or switching schools for another reason (e.g., because the child needed special education), decreasing the sample size to a total of 207 participants at the end of the school year (115 experimental participants; 92 controls). Although there was 4.6% attrition from the original sample, bivariate correlations suggested no significant differences between dropouts and the remainder of our sample on key background characteristics (i.e., parental educational attainment and migration background, gender of the child). There was one exception: home language was significantly related to drop-out (r = 0.202, p < 0.01), indicating that native Dutch speaking parents remained in the sample more often compared to non-native Dutch speakers.

2.2. Measures

Language and literacy skills. Three measures were included to assess child language and emergent literacy skills. First, we used a standardized language test [28] that was part of the participating schools’ student monitoring system and that was administered (pre- and post-test) by teachers in a regular class setting (approx. 30 min.). This test included measures of receptive vocabulary, critical listening, phonemic and rhyme awareness, print knowledge, and auditory synthesis abilities. To measure receptive vocabulary, for example, children were asked to select out of four images the image corresponding to a word that was read aloud by the teacher. Rhyme awareness was measured, for example, by the teacher reading a word aloud followed by four other words and children were asked which of the final four words had the same starting sound as the first word. Cronbach’s alpha for the total score is 0.87 [28]. Information on children’s emergent literacy skills was additionally obtained by teacher ratings via a questionnaire. We used an emergent literacy instrument with a five-point Likert scale based on Van Steensel [26], which consists of three subscales with a total of 15 items: oral language, phonological awareness, and print knowledge. To assess oral language, teachers were, for instance, asked to indicate to what extent a child could tell a coherent story. For our data, the composite alpha was 0.97 (averaged across pre- and post-test).

Additional information on children’s curriculum-based vocabulary was obtained from a receptive vocabulary test (similar in format to the Peabody Picture Vocabulary Test) designed for this study by the first and third author. We incorporated 43 words from EEH program themes. Children were tested individually (approx. 5 min.) by a research assistant, in a quiet one-on-one setting. Cronbach’s alpha for this test was 0.71 (averaged across pre- and post-test).

Implementation. For an overview of our measurements of program implementation following the conceptual framework of Powell and Carey [19], see

Table 3. Receipt and enactment [19].

Element of Implementation	Dimension	Aspect
Receipt	Quantity	Attendance at training sessions
		Number of diaries handed in; activities completed
	Quality	Quality of parent behavior and language during a program activity (Program Activity; shared reading)
Enactment	Quantity	Frequency of literacy-related activities outside program time
	Quality	Quality of parent behavior and language during a non-program activity (Non-Program Activity; prompting board)
		Parent self-efficacy in helping the child succeed in school

.

Regarding receipt quantity, attendance at group meetings was registered for each session by the teachers who delivered the program to parents. Additionally, parents were given diaries for every activity booklet, in which they were instructed to register completed program activities on a checklist. We counted the number of diaries handed in by parents and the number of activities completed, based on what parents reported in the diaries.

All other implementation measures were administered twice: at the beginning and at the end of the intervention period (see

Table 4. Study overview—intervention and measurements.

	T1	T2	T3	T4
	June–August 2014	September–October 2014	January 2015	June 2015
EEH	Teacher training	start Early Education at Home (EEH) Teacher coaching		end EEH
CHILD OUTCOMES	Language test [28]	Emergent Literacy scale [31] Receptive vocabulary test	Language test	Language test Emergent Literacy scale Receptive vocabulary test
PARENT QUESTIONNAIRE		Enactment Home Literacy Environment (HLE) [31,32] Parent self-efficacy [29]		Enactment HLE Parent self-efficacy
PARENT–CHILD OBSERVATIONS		Receipt & enactment Quality of parent behavior and language during a Non-Program Activity (NPA) and a Program Activity (PA) [29]		Receipt & enactment Quality of parent behavior and language during a NPA and a PA

). The quality dimension of intervention receipt was measured by observing parent–child interactions during a program activity (shared reading) at pre- and post-test and scoring the quality of parents’ behaviors and language. Most observations took place at school, whereas a few parents preferred to be observed in their home. In order to rate the observations of both the program and non-program activities (see ‘Enactment quality’ below for the non-program activity) we used an observation scheme developed by Kenney [29] (later used by Mol & Neuman [30]), which we translated from English to Dutch. This observation scheme includes the following six categories of parent behavior features and language: Labeling, generalizing, repetition and paraphrasing, scaffolding, fostering child autonomy, and quantity and variety of language (see Appendix A for examples). These are all aspects targeted in EEH and were scored on a scale ranging from 1 (not at all characteristic) to 4 (very characteristic). Cronbach’s alpha reliabilities of the scales were 0.83 for the non-program activity (NPA) and 0.87 for the program activity (PA).

Enactment quantity was measured by an HLE-questionnaire consisting of eight items derived from Van Steensel [31]. Van Steensel found support for the construct and predictive validity of the HLE questionnaire in a previous study of a comparable sample. The questionnaires were available in Dutch, English, Arabic, Turkish, and Polish. The parents were asked to indicate how many times they engaged in literacy-related activities with their child on a scale ranging from 1 (daily) to 4 (almost never/never). The following activities were included: shared reading, going to the library, singing songs, writing alphabet letters, storytelling, visiting a bookstore, playing educational (online) games, and watching educational TV shows together. The Cronbach’s alpha coefficient for this scale was 0.63.

Enactment quality was measured by scoring the quality of parents’ behaviors and language during a non-program activity (NPA), which was a prompting board task very much like those in EEH, but without written (program) instructions. A prompting board is a complex picture, suggesting a sequence of events, and is designed to elicit child speech. We selected a picture of a busy park on a summer day (pre-test) and a picture of a zoo (post-test) from an existing prompting board book [33]. We invited parents to engage in a conversation with their child as they would normally do when looking at pictures together.

Because of the large number of observations (n = 156 at pre-test and n = 148 at post-test for non-program and program activities combined), ratings were given by a total of 12 coders during the observations (one coder per parent–child dyad). In order to assess interobserver agreement, Cronbach’s alphas were calculated for the NPA and the PA on both the pre- and post-test, based on a random selection of 12% of all ratings. This selection of observations was double-coded by the first author, using the video-recordings that were made during the observations. The average alphas across the observed categories of parent behavior and language for the NPA and the PA ($\mathsf{\alpha }$ = 0.87) at pre-test ($\mathsf{\alpha }$ = 0.86 and 0.87, respectively) and posttest ($\mathsf{\alpha }$ = 0.78 and 0.83) indicated sufficient agreement. As our sample contained parents with limited Dutch language proficiency, observations were conducted in parents’ self-reported home language, with the aid of bilingual research assistants who spoke one or more of the following languages: Arabic, Berber, English, Polish, and Turkish.

To measure parental self-efficacy (PSE) as an aspect of enactment, we administered the ‘How to help my child succeed in school scale’ at pre- and post-test. This scale was developed by Hoover-Dempsey and colleagues [26,32] and consists of 12 items that measure parents’ perceptions of personal efficacy, specifically in relation to supporting their children’s school success. This scale contains items such as ‘I know how to help my child do well in school’ and ‘If I try hard, I can get through to my child even when he or she has trouble understanding something.’ Parents were asked to rate their sense of efficacy per item on a 5-point scale ranging from 1 (strongly disagree) to 5 (strongly agree). Cronbach’s alpha for this scale was 0.76.

Parent and child background variables. To obtain relevant background information from parents and answer our second and third research question, we added questions to our parent questionnaire pertaining to SES, ethnic-minority status, and home language. SES was operationalized as the highest level of education that parents had completed. Ethnic-minority status was distilled from parents’ country of birth. Home language was operationalized as the parents’ best oral language. Furthermore, we asked teachers to provide us with information regarding children’s age and gender.

2.3. Procedure

Table 4 provides information on the overall planning of the study. The experimental teachers were trained and coached four times from June to October 2014. The measurements took place at four points in time. The pre-tests were spread across two periods: the language test was administered in June 2014 (T1), since this is the administration time prescribed [28], whereas the other measures were administered in September and October 2014 (T2), closer to the start of the intervention period (September 2014). Subsequently, data were collected half-way through the school year (T3: January 2015), and at post-test (T4: June 2015).

2.4. Intervention

EEH is a government-funded program conducted in major cities and suburban districts across The Netherlands, Belgium, and Germany. EEH is mostly conducted in schools with many children from low-educated and/or ethnic-minority families, although—because the program often targets whole classes—higher-educated, native Dutch families also take part. EEH involves literacy-related activities such as shared reading, prompting board activities [34] and arts and crafts activities which parents are stimulated to conduct with their child at home. Activities can also be part of daily routines outside the home, such as when parents take their child on a walk to the park and discuss what kind of things and animals can be found there (such as leaves, squirrels). Parents are instructed to pose stimulating questions (such as “during what season are the leaves falling and why do you think that happens?”) and, for example, gather objects to bring home and use to create art or a drawing (such as leaves and beechnuts). During playful activities such as these, parents can stimulate their child’s literacy development by exposing the child to sophisticated vocabulary, abstract language, carefully adjusted to the child’s developmental level. Parents are trained by their child’s teacher during group meetings at school in which literacy activities are discussed, modeled and role-played. EEH is assumed to affect children’s literacy outcomes by means of improving both the frequency and the quality of literacy related activities in the home. With respect to the quality of shared activities, stimulating child autonomy, variety of language, and out-of-context language (also referred to as abstract language in the literature) are targeted. In addition, parent training is assumed to increase parents’ self-efficacy. By enabling parents to create successful learning experiences with their child, they are expected to gain confidence in their role.

Materials. Participants in EEH are provided with a colorful bag holding a multi-order with activity booklets (one for each theme, with a total of seven themes per year) which include eight literacy-related activities and instructions, as well as materials for conducting these activities, such as storybooks, prompting boards [35], and materials for arts and crafts (e.g., colored paper, crayons, paint, scissors, etc.). The storybooks used in the intervention were written specifically for this age group by well-known (Dutch) children’s book writers. Each activity comes with a sheet of instructions for parental guidance and suggestions for questions aimed to trigger stimulating parent–child interactions, characterized by responsiveness, open-ended questions, scaffolding, and exposure to new vocabulary. As a considerable part of the target group of EEH consists of children of ethnic-minority parents who are more proficient in other languages than Dutch, some materials (i.e., storybooks) are also available in a selection of other languages (i.e., Arabic, English, and Turkish).

Teacher training. The teachers who delivered the program to parents were trained by the first author in two phases. Phase 1 was a three-hour session in which teachers were instructed on the specific contents of EEH, and on delivering the program to parents. The teachers were trained in using four techniques: explaining activities to parents in an interactive manner, modeling interaction strategies, conducting program activities together with all attending parents, and role-play (i.e., enacting activities with a colleague and/or parent(s), where one plays the parent and another the child). They were trained to invite parents to actively share their experiences with the program during the meetings. Finally, the teachers were asked to provide parents with ideas for them to transfer skills mastered during the intervention period to their daily lives (enactment). This included suggestions for turning a regular daily activity, such as shopping, into a learning experience. Parents could, for example, be encouraged to discuss pieces of clothing with their child, and to ask open-ended questions, such as ‘What pieces of clothing are suitable for winters?’.

Additionally, the teachers were stimulated to adapt their instructions to low-educated and low-literate parents, and parents with limited Dutch language proficiency, through the use of pictures, repetition, monitoring parents’ understanding, and, when possible, allowing time for parents to translate for others. EEH assumes that parents are best able to support their children’s development by using the language they are most proficient in, and that knowledge and skills acquired in the first language can be transferred to the second language [34]. Hence, teachers were encouraged to stimulate parents with limited Dutch proficiency to make use of the materials available in other languages and conduct activities in their home language.

After this first training session, the intervention commenced. Phase 2 of the teacher training consisted of coaching. After the second and the third parent meeting, which were observed by the first author, teachers were provided with immediate feedback regarding their performance based on these observations (1.5 h per session).

Parent meetings. Parents in the intervention group were trained by their child’s teacher. Teachers were requested to organize seven six-weekly group meetings, lasting between 60 and 90 min each. Teachers worked from a scripted outline to ensure fidelity across schools. The first part of a standard EEH parent meeting is dedicated to informing parents about what children have been learning in class during the previous period, and what they will be learning during the upcoming period. During the second part of the meeting, teachers evaluated the activities that parents completed with their child over the preceding period, in order to identify difficulties and suggestions for improvement for upcoming program themes. In the final part, teachers provided parents with an overview of the activities in the new workbook and explained how to conduct these activities.

Treatment fidelity (delivery). Adherence to the proposed number and duration of parent meetings was checked by contacting experimental group teachers after each planned meeting, asking them to provide basic quantitative information. Almost all teachers were able to organize all intended seven meetings. One teacher organized six meetings. The average duration of group meetings across all schools was 52 min, with quite a large range (15–80 min), suggesting that, generally, schools did not stay within the range that is prescribed by the program (60–90 min).

In order to assess the quality of intervention delivery, the first author observed three out of seven parent meetings at each participating school, using a checklist to assess adherence to program guidelines. This checklist entailed topics such as evaluation of the previous EEH theme, use of the trained techniques (e.g., role-play, modeling), and use of open questions and concrete examples. Overall, the quality of delivery was quite high across schools, with one exception. In this school the teacher failed to adequately address the new theme in class and the activities for the upcoming EEH theme in all of the observed meetings. In the remaining schools, most meetings were in line with program guidelines. Nonetheless, all teachers largely ignored transfer of program skills to daily situations outside of program time, with one exception. Regarding explanation of the upcoming EEH theme, three of the proposed techniques—modeling, enacting, and role-play—were hardly ever used by any of the teachers.

Control group. The control group was a “business as usual” control group, meaning that the children in the control group followed the same school curriculum as the experimental group children. However, there were no family literacy type programs offered to the control group children. One of the inclusion criteria for participation in this study was that the school was not working with a family literacy or other type of parental involvement program in which systematic parent–child activities and training for parents were offered.

2.5. Analyses

We estimated, a priori, that a sample of 128 children was needed to test the intervention effects with a two-sided test, an alpha of 0.05 and a statistical power 0.80 [36]. The power analyses were based on the overall moderate effect size (Cohen’s d = 0.50) of FLPs found in recent meta-analyses [6,9] and were conducted in in G*Power Version 3.1.9.2 using a two-sided t-test [37]. For all research questions we employed regression analyses using the program MLWin Version 2.36 [38]. When significantly related to outcome measures, relevant background variables (child gender, child age, parent SES, parent ethnic-minority status, and home language) were added to the models as covariates. Our data are hierarchical, that is, measurements are nested within pupils, pupils are nested within classes, and classes are nested within schools. Because of this hierarchical structure, we first of all tested for significant variance on the upper levels, to determine whether or not we should employ multi-level analyses. For each set of outcome measures, which differed per research question, decisions were made regarding the most appropriate strategy to model growth (growth model, pre-test as covariate, or change scores).

Research Question 1 and 3 involved effects on children’s development, and thus included language, emergent literacy and receptive vocabulary as outcome measures. To analyze language scores we fitted a growth model to the data, as we had three measurement points (see Table 4). Exploration of intercept-only models showed significant variance in language scores on all four levels (see Appendix B,

Table A1. Exploration Multi-level Structure—Language Development.

Model	0	1	2
Fixed part
Intercept	45.428 (0.973)	45.498 (1.267)	44.949 (1.733)
Time (gm)	7.610 (0.337)	7.615 (0.337)	7.610 (0.336)
Random part (variances)
Repeated measures	43.639 (3.143)	43.645 (3.139)	43.637 (3.138)
Pupil	93.697 (10.626)	82.726 (10.016)	82.567 (9.987)
Class		12.160 (6.970)	0.921 (4.008)
School			13.630 (9.701)
Deviance	4404.448	4397.702	4393.679
Reference model		0	1
Fit improvement		χ2 = 6.746 df = 1 p < 0.01	χ2 = 4.023 df = 1 p < 0.05

Note: N repeated measures = 603; N pupils = 217; N classes = 18; N schools = 7. gm = grand mean centered.

). Hence, we proceeded with a four-level growth model. Emergent literacy and receptive vocabulary were measured at two time points. As we were dealing with: (1) a quasi-experimental setting without randomization, and (2) existing groups (i.e., classes), we used change scores to conduct these analyses [39]. Because we found significant variance in change scores for emergent literacy on the class level, we proceeded with a two-level model (pupils and classes; see Appendix B,

Table A3. Exploration Multi-level Structure—Literacy Development.

Model	0	1	2
Fixed part
Intercept	0.868 (0.038)	0.839 (0.078)	0.834 (0.108)
Random part (variances)
Pupil	0.294 (0.029)	0.192 (0.037)	0.192 (0.020)
Class		0.091 (0.020)	0.051 (0.029)
School			0.052 (0.045)
Deviance	333.732	278.619	277.132
Reference model		0	1
Fit improvement		χ2 = 55.11 df = 1 p < 0.001	χ2 = 1.487 df = 1 p = n.s.

Note: N pupils = 207; N classes = 18; N schools = 7. n.s. = non-significant.

). We found no significant variance on the class- or school-level for receptive vocabulary, and hence the analyses involving this outcome measure were conducted at uni-level (see Appendix B,

Table A5. Exploration Multi-level Structure—Vocabulary Development.

Model	0	1	2
Fixed part
Intercept	4.400 (0.281)	4.450 (0.345)	4.450 (0.345)
Random part (variances)
Pupil	16.191 (1.599)	15.457 (1.595)	15.457 (1.595)
Class		0.733 (0.714)	0.733 (0.714)
School			0.000 (0.000)
Deviance	1152.581	1150.700	1150.700
Reference model		0	1
Fit improvement		χ2 = 1.881 df = 1 p = n.s.	χ2 = 0.000 df = 1 p = n.s.

Note: N pupils = 205; N classes = 18; N schools = 7. n.s. = non-significant.

).

The second research question regarded the prediction of implementation and thus included both receipt and enactment variables as outcomes. For receipt we were interested in examining whether parent characteristics would predict overall implementation. Therefore, we analyzed sum scores for attendance, diaries, and activities, and mean scores for program activity (PA). Attendance scores were analyzed with a multilevel model with two levels: schools and pupils (see Appendix C,

Table A7. Exploration Multi-level Structure—Attendance.

Model	0	1	2
Fixed part
Intercept	4.969 (0.208)	4.869 (0.357)	4.696 (0.433)
Random part (variances)
Pupil	4.234 (0.605)	3.513 (0.527)	3.447 (0.511)
Class		0.818 (0.541)	0.000 (0.000)
School			1.019 (0.699)
Deviance	419.535	412.534	410.284
Reference model		0	1
Fit improvement		χ2 = 7.001 df = 1 p < 0.01	χ2 = 2.25 df = 1 p = n.s.

Note: N pupils = 98; N classes = 9; N schools = 7. n.s. = non-significant.

). Diary scores were analyzed with a pupil and a class level (see Appendix C,

Table A8. Exploration Multi-level Structure—Diaries.

Model	0	1	2
Fixed part
Intercept	4.328 (0.219)	4.353 (0.415)	4.353 (0.415)
Random part (variances)
Pupil	5.699 (0.739)	4.429 (0.597)	4.429 (0.597)
Class		1.179 (0.727)	1.179 (0.727)
School			0.000 (0.000)
Deviance	544.809	527.955	527.955
Reference model		0	1
Fit improvement		χ2 = 16.854 df = 1 p < 0.001	χ2 = 0.000 df = 1 p = n.s.

Note: N pupils = 119; N classes = 9; N schools = 7. n.s. = non-significant.

). Sum scores for activities conducted were analyzed uni-level (see Appendix C,

Table A9. Exploration Multi-level Structure—Activities.

Model	0	1	2
Fixed part
Intercept	33.838 (1.341)	33.757 (1.835)	33.757 (1.835)
Random part (variances)
Pupil	177.974 (25.296)	164.383 (24.454)	164.383 (24.454)
Class		14.258 (14.120)	14.258 (14.120)
School			0.000 (0.000)
Deviance	793.932	791.947	791.947
Reference model		0	1
Fit improvement		χ2 = 1.985 df = 1 p = n.s.	χ2 = 0 df = 1 p = n.s.

Note: N pupils = 99; N classes = 9; N schools = 7. n.s. = non-significant.

). PA was analyzed uni-level, as no significant variance was found on the upper levels (see Appendix C,

Table A10. Exploration Multi-level Structure—Mean Program Activity.

Model	0	1	2
Fixed part
Intercept	2.766 (0.082)	2.780 (0.102)	2.730 (0.107)
Random part (variances)
Pupil	0.445 (0.077)	0.410 (0.076)	0.405 (0.074)
Class		0.037 (0.044)	0.000 (0.000)
School			0.035 (0.042)
Deviance	135.898	134.923	133.742
Reference model		0	1
Fit improvement		χ2 = 0.975 df = 1 p = n.s.	χ2 =1.181 df = 1 p = n.s.

Note: N pupils = 67; N classes = 9; N schools = 7. n.s. = non-significant.

). With respect to enactment variables (HLE, PSE, NPA), we were interested in examining whether targeted behaviors and practices increased and whether low-SES and ethnic-minority families’ growth on enactment variables was different from that of higher-SES and native Dutch parents. Therefore, we analyzed change scores. For change in HLE, PSE and NPA, no significant multi-level structures were found, and hence, all analyses including these variables as dependent variables were conducted uni-level (see Appendix C,

Table A14. Exploration Multi-level Structure—Change in HLE.

Model	0	1	2
Fixed part
Intercept	−0.054 (0.027)	−0.059 (0.029)	−0.059 (0.029)
Random part (variances)
Pupil	0.127 (0.014)	0.121 (0.015)	0.121 (0.015)
Class		0.006 (0.008)	0.006 (0.008)
School			0.000 (0.000)
Deviance	136.863	136.377	136.377
Reference model		0	1
Fit improvement		χ2 = 0.486 df = 1 p = n.s.	χ2 = 0 df = 1 p = n.s.

Note: N pupils = 176; N classes = 18; N schools = 7. n.s. = non-significant.

,

Table A15. Exploration Multi-level Structure—Change in PSE.

Model	0	1	2
Fixed part
Intercept	−0.026 (0.032)	−0.025 (0.034)	−0.018 (0.038)
Random part (variances)
Pupil	0.176 (0.019)	0.171 (0.021)	0.169 (0.019)
Class		0.006 (0.011)	0.000 (0.000)
School			0.007 (0.009)
Deviance	190.574	190.433	188.842
Reference model		0	1
Fit improvement		χ2 = 0.141 df = 1 p = n.s.	χ2 = 1.591 df = 1 p = n.s.

Note: N pupils = 173; N classes = 18; N schools = 7. n.s. = non-significant.

and

Table A16. Exploration Multi-level Structure—Change in NPA.

Model	0	1	2
Fixed part
Intercept	−0.156 (0.052)	−0.156 (0.052)	−0.156 (0.052)
Random part (variances)
Pupil	0.339 (0.043)	0.339 (0.043)	0.339 (0.043)
Class		0.000 (0.000)	0.000 (0.000)
School			0.000 (0.000)
Deviance	217.579	217.579	217.579
Reference model		0	1
Fit improvement		χ2 = 0 df = 1 p = n.s.	χ2 = 0 df = 1 p = n.s.

Note: N pupils = 124; N classes = 18; N schools = 7. n.s. = non-significant.

).

The fourth research question involved the same child outcome measures as in Research Question 1 and 3, but now the sample included only experimental group children (see Figure 1), and hence the multi-level structures were explored separately. Language scores were analyzed using a growth model with two levels: time and pupil (see Appendix E,

Table A23. Exploration Multi-level Structure—Language Development.

Model	0	1	2 a
Fixed part
Intercept	45.467 (1.302)	45.384 (1.534)	44.909 (1.582)
Time (gm)	7.527 (0.459)	7.528 (0.459)	7.527 (0.459)
Random part (variances)
Repeated measures	45.315 (4.357)	45.360 (4.360)	45.366 (4.361)
Pupil	86.827 (13.538)	81.674 (13.365)	81.574 (13.354)
Class		5.790 (6.412)	-
School			-
Deviance	2446.743	2445.946	2445.352
Reference model		0	1
Fit improvement		χ2 = 0.797 df = 1 p = n.s.	χ2 = 0.594 df = 1 p = n.s.

Note: N repeated measures = 335; N pupils = 119; N classes = 9; N schools = 7. gm = grand mean centered. n.s. = non-significant. ^a This model does not converge fully as for 5 out of 7 schools, there is only one class available for analyses that include receipt variables.

). Change scores on emergent literacy were analyzed using a multilevel model with two levels: pupils and schools (Appendix E,

Table A26. Exploration Multi-level Structure—Literacy Development.

Model	0	1	2
Fixed part
Intercept	0.875 (0.055)	0.847 (0.129)	0.823 (0.162)
Random part (variances)
Pupil	0.350 (0.046)	0.207 (0.028)	0.203 (0.028)
Class		0.132 (0.070)	0.000 (0.000)
School			0.169 (0.099)
Deviance	205.595	164.348	161.081
Reference model		0	1
Fit improvement		χ2 = 41.247 df = 1 p < 0.001	χ2 = 3.267 df = 1 p <.10

Note: N pupils = 115; N classes = 9; N schools = 7.

), and change scores on vocabulary were analyzed uni-level (Appendix E,

Table A28. Exploration Multi-level Structure—Vocabulary Development.

Model	0	1	2
Fixed part
Intercept	4.430 (0.354)	4.561 (0.470)	4.451 (0.547)
Random part (variances)
Pupil	14.236 (1.889)	13.418 (1.849)	13.242 (1.808)
Class		0.861 (0.932)	0.000 (0.000)
School			1.142 (1.094)
Deviance	626.489	624.745	623.835
Reference model		0	1
Fit improvement		χ2 = 1.744 df = 1 p = n.s.	χ2 = 0.91 df = 1 p = n.s.

Note: N pupils = 114; N classes = 9; N schools = 7. n.s. = non-significant.

).

The fifth research question involved possible mediation of intervention effects by enactment variables (see Figure 2), which we tested following a widely used method [39]. We first examined relations between EEH and child development, followed by relations among EEH and growth on enactment variables (mediators), and finally, relations among growth in enactment variables and child development, after controlling for the effect of condition. The first and third steps for testing mediation included language, emergent literacy and receptive vocabulary as outcome measures and hence were analyzed according to the procedure described for Research Question 1. Change scores on our mediator variables (HLE, PSE, and NPA) showed no significant variance on the upper levels and thus were analyzed uni-level (see Appendix F,

Table A30. Exploration Multi-level Structure—Change in HLE.

Model	0	1	2
Fixed part
Intercept	−0.039 (0.028)	−0.039 (0.028)	−0.039 (0.028)
Random part (variances)
Pupil	0.145 (0.015)	0.145 (0.015)	0.145 (0.015)
Class		0.000 (0.000)	0.000 (0.000)
School			0.000 (0.000)
Deviance	167.463	167.463	167.463
Reference model		0	1
Fit improvement		χ2 = 0.000 df = 1 p = n.s.	χ2 = 0.000 df = 1 p = n.s.

Note: N pupils = 184; N classes = 18; N schools = 7. n.s. = non-significant.

,

Table A31. Exploration Multi-level Structure—Change in PSE.

Model	0	1	2
Fixed part
Intercept	−0.021 (0.031)	−0.021 (0.031)	−0.021 (0.031)
Random part (variances)
Pupil	0.172 (0.018)	0.172 (0.018)	0.172 (0.018)
Class		0.000 (0.000)	0.000 (0.000)
School			0.000 (0.000)
Deviance	192.404	192.404	192.404
Reference model		0	1
Fit improvement		χ2 = 0.000 df = 1 p = n.s.	χ2 = 0.000 df = 1 p = n.s.

Note: N pupils = 179; N classes = 18; N schools = 7. n.s. = non-significant.

and

Table A32. Exploration Multi-level Structure—Change in NPA.

Model	0	1	2
Fixed part
Intercept	−0.156 (0.051)	−0.156 (0.051)	−0.156 (0.051)
Random part (variances)
Pupil	0.331 (0.041)	0.331 (0.041)	0.331 (0.041)
Class		0.000 (0.000)	0.000 (0.000)
School			0.000 (0.000)
Deviance	223.455	223.455	223.455
Reference model		0	1
Fit improvement		χ2 = 0.000 df = 1 p = n.s.	χ2 = 0.000 df = 1 p = n.s.

Note: N pupils = 129; N classes = 18; N schools = 7. n.s. = non-significant.

).

3. Results

3.1. Descriptive Statistics

The descriptive statistics for child outcomes, receipt, and enactment variables are presented in

Table 5. Possible scores, mean scores, and standard deviations for study variables.

		Control Group				Intervention Group
		Pre-Test		Post-Test		Pre-Test		Post-Test
Measures	Possible scores	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n
Child outcomes
1. Language a	0–108	51.63 (12.31)	92	67.06 (10.99)	85	52.08 (12.28)	112	67.09 (10.61)	108
2. Literacy	1–5	2.78 (0.85)	98	3.67 (0.87)	92	2.82 (0.86)	119	3.73 (0.88)	115
3. Vocabulary	0–43	30.27 (5.42)	97	34.6 (4.04)	91	30.93 (4.76)	119	35.41 (3.68)	114
Implementation
Enactment
4. HLE b	1–4	2.11 (4.1)	95	2.12 (0.46)	75	2.16 (0.44)	119	2.12 (0.42)	110
5. PSE	1–5	3.86 (0.5)	93	3.82 (0.45)	74	3.81 (0.47)	116	3.81 (0.38)	108
6. NPA	1–4	2.80 (0.58)	72	2.70 (0.62)	65	2.95 (0.63)	81	2.75 (0.63)	81
Receipt
7. PA	1–4					2.62 (0.77)	81	2.83 (0.77)	80
						Overall
8. Attendance	0–100%					71.43 (29.53)	113 c
9. Diaries	0–7					4.33 (2.40)	119
10. Activities	0–100%					61.03 (23.72)	75 d

Note: HLE = Home Literacy Environment; PSE = Parent self-efficacy; NPA= Non-Program Activity; PA = Program Activity. ^a Mid-intervention language scores are not presented above to improve readability. Control group: M (SD) = 63.85 (11.63), intervention group: M (SD) = 62.24 (11.52). ^b HLE is coded from daily (1) to almost never (4) whereas for the other measures a higher score indicates better scores. ^c Average sample size for mean attendance (range 111–118). ^d Average sample size for activities (range 50–93).

. First, the table shows that language and literacy scores were higher at post-test than at pre-test. This was not the case for the three parent enactment variables that were expected to mediate intervention effects. For example, the overall quality of parents’ behavior and language during the NPA slightly decreased over the year. Average scores on receipt quantity variables revealed that parental engagement in the program was not optimal: on average, parents returned approximately 4 out of 7 diaries and conducted about 60% of program activities. As the end of the intervention year approached, we witnessed a decline in attendance, diaries handed in, and activities conducted. Conversely, there was a slight increase in receipt quality (PA).

Bivariate correlations between study variables (at post-test) are presented in

Table 6. Bivariate correlations between study variables at post-test.

Measures	1	2	3	4	5	6	7	8	9	10	11	12	13	14
Child outcomes
1. Language	−
2. Literacy	0.697 **	−
3. Vocabulary	0.594 **	0.560 **	−
Implementation
4. HLE	−0.023	−0.078	0.077	−
5. PSE	0.141	0.229 **	0.090	−0.250 **	−
6. Prompting board	0.328 **	0.268 **	0.252 **	0.104	−0.056	−
7. Attendance	0.119	0.153	0.126	−0.004	−0.021	0.163	−
8. Diaries	0.161	0.166	0.185 *	−0.055	−0.032	0.292 **	0.682 **	−
9. Activities	0.027	−0.053	0.096	0.012	−0.134	0.212	0.547 **	0.905 **	−
10. Shared reading	0.380 **	0.303 **	0.360 **	−0.020	0.181	0.696 **	0.255 *	0.303 **	0.161	−
Child characteristics
11. Age (months)	−0.024	0.019	0.049	0.126	−0.101	−0.124	0.005	0.032	−0.008	0.056	−
12. Gender	0.128	0.128	0.040	−0.200 **	0.029	0.012	−0.005	0.092	0.102	−0.143	−0.222 **	−
Parent characteristics
13. Education	0.377 **	0.404 **	0.273 **	−0.045	0.100	0.317 **	−0.019	0.027	−0.147	0.341 **	−0.010	0.019	−
14. Ethnicity	−0.279 **	−0.272 **	−0.277 **	−0.032	−0.021	−0.420 **	0.013	−0.173	0.073	−0.281 *	−0.099	0.074	−0.412 **	−
15. Best language a	−0.379 **	−0.312 **	−0.354 **	−0.112	−0.029	−0.409 **	−0.118	−0.209	−0.019	−0.356 **	−0.101	0.076	−0.373 **	0.781 **

Note: HLE = Home Literacy Environment; PSE = Parent self-efficacy. Prompting board = parent behavior and language during prompting board. Shared reading = parent behavior and language during shared reading. Spearmans correlations are presented for education and best language, for all other variables Pearsons correlations are shown. ^a Best oral language. * p < 0.05. ** p < 0.01.

. All child outcome measures showed to be significantly and moderately to strongly correlated. This is to be expected, as all child outcomes were to measure (a specific part of) language ability. Three of our measures of implementation quality—parent self-efficacy and parent behavior and language during a prompting board activity and shared reading—showed to be significantly related to one or more child outcome measures. The strongest significant relation was found between two measures of receipt: the number of diaries handed in and the percentage of activities conducted. This strong association reflects the fact that the percentage of activities conducted was derived from the diaries that were handed in by parents. Furthermore, parent behavior and language during the program task (shared reading) was significantly, though weakly, associated with parents’ attendance at group meetings and the number of diaries handed in. Parent behavior and language during the non-program activity (prompting board) was associated only with the number of diaries handed in. Moreover, parent ethnicity correlated significantly with our child outcome measures and parent behavior and language (during both activities), in the expected direction.

3.2. Effects of the EEH-Intervention

Regarding our first research question, we found no direct effects of the EEH intervention on children’s language skills as measured by a standardized language test, their (teacher-reported) emergent literacy skills, and their curriculum-based receptive vocabulary scores.

Tables with parameter estimates are presented in Appendix B (

Table A2. Multi-level Regression—Effect of EEH on Language Development.

Model	1 B (SE)	2 B (SE)	3 B (SE)
Fixed part
Intercept	62.195 *** (1.333)	62.255 *** (1.647)	62.252 *** (1.646)
Time	7.711 *** (0.404)	7.711 (0.404)	7.567 *** (0.627)
Child gender (girl = 1)	3.949 ** (1.305)	3.949 ** (1.305)	3.947 ** (1.305)
Child age (gm)	0.427 * (0.168)	0.427 * (0.169)	0.427 ** (0.169)
Parent SES (gm)	2.441 *** (0.618)	2.441 *** (0.618)	2.444 *** (0.618)
Parent home language: Dutch and other equal	−7.405 ** (1.999)	−7.398 ** (2.001)	−7.398 ** (2.001)
Parent home language: other	−5.219 ** (1.763)	−5.222 ** (1.764)	−5.2222 ** (1.765)
Condition (EEH = 1)		−0.112 (1.816)	−0.108 (1.813)
Condition × Time			0.247 (0.821)
Random part (variances)
Repeated measures	46.044 (3.877)	46.043 (3.877)	46.029 (3.876)
Pupil	40.663 (7.100)	40.644 (7.097)	40.662 (7.099)
Class	7.338 (5.983)	7.380 (6.000)	7.328 (5.981)
School	1.567 (4.306)	1.574 (4.319)	1.592 (4.320)
Deviance	3098.467	3098.463	3098.372
Reference model		1	2
Fit improvement		χ2 = 0.004 df = 1 p = n.s.	χ2 = 0.091 df = 1 p = n.s.

Note: N repeated measures = 434; N pupils = 152; N classes = 18; N schools = 7. gm = grand mean centered. Parent home language reference category = Dutch. EEH = Early Education at Home. n.s. = non-significant. * p < 0.05; ** p < 0.01; *** p < 0.001.

,

Table A4. Multi-level Regression—Effect of EEH on Literacy Development.

Model	1 B (SE)	2 B (SE)
Fixed part
Intercept	0.863 *** (0.084)	0.833 *** (0.119)
Child age (gm)	−0.027 ** (0.009)	−0.027 ** (0.009)
Condition (EEH = 1)		0.060 (0.167)
Random part (variances)
Pupil	0.104 (0.042)	0.174 (0.021)
Class	0.174 (0.021)	0.103 (0.042)
Deviance	209.386	209.257
Reference model		1
Fit improvement		χ2 = 0.129 df = 1 p = n.s.

Note: N pupils = 162; N classes = 18; N schools = 7. gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. ** p < 0.01; *** p < 0.001.

and

Table A6. Regression—Effect of EEH on Vocabulary Development.

Model	1 B (SE)	2 B (SE)
Intercept	4.359 *** (0.297)	4.391 *** (0.459)
Child age (gm)	−0.249 *** (0.075)	−0.249 *** (0.076)
Condition (EEH = 1)		−0.055 (0.603)
Variance	14.065 (1.573)	14.064 (1.572)
Deviance	877.053	877.044
Reference model		1
Fit improvement		χ2 = 0.009 df = 1 p = n.s.

Note: N = 160. gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. *** p < 0.001.

).

3.3. SES, Ethnic-Minority Status, Home Language, and Implementation

Regarding our second research question, we first did not find significant relationships among parents’ SES, ethnic-minority status, and home language, and the following receipt variables: attendance at training sessions, the number of diaries handed in, and the number of activities conducted. Tables with parameter estimates are presented in Appendix C (

Table A11. Multi-level Regression—Predicting Attendance with Parent Background Variables.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Fixed part
Intercept	4.696 *** (0.433)	4.696 *** (0.433)	4.976 *** (0.476)	4.725 *** (0.468)
Parent SES (gm)		0.007 (0.184)
Parent home language: Dutch and other equal			−0.803 (0.557)
Parent home language: other			−0.592 (0.483)
Parent ethnic-minority status (ethnic-minority = 1)				−0.074 (0.433)
Random part (variances)
Pupil	3.447 (0.511)	3.447 (0.511)	3.331 (0.494)	3.444 (0.510)
School	1.020 (0.700)	1.019 (0.700)	1.089 (0.731)	1.029 (0.705)
Deviance	410.284	410.283	407.490	410.255
Reference model		0	0	0
Fit improvement		χ2 = 0.001 df = 1 p = n.s.	χ2 = 2.794 df = 2 p = n.s.	χ2 = 0.029 df = 1 p = n.s.
N Pupil	98	98	98	98
N School	7	7	7	7

Note: gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant. *** p < 0.001.

,

Table A12. Multi-level Regression—Predicting Diaries with Parent Background Variables.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Fixed part
Intercept	4.350 *** (0.415)	4.347 *** (0.414)	4.648 *** (0.424)	4.394 *** (0.440)
Parent SES (gm)		0.108 (0.182)
Parent home language: Dutch and other equal			−0.362 (0.579)
Parent home language: other			−0.998 (0.492)
Parent ethnic-minority status (ethnic-minority = 1)				−0.122 (0.448)
Random part (variances)
Pupil	4.174 (0.582)	4.163 (0.580)	4.055 (0.565)	4.179 (0.582)
Class	1.192 (0.728)	1.179 (0.722)	1.033 (0.652)	1.155 (0.715)
Deviance	491.235	490.884	487.229	491.164
Reference model		0	0	0
Fit improvement		χ2 = 0.351 df = 1 p = n.s.	χ2 = 4.006 df = 2 p = n.s.	χ2 = 0.71 df = 1 p = n.s.
N Pupils	112	112	112	112
N Classes	9	9	9	9

Note: gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant. *** p < 0.001.

and

Table A13. Regression—Predicting Activities with Parent Background Variables.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Intercept	33.838 (1.341)	33.974 (1.337)	34.047 (1.667)	33.246 (1.652)
Parent SES (gm)		−1.452 (1.242)
Parent home language: Dutch and other equal			−0.190 (3.935)
Parent home language: other			−0.856 (3.354)
Parent ethnic-minority status (ethnic-minority = 1)				1.724 (2.818)
Variance	177.974 (25.296)	175.550 (24.952)	177.857 (25.279)	177.303 (25.201)
Deviance	793.932	792.575	793.867	793.558
Reference model		0	0	0
Fit improvement		χ2 = 1.357 df = 1 p = n.s.	χ2 = 0.065 df = 1 p = n.s.	χ2 = 0.374 df = 1 p = n.s.
N Pupil	99	99	99	99

Note: gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant.

). We did, however, find significant relations between SES and receipt quality, as measured by parents’ behavior and language during a program activity (shared reading; see

Table 7. Regression—predicting parents’ behavior and language during a program activity.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Intercept	2.766 *** (0.082)	2.690 *** (0.079)	2.977 *** (0.088)	2.932 *** (0.093)
Parent SES (gm)		0.320 *** (0.097)
Parent home language: Dutch and other equal			−0.569 ** (0.207)
Parent home language: other			−0.706 *** (0.193)
Parent ethnic-minority status (ethnic-minority = 1)				−0.508 ** (0.162)
Variance	0.445 (0.077)	0.383 (0.066)	0.352 (0.061)	0.388 (0.067)
Deviance	135.898	125.802	120.218	126.724
Reference model		0	0	0
Fit improvement		χ2 = 10.096 df = 1 p < 0.01	χ2 = 15.680 df = 2 p < 0.001	χ2 = 9.174 df = 1 p < 0.005
R2		0.139	0.209	0.128

Note: N = 67. gm = grand mean centered. Parent home language reference category = Dutch. ** p < 0.01; *** p < 0.001.

). A higher level of education was associated with a higher mean score on PA. In addition, non-native Dutch parents scored significantly lower on PA than native Dutch parents. Finally, parents whose home language was different from Dutch and parents who were equally proficient in Dutch and their mother tongue, scored significantly lower on PA than parents who indicated their home language to be Dutch. Regarding enactment variables, we found no significant relations between SES, ethnic-minority status, and home language, and change in HLE, PSE, and NPA (see Appendix C,

Table A17. Regression—Predicting Change in HLE with Parent Background Variables.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Intercept	−0.054 (0.027)	−0.055 (0.027)	−0.014 (0.033)	−0.045 (0.033)
Parent SES (gm)		0.029 (0.025)
Parent home language: Dutch and other equal			−0.139 (0.081)
Parent home language: other			−0.091 (0.064)
Parent ethnic-minority status (ethnic-minority = 1)				−0.024 (0.056)
Variance	0.127 (0.014)	0.126 (0.013)	0.125 (0.013)	0.127 (0.014)
Deviance	136.863	135.469	132.799	136.680
Reference model		0	0	0
Fit improvement		χ2 = 1.394 df = 1 p = n.s.	χ2 = 4.064 df = 2 p = n.s.	χ2 = 0.183 df = 1 p = n.s.

Note: N = 176. gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant.

,

Table A18. Regression—Predicting Change in PSE with Parent Background Variables.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Intercept	−0.026 (0.032)	−0.025 (0.032)	−0.027 (0.039)	−0.012 (0.039)
Parent SES (gm)		−0.009 (0.031)
Parent home language: Dutch and other equal			−0.103 (0.095)
Parent home language: other			0.068 (0.078)
Parent ethnic-minority status (ethnic-minority = 1)				−0.043 (0.067)
Variance	0.176 (0.019)	0.176 (0.019)	0.174 (0.019)	0.176 (0.019)
Deviance	190.574	190.490	188.154	190.171
Reference model		0	0	0
Fit improvement		χ2 = 0.084 df = 1 p = n.s.	χ2 = 2.42 df = 2 p = n.s.	χ2 = 0.403 df = 1 p = n.s.

Note: N = 173. gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant.

and

Table A19. Regression—Predicting Change in NPA with Parent Background Variables.

Model	0 B (SE)	1 B (SE)	2 B (SE)	3 B (SE)
Intercept	−0.156 (0.052)	−0.154 (0.052)	−0.108 (0.065)	−0.111 (0.066)
Parent SES (gm)		−0.018 (0.050)
Parent home language: Dutch and other equal			−0.118 (0.168)
Parent home language: other			−0.142 (0.124)
Parent ethnic-minority status (ethnic-minority = 1)				−0.121 (0.108)
Variance	0.339 (0.043)	0.338 (0.043)	0.334 (0.042)	0.335 (0.043)
Deviance	217.579	217.447	216.036	216.317
Reference model		0	0	0
Fit improvement		χ2 = 0.132 df = 1 p = n.s.	χ2 = 1.543 df = 2 p = n.s.	χ2 = 1.262 df = 1 p = n.s.

Note: N = 124. gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant.

).

3.4. Moderation of EEH Effects by Parent Variables

With respect to our third research question, we tested whether effects of EEH were moderated by parental SES, ethnic-minority status, and home language. This proved not to be the case (See Appendix D,

Table A20. Multi-level Regression—Testing for Moderation of EEH Effect on Language Development by Parent Background.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)
Fixed part
Intercept	61.467 *** (1.552)	61.464 *** (1.548)	62.102 *** (1.663)	61.789 *** (1.655)
Time	7.554 *** (0.619)	7.547 *** (0.617)	6.566 *** (0.789)	6.711 *** (0.766)
Child gender (girl = 1)	4.132 ** (1.305)	4.156 ** (1.303)	4.209 ** (1.307)	4.185 ** (1.301)
Child age (gm)	0.446 ** (0.175)	0.447 ** (0.174)	0.455 ** (0.175)	0.442 ** (0.174)
Parent SES (gm)	2.089 ** (0.764)	2.487 ** (0.939)	2.058 ** (0.761)	1.885 * (0.780)
Parent home language: Dutch and other equal	−7.243 *** (2.158)	−7.155 *** (2.158)	−7.546 *** (2.631)	−11.019 *** (3.717)
Parent home language: other	−5.292 ** (1.795)	−5.405 ** (1.796)	−5.591 ** (2.652)	−5.394 * (2.439)
Condition (EEH = 1)	0.547 (1.297)	0.552 (1.295)	−0.536 (1.638)	0.045 (1.603)
Condition × Time	0.117 (0.811)	0.118 (0.810)	1.066 (1.013)	0.730 (1.003)
Time × SES		0.054 (0.520)
Condition × SES		−0.820 (1.143)
Condition × Time × SES		−0.619 (0.716)
Parent ethnic-minority status (ethnic-minority = 1)			−1.243 (3.075)
Time × Ethnic-minority status			2.483 (1.256)
Condition × Ethnic-minority status			3.107 (2.770)
Condition × Time × Ethnic-minority status			−2.366 (1.675)
Time × Home language: Dutch and other equal				2.602 (2.170)
Time × Home language: other				2.224 (1.405)
Condition × Home language: Dutch and other equal				5.384 (4.316)
Condition × Home language: other				−0.329 (3.184)
Condition × Time × Home language: Dutch and other equal				−3.135 (2.605)
Condition × Time × Home language: other				−0.572 (1.948)
Random part (variances)
Repeated measures	44.732 (3.781)	44.540 (3.765)	44.023 (3.721)	43.798 (3.702)
Pupil	40.722 (7.252)	40.586 (7.223)	41.446 (7.287)	40.311 (7.160)
Class	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)
School	12.071 (6.896)	11.937 (6.857)	10.270 (6.315)	12.849 (7.143)
Deviance	3071.866	3070.020	3066.605	3064.921
Reference model		1	1	1
Fit improvement		χ2 = 1.846 df = 3 p = n.s.	χ2 = 5.261 df =4 p = n.s.	χ2 = 6.945 df = 6 p = n.s.

Note: N repeated measures = 43; N pupils =152; N classes = 18; N schools = 7. gm = grand mean centered. Parent home language reference category = Dutch. EEH = Early Education at Home. n.s. = non-significant. * p < 0.05; ** p < 0.01; *** p < 0.001.

,

Table A21. Multi-level Regression—Testing for Moderation of the EEH Effect on Literacy Development by Parent Background.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)
Fixed part
Intercept	0.847 *** (0.084)	0.853 *** (0.085)	0.726 *** (0.101)	0.774 *** (0.101)
Child age (gm)	−0.030 ** (0.010)	−0.030 ** (0.010)	−0.029 ** (0.010)	−0.029 ** (0.010)
Condition (EEH = 1)	0.037 (0.114)	0.043 (0.114)	0.198 (0.131)	0.102 (0.135)
Parent SES (gm)		0.021 (0.060)
Condition × SES		0.032 (0.085)
Parent ethnic-minority status (ethnic-minority = 1)			0.269 (0.136)
Condition × Ethnic-minority status			−0.380 (0.178)
Parent home language: Dutch and other equal				0.354 (0.231)
Parent home language: other				0.107 (0.153)
Condition × Home language: Dutch and other equal				−0.400 (0.278)
Condition × Home language: other				−0.042 (0.209)
Random part (variances)
Pupil	0.186 (0.026)	0.186 (0.026)	0.184 (0.026)	0.183 (0.026)
Class	0.094 (0.034)	0.090 (0.033)	0.083 (0.032)	0.091 (0.033)
Deviance	219.002	218.127	214.276	216.197
Reference model		1	1	1
Fit improvement		χ2 = 0.875 df = 2 p = n.s.	χ2 = 4.726 df = 2 p = n.s.	χ2 = 2.805 df = 4 p = n.s.

Note: N pupils = 151; N classes = 18; N schools = 7. gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. ** p < 0.01; *** p < 0.001.

and

Table A22. Regression—Testing for Moderation of the EEH Effect on Vocabulary Development by Parent Background.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)
Intercept	4.379 *** (0.477)	4.370 *** (0.472)	3.793 *** (0.596)	3.814 *** (0.576)
Child age (gm)	−0.210 * (0.080)	−0.211 * (0.079)	−0.184 * (0.079)	−0.192 * (0.078)
Condition (EEH = 1)	0.050 (0.629)	0.063 (0.622)	0.088 (0.769)	0.090 (0.758)
Parent SES (gm)		−0.673 (0.388)
Condition × SES		0.418 (0.542)
Parent ethnic-minority status (ethnic-minority = 1)			1.518 (0.965)
Condition × Ethnic-minority status			0.191 (1.285)
Parent home language: Dutch and other equal				1.446 (1.640)
Parent home language: other				1.663 (1.092)
Condition × Home language: Dutch and other equal				−1.329 (1.961)
Condition × Home language: other				0.938 (1.490)
Variance	14.312 (1.658)	13.986 (1.620)	13.722 (1.590)	13.459 (1.559)
Deviance	819.348	815.916	813.079	810.186
Reference model		1	1	1
Fit improvement		χ2 = 3.432 df = 2 p = n.s.	χ2 = 6.269 df = 2 p < 0.05	χ2 = 9.162 df = 4 p = n.s.

Note: N = 149 gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant.* p < 0.05; *** p < 0.001.

).

3.5. The Role of ‘Receipt’

In answering our fourth research question, we analyzed relations among receipt variables and children’s language and literacy development. None of our receipt variables—attendance at training sessions, diaries handed in, activities conducted, and quality of behavior and language during a program activity (PA)—significantly predicted children’s language and literacy development. Tables with parameter estimates are presented in Appendix E (

Table A24. Multi-level Regression—Predicting Language Development with Program Activity and Attendance Scores.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)	5 B (SE)	6 B (SE)
Fixed part
Intercept	62.827 *** (1.520)	62.822 *** (1.623)	62.823 *** (1.623)	62.437 *** (1.579)	61.791 *** (1.552)	61.798 *** (1.555)
Time	7.563 *** (0.670)	7.563 *** (0.670)	7.561 *** (0.671)	8.161 *** (0.567)	8.133 *** (0.568)	8.220 *** (0.568)
Child gender (girl = 1)	4.160 * (1.902)	4.164 * (1.964)	4.163 * (1.964)	3.727 (2.009)	4.281 * (1.956)	4.279 * (1.960)
Child age (gm)	0.459 (0.309)	0.459 (0.309)	0.459 (0.309)	0.431 (0.270)	0.429 (0.261)	0.433 (0.262)
Parent SES (gm)	3.197 * (1.428)	3.193 * (1.519)	3.193 * (1.519)	2.327 * (1.009)	2.342 * (0.975)	2.360 * (0.977)
Parent home language: Dutch and other equal	−9.975 *** (2.836)	−9.964 *** (3.120)	−9.964 *** (3.120)	−5.405 (2.810)	−4.401 (2.755)	−4.354 (2.761)
Parent home language: other	−6.842 * (2.889)	−6.836 * (2.986)	−6.837 * (2.986)	−6.523 * (2.840)	−6.341 * (2.745)	−6.261 * (2.751)
Program activity (gm)		0.015 (1.766)	0.015 (1.766)
Program activity (gm) × Time			0.051 (0.977)
Attendance (gm)					1.051 * (0.470)	1.041 * (0.471)
Attendance (gm) × Time						−0.370 (0.283)
Random part (variances)
Repeated measures	46.408 (6.426)	46.408 (6.426)	46.406 (6.425)	47.010 (5.477)	47.204 (5.501)	46.567 (5.428)
Pupil	31.088 (9.262)	31.088 (9.262)	31.090 (9.262)	56.452 (11.830)	51.507 (11.051)	52.001 (11.086)
R2 Repeated measures level					--
R2 Pupil Level					0.088
Deviance	1127.487	1127.487	1127.484	1628.285	1623.471	1621.776
Reference model		1	2		4	5
Fit improvement		χ2 = 0.000 df = 1 p = n.s.	χ2 = 0.003 df = 1 p = n.s.		χ2 = 4.814 df = 1 p < 0.05	χ2 = 1.695 df = 1 p = n.s.
N measurement	160	160	160	226	226	226
N pupils	56	56	56	79	79	79
N classes	9	9	9	9	9	9

Note: gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant. * p < 0.05; *** p < 0.001.

,

Table A25. Multi-level Regression—Predicting Language Development with Diaries and Activities Scores.

Model	7 B (SE)	8 B (SE)	9 B (SE)	10 B (SE)	11 B (SE)	12 B (SE)
Fixed part
Intercept	62.157 (1.489)	61.935 (1.513)	61.936 (1.513)	62.382 (1.456)	62.340 (1.445)	62.339 (1.444)
Time	7.827 *** (0.536)	7.819 *** (0.537)	7.834 *** (0.547)	7.746 *** (0.564)	7.737 *** (0.565)	7.732 *** (0.567)
Child gender (girl = 1)	4.288 * (1.814)	4.265 * (1.807)	4.266 * (1.807)	4.368 * (1.826)	4.209 * (1.817)	4.210 * (1.817)
Child age (gm)	0.376 (0.246)	0.363 (0.245)	0.363 (0.245)	0.356 (0.243)	0.352 (0.241)	0.352 (0.241)
Parent SES (gm)	2.061 * (0.948)	2.077 * (0.945)	2.079 * (0.945)	2.421 * (1.005)	2.578 * (1.009)	2.578 * (1.009)
Parent home language: Dutch and other equal	−6.361 * (2.514)	−6.050 * (2.540)	−6.046 * (2.541)	−5.163 * (2.609)	−4.948 (2.597)	−4.953 (2.597)
Parent home language: other	−5.592 * (2.611)	−5.300 * (2.631)	−5.293 * (2.632)	−7.109 ** (2.679)	−6.997 *** (2.659)	−7.000 *** (2.659)
Diaries (gm)		0.309 (0.417)	0.308 (0.417)
Diaries (gm) × Time			−0.33 (0.247)
Activities (gm)					0.070 (0.069)	0.070 (0.069)
Activities (gm) × Time						0.005 (0.043)
Random part (variances)
Repeated measures	47.294 (5.201)	47.334 (5.206)	47.323 (5.205)	48.284 (5.531)	48.385 (5.544)	48.391 (5.544)
Pupil	52.745 (10.647)	52.188 (10.565)	52.210 (10.567)	48.053 (10.429)	46.942 (10.258)	46.916 (10.255)
R2 Level 1
R2 Level 2
Deviance	1819.168	1818.622	1818.604	1673.247	1672.220	1672.220
Reference model		7	8		10	11
Fit improvement		χ2 = 0.546 df = 1 p = n.s.	χ2 = 0.018 df = 1 p = n.s.		χ2 = 1.027 df = 1 p = n.s.	χ2 = 1.027 df = 1 p = n.s.
N measurement	253	253	253	233	233	233
N pupils	88	88	88	81	81	81
N classes	9	9	9	9	9	9

Note: gm = grand mean centered. Parent home language reference category = Dutch. n.s. = non-significant. * p < 0.05; *** p < 0.001.

,

Table A27. Multi-level Regression—Predicting Literacy Development with Receipt Variables.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)	5 B (SE)	6 B (SE)	7 B (SE)	8 B (SE)
Fixed part
Intercept	0.911 *** (0.079)	0.909 *** (0.079)	0.861 *** (0.080)	0.861 *** (0.081)	0.902 *** (0.113)	0.903 *** (0.114)	0.901 *** (0.066)	0.893 *** (0.066)
Child age (gm)	−0.035 (0.022)	−0.032 (0.022)	−0.028 (0.018)	−0.028 (0.018)	−0.047 (0.018)	−0.047 (0.018)	−0.039 (0.017)	−0.039 (0.017)
Parent SES (gm)	0.262 * (0.108)	0.315 ** (0.117)	0.017 (0.055)	0.017 (0.055)	0.048 (0.044)	0.050 (0.044)	0.079 (0.060)	0.100 (0.067)
Program activity (gm)		−0.136 (0.137)
Attendance (gm)				−0.001 (0.036)
Diaries (gm)						−0.005 (0.022)
Activities (gm)								0.006 (0.005)
Random part (variances)
Pupil	0.179 (0.085)	0.167 (0.079)	0.130 (0.032)	0.130 (0.032)	0.165 (0.027)	.0164 (0.027)	0.278 (0.113)	0.294 (0.116)
School	0.148 (0.096)	0.157 (0.092)	0.214 (0.067)	0.214 (0.067)	0.157 (0.071)	0.160 (0.072)	0.073 (0.110)	0.054 (0.111)
Deviance	97.818	96.850	125.112	125.112	122.123	122.067	150.076	147.784
Reference model		1		3		5		7
Fit improvement		χ2 = 0.968 df = 1 p = n.s.		χ2 = 0.000 df = 1 p = n.s.		χ2 = 0.058 df = 1 p = n.s.		χ2 = 2.292 df = 1 p = n.s.
N pupils	58	58	82	82	92	92	84	84
N classes	9	9	9	9	9	9	9	9

Note: gm = grand mean centered. n.s. = non-significant. * p < 0.05; ** p < 0.01; *** p < 0.001.

and

Table A29. Regression—Predicting Vocabulary Development with Receipt Variables.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)	5 B (SE)	6 B (SE)	7 B (SE)	8 B (SE)
Intercept	4.797 *** (0.471)	4.797 *** (0.471)	4.309 *** (0.379)	4.321 *** (0.379)	4.430 *** (0.354)	4.429 *** (0.354)	4.404 *** (0.380)	4.391 *** (0.380)
Program activity (gm)		0.117 (0.710)
Attendance (gm)				−0.140 (0.148)
Diaries (gm)						0.008 (0.149)
Activities (gm)								0.020 (0.029)
Variance	15.292 (2.604)	15.286 (2.602)	13.966 (2.005)	13.921 (1.999)	14.263 (1.889)	14.262 (1.889)	14.301 (2.033)	14.233 (2.023)
Deviance	384.000	383.973	531.028	530.712	626.489	626.487	544.325	543.847
Reference model		1		3		5		7
Fit improvement		χ2 = 0.027 df = 1 p = n.s.		χ2 = 0.316 df = 1 p = n.s.		χ2 = 0.002 df = 1 p = n.s.		χ2 = 0.478 df = 1 p = n.s.
N Pupils	69	69	97	97	114	114	99	99

Note: gm = grand mean centered. n.s. = non-significant. *** p < 0.001.

).

3.6. Mediation of EEH Effects by ‘Enactment’

To test whether enactment variables mediated intervention effects, we first analyzed relations among EEH and growth in enactment variables HLE, PSE and NPA. These were non-significant, indicating that the intervention did not succeed in improving these attributes in parents. Second, we analyzed effects of change in enactment on children’s language development. Growth in HLE, PSE and NPA was found to not significantly predict children’s development on any of the language measures. Finally, no significant relations were found among the mediator variables and child outcome measures, while controlling for condition. These results indicate that program effects were not mediated by enactment variables. Tables with parameter estimates are presented in Appendix F (

Table A33. Regression—Effects of EEH on Change in HLE, PSE, and NPA.

Model	1 HLE B (SE)	2 HLE B (SE)	3 PSE B (SE)	4 PSE B (SE)	5 NPA B (SE)	6 NPA B (SE)
Intercept	−0.040 (0.035)	−0.038 (0.049)	0.009 (0.033)	−0.025 (0.053)	−0.156 ** (0.051)	−0.076 (0.074)
Child age (gm)			0.021 ** (0.008)	0.022 ** (0.008)
Parent ethnic-minority status (ethnic-minority = 1)	−0.000 (0.059)	−0.000 (0.059)
Condition (EEH = 1)		−0.004 (0.058)		0.056 (0.068)		−0.147 (0.101)
Variance	0.146 (0.015)	0.146 (0.015)	0.159 (0.019)	0.158 (0.019)	0.331 (0.041)	0.326 (0.041)
Deviance	167.364	167.359	143.436	142.752	223.455	221.346
Reference model		1		3		5
Fit improvement		χ2 = 0.005 df = 1 p = n.s.		χ2 = 0.684 df = 1 p = n.s.		χ2 = 2.109 df = 1 p = n.s.
N Pupils	183	183	144	144	129	129

Note: gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. ** p < 0.01.

,

Table A34. Multi-level Regression—Effect of HLE change on Language Development.

Model	1 B (SE)	2 B (SE)	3 B (SE)
Fixed part
Intercept	62.583 *** (1.379)	62.337 *** (1.375)	62.341 *** (1.278)
Time (gm)	7.659 *** (0.428)	7.662 *** (0.428)	7.654 *** (0.429)
Child gender (girl = 1)	4.493 *** (1.340)	4.657 *** (1.330)	4.653 *** (1.330)
Child age (gm)	0.465 * (0.186)	0.462 * (0.184)	0.463 * (0.184)
Parent SES (gm)	2.455 *** (0.667)	2.383 *** (0.662)	2.382 *** (0.662)
Parent home language: Dutch and other equal	−6.538 ** (2.162)	−6.297 ** (2.145)	−6.298 ** (2.145)
Parent home language: other	−6.301 *** (1.804)	−6.315 *** (1.785)	−6.309 *** (1.785)
Condition (EEH = 1)	−0.126 (1.461)	0.002 (1.445)	−0.001 (1.445)
HLE change (gm)		3.498 (2.097)	3.488 (2.097)
HLE change (gm) × Time			0.385 (1.278)
Random part (variances)
Repeated measures	47.991 (4.186)	48.040 (4.190)	48.033 (4.189)
Pupil	6.962 (10.878)	6.705 (10.755)	6.678 (10.733)
Class	42.112 (13.008	39.035 (20.204)	39.050 (20.190)
School	0.000 (0.000)	1.897 (15.868)	1.889 (15.864)
Deviance	2897.495	2894.768	2894.677
Reference model		1	2
Fit improvement		χ2 = 2.727 df = 1 p = n.s.	χ2 = 0.091 df = 1 p = n.s.

Note: N repeated measures = 404; N pupils = 142; N classes = 18; N schools = 7. gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. * p < 0.05; ** p < 0.01; *** p < 0.001.

,

Table A35. Multi-level Regression—Effect of PSE Change on Language Development.

Model	1 B (SE)	2 B (SE)	3 B (SE)
Fixed part
Intercept	62.782 *** (1.502)	62.677 *** (1.572)	62.686 *** (1.570)
Time (gm)	7.677 *** (0.432)	7.684 *** (0.431)	7.687 *** (0.432)
Child gender (girl = 1)	4.261 *** (1.371)	4.272 *** (1.366)	4.265 *** (1.366)
Child age (gm)	0.405 * (0.178)	0.448 * (0.180)	0.450 * (0.180)
Parent SES (gm)	3.137 * (1.502)	3.107 *** (0.722)	3.105 *** (0.722)
Parent home language: Dutch and other equal	−6.167 ** (2.170)	−6.242 ** (2.167)	−6.243 ** (2.166)
Parent home language: other	−6.740 *** (1.834)	−6.502 *** (1.839)	−6.495 *** (1.839)
Condition (EEH = 1)	−0.673 (1.649)	−0.597 (1.650)	−0.597 (1.654)
PSE change (gm)		−2.094 (1.671)	−2.078 (1.671)
PSE change (gm) × Time			−0.338 (1.068)
Random part (variances)
Repeated measures	48.156 (4.232)	48.069 (4.224)	48.058 (4.223)
Pupil	42.682 (7.796)	41.586 (7.657)	41.569 (7.655)
Class	3.504 (4.657)	3.600 (4.793)	3.660 (4.811)
School	0.359 (2.799)	1.441 (3.573)	1.373 (3.540)
Deviance	2853.256	2851.749	2851.649
Reference model		1	2
Fit improvement		χ2 = 1.507 df = 1 p = n.s.	χ2 = 0.100 df = 1 p = n.s.

Note: N repeated measures = 398; N pupils = 140; N classes = 18; N schools = 7. gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. * p < 0.05; ** p < 0.01; *** p < 0.001.

,

Table A36. Multi-level Regression—Effect of NPA Change on Language Development.

Model	1 B (SE)	2 B (SE)	3 B (SE)
Fixed part
Intercept	62.575 *** (1.855)	62.692 *** (1.886)	62.683 *** (1.889)
Time (gm)	7.328 *** (0.506)	7.329 *** (0.506)	7.299 *** (0.507)
Child gender (girl = 1)	3.482 * (1.491)	3.423 (1.499)	3.430 (1.499)
Child age (gm)	0.597 ** (0.207)	0.592 ** (0.207)	0.590 ** (0.207)
Parent SES (gm)	2.506 *** (0.728)	2.495 *** (0.728)	2.491 *** (0.728)
Parent home language: Dutch and other equal	−8.999 *** (2.473)	−9.137 *** (2.506)	−9.134 *** (2.505)
Parent home language: other	−5.001 *** (1.977)	−5.122 *** (2.008)	−5.112 *** (2.008)
Condition (EEH = 1)	0.143 (1.871)	0.047 (1.898)	0.061 (1.893)
NPA change (gm)		−0.431 (1.303)	−0.405 (1.303)
NPA change (gm) × Time			−0.759 (0.910)
Random part (variances)
Repeated measures	45.215 (4.792)	45.218 (4.792)	45.060 (4.775)
Pupil	28.312 (7.238)	28.219 (7.224)	28.236 (7.217)
Class	6.200 (6.556)	6.300 (6.591)	6.214 (6.554)
School	5.129 (6.504)	5.072 (6.494)	5.227 (6.550)
Deviance	1948.371	1948.262	1947.569
Reference model		1	2
Fit improvement		χ2 = 0.109 df = 1 p = n.s.	χ2 = 0.700 df = 1 p = n.s.

Note: N repeated measures = 276; N pupils = 99; N classes = 18; N schools = 7. gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. * p < 0.05; ** p < 0.01; *** p < 0.001.

,

Table A37. Multi-level Regression—Effects of Enactment Variables on Literacy Development.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)	5 B (SE)	6 B (SE)
Fixed part
Intercept	0.868 *** (0.071)	0.864 *** (0.071)	0.864 *** (0.120)	0.865 *** (0.120)	0.834 *** (0.123)	0.834 *** (0.123)
Child age (gm)	−0.030 ** (0.011)	−0.030 ** (0.011)	−0.027 ** (0.009)	−0.026 ** (0.009)	−0.027 ** (0.011)	−0.027 ** (0.011)
Condition (EEH = 1)	0.041 (0.092)	0.041 (0.092)	0.030 (0.166)	0.032 (0.166)	0.075 (0.172)	0.073 (0.173)
HLE change (gm)		0.152 (0.120)
PSE change (gm)				−0.041 (0.091)
NPA change (gm)						−0.007 (0.075)
Random part (variances)
Pupil	0.122 (0.056)	0.123 (0.056)	0.170 (0.021)	0.170 (0.021)	0.160 (0.024)	0.160 (0.024)
Class	0.165 (0.063)	0.160 (0.063)	0.100 (0.041)	0.100 (0.041)	0.101 (0.044)	0.101 (0.044)
Deviance	231.254	229.660	184.242	184.042	131.125	131.117
Reference model		1		3		5
Fit improvement		χ2 = 1.594 df = 1 p = n.s.		χ2 = 0.200 df = 1 p = n.s.		χ2 = 0.008 df = 1 p = n.s.
N pupils	148	148	144	144	104	104
N classes	18	18	18	18	18	18

Note: gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. ** p < 0.01; *** p < 0.001.

and

Table A38. Regression—Effects of Enactment Variables on Vocabulary Development.

Model	1 B (SE)	2 B (SE)	3 B (SE)	4 B (SE)	5 B (SE)	6 B (SE)
Intercept	4.302 *** (0.478)	4.341 *** (0.478)	4.267 *** (0.492)	4.266 *** (0.492)	4.766 *** (0.550)	4.826 *** (0.550)
Child age (gm)	−0.282 ** (0.077)	−0.281 ** (0.077)	−0.284 ** (0.078)	−0.288 ** (0.080)	−0.326 ** (0.095)	−0.327 ** (0.094)
Condition (EEH = 1)	0.180 (0.613)	0.179 (0.612)	0.163 (0.629)	0.155 (0.630)	−0.434 (0.733)	−0.568 (0.740)
HLE change (gm)		−0.457 (0.808)
PSE change (gm)				0.173 (0.783)
NPA change (gm)						−0.669 (0.641)
Variance	13.036 (1.526)	13.007 (1.522)	13.276 (1.576)	13.271 (1.575)	13.470 (1.868)	13.330 (1.849)
Deviance	789.213	788.894	770.179	770.131	565.584	564.499
Reference model		1		3		5
Fit improvement		χ2 = 0.319 df = 1 p = n.s.		χ2 = 0.048 df = 1 p = n.s.		χ2 = 1.085 df = 1 p = n.s.
N pupils	146	146	142	142	104	104

Note: gm = grand mean centered. EEH = Early Education at Home. n.s. = non-significant. ** p < 0.01; *** p < 0.001.

).

4. Discussion

Earlier results of family literacy programs underline the importance of examining implementation [7,10], as this seems key to understanding variability in intervention effects [16]. We evaluated the outcomes of EEH, a program that aims to stimulate kindergartners’ language and literacy skills by improving the frequency and quality of home literacy practices as well as the degree of parent self-efficacy and analyzed whether possible effects were associated with two aspects of parental implementation: receipt and enactment.

Regarding our first research question, contrary to our first hypothesis, results indicated no main intervention effects on children’s language and literacy skills. With respect to our second research question, we found significant relations among parent background variables and receipt quality, as measured by the observed quality of behavior and language during a program activity (shared reading). Higher-educated parents, native Dutch parents, and parents who had indicated that Dutch is their home language generally had higher scores on quality measures. This was in line with our hypothesis (hypothesis 2). No significant relations were found among parental SES, ethnic-minority status, and home language, and the receipt quantity variables: attendance at training sessions, the number of diaries handed in, and the number of activities conducted. Regarding change in the enactment variables (HLE, PSE, NPA) we found no significant relations with parent background characteristics, indicating that change in targeted behaviors and practices did not differ for low- and high-SES parents, for ethnic-minority and native Dutch parents, and for parents speaking Dutch versus another language. Analyses for research question three showed no moderation of program effects by parental SES, ethnic-minority status, or home language, indicating that the intervention was equally (in)effective for different groups of families. This is partly in line with our third hypothesis, expecting small to non-significant effects for at-risk families. Our results are not in line with other previous studies that have found various parental background variables to influence the magnitude of FLP effects (e.g., parents’ educational attainment and ethnicity [40]). Regarding our fourth research question, we found that none of our receipt variables—attendance at training sessions, diaries handed in, activities conducted, and quality of behavior and language during a program activity—significantly predicted experimental children’s language and literacy development. Our fourth hypothesis was thus not supported. Finally, results for our fifth research question showed that the EEH intervention did not improve the home literacy environment, parent self-efficacy, and quality of parents’ behavior and language during a non-program activity (prompting board). Moreover, changes in these enactment variables were not associated with children’s language and literacy development. In other words, the intervention did not contribute to children’s language and literacy skills, neither directly nor through an impact on parent variables. Thus, we found no support for our fifth hypothesis.

To some extent, our results thus support the assumption of a relationship between parental implementation and background characteristics. However, they evidently do not support the hypothesis that parental implementation has an impact on program effects. One possible explanation is that our selected variables were not the most relevant indicators of program implementation. This might be particularly true for attendance at parent meetings, as attending a training session does not necessarily mean that parents understand what is being explained to them [20]. The same can be said for parents’ engagement in activities; even if parents report a high engagement in program activities, this does not necessarily mean that the quality of these activities is sufficient for children to benefit from the program. However, parents’ engagement in training and activities can be seen as a prerequisite; if engagement is low, it is unlikely that parents will learn and apply program strategies. Furthermore, other variables possibly influenced children’s development, such as parents’ understanding of program content, and parental beliefs regarding how to best stimulate their child’s development. A match between parents’ goals and program objectives, for example, has been shown to promote positive intervention outcomes [41]. Moreover, certain variables might be viewed as prerequisites for actual program receipt to be of influence when attempting to explain intervention effects. For example, if parental beliefs regarding involvement in the child’s literacy related development are not in line with program theory (e.g., if parents do not believe the type of activities that the program offers are helpful to their child’s literacy development), the actual conducting of activities might not lead to improved child outcomes. Parental beliefs regarding their child’s development have been found to predict actions regarding parent’s literacy behaviors when around their children [42,43]. Therefore, it is likely that when parents believe that program activities can be effective in improving their child’s development, they are more likely to be able to realize desired outcomes.

Another explanation is that program effects were hampered by flaws in delivery. Information we gathered about the delivery of the program provides some grounds for this explanation. Firstly, transfer of program skills to daily situations outside of program time was largely ignored by teachers in parent meetings. In other words, the delivery of the EEH intervention was not focused on stimulating parents’ enactment of learnt skills to their day-to-day lives. This possibly (partly) explains why we did not find the intervention to enhance the quality or quantity of the HLE, by which parents supposedly would have influenced child outcomes.

A second, related issue, is that although EEH is specifically developed for low-educated and ethnic-minority parents, in practice, a broad range of parents is involved. Such high group diversity poses significant challenges for program delivery. During the parent meetings we observed, this sometimes resulted in trainers focusing on a certain subgroup of parents, while ignoring the needs of others. We observed that techniques specifically targeting low-SES and ethnic-minority parents (modeling, enacting, and role-play) were hardly used during training sessions by our intervention group teachers, even though we had trained and coached teachers in using these techniques. Several explanations for this finding were offered by teachers in post-intervention interviews. Some teachers were apprehensive of using role-play and enacting because they expected higher-educated parents to perceive these techniques as unnecessary and childish. Similar findings have been reported in previous studies: program deliverers placed little emphasis on changing parenting behavior, even though this was an explicit goal of the program [44]. Trainers sometimes feel reluctant to endorse certain program components [15]. It is possible that the observed differences between low- and high-SES parents, native Dutch and ethnic-minority parents, and native Dutch speaking parents and parents who speak another language regarding quality of program receipt would have been smaller, had the delivery been more tailored to these subgroups (hypothesis 2). In turn, this might have led to improved outcomes for children with low-SES or ethnic-minority parents, as a function of the intervention.

Third, the language of instruction during parent meetings was Dutch, which likely resulted in less optimal program delivery for parents with limited Dutch proficiency (in line with hypothesis 2). It is probable that at least part of the 13% of parents who indicated in the parent questionnaire that Dutch was not their home language, experienced difficulties in comprehending the training sessions. This might (partly) explain the lack of intervention effects found, at least for some children from ethnic-minority families. Limited language proficiency may have also played a role in implementing program activities. Although program theory recommends parents to conduct the program in their dominant language, only two parents made use of translated materials. Three parents reported to conduct program activities in their mother tongues and seven parents used a mixture of Dutch and their mother tongues. Some parents might have struggled to conduct the program in Dutch. This is supported by our finding that parents with a mother tongue other than Dutch were able to realize less stimulating parent–child interactions than native Dutch speaking parents.

Finally, regarding the quantity of delivery, we observed the duration of the parent meetings, and thus the opportunity to give in-depth instructions for program activities, to vary significantly across schools. During post-intervention interviews, teachers reported believing that the prescribed time for parent meetings would be too long for parents. A previous study reported similar findings [45]. Across multiple FLPs implemented in the UK, the amount of contact time was shorter than program guidelines suggested.

These observations stress that the way programs are transferred to parents, delivery, is an important variable to consider. It could be that the issues described here have prevented parents to cross an implementation threshold necessary for the program to yield effects: suboptimal delivery may have resulted in the suboptimal receipt and enactment we observed, and this may have led to non-effects.

4.1. Limitations

Several study limitations are worthy of note. First, we were not able to randomly assign groups to conditions. However, no significant differences were found between conditions on pre-test scores and relevant background characteristics of parents and children (child gender and age, parental SES, ethnic-minority status, and home language, richness of the HLE and parent self-efficacy), suggesting that conditions were comparable on possible confounding variables.

Second, although we included large scale observational data in our analyses, we also partly relied on parent self-report measures. Naturally, these must be interpreted cautiously. For example, there are concerns about the validity of parent reports of home literacy practices because of the high cultural value placed on these activities that may lead parents to exaggerate their self-perceptions [46].

Third, due to the limited number of schools and classes in our sample, we were not able to include delivery in our analyses, even though the variety we observed may have affected the manner in which program contents were transferred to parents. Future research should address the role of delivery in explaining program effects in a quantitative manner.

Finally, EEH program theory recommends parents to conduct the program in their most proficient language. Knowledge and skills acquired by children in the first language can then be transferred to the second language [34]. However, we were not able to test whether this transfer occurred, as our sample included a very limited number of ethnic-minority parents who conducted the program in their mother tongue. Furthermore, we did not test children’s language and literacy skills in other languages than Dutch.

4.2. Implications for Future Research

For future family literacy intervention research, we recommend a random assignment of groups to conditions with a sufficiently large sample size on the highest level (schools in the case of school-based interventions) to enable testing delivery effects. Moreover, we recommend future researchers to test the transfer-hypothesis [34] described in the previous paragraph in the context of multilingual family literacy interventions (for an example, see [47]).

4.3. Implications for Policy and Practice

Although teachers were trained during multiple well-prepared sessions, coached, and observed during program delivery to ensure fidelity, the delivery of the intervention was found to be suboptimal relative to program guidelines. The transfer of program skills to daily situations outside of program time was largely ignored, techniques specifically targeting low-SES and ethnic-minority parents were hardly used, and the majority of teachers were found to deviate from duration guidelines for parent meetings. A certain level of apprehension was found among trainers to follow guidelines, which was also reported in previous studies [15,44,45]. The main reason for this appeared to be the large diversity among parents. Program deliverers (teachers in the case of EEH) are often confronted with a very diverse group of parents. However, teachers often feel ill-prepared when communicating with parents and they may not know what to say or how to react [48]. Previous research on this topic has established that teacher education falls short of helping teachers acquire and develop the necessary interpersonal skills to engage and communicate with parents [49,50]. We therefore argue that teachers need additional training in engaging and communicating with families from various socio-economic and cultural backgrounds, in order to successfully deliver FLPs to parents and children. Recognizing this and incorporating additional training when preparing teachers to deliver FLPs can be a valuable step for FLP-developers.

Another approach to realize more differentiation in program delivery could be to deliver the program via additional home visits. This approach has been found to be more effective than a center-based approach [6] and has the advantage that delivery can be tailored to the individual needs of parents. A number of FLPs that made use of home visits with disadvantaged families showed significant effects on child outcomes [11,47,51,52]. Moreover, provided that bilingual deliverers are available for intervention implementation, parents can be instructed in their home language. Some findings suggest that home visits in parents’ home language is a beneficial way of delivering FLPs in these families [47].

Finally, an additional reason for delivery issues might be that program activities were insufficiently aligned with participating families’ literacy practices. In the late eighties, FLPs have been criticized by researchers who pointed out that these programs were mostly based on mainstream Western pedagogies and ignored the cultural capital of ethnic-minority families [53,54]. More recently, scholars have argued for a more partnership-driven approach to intervention research [6]. Such an approach relies heavily on the active involvement of stakeholders (e.g., parents and children), in order to form theories and methods that underlie study designs [55]. Specifically, incorporating more culturally sensitive program materials, for example by co-constructing activities with participating families [56] might benefit program implementation for parents from various cultural backgrounds.

In conclusion, this study has shown that high quality implementation of FLPs is not self-evident. It can be argued that examining implementation is of particular significance in the field of family literacy interventions [19,57,58]. Many FLPs have a phased design: trainers (e.g., teachers, social workers) are trained to deliver the program to parents, and parents are expected to transfer what they have learnt to their children. All these phases need to be implemented as intended to be able to realize desired program effects. If delivery is flawed, suboptimal receipt and enactment are a likely consequence. Future studies should therefore comprehensively examine the role of implementation in program evaluations, while in practice, systematic preparation for implementation of FLPs is necessary to help parents in supporting their young children’s literacy development.