Parental Stress Assessment with the Parenting Stress Index (PSI): A Systematic Review of Its Psychometric Properties

Parental stress is a construct related to people’s perception of difficulties and feelings of not being able to cope with the demands of parenting. This construct is often experienced as a negative or aversive response to parental obligations, and the available evidence also suggests that excessive parenting stress reduces the use of positive parenting behaviors and are related to dysfunctional parenting. Different instruments exist to assess parental stress. This article is part of a project to translate and adapt the Parenting Stress Index (PSI), fourth edition, in its two forms (full and short). The aim of this research is to identify the psychometric indicators obtained by this instrument and to review the evidence they can provide. Method: Following the PRISMA guide (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), the studies related to the PSI were identified in different databases (ERIC, PsycArticles, PubMed, Scopus and Web of Science). Results: The screening process resulted in 16 articles; four have analyzed the psychometric properties of the PSI-4 and the rest have studied the PSI-3. Although version 4 was published in 2012, the studies are scarce. However, they follow the line noted in the previous short version, a high internal consistency and a factor structure of three factors. Conclusions: The difficulties of working with a measurement instrument with 101 items means that the full version of the PSI has been little studied, except in translation and linguistic adaptation studies.


Introduction
Parental stress has been defined as stress related to parenting, as opposed to other forms of stress experienced by parents, such as economic hardship, work or academic stress or negative life events. The study of that construct is of great clinical and research interest because it has been linked to negative parenting characteristics, such as low levels of parental warmth, unhealthy parenting styles, harsh discipline and potential child neglect or abuse [1,2]. The impact of high levels of parental stress can have a direct impact on a child's socio-affective and cognitive development [3][4][5]. Parenting stress has also been associated with other disorders such as parental anxiety and depression [6][7][8], marital conflict [9,10], poorer physical health [11,12], reduced parenting effectiveness [13,14] and as a source, origin or reinforcement of children's behavioral problems [15][16][17][18]. From toddlerhood to adolescence, increased parental stress can create a chaotic family environment that would contribute to an increase in children's behavioral problems [19].
Different instruments exist to measure parental stress with different levels of evidence [20] (e.g., Caregiver Strain Questionnaire (CGSQ; [21]); Family Impact Questionnaire (FIQ; [17]); Parenting Daily Hassles Scale (PDH; [22]); Parental Stressor Scale [23]; Stress Index for Parents of Adolescents (SIPA; [24]); Questionnaire on Resources and Stress (QRS-F; [25]) and Parental Stress Scale (PSS; [26] or related to specific times such as labor or postpartum or preterm birth [27][28][29][30]). The Parental Stress Index (PSI) [31] is one of the most common and widely used instruments for both clinical and research purposes [32,33] especially in its short forms due to its ease of application. The PSI has been used as a gold standard to validate other instruments [24,26]. In specific areas such as neurodevelopmental disorders (e.g., intellectual disability, autism spectrum disorders, hyperactivity, among others), severe or chronic health problems in children, the use of the PSI is very common [34]. There is currently no official version of the PSI (full or short form) available in Spanish, which has motivated the project of translation and adaptation of this instrument of which this article is a part.
The PSI model [31] considers parental stress to be composed of two dimensions: general stress associated with parental demands and stress that is specifically derived from the child's own demands. This scale assesses areas of dysfunctional parent-child relationships with children between 1 month and 12 years of age, in order to identify dysfunctional relationships [31]. The original PSI [31,35] measures parenting stress perceived by the caregivers. It was a five-point Likert-type scale including 120 items that consisted of two domains: the child and parent characteristics domain (101 items) and the optional stressful life events domain (19 items). Specifically, the 101 items are divided into 47 items for the child domain and 54 items in the parent domain.. The two domains are structured in a total of 13 sub-domains. The PSI-Short Form (PSI-SF) is composed of 36 items [36], which is a direct derivation of the full-length PSI and three-factor structured (Parental Distress, Difficult Child and Parent-Child Dysfunctional Interaction). Since the third edition of the PSI was published in 1995, a considerable body of literature has accumulated evidence that confirmed its clinical utility and good psychometric properties [20]. Given major demographic and family changes in recent decades, a revision was proposed which has resulted in the PSI 4th edition [37]. The changes introduced in the 4th version focus on two fundamental points: (a) in contrast to the lack of specification in the 3rd version, the 4th version evaluates a particular child, and (b) the language has been adapted so that it can be used in the various current family forms (single-parent families, reconstituted families, foster families, homoparental, etc.). Since the first version of the PSI, different studies have been published showing good internal consistency and adequate test-retest reliability, as this instrument has been translated (in their forms and versions) into different languages and validated in different cultural groups (e.g., China [38,39], Portugal [40], France [41], Canada [42], Finland [43] and the Netherlands [44]).
The main goal of the PSI-4 is to address weak items and outdated language without altering the empirically validated and clinically relevant structure in assessing the overall level of parental stress. The main improvements included in the PSI-4 are cultural sensitivity of language in the items, increased internal consistency of the scales and item loading factor in the scales; moreover, norms based on age, mastery level and subscales have been added, and parents have been included in the standardization sample; finally, T-scores have been added to improve interpretation of the PSI [37]. Another significant change in PSI-4 is that the answers are made regarding the stress produced by raising a particular child. Earlier versions and other questionnaires ask about parenting in general. This fact makes this new version more efficient and specific, since raising all children does not generate the same level of anxiety. In this way, PSI-4 allows to identify, at a clinical and non-clinical level, the problem areas and strengths in relation to the child, the parents and the family system in order to be able to carry out the adequate design of a treatment and follow-up plan [37].

Present Study
Overall, the PSI has become one of the most widely used instruments to measure parental stress across a wide range of families and children (including those with disabilities). Although the PSI has been applied in a variety of studies, few studies have examined its psychometric integrity [20]. So, this article is a systematic review of the literature related to the psychometric properties of the PSI instrument (both third and fourth versions) in its two modalities: long and short form. Our main objective is to identify studies that analyze PSI from a psychometric point of view and cross-cultural studies to review the evidence that these articles can provide in order to prepare the design of the PSI-4 Spanish version.

Search Strategy
Following the PRISMA guide (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) (Figure 1), one database has been selected for each adjoining area of knowledge: one for health (PubMed), one for psychology (PsycArticles) and one for education (ERIC) and two general databases (Scopus and Web of Science) All searches included the English key terms parenting, stress and index. The search was performed by accessing all the databases using the online search interface TROBES of the Documentation and Library Service of the University of Valencia (Spain).
Children 2022, 9, x FOR PEER REVIEW 3 of 12 related to the psychometric properties of the PSI instrument (both third and fourth versions) in its two modalities: long and short form. Our main objective is to identify studies that analyze PSI from a psychometric point of view and cross-cultural studies to review the evidence that these articles can provide in order to prepare the design of the PSI-4 Spanish version.

Search Strategy
Following the PRISMA guide (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) (Figure 1), one database has been selected for each adjoining area of knowledge: one for health (PubMed), one for psychology (PsycArticles) and one for education (ERIC) and two general databases (Scopus and Web of Science) All searches included the English key terms parenting, stress and index. The search was performed by accessing all the databases using the online search interface TROBES of the Documentation and Library Service of the University of Valencia (Spain).

Inclusion and Exclusion Criteria
The search was restricted to studies published between 2012 and 2022, so all studies were developed after the publication of the PSI-4 [37]. We excluded all publications that were not published in publishers following blind peer review (e.g., books, book chapters, working papers, conference proceedings and master's theses). Review studies or clinical trials were also excluded if they did not provide psychometric data on the PSI-4.

Procedure
A total of 79 articles were found in the initial keyword search. After removing duplicate articles, 45 articles remained. Subsequently, the articles were screened by reading the titles, selecting only those in which all the keywords (parenting, stress and index) appear in the title. Finally, 25 articles were excluded in the screening phase. The remaining articles

Inclusion and Exclusion Criteria
The search was restricted to studies published between 2012 and 2022, so all studies were developed after the publication of the PSI-4 [37]. We excluded all publications that were not published in publishers following blind peer review (e.g., books, book chapters, working papers, conference proceedings and master's theses). Review studies or clinical trials were also excluded if they did not provide psychometric data on the PSI-4.

Procedure
A total of 79 articles were found in the initial keyword search. After removing duplicate articles, 45 articles remained. Subsequently, the articles were screened by reading the titles, selecting only those in which all the keywords (parenting, stress and index) appear in the title. Finally, 25 articles were excluded in the screening phase. The remaining articles were examined independently by two researchers, who agreed to exclude four more because they did not provide psychometric data. The remaining 16 articles were used to prepare the present paper. Initially, the aim was to include articles that examined the psychometric properties of the latest version (PSI-4); however, given the small number of identified studies of the 4th version, and understanding that it was relevant to compare the psychometric data with those of the 3rd version, it was decided to include the items of the 3rd version as well. All articles were included regardless of the form (SF or LF) studied. The complete reading and selection of the articles was carried out by the first two authors of this work and supervised by the last two of the authors. Table 1 presents a brief description of the main characteristics of each of the 16 selected articles, including the author(s), type of study and sample and different psychometrics data (e.g., internal consistency; test-retest reliability, convergent validity and construct validity).

Results
Luo et al. [45] explained that, despite its popularity, there is a lack of consensus on the factor structure of the PSI-SF (e.g., [32,46] As noted above, very few studies have studied the PSI-SF in the Spanish population. All of them are from version 3, although they were developed many years after version 4 was published; this is justified by the lack of an official 4th version in Spanish. Of these, only five studies have been developed on the Spanish population; the study by [47], which analyses the psychometric properties using the Rasch model of the Spanish version of the PSI-SF, with a sample of 542 participants, male and female, parents of children with intellectual disabilities; the research by [48] studying the factor structure and psychometric properties of the Spanish version of PSI-SF with a sample of 309 mothers (203 with difficulties managing their children's behavior and 106 from the general population); and finally, the study by [49] on the validity of the PSI-SF in a sample of at-risk mothers with a sample of 149 participants.  Four studies have analyzed psychometric properties of the PSI-4. Two of them have focused on the short version of the instrument [41,57] and the other two on the long version [58,59].
Barroso et al. [57] conducted a three-moment follow-up study of 58 mothers of preterm infants aged 12-15 months with behavioral problems and low-income Hispanic origin. English and Spanish versions of the PSI-4-SF are used. Overall internal consistency (alpha < 0.91) as well as test-retest stability (r < 0.58) and ICC intraclass correlations (<0.77) show adequate psychometric performance. The second study [41] presents the validation of the French version of the PSI-4-SF with a sample of 318 participants (cross-sectional design with three samples collected from the background population and a sample of parents who have a child with a chronic illness). The data show that the internal consistency is adequate (alpha < 0.89); moreover, the CFA developed on the original three factors model allows us to accept this structural hypothesis.
Given the length of the full version of the PSI-4, it is difficult to replicate the factor structure and, thus, the construct validity. Not surprisingly, no studies of PSI-3 LF were found. Çekiç et al. [58] have translated and adapted the PSI-4 long version into Turkish, with a sample of 386 parents of children with psychological problems. The internal consistency is adequate (although the study does not provide the alpha for the total test) in each domain (Child Domain 0.92 and Parents Domain 0.95), decreasing slightly in each of the thirteen subdomains depending on the number of items that compose them, and stability of the measure (test-retest) is adequate (0.78). To test construct validity, a separate CFA was conducted for each of the two domains. The results confirm that each domain is formed by the corresponding subdomains; however, an AFC was not performed on all 101 items to confirm the structure of the two domains as second-order factors.
Pereira et al. [59] developed a Portuguese version with 53 Brazilian mothers of children born preterm. The internal consistency coefficients (alpha) are adequate (in all domains and subdomains above 0.80). Given the size of the sample, it was decided to perform a principal component analysis (PCA with varimax rotation) on the scores of the 13 subdomains, finding a bifactorial structure compatible with the two domains (parents and children).

Discussion and Conclusions
Over the past few decades, there has been an increase in both clinical and research interest in measuring and understanding the effects of parental stress [20]. Testing the existence of reliable and valid measurement instruments is one of the fundamental aims of this work. If we have these tools, we can consider evaluating the level of effectiveness of our interventions in family, clinical or preventive settings. Although scarce, there is evidence on the value of the PSI in its different versions [31,35,37,60] for measuring parental stress and, thus, identifying dysfunctional parental relationships. Over the last decades, efforts to understand parental stress have steadily increased. Many studies have attempted to analyze the psychometric properties of the PSI, which assesses parent-child problem areas in children between one month and twelve years of age to identify dysfunctional parent-child relationships [35].
Holly et al. [20] provide a comparative study of eight parental stress measurement instruments (considering the PSI and PSI-SF as two different instruments). The frequency of use in the analysis of the found data shows that the PSI (in its two forms) is not only the most widely used tool, but also the one that provides the best evidence on its norms, validity, reliability and usefulness of its measures.
In adaptations of scales to cultural and linguistic environments other than the original one, according to the International Testing Commission (ITC) [61,62], the confirmation guidelines propose to carry out studies on the reliability, validity and metric equivalence between each of the items that make up the test and the dimension they represent.
Thus, to avoid methodological bias, given that Likert-type items are polytomous, Pearson's correlation was not used. This statistic requires continuity of the variable and measurement on an interval scale, but polytomous item categories do not meet either of these qualities, so it is necessary to use a factor estimation method suitable for this type of variable. In this case, two very common methods can be used: the unweighted least squares method or the robust weighted least squares method (e.g., [63]). The latter is better; the drawback is that it is only implemented in Mplus [64], but the first method can produce a good approximation of the factor structure of an instrument when performing an EFA.
A clearly neglected aspect in psychometric studies with the classic test model is the study of the goodness of fit of the number of categories used to obtain information in the items. That is, the classic test model assumes that categories are set by authors regardless of their ability to obtain the most appropriate information from individuals about the measured attribute. However, there are multiple studies showing that the use of fewer or more categories, whether odd or even, produces clear discrepancies in obtaining sensitive information about the measured psychological trait. Nevertheless, for the past four decades, methodology based on Rasch and IRT modelling has made it possible to study the functioning of item categories in detail [65,66].
Factor studies of the long version of the PSI are scarce and methodologically weak, and are generally associated with translation, adaptation and typing studies in which the sample must be sufficiently large. It is within the framework of these studies that the construct validity of the full PSI should be confirmed and the same for the short version. Moreover, most of the found studies have been conducted on clinical or at-risk samples. Sometimes, the factor structure is not replicated in such samples, so it is advisable to conduct validation studies on general population samples, without known problems. It should be understood that parental stress is a construct that manifests a continuum of measurement, so that higher or lower scores can be used as an indicator of dysfunction. The use of clinical samples or samples drawn from the at-risk population to estimate diagnostic validity is appropriate; however, the factor structure of the scales calculated solely from the scales may lead to the erroneous assumption that the scale structure is invalid when, perhaps, what is revealed is only the structure of the sample used, with a specific pathology.
Consequently, the importance of continuing research on the PSI and developing, if possible, a coherent model for the two forms (LF and SF) should be emphasized. The clinical use of the PSI has shown that it is a useful tool for the early detection of high levels of stress in families and the analysis of its origin (both in clinical and non-clinical samples), which may allow them to improve their coping strategies and skills through the design and creation of preventive and family intervention programmers, as well as individualized psychosocial and educational resources adapted to their circumstances in order to reduce their stress to an optimal level [37,47,67].