Parental Alienation-Development and Validation of a Behavioral Anchor Scale

Abstract

(1) Objective: Parental alienation phenomenon was given an increased attention during the recent years in both the clinical and the legal fields. This study aims to introduce the development of a new parental alienation questionnaire. (2) Method: Four studies were performed. In the first two studies in which five specialists participated, the questionnaire was defined, and the inter-rater reliability was verified. The next study, which included 267 participants, sought to establish the construct validity by applying Exploratory Factor Analysis and Confirmatory Factor Analysis. The last study that enrolled 200 participants considered predictive validity as well as test-retest reliability. (3) Results: Eight criteria of parental alienation have been identified: Campaign of denigration, frivolous, weak, or absurd rationale for the alienation, lack of ambivalence towards the alienating, lack of guilt or remorse about the alienation, borrowed scenarios, independent thinker phenomenon, taking the alienating parent’s side in the conflict, and spread of alienation to the extended family of the targeted parent. (4) Conclusion: The results highlighted the proper psychometric qualities of the questionnaire. The Parental Alienation Questionnaire (PAQ) seems to be a promising tool not only for clinical and judicial practice, but also for research.

1. Introduction

Parental alienation occurs when one of the parents intentionally interferes with the relationship between the child and the other parent. This interference from one of the parents involves behaviors designed to denigrate the alienated parent and to reduce the contact between the child and that alienated parent, a fact that leads to the rejection of that parent by the child [1]. Parental alienation phenomenon was initially mentioned in the scientific literature as the “pathological alignment”, a phrase introduced in 1976 by Wallerstein and Kelly. They described “the pathological alignment of a parent and a child, having as a result of the rejection of the alienated parent by the child” [2]. As a result, in family conflicts emerging from the separation of a family with children, rejection of a parent without obvious reasons was identified as a phenomenon with a significant presence. Consequently, the experts’ concerns regarding this phenomenon became more and more obvious, so the specialized literature documented ways not only to define the concept but also to identify the impact the parental alienation has upon the child [3,4,5,6,7,8,9,10].

There are certain authors [8] who see parental alienation as a form of emotional abuse where the child/children is/are used as means of revenge by one parent against the other parent who, more frequently, was a good parent. Besides the concept of parental alienation, within the scientific literature there are also mentioned the concepts of parental alienation syndrome (PAS) [3] and parental alienation disorder (PAD) [11].

Parental alienation syndrome was initially conceptualized by Gardner [3], being subsequently developed in his studies as well as in the research authored by other experts. Thus, he defines PAS as “a disorder that arises primarily in the context of child-custody disputes. The disorder results from the combination of alienating parent’s indoctrinations involving programming (brain-washing) the child’s mind and the child’s own contributions to the vilification of the alienated (targeted) parent” [4,12]. Parental alienation syndrome refers to a relational pathology that usually occurs in circumstances related to marriage dissolutions, but it may also precede the divorce [13].

PAD has been defined as an attachment disorder [11]. The authors hold that this disorder is not predominantly genetic or constitutional but is mainly caused by what parents or caregivers do and say to their children [11]. Walker and Shapiro (2010) argue that PAD is a newer form of what was originally called as Gardner’s Parental Alienation Syndrome (PAS).

As a response to the need of developing new suitable tools to assess parental alienation and its efficacy, in 2002, Laughrea [14] developed a self-reporting scale called “Alienated Family Scale Relationship “(AFRS). The scale aimed to identify the dynamics of PA within the family, from the perspective of young adults. Monè and Biringen, in 2006 focused on measuring the perceptions of young adults on the way they perceived the parental alienation (PA) experience during their childhood. The tool, known as the Relationship Distancing Questionnaire (RDQ), assesses the extent to which an individual felt alienated from one or both parents during childhood due to parental alienation [15].

Baker and Chambers (2011) developed the Baker Strategy Questionnaire (BSQ), which aimed to measure specific parental alienation behaviors that parents may have in relation to their children [16]. Rowlands (2018) developed the Rowlands Parental Alienation Scale, a tool built to capture the eight areas of parental alienation, as they are described in the literature and which may be applied to parents during the psychological assessment process [10].

As it can be seen the questionnaires that were developed for measuring PA are addressed either to children subjected to PA, or adults who as children have undergone PA situations caused by a parent/caregiver or to their parents. All the measures were based on self reports of the actors (children or parents/caregivers) of the PA situations. What has not been done at all is a tool, an objective measure for psychologists evaluating parental alienation.

2. Results

2.1. Study 1

The first study aimed to develop a psychological analysis tool designed identify the degree of parental alienation in children who have been subjected to dysfunctional relationships with their parents [13], in relation with eight criteria (a campaign of denigration of one of the parents; weak, absurd, or frivolous rationalizations for the deprecation; lack of ambivalence; “the independent thinker” phenomenon; reflexive support of the alienating parent in the parental conflict; absence of guilt over cruelty to and/or exploitation of the alienated parent; the presence of borrowed scenarios; and the spread of animosity to the friends and/or extended family of the alienated parent), depending on the three stages of PAS: Mild, moderate, and severe PA [16].

Procedure. A number of five Romanian experts, four females (75%) and one male (25%), with an average age of 43.5 years old and an average professional seniority for 16 years, all clinical psychologists, with a specialization level varying from specialist to consultant, participated in this phase.

These eight criteria were in fact the elements upon which the questionnaire was developed: The campaign of denigration of the rejected/target parent, the child’s unconvincing arguments, the absence of the child’s ambivalence towards the rejected/target parent, the “independent thinker” phenomenon, the unconditional support of the preferred/denigrating parent, the absence of guilt, the borrowed scenarios, and the extension of animosity upon the rejected parent’s family [17]. The tool was designed as a behaviorally anchored rating scale, depending on the frequency of the occurrence of such behavior (where 0 refers to the absence of behavior and 4 refers to the highest frequency of such behavior): Absent-0, Rare (1–3 times)-1, Moderate (3–6 times)-2, Often (6–8 times)-3, All the time (over 10 times)-4. After defining the criteria that are characteristic for parental alienation [17], 119 items were designed to cover all the aforementioned eight areas.

These items were submitted for analysis to the group of experts in order for them to carry out a qualitative analysis. The task assigned to the experts was to analyze the extent to which the content of these items reflects the criteria and dimensions that have been put forward, using an assessment scale from 1 to 10, where 1 represents the total lack of relevance of the item for the criterion/dimension that was developed, and 10 represents the absolute relevance of the item for the respective criterion/dimension.

Results. Only the items that received an average score of 8 to10 points were considered significant and were kept to the final form of the questionnaire. Following the analysis conducted by the experts, the number of items was reduced from 119 to 101, a number of 18 items being thus eliminated, as they scored less than 8 points. We named the instrument of 101 items resulting from Study 1 the Parental Alienation Questionnaire (PAQ).

2.2. Study 2

The second study aimed to analyze the inter-rater case reliability in order to determine the tool’s reliability.

Procedure. Under the analysis focused on the psychometric properties of the PAQ, the inter-rater reliability was verified by analyzing their evaluation consent for a number of 10 cases. In this stage, a number of five Romanian experts, four females (80%) and one male (20%), with an average age of 43.4 years and an average professional seniority of 16.2 years, all acting as clinical psychologists, with a specialization level that ranged from specialist to consultant psychologist, had the task to assess individually with PAQ a number of 10 cases.

Every expert individually assessed the same 10 cases involving children (both boys and girls) who were subject to the psychological assessment procedure ordered by a court of law in relation to custody proceedings. All five evaluators received information on how to rate and use the tool; subsequently, the inter-rater reliability of case analyzes was studied. The inter-rater reliability coefficient (r_wg) was used to measure the inter-rater reliability. Its values were in the range [0.84, 1.00].

Results. The results that have been obtained outlined the existence of a proper inter-rater reliability. The minimum value was 0.84 in two cases (20%), with most of the cases having a r_wg of 0.94 (70%). The maximum value was 1.00, indicating a complete agreement among the raters. This shows that the description of both the tool and the assessment method is clear, so that different raters obtain results which are closer in terms of rating (

Table 1. Inter-rater reliability at the level of items.

Case	Age	Gender	rWG Values for 5 Independent Raters at the Item Level
			Min	Max	Mean
Case 1	6 years	M	0.94	1.00	0.95
Case 2	8 years	M	0.94	1.00	0.95
Case 3	10 years	F	0.94	1.00	0.95
Case 4	17 years	M	0.94	1.00	0.95
Case 5	12 years	F	0.94	1.00	0.95
Case 6	9 years	F	0.94	1.00	0.95
Case 7	11 years	M	0.94	1.00	0.95
Case 8	7 years	M	0.86	1.00	0.95
Case 9	13 years	F	0.84	1.00	0.94
Case 10	7 years	F	0.84	1.00	0.94

).

2.3. Study 3

The third study was concerned with optimizing PAQ in order to explore and confirm its theoretical structure.

Procedure. A number of 267 subjects of Romanian origin, i.e., 157 boys (58.80%) and 110 girls (41.20%), aged from 3 to 18 years old (girls’ average age: 10.94; boys’ average grade: 10.45, and overall average age: 10.74 years, SD = 3.94), who came from divorced/separated families and/or families undergoing divorce/separation proceedings, attended this stage. Their cases were analyzed using PAQ by clinical psychologists specialized in assessing the child and the divorce circumstances, as well as by clinical psychologists/psychotherapists specialized in psychological interventions in children. All these assessments or psychological interventions took place within the context of certain procedures required for determining and setting the custody or residence of the child, upon the order of a law court, the police, or the prosecutor’s office.

Results. One of the main objectives of the study was to determine the internal attributes that underlie the aforementioned items of PA and that influence evolution thereof. To do this, we employed the SPSS 20 program, using the principal axis factor analysis to extract the common factors. Prior to implementing this analysis, we analyzed the applicability of the method in relation to the set of data we considered. Therefore, the overall result for the Kaiser–Meyer–Olkin test was 0.97, all individual results being also higher than 0.9. According to Field (2000) these values are excellent so that one may conclude that the sample is eligible for the application of the factor analysis. To verify the extent to which we are dealing with an identity matrix, the Bartlett’s test of Sphericity was applied. The statistically significant result (p < 0.0005) of this test also indicated that it is possible to apply the factor analysis. As the applicability of this method was successfully verified, the analysis was applied onto the 101-item questionnaire that quantified the characteristics of parental alienation for 267 respondents.

We obtained the factorial solution after the completion of the principal axis factoring and the Direct Oblimin oblique rotation. This type of rotation was chosen because, according to the theoretical model that has been outlined, we expected to find correlations between factors. Twenty-four items with a saturation over 0.50 were selected and a second factor analysis was applied to obtain the final solution for them. The degree of commonality for each item was then analyzed; subject to analysis was also the saturation of each factor that has been extracted.

The degree of communality of the variables is highlighted in

Table 2. Degree of commonality of items.

Item	Commonality Extraction
i01	0.92
i02	0.91
i05	0.88
i22	0.89
i25	0.91
i26	0.90
i30	0.91
i31	0.95
i32	0.94
i40	0.94
i41	0.93
i44	0.88
i53	0.92
i54	0.94
i55	0.93
i66	0.91
i67	0.95
i68	0.90
i83	0.86
i84	0.94
i85	0.92
i95	0.95
i96	0.96
i97	0.94

Note: Extraction method: Principal axis factoring.

. For all items, the communality presented values that were higher than 0.50; so, these values indicate that each of these items was largely influenced by the factors that have been extracted.

Table 3. Total variance explained.

Component	Initial Eigen Values			Extraction Sums of Squared Loadings			Rotation
	Total	% of Variance	Cumulative%	Total	% of Variance	Cumulative%	Total
1	18.33	76.38	76.38	18.33	76.38	76.38	11.26
2	1.17	4.90	81.29	1.17	4.90	81.29	12.79
3	0.87	3.65	84.95	0.87	3.65	84.95	8.00
4	0.49	2.08	87.03	0.49	2.08	87.03	14.29
5	0.39	1.62	88.65	0.39	1.62	88.65	10.42
6	0.34	1.44	90.10	0.34	1.44	90.1	10.59
7	0.31	1.29	91.39	0.31	1.29	91.39	11.94
8	0.25	1.07	92.46	0.25	1.07	92.46	14.60
9	0.21	0.90	93.36
10	0.19	0.80	94.17
11	0.16	0.69	94.86
12	0.16	0.68	95.55
13	0.13	0.57	96.13
14	0.12	0.53	96.66
15	0.11	0.46	97.13
16	0.10	0.43	97.57
17	0.09	0.41	97.98
18	0.09	0.38	98.37
19	0.08	0.35	98.72
20	0.08	0.34	99.06
21	0.06	0.27	99.34
22	0.06	0.25	99.59
23	0.05	0.21	99.8
24	0.04	0.19	100.00

highlights the total variance obtained by principal axis factoring. The result is saturated items in eight factors that, taken all together, account for 92.47% of the total dispersion of the items. Each factor influences 76.4%, 4.91%, 3.65%, 2.08%, 1.63%, 1.45%, 1.29%, and 1.07% of the total variance, respectively.

As can be noticed from Table 3, the factor analysis shows as the most significant factor, covering 76.38% of the variance, the unconvincing arguments that a child brings in order to explain the reasons of rejecting the meetings with the parent. The second main factor covering 4.90% of the variance refers to extended animosity related to the parent’s extended family members. The factor correlations in the rotated solution are presented in

Table 4. Component correlation matrix.

	Factors
	F1	F2	F3	F4	F5	F6	F7	F8
F1	1
F2	0.57	1
F3	−0.50	−0.47	1
F4	0.68	0.67	−0.59	1
F5	0.47	0.52	−0.28	0.57	1
F6	0.50	0.48	−0.33	0.61	0.65	1
F7	0.54	0.64	−0.48	0.61	0.57	0.52	1
F8	0.66	0.71	−0.52	0.75	0.6	0.63	0.67	1

Note: Extraction Method: Principal Component Matrix. Rotation method: Oblimin with Kaizer Normalization.

. As can be seen, factor correlations are acceptable for the oblique rotation. Taking into consideration the theoretical model of eight factors that guided the construction of the behavioral anchor scale and also the value of correlation coefficients for the items (

Table 5. Exploratory Factor Analysis (EFA)—final factor loadings.

PAQ	Factor Loading
	1	2	3	4	5	6	7	8
i01			−0.64
i02			−0.51
i05			−0.43
i22	0.52
i25	0.53
i26	0.48
i30				0.32
i31				0.90
i32				0.91
i40						0.71
i41						0.81
i44						0.33
i53					0.58
i54					0.65
i55					0.74
i66							0.60
i67							0.70
i68							0.52
i83								0.51
i84								0.87
i85								0.73
i95		0.94
i96		0.97
i97		0.93

Note: PAQ-Parental Alienation Questionnaire. 1. Unconvincing arguments, 2. Extension of animosity, 3. Denigration of the rejected parent, 4. Absence of ambivalence, 5. Unconditional support, 6. Independent thinker, 7. Guilt free, 8. Borrowed scenarios.

) we focused on a model with two main factors and six subsidiary ones.

The final factor loading is outlined by Table 5.

The content analysis indicates that three items are related to factor one and highlight the child’s unconvincing arguments regarding the rejection of the alienated parent. When giving these arguments, the child is staring with an accusing look, at the same time with a convincing but yet tense and defensive attitude. Thus, the factor one may be labelled as “unconvincing arguments”. This is the main factor that is explaining most of the scale. The high importance of this factor explains the need of systematic observation of the situations that generated the child’s arguments. This is one of the two main factors of the model, the second being animosity towards parents’ close relatives.

Three other items are related to the second factor and express the extension of the child’s animosity upon the rejected parent’s friends or extended family. The animosity is manifested by gestures and face grimaces of depreciation and scorn, by a defensive bodily attitude, and the avoidance of or no eye contact when talking about them. This factor may be labelled as “animosity expansion”.

Three items saturate the third factor, and they are focused on the denigration of the rejected parent. The children affected in this direction are using insulting names in respect to this parent, they criticize him/her and have a sharp tone when talking about him/her. Thus, factor three may be called “denigration of the rejected parent.”

The absence of the child’s ambivalence towards the rejected parent is manifested by the sharp way of bringing up counterarguments but also by the lack of spontaneous or challenge-based verbalization as well as the lack of the positive experiences shared with this parent. Items describing these issues are loaded into factor four, which may be titled as the “absence of ambivalence”.

The three items related to the factor five are focused on the unconditional support of the preferred parent. The children who were subject to this criterion are verbally approving the alienating parent’s negative attitude towards the other parent, they even justify it and find arguments to support it. Therefore, this factor may be called “unconditional support”.

Factor six, due to the items it usually loads, seems to be closely linked to the fact that the child states that the parent’s rejection is a personal rejection based on his/her own opinions and, at the same time, he/she is building several scenarios to support this. This factor may be called “the independent thinker phenomenon”.

Three other items reflect the lack of guilt towards the rejected parent, which is manifested not only by biased statements regarding the fact that the parent deserves to be punished this way but also by verbalizations, which substantiate the fact that this behavior towards the alienated parent is normal. The seventh factor to which these items are loaded may be labeled as “guilt-free” factor.

The eighth factor is related to the items that describe the defensive posture, the convincing tone, and the direct, sustained contact of the child when using negative terms and words to the alienated parent. This factor may be called as the “borrowed scenarios” factor.

Therefore, we can say that the interpretation of the data is consistent with the characteristics of parental alienation for whose measurement the questionnaire was developed; furthermore, the factors “unconvincing arguments”, “extension of animosity”, “denigration of the rejected parent”, “absence of ambivalence”, “unconditional support”, “independent thinker phenomenon”, “guilt-free” and “borrowed scenarios” have high loads (Table 5).

The internal consistency of the factors was verified using the Cronbach’s alpha coefficient on a sample of 267 participants 157 boys (58.80%) and 110 girls (41.20%), overall average age: 10.74 years, SD = 3.94). As one may see from

Table 6. Alpha Cronbach.

Factor	Factor Name	Items	Alpha Cronbach
1	Unconvincing arguments	i22, i25, i26	0.95
2	Extension of animosity	i95, i96, i97	0.98
3	Denigration of the rejected parent	i01, i02, i05	0.94
4	Absence of ambivalence	i30, i31, i32	0.95
5	Unconditional support	i53, i54, i55	0.96
6	Independent thinker	i40, i41, i44	0.94
7	Guilt free	i66, i67, i68	0.95
8	Borrowed scenarios	i83, i84, i85	0.94

, all eight factors have internal consistency, the alpha coefficients varying from 0.94 to 0.98, a fact that demonstrates a strong reliability.

The statistics afferent to every sub-scale are outlined in

Table 7. The sub-scale statistics.

	F1	F2	F3	F4	F5	F6	F7	F8
Means	6.59	4.97	6.91	6.99	7.11	7.48	6.76	6.66
Standard deviations	4.33	4.51	4.46	4.39	4.44	4.41	4.51	4.19
No of items	3	3	3	3	3	3	3	3

.

After completing the exploratory factor analysis, confirmatory factor analysis was used to test the hypotheses. Four different models were specified and subsequently tested using a hierarchical factor solution. The first model outlined a single-factor structure that included all 24 items of the scale while the all other three models had an eight-factor structure containing three items. Therefore, the structure of the second specified model included eight uncorrelated factors, the structure of the third model included the eight correlated factors, and the structure of the fourth model contained eight uncorrelated factors and a second-order factor. Consequently, Figure 1 graphically represents the hierarchical factor solution with eight uncorrelated primary factors and a second-order factor.

The criterion was verified for each of these models [18] and it has been found that each model is testable.

The actual testing of the models was carried out by using the lavaan version-0.6-5 in the R Core Team, 2014. One may notice that χ², which indicates the matching degree, is statistically significant for all the models we have considered. Knowing that this indicator is sensitive to the volume of the sample and taking into account the large number of data that has been analyzed (N = 267), this result was interpreted with caution and other indicators were further analyzed. For both the one-factor model and the eight uncorrelated factor model, the indicators show a weak matching. However, a better match was obtained for the model with eight correlated factors and for the model with eight uncorrelated factors and a second-order factor, but without having the indicators reaching the optimal values. It has also been noticed that these two models are comparable in terms of their index values, the latter model having the advantage that it is much simpler. Therefore, in order to obtain an optimized model, the latter model was considered, and it has been subject to the application of the index modification procedure. Following this approach, the fifth model was obtained with a structure with eight factors, of which two factors (factor 4 and factor 5, factor 5 and factor 6) and four items (i31-i32; i40-i41; i66-i67; i84-i85) were correlated, and a second-order factor. The results we have obtained were summarized in

Table 8. Confirmatory Factor Analysis (CFA) results: Goodness of fit statistics for the five models of the PAQ.

Model	χ2	df	χ2/df	CFI	TLI	RMSEA (90% CI)	SRMR	∆χ2	∆df
24 items, one factor	2320.621 **	252	9.21	0.79	0.77	0.18 (0.17, 0.18)	0.05	-	-
24 items, eight factors a	3562.702 **	252	14.14	0.67	0.63	0.22 (0.22, 0.23)	0.68	1242.08	0
24 items, eight factors correlated b	722.937 **	224	3.23	0.95	0.94	0.09 (0.09, 0.10)	0.03	1567.68 *	28
24 items, eight uncorrelated primary factors	892.261 **	244	3.66	0.94	0.93	0.10 (0.09, 0.11)	0.04	1428.36 *	8
& 1 second order factor c
24 items, eight uncorrelated primary factors	644.552 **	276	2.34	0.96	0.96	0.08 (0.07, 0.09)	0.03	1676.07 *	24
& 1 second order factort after mi d

Note. * p < 0.05; ** p < 0.01, ^a = uncorrelated factors, ^b = correlated factors, ^c = second order factor; ^d = second order factor + correlated factors and correlated items; all models are compared to the 24 items, eight factors model.

.

We noticed that the best match was reached by the solution involving eight factors with two factors and four correlated items and a second-order factor. For this model, all indices have reached the optimal values. This model was compared to the previous models and it was found that ∆χ² was statistically significant for each and every of these comparisons, a fact that confirms its superiority over them. Therefore, it seems that the latest model is the most representative solution. Furthermore, the questionnaire shows gender invariance (∆CFI < 0.10), so it may be applied to both girls and boys. More detailed results of Multi-group Confirmatory Factor Analysis (MGCFA) are presented in

Table 9. Fit indices of models estimated to check the measurement invariance of the PAQ.

Invariance	χ2	df	Δχ2	Δdf	RMSEA	ΔRMSEA	CFI	ΔCFI	SRMR	ΔSRMR
Configural	1229.08 *	480	-	-	0.084	-	0.925	-	0.043	-
Metric	1293.29 *	503	64.21	23	0.086	0.002	0.921	0.004	0.071	0.028
Scalar	1303.89 *	518	74.81	38	0.078	0.006	0.922	0.003	0.082	0.039
Strict	1398.31 *	546	169.23	66	0.092	0.008	0.915	0.01	0.094	0.051

Note. * p < 0.01.

.

To test configural invariance, we conducted a Multi-group Confirmatory Factor Analysis (MGCFA), to check whether responses from both genders supported the obtained model structure. The goodness-of-fit indices (see Table 8) suggest an acceptable fit to the data (χ² (480) = 1229.08, p < 0.001, CFI = 0.93, RMSEA = 0.08, SRMR = 0.05), with all items loading significantly on their specific factors and all item scores significantly loaded on the factors they were supposed to load on. Therefore, the model has the exact number of factors and configuration of item loadings across gender. MGCFA was then conducted to test metric invariance. The results of the analysis indicate an acceptable fit of the model to the data (χ² (503) = 1293.29, p < 0.001, CFI = 0.92, RMSEA = 0.09, SRMR = 0.07). Δ CFI is smaller than 0.01 suggesting that items have identical saturation/loadings in factors across gender. To assess scalar invariance, an additional constraint to the item intercepts was included, followed by another MGCFA. The fit indices are, again, acceptable (χ² (518) = 1303.89, p < 0.001, CFI = 0.92, RMSEA = 0.08, SRMR = 0.08), with ΔCFI smaller than. Thus, the intercept levels of the items are similar across gender. MGCFA was again conducted to test strict invariance, by additionally constraining the residual error variances of the items. Data showed almost acceptable fit indices (χ² (546) = 1398.31, p < 0.001, CFI = 0.92, RMSEA = 0.09, SRMR = 0.09) with a Δ CFI of 0.01. Residual errors are, therefore, similar across respondents’ gender. Overall, these results suggest that the scale can be used invariant of gender.

2.4. Study 4

The purpose of the fourth study was to analyze the psychometric qualities of the PAQ.

Procedure. This stage was attended by a number of 200 cases of Romanian children, boys (51.50%) and girls (48.50%) aged 8 to 18 years (the boys’ average age being 11.45; the girls’ average age obeying 11.41 years, and the general average age being 11.43 years), who came from divorced/separated families and/or families pending divorce/separation proceedings.

The testing-retesting phase was completed for a number of 40 cases, subjects of Romanian origin, i.e., 23 boys (57.5%) and 17 girls (42.5%), with an average age of 11.17 years for boys, 11.65 years for girls, and with a general average age of 11.37 years. There were two weeks between the test and retest. Ten clinical psychologists (seven women and three men) with an average age of 37.2 and 11.9 years of experience in the field of clinical psychology have completed PAQ twice for the same four children each. Besides the PAQ, two other questionnaires—The Behavior Rating Index for Children (BRIC) [19] and the Spence Scale of Anxiety for Children [20,21]—were also applied. The questionnaires were administered by clinical psychologists specialized in assessing the child and the divorce circumstances, upon the order of a law court, the police, and the prosecutor’s office, and by clinical psychologists/psychotherapists specialized in psychological intervention in children.

Results. Predictive validity was determined using the Pearson correlations between the Parental Alienation Questionnaire and The Behavior Rating Index for Children and the Spence Scale of Anxiety for Children, respectively. Most of these correlations were statistically significant with a medium to large effect size. The values of these correlations ranged from r = 0.31, p < 0.01, r² = 0.09 (the association between social phobia and the independent thinker phenomenon) to r = 0.53, p < 0.01, r² = 0.28 (the association between obsessive compulsive symptoms and the extension of animosity). The only exceptions were represented by the correlations obtained between the Spence Anxiety Separation Anxiety Scale for Children and the unconditional support subscales (r = 0.11, p > 0.05, r² = 0.01) and the borrowed scenarios (r = 0.11, p > 0.05, r² = 0.01) of the Parental Alienation Questionnaire for which the correlations were statistically insignificant. Also, the Anxiety Separation Sub-Scale of the Childhood Anxiety Scale has made a significant correlation with the rest of the Parental Alienation Questionnaire subscales, but the effect size was small. The values of these correlations ranged from r = 0.14, p < 0.05, r² = 0.01 (the association between separation anxiety and the independent thinker phenomenon, respectively from the separation anxiety and the unconvincing arguments) to r = 0.17, p < 0.05, r² = 0.02 (the association between separation anxiety and lack of ambivalence). Consequently, as we have anticipated, the PAQ has significantly predicted the presence of anxiety in children as well as the existence of behavioral issues. The results of these analyzes are highlighted in

Table 10. Correlation matrix: Factors correlation, convergent, and predictive validity.

Variables	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17
1. PAQ Denigration of the rejected parent	10.44	1.95	0.90
2. PAQ Unconvincing argumnets	9.97	2.38	0.92 **	0.96
3. PAQ Absence of ambivalence	10.41	2.26	0.85 **	0.82 **	0.93
4 PAQ Independent thinker	10.98	2.04	0.80 **	0.76 **	0.89 **	0.97
5. PAQ Unconditionnal support	10.68	2.27	0.82 **	0.80 **	0.89 **	0.91 **	0.98
6. PAQ Guilt free	10.58	2.31	0.80 **	0.76 **	0.90 **	0.89 **	0.95 **	0.98
7. PAQ Borrowed scenarios	9.96	2.65	0.85 **	0.85 **	0.85 **	0.81 **	0.86 **	0.88 **	0.98
8. PAQ Extenstion of animosity	9.83	2.95	0.83 **	0.80 **	0.79 **	0.76 **	0.80 **	0.82 **	0.93 **	0.98
9. PAQ Total alienation	82.83	17.47	0.92 **	0.90 **	0.94 **	0.91 **	0.94 **	0.94 **	0.95 **	0.92 **	0.99
10. SCAI Separation anxiety	16.22	3.75	0.15 *	0.14 *	0.17 *	0.14 *	0.11	0.15 *	0.11	0.15 *	0.15 *	0.88
11. SCAI Social phobia	17.26	2.79	0.36 **	0.37 **	0.38 **	0.31 **	0.29 **	0.33 **	0.36 **	0.38 **	0.37 **	0.65 **	0.83
12. SCAI Obsessive compulsive	14.53	3.21	0.47 **	0.44 **	0.42 **	0.35 **	0.38 **	0.41 **	0.49 **	0.53 **	0.47 **	0.47 **	0.68 **	0.9
13. SCAI Panic	20.93	3.95	0.40 **	0.38 **	0.38 **	0.33 **	0.32 **	0.34 **	0.40 **	0.46 **	0.41 **	0.66 **	0.75 **	0.85 **	0.88
14. SCAI Physical injury fears	13.17	2.81	0.33 **	0.32 **	0.34 **	0.30 **	0.31 **	0.32 **	0.33 **	0.36 **	0.35 **	0.74 **	0.68 **	0.69 **	0.80 **	0.75
15. SCAI Generalized anxiety	17.23	2.89	0.42 **	0.43 **	0.43 **	0.37 **	0.35 **	0.38 **	0.41 **	0.43 **	0.43 **	0.71 **	0.77 **	0.76 **	0.83 **.	0.80 **	0.89
16. SCAI total	98.36	17.02	0.40 **	0.39 **	0.40 **	0.33 **	0.33 **	0.36 **	0.39 **	0.43 **	0.41 **	0.81 **	0.85 **	0.85 **	0.94 **	0.89 **	0.92 **	0.94
17. BRIC total	64.58	18.36	0.52 **	0.52 **	0.53 **	0.52 **	0.50 **	0.48 **	0.47 **	0.47 **	0.54 **	0.38 **	0.49 **	0.49 **	0.51 **	0.53 **	0.59 **	0.56 **	0.85

Note: N = 200; * p < 0.05; ** p < 0.01, Cronbach alphas are presented on the main diagonal.

.

Testing-retesting reliability was determined by calculating the Pearson correlation between the scores obtained after the two successive administrations of the PAQ. The result obtained was statistically significant with a large effect size (r = 0.84, p < 0.01, r² = 0.7). The results of test-retest reliability with a two-week interval on 40 subjects are presented in

Table 11. Test-retest reliability for PAQ.

Variables	Test-Retest Correlations	Interclass Correlation
	R	ICC (95% CI)
1. Denigration of the rejected parent	0.56 *	0.72 (0.47, 0.85)
2. Unconvincing argumnets	0.52 *	0.76 (0.38, 0.83)
3. Absence of ambivaence	0.69 *	0.82 (0.65, 0.90)
4 Independent thinker	0.82 *	0.87 (0.76, 0.93)
5. Unconditional support	0.70 *	0.82 (0.67, 0.91)
6. Guilt free	0.78 *	0.86 (0.73, 0.92)
7. Borrowed scenarios	0.66 *	0.79 (0.61, 0.89)
8. Extention of animosity	0.43 *	0.60 (0.24, 0.79)
9. Total alienation	0.85 *	0.92 (0.84, 0.96)

Note: N = 40; * p < 0.05.

. The final version of the questionnaire as well as the scoring method are described in the Appendix A.

Test-retest reliability at two weeks on 40 cases rated by the same clinicians showed the fact that overall, the results of PAQ are consistent over time. The highest reliability is shown for the whole instrument as a measure of Parental Alienation, the interclass correlation being higher than 0.90, which proves an excellent stability. The scales show good reliability all the coefficients being over 0.70 except for scale Spread of animosity that has a value of 0.60 that is in the area of questionable reliability, but not poor. In this regard, we recommend that the ratters focus for evaluation and interpretation to be on PAQ total score and on the scales with high reliability.

3. Discussions and Conclusions

The objective of this study was to develop, administer, and test the psychometric qualities of a questionnaire—the Parental Alienation Questionnaire (PAQ), through which psychologists may directly analyze the children’s behaviors which signal the presence of parental alienation (by means of the three steps: Mild, moderate, and severe alienation). Both the results of the exploratory factor analysis and the results of the confirmatory factor analysis are promising. Following the consultation of experts in this field, a number of 101 items resulted, which then represented an analysis criterion for a number of 267 children. By applying the exploratory factor analysis, there have been extracted two main factors and six subsidiaries in correspondence with the eight criteria presented by Gardner. These factors loaded 24 items, 3 items on each factor and explained 92.47% of the total item dispersion, with the two main ones explaining 81,29% from it. The highlighted factors maintained the initial structure proposed by Gardner (2004) and were labeled as follows: “unconvincing arguments”, “extension of animosity”, “denigration of the rejected parent”, “absence of ambivalence”, “unconditional support”, “independent thinker phenomenon”, “absence of guilt”, and “borrowed scenarios”. These factors are consistent with the criteria proposed by Gardner (2004) for measuring parental alienation. Using the same theoretical background as RPAS [10] that is a tool completed by alienated parents, PAQ was developed for clinical psychologists evaluating the situation of the child. Confirmatory factor analysis acknowledged that the PAQ may be used by psychologists to quantify parental alienation in children with gender invariance.

All factors present a high internal consistency and a high predictive validity in terms of anxiety and behavioral issues in children. The predictive validity highlighted for this questionnaire is also consistent with those noticed in and outlined by the literature. There are dates that adults that experienced parental alienation during childhood report in their young adulthood problems of depression and self-esteem [22], or depression, anxiety, conduct disorders, and substance abuse [23]. However, there were studies that found no relation between parental alienation in childhood and depression and self-esteem at adult age [4,24]. Thus, as research shows, the negative impact of parental alienation on children leads to results ranging from manifestations in the field of psychopathology (anxiety, depression, substance abuse, and conduct disorders), to decreased academic performance and self-esteem, which are factors that have a negative impact upon the quality of life [23]. The test-retest reliability was also very good, so it may be said that the psychometric properties of the questionnaire appear to be solid.

The high predictive validity of anxiety and behavioral issues is relevant to clinical practice because it points out the prevention of parental alienation as a possible direction of intervention in order to prevent anxiety and behavioral problems in children. Children being in the situation of parental alienation have shown signs of separation anxiety, lack of social skills, and behavior problems [7,25].

Furthermore, the development of this standardized tool may contribute to the clinical and judicial acceptance of parental alienation. This tool may be extremely useful for the evaluators involved in the custody proceedings as well as the mental health professionals and all those who contribute to decisions regarding the treatment and custody of children and families.

Like any other analysis tool, PAQ has certain limitations. This tool has been tested only on cases involving Romanian subjects and therefore we do not yet know the inter-cultural compatibility of the tool, a fact that may also be verified by further studies.

At the prediction level, there has been acknowledgement of the fact that PAQ’s results predict behavioral disorders and anxiety, but the effects within the context of cognitive development or social skills were not yet studied, and these effects may be investigated by further studies.

In addition to the clinical applicability highlighted above, the development of this scale may encourage the development of a new line of research able to explore the relationship between parental alienation and other psychological processes in children.

4. Materials and Methods

In this regard, we developed and validated a psychological analysis tool that enables the psychologist, while carrying out the psychological assessment process, to directly perform an evaluation of behaviors that are specific to the onset of PA. This psychological analysis tool allows the identification of the degree of parental alienation in children who have been subjected to these dysfunctional relationships with their parents [13], depending on the three stages of parental alienation syndrome: Mild, moderate, and severe [17].

The development of this tool involved the completion of four distinct stages, which have been materialized in four studies, as follows: Defining the criteria to develop the tool, evaluating the inter-rater reliability, exploratory factor analysis and confirmatory factor analysis, as well as assessing how the parental alienation construct correlates with other significant constructs for this phenomenon, namely anxiety and behavioral disorders.

5. Conclusions

In conclusion, the results highlighted the proper psychometric qualities of the questionnaire and the Parental Alienation Questionnaire (PAQ) seems to be a promising tool for clinical and judicial practice, but also for researchers.