Measurement Invariance of Prayer Importance Scale: Religiosity, Gender, Age

Abstract

The main goal of the research was to increase knowledge on the psychometric properties of the Prayer Importance Scale (PIS). The study analyzed the structure of test items using item response theory (IRT), Mokken scale analysis (MSA), and confirmatory factor analysis (CFA) to establish factorial structure of the method. Measurement invariance (MI) was calculated in groups differentiated by three criteria: religiosity, gender, and age. MI verifies whether test items measure the construct in the same way across different groups. The Structure and Level of Religiosity Test (SLRT) was used to examine the level of religiosity in participants. In a study conducted with a sample of n = 566 adults (M_age = 49.16 years; SD_age = 15.72), two religious groups were identified based on a median distribution: with low (n = 275) and high (n = 291) levels of religiosity. Two groups differentiated by gender, men (n = 284) and women (n = 282), were equivalent per age period: early (age 25–39; n = 192), middle (age 40–59; n = 187), and late adulthood (age 60+; n = 187). Results of the analyses show high homogeneity of items comprising PIS, as well as strict invariance for the three distinguished criteria. Results of the study provide extended knowledge about psychometric properties of PIS and the ability to compare results due to religiosity, gender, and age.

1. Introduction

Prayer is defined as a personal, autonomous and authentic ability to enter into a dialogue with God (Zarzycka et al. 2022). Prayer occupies a central place in one’s religious life, permeating its structure and path from within (Bänziger et al. 2008). It leads to an increasingly fuller integration and contributes to establishing spiritual bonds with God as well as mutual bonds between people (Kulpaczyński and Tatala 2000; Ladd and McIntosh 2008). Constituting a rich and highly varied field of the human inner experience, it accompanies thoughts and sheds light on relations with other people and the outside world (cf. Walesa and Tatala 2020; Wojciechowska 2017). Among the most common typologies associated with the study of prayer is the analysis of its manifestations. With regard to the content, many types of prayers are distinguished, among which the most common are prayers of praise, thanksgiving, propitiation and petition (Kulpaczyński and Tatala 2000; Szcześniak et al. 2021). In addition, perception of importance of prayer in human life has become a subject of interest lately (Tatala and Wojtasiński 2021). Despite the fact that tangible, positive results of prayer bring positive reinforcement, thereby causing an increase in the subjective importance attributed to it, Tatala and Wojtasiński (2021) noted that the subject of prayer importance has played a minor role in the literature. The topic has often been discussed explicitly (Huber and Huber 2012) or marginally (Rydz et al. 2017), which prompted the authors to develop the Prayer Importance Scale (PIS). Its model and results were subjected to convergent validity analysis (Tatala and Wojtasiński 2021). However, it is still unknown whether results obtained with PIS are comparable due to a differentiation in basic and fundamental variables, such as the overall level of religiosity, gender of the respondents or their age.

Recent polls conducted by the Social Opinion Research Centre in Poland (Social Opinion Research Centre 2022a) showed that there has been a slow decline in the general level of religious beliefs among Poles with a relatively faster decline in religious practices observed at the same time. A series of surveys conducted from March 1992 to June 2022 showed that the percentage of adults who describe themselves as believers dropped from 94% to 84%, while the percentage of those who practice regularly (once a week or more) dropped from nearly 70% to 42%. Simultaneously, the percentage of non-practitioners increased from 9% to 19%. Among the significant reasons for secularization in Poland is the increasingly growing interconnection of the religious and political sphere (Wojtasik 2021). Another survey carried out by the Social Opinion Research Centre (Social Opinion Research Centre 2022b) indicated that 82% of Poles have never attended meetings of any religious community, prayer group, or religious movement; less than 9% said they used to belong to a religious community or prayer group; and 7% stated they currently belong to a religious group. Out of this 7% of respondents, 4% regularly attend meetings, and slightly fewer (3%) do so occasionally. It was noted that the decline in the level of religiosity has been fastest among respondents aged 18–24. It should be noted, however, that in the case of Poland a decline in religiosity is not always associated with a complete departure from the Church (Szymczak et al. 2022). Regardless, the fact that the observed decline appears to be one of the highest in the world is alarming (Pew Research Center 2018). This phenomenon is noted mainly in large cities and among educated people.

Cohort analysis revealed the occurrence of intergenerational changes: from one generation to the next, the level of religious belief and regular practice decreases, and non-practice increases. Moreover, intra-generational changes were observed: as people enter adulthood and with the passage time, the level of religious belief and regular practice decreases and non-practice increases (Social Opinion Research Centre 2022a). The youngest age cohort (18–24-year-olds) differed noticeably from other age groups, such as their parents and grandparents (cf. Borowska 2020; Social Opinion Research Centre 2022a). In addition to the issue of religiosity over the course of life, a number of studies have been undertaken to analyze gender differences (Walesa 2014). Women were found to be more religious and to practice more regularly than men (Walesa and Tatala 2020).

Prayer and its importance, as an expression of religious practice, is conditioned by religiosity, gender and age (Walesa and Tatala 2020). This creates a wide verification field regarding potential differences in the level of prayer importance for different values of the above-mentioned variables (religiosity, gender and age). At the same time, the possibility and methodological validity of such comparisons needs to be verified, as so far, the Prayer Importance Scale has not been subjected to invariance analysis with participants representing different levels of religiosity, gender and age (see Figure 1).

With the purpose of conducting a methodologically valid inspection of PIS, it was necessary to analyze the factor structure of the method. Therefore, dimensionality of the scale and the quality of items were examined. To analyze the factor structure of the method, we used: Mokken scale analysis (MSA), parallel analysis (PA), Very Simple Structures analysis (VSS) and confirmatory factor analysis (CFA). Based on MSA, it was possible to determine scalability, homogeneity and monotonicity of the items. PA and VSS enabled establishing the factorial structure of the tool. CFA made it possible to calculate the goodness-of-fit criteria between the model and the data and to confirm the structure of PIS. Once the above analyses were conducted, invariance analysis allowed us to draw conclusions about comparisons between groups representing different levels of religiosity, gender and age.

2. Results

2.1. Method

Two methods were used in the present study:

The Prayer Importance Scale (PIS) by Tatala and Wojtasiński (2021; Appendix A) consists of six items and is used for a speedy diagnosis of religious people. Responses are provided on a five-point scale from 1 = “strongly disagree” to 5 = “strongly agree”. Satisfactory properties of fit indices measurements (CFI, RMSEA, PCLOSE) were obtained in two studies and confirmed the proposed model (English and Polish versions of the scale can be found in the Appendix A). The standardized reliability of the method in the present study was α = 0.87, which is considered satisfactory.

The Structure and Level of Religiosity Test (SLRT) by Rydz et al. (2017) was used to measure levels of eight parameters of religiosity by Walesa (2005). Walesa (2005), in his cognitive-developmental theory of the integral development of religiosity defines religiosity as a personal and positive relationship with God that is expressed through religious awareness, religious feelings, religious decisions, bonds with the fellowship of believers, religious practices, religious morality, religious experience and forms of religious beliefs. The SLRT consists of 40 test items, five for each parameter of religiosity (more can be found at Rydz et al. 2017, p. 23). Response options range from 1 = “definitely not applicable” to 5 = “definitely applicable”. Cronbach’s α for the total score was α = 0.90 in the present study, which is consistent with previous results (α = 0.95) obtained by Rydz et al. (2017).

2.2. Participants

The study was conducted with a group of 566 participants in early (25–39), middle (40–59) and late adulthood (60+). The overall mean age was M = 49.16 years, SD = 15.72 (in early adulthood M = 31.32 years, SD = 4.33; in middle adulthood M = 49.19 years, SD = 5.65; in late adulthood M = 67.44; SD = 5.90). The youngest person was 25 years old and the oldest was 84. The groups of men (n = 284) and women (n = 282) were balanced in age: early (n_men = 97; n_women = 95), middle (n_men = 95; n_women = 92) and late adulthood (n_men = 92; n_women = 95).

In addition to the variables of age and gender, religiosity was also analyzed. Two religious groups were identified based on a median distribution of results in the SLRT with low (n = 275; M = 144.58; SD = 10.87) and high (n = 291; M = 172.53; SD = 9.11) levels of religiosity. Both groups were represented by people belonging to Catholic religious communities and involved in the religious life of their parishes. No clergy, seminary students, or religious brothers/sisters were examined in order to keep the sample homogeneous. However, access to the target groups was possible through the above persons. As such, a control of membership to the specific religious group has allowed to reduce the proportion of uncontrolled variance in the results already at the study design stage.

2.3. Procedure

The study was conducted individually among Roman Catholics who belonged to religious communities in Poland. Respondents were informed of the subject and purpose of the study and were assured of the confidentiality of the collected data and the use of the results for scientific purposes only. There was no time limit so that the participants could thoroughly familiarize themselves with the test items and answer them reliably. Meetings ended with a short conversation, during which the respondents could express their opinions on the completed questionnaires and were thanked for their time and cooperation.

2.4. Analysis

A series of psychometric analyses were conducted to verify the structure of the method: MSA, PA, VSS and CFA. MSA refers to a series of procedures that verify the nature of test items against the scale. As recommended by Sijtsma and van der Ark (2017), the steps of MSA consist of homogeneity analysis, scalability analysis with the use of automated item selection procedure (AISP) and monotonicity. In the second stage, three sets of invariance analysis were conducted in groups differentiated by religiosity (low or high scores on the SLRT scale), gender (men and women) and age (period of early, middle and late adulthood). In the next step, unidimensionality and multidimensionality of the tool was verified using PA and VSS as recommended by Dima (2018) and van der Linden (2016).

The final step was to conduct a series of CFAs to verify measurement invariance. All analyses were performed in R Program (R Core Team 2022) with RStudio (RStudio Team 2022) using the following packages: corrplot (Wei and Simko 2021), dplyr (Wickham et al. 2022b), EFAtools (Steiner and Grieder 2020), ellipse (Murdoch and Chow 2022), ggExtra (Attali and Baker 2022), ggplot2 (Wickham 2016), ggpubr (Kassambara 2022), GPArotation (Bernaards and Jennrich 2005), haven (Wickham et al. 2022a), here (Kirill and Bryan 2020), Hmisc (Harrell and Dupont 2022), jtools (Long 2022), lavaan (Rosseel 2012), lavaanPlot (Lishinski 2021), magrittr (Bache et al. 2022), mokken (van der Ark 2007, 2012), nFactors (Raiche and Magis 2022), psych (Revelle 2022), RColorBrewer (Neuwirth 2022), semPlot (Epskamp et al. 2022), semTools (Jorgensen et al. 2022), tidySEM (van Lissa 2022) and tidyverse (Wickham and RStudio 2022).

Homogeneity analysis is the first step of MSA. In the present study, it showed that all items exceeded the value of the minimum scalability coefficient, which, according to the literature, is 0.30 (Sijtsma and van der Ark 2017; see

Table 1. Homogeneity analysis for PIS items.

Item Name	Homogeneity Index	Standard Error
PIS_1	0.575	0.034
PIS_2	0.618	0.028
PIS_3	0.620	0.025
PIS_4	0.577	0.035
PIS_5	0.587	0.027
PIS_6	0.582	0.029

).

In the second step, the ability to assign PIS test items to potential factors with rising levels of homogeneity was checked (see

Table 2. Scalability analysis for different homogeneity indices.

Item	Index of Homogeneity
	0.10	0.20	0.30	0.40	0.50	0.60	0.70	0.80
PIS_1	1	1	1	1	1	1	1	1
PIS_2	1	1	1	1	1	1	1	1
PIS_3	1	1	1	1	1	1	1	1
PIS_4	1	1	1	1	1	1	1	1
PIS_5	1	1	1	1	1	1	1	1
PIS_6	1	1	1	1	1	1	1	1

). This analysis shows how the average homogeneity of the entire scale changes when items explain different coefficients.

As a result of the scalability analysis, it was observed that it is possible to assign all test items to a single scale, regardless of the established homogeneity threshold (0.10–0.80). On the other hand, in no case was the value of zero (“0”) obtained, indicating the impossibility of assigning a test item to any scale, nor a value of two (“2”), indicating the possibility of assigning an item to two scales. Therefore, the results presented in Table 2 prove that the one-factor solution dominates within test items and homogeneity thresholds.

In the subsequent step, monotonicity analysis was conducted. It captures the specificity of a function, describing the relationship between rest score groups and item response profiles based on the item step response function (Figure 2).

Monotonicity analysis includes both a graphical inspection of the above graphs and a formal analysis of significant deviations from the linearity of the monotonicity function (see

Table 3. Item test for deviations from the linearity.

Item	ac	vi	vi/ac	zsig	crit
PIS_1	112	0	0.00	0	0
PIS_2	72	3	0.04	0	18
PIS_3	46	0	0.00	0	0
PIS_4	81	0	0.00	0	0
PIS_5	83	2	0.02	0	1
PIS_6	58	0	0.00	0	0

Legend: ac = number of accordances; vi = number of violations; vi/ac = ratio of accordances to violations; zsig = significance of violations.

). The graphical inspection includes verification of the degree of skewness of the curve illustrating the shape of the function, as well as observation of the presence of zig-zag patterns. Although the greatest skewness was observed for PIS_2 item, additional support in the form of a significant violations test (see Table 3) showed no significant violations in the linearity.

In the next step, PA and VSS were conducted to verify the uni- or multidimensionality of the scale. Both of these analyses suggested a one-factor solution (Figure 3 on the left for parallel analysis). The first parameter of VSS, VSS1, reached a maximum (0.91) for one factor with fit index RMSEA = 0.078, and although VSS2 reached a maximum (0.93) for two factors, it showed a lower fit index (RMSEA = 0.084). The Velicer MAP test reached a minimum of 0.05 for one factor and the BIC reached a minimum of −16.95 also for one factor (Figure 3 on the right for VSS).

Before measurement invariance, fit between the general model of PIS and the data was verified with CFA. We used the following measures of fit: model chi-squared (χ²), comparative fit index (CFI), incremental fit index (IFI), normed fit index (NFI), non-normed fit index (NNFI), parsimony goodness of fit (PGFI), relative fit index (RFI), root mean square error of approximation (RMSEA), McDonald and Marsh’s relative noncentrality index (RNI), standardized root mean square residual (SRMR) and the Tucker–Lewis index (TLI). The model was found to be a very good fit to the data (χ²(9) = 40.27, CFI = 0.98, IFI = 0.98, NFI = 0.97, NNFI = 0.96, PGFI = 0.33, RFI = 0.95, RMSEA = 0.057 (90% CI [0.04, 0.08]), RNI = 0.98, SRMR = 0.02 and TLI = 0.97).

Reliability analysis of the scale was also conducted using two methods: Cronbach’s alpha and Tarkkonen’s rho, which provided satisfactory scores (α = 0.87; ρ = 0.77). In addition, correlations between items were calculated. The plot (Figure 4) shows that PIS items are moderately correlated with each other.

Invariance analysis was conducted separately for the following variables: religiosity, gender and age.

2.4.1. Religiosity

To verify the measurement invariance, tests comparing fit indices as well as the invariance between factor loadings, intercepts and the unexplained variances for the religiosity variable were conducted (

Table 4. Measurement invariance steps across religiosity groups.

	chisq.	df	p Value	RMSEA	CFI	TLI	SRMR
Configural model	27.37	18	0.072	0.043	0.985	0.976	0.031
Metric model	39.86	23	0.016	0.051	0.974	0.966	0.046
Scalar model	62.31	28	0.000	0.066	0.947	0.943	0.055
Adjusted scalar model PIS_6 ~ 1	47.86	27	0.008	0.053	0.968	0.964	0.050
Strict model (with free PIS_6 ~ 1)	71.16	33	0.000	0.064	0.941	0.946	0.070
Adjusted strict model PIS_6 ~~ PIS_6 (with free PIS_6 ~ 1)	50.18	32	0.021	0.045	0.972	0.974	0.053

Legend: chisq = chi-squared test score; df = degrees of freedom; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual.

and

Table 5. Differences between chi-square statistics for religiosity groups.

	chisq. diff.
Configural model—metric model	0.091
Metric model—scalar model	<0.001
Metric model—adjusted scalar model	0.078
Adjusted Scalar model—strict model	<0.001
Adjusted scalar model—adjusted strict model	0.562

Legend: chisq. diff. = significance of difference between two chi-square statistics.

).

It can be concluded that partial strict invariance across low- and high-level religiosity (with a free intercept and residuals for item 6) was established for PIS.

2.4.2. Gender

To verify the measurement invariance, tests comparing fit indices as well as the invariance between factor loadings, intercepts and the unexplained variances for the gender variable were conducted (

Table 6. Measurement invariance steps across gender.

	chisq.	df	p Value	RMSEA	CFI	TLI	SRMR
Configural model	31.58	18	0.025	0.052	0.977	0.961	0.029
Metric model	27.53	23	0.234	0.027	0.992	0.990	0.034
Scalar model	38.27	28	0.093	0.036	0.982	0.981	0.039
Strict model	42.29	34	0.156	0.030	0.986	0.987	0.046

Legend: chisq = chi-squared test score; df = degrees of freedom; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual.

and

Table 7. Differences between chi-square statistics for gender.

	chisq. diff.
Configural model—metric model	0.588
Metric model—scalar model	0.056
Scalar model—strict model	0.503

Legend: chisq. diff. = significance of difference between two chi-square statistics.

).

It can be concluded that strict invariance across female and male groups was established for PIS.

2.4.3. Age

To verify the measurement invariance, tests comparing fit indices as well as the invariance between factor loadings, intercepts and the unexplained variances for the age variable were conducted (

Table 8. Measurement invariance steps across age groups.

	chisq.	df	p Value	RMSEA	CFI	TLI	SRMR
Configural model	40.96	27	0.042	0.053	0.972	0.953	0.031
Metric model	37.01	37	0.468	0.001	1.000	1.000	0.045
Scalar model	51.82	47	0.291	0.024	0.990	0.991	0.053
Strict model	73.77	59	0.093	0.037	0.970	0.977	0.089
Adjusted strict model PIS_6 ~~ PIS_6	64.67	57	0.227	0.027	0.984	0.988	0.072

Legend: chisq = chi-squared test score; df = degrees of freedom; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual.

and

Table 9. Differences between chi-square statistics for age groups.

	chisq. diff.
Configural model—metric model	0.560
Metric model—scalar model	0.137
Scalar model—strict model	0.042
Scalar model—adjusted strict model	0.224

Legend: chisq. diff. = significance of difference between two chi-square statistics.

).

It can be concluded that partial strict invariance across three age periods: early, middle and late adulthood (with free residual of item 6) was established for PIS.

3. Discussion

The purpose of the present study was to build on the existing validation of the Prayer Importance Scale (PIS), as the already validated model was characterized by satisfactory psychometric indices (Tatala and Wojtasiński 2021). Due to low availability of tools testing the importance of prayer in one’s life, the quality of the items underwent Mokken scale analysis (van der Linden 2016). A series of subsequent and independent steps were taken to examine the specifics of the scale items. It was proved that PIS passed homogeneity, scalability and monotonicity analyses for the individual items. In addition, parallel analysis with Very Simple Structures showed that the scale is unidimensional. Finally, measurement invariance was conducted of three variables that seem to generate high variance in results: religiosity, gender and age.

The analyses confirmed at least partial strict invariance for each of the variables. This means that results of values of the variables are methodologically comparable on several levels. First, it is possible to compare the structure of indicator configurations, which is a prerequisite in order to accept the common nature of the construct (configural invariance). Thus, for each of the groups, a one-factor model consisting of six items indicates a good fit to the data. Second, it is valid to conduct further statistical analyses to test hypotheses on differences between individuals representing different religiosity levels, genders and ages (Laakasuo et al. 2022).

This is directly related to fulfillment of the second and third invariance criteria, that is, the equivalence of factor loadings (metric invariance) as well as test items (scalar invariance). Establishing scalar invariance for PIS means that the significance of items is balanced for values of test variables that are the subject of equivalence (scalar invariance). Finally, partial strict invariance associated with correlated residuals (unexplained variance) means that the results for each group are fully comparable (Luong and Flake 2022).

Although the results presented in this article seem to support high psychometric properties of PIS, there are a number of limitations that are worth considering when planning further research. First, the tool provides a starting point, rather than a final one, for an in-depth analysis of the importance of prayer in people’s lives. The unidimensional nature of the scale may be attributed to the small but satisfactory number of items. The authors intended PIS to be used for the diagnosis of religious individuals. Importance, however, implies a kind of attitude toward a certain phenomenon, which means that it can represent a cognitive, emotional or a social angle. The second issue is the lack of discriminant validity, which could further establish the value of the method (Sijtsma and van der Ark 2017). For example, correlations with a religious crisis or deconversion could be calculated. Thirdly, there is a risk of generalizing the results obtained with PIS to other religions or faiths without performing separate invariance analyses for different groups. However, the results provide a valuable psychometric basis for making reasonable inferences for Roman Catholics with respect to the three measured variables: religiosity level, gender and age. Finally, the religiosity in participants of the study was assessed based on a multidimensional construct of religiosity by Walesa (2005); nonetheless, other ways of studying the degree of religiosity are proposed in the literature. In addition to questionnaire methods, such as Huber’s CRS (Huber and Huber 2012), other religious practices (besides prayer) may play a large role here.

The goal of the presented article was to provide extended validation of the Prayer Importance Scale psychometrically. The method was proved to show satisfactory indices at both the item structure and factor structure levels. Additionally, it met invariance requirements for three tested criteria: religiosity, gender and age. The scale can therefore be successfully used without risking the violation of methodological assumptions regarding Mill’s method of difference, among others. As a short, six-item scale, it is suitable for an initial diagnostic of prayer importance which may be the key to understanding the motivational aspects of religiosity fluctuations in future research.