1. Introduction
In the field of education, it is common to use tests that attempt to assess both reflective psychological constructs (e.g., emotional intelligence) and formative constructs related to learning and academic achievement. Although the reflective or formative nature of the measures is rarely addressed [
1,
2,
3], this differentiation is a fundamental issue in choosing the most appropriate statistical model for assessing the validity of evidence regarding the constructs measured and their relationships.
In a reflective measure such as emotional intelligence (EI), it is assumed that there is a latent variable in which the items are imperfect indicators that are directly deduced from the latent variable, which means that a larger number of indicators usually leads to a more reliable estimate of the latent variable. This requires that the indicators have an adequate discrimination value (i.e., a high relationship with the latent variable), which entails a decrease in measurement error.
In a formative measure, the indicators or items are chosen by the researcher for their own interest in relation to a particular learning outcome. For example, in a measure of reading meta-comprehension, it may be of interest to know whether the person knows and uses strategies such as underlining parts of the text, marking main ideas or other strategies that improve comprehension. Also, in order to assess oral communication skills, the use and knowledge of various strategies can be investigated. Such constructs can be conceptualised as formative, as the constructed variable depends critically on the items chosen by the researcher and, therefore, a different set of strategies necessarily leads to a different constructed latent variable.
In this study, we conducted a Partial Least Squared Structural Equation Modeling (PLS-SEM) [
4] as an analysis procedure, which allows both formative and reflective variables to be analysed jointly. More specifically, we showed how to set up a reflective-formative model and the consequences derived from this analysis. To this end, the model combines a reflective psychological variable for the assessment of EI (measured by the Trait Meta-Mood Scale—TMMS-24 [
5]), one of the most widely used assessment measures of EI, and two formative variables on different strategies taught in the educational environment: strategies for oral communication (the Test of Self-Perceived Oral Competence—TSOC) [
6] and strategies for reading meta-comprehension (Revised Metacognitive Awareness of Reading Strategies Inventory—MARSI-R) [
7]. For these reflective and formative latent variables, the predictive capacity is analysed with respect to two criterion variables, reading skills (self-perceived measure) and reading comprehension assessed by a test corresponding to the Programme for International Student Assessment (PISA) [
8].
There is controversy about the role of EI in academic achievement, with results that have observed a relationship with academic success [
9,
10], and others in which no such relationship was observed [
11,
12,
13]. These contradictory results highlight the object of study and the variables involved, making it necessary to implement methods of analysis that allow the problem evaluation in its complexity while respecting the formative or reflective nature of the variables involved.
PLS-SEM, due to its exploratory nature, is designed to model complex relationships that maximise the predictive capacity of the models, making it a suitable tool to try to explain how academic performance, specifically in the area of reading comprehension, is influenced by IE, taking into account mediating variables such as oral and reading strategies, respecting the reflective or formative nature of these variables.
We operationalize all these variables in the following sections. Finally, the model tested is justified in relation to evidence from previous studies.
1.1. Emotional Intelligence (EI)
Emotions foster the satisfaction of social functions, and contribute to the determination of a collective as a social group [
14]. Previous research suggests that “emotions, like feelings, are part of what we are, personally and socially” [
15]. Therefore, the importance of delving deeper into this construct has increased in the psycho–pedagogical area. Emotions are part of the human being, and one of the ways of studying them has been through EI.
In the early 1990s, Salovey and Mayer [
16], inspired by earlier research by Gardner [
17], coined the term EI for the first time. These authors understand EI as a new approach that goes beyond cognition, highlighting the importance of the emotional, as well as the intellectual. Thus, they define EI as the ability to perceive, know, access and generate emotions, as well as to reflect and regulate emotions that activate both emotional and intellectual growth [
18].
The most commonly used instruments for the assessment of EI are self-reports. They consist of a series of short verbal statements, in which the person evaluates his or her level of emotional competences and skills. Their usefulness lies in the exercise of introspection that is carried out, which implies knowledge about intrapersonal ability. Their main drawback is that the response may be influenced by the perceptual bias of the person who is self-assessing, as well as by social desirability. Within self-reports, Salovey et al. [
19] developed the TMMS, one of the first questionnaires to measure metacognition of emotional states, based on the original model of Salovey and Mayer [
16]. Both the original version and the reduced 24-item version adapted to Spanish (TMMS-24) [
20] are self-report measures for assessing perceived intrapersonal EI (beliefs that people hold about their ability) in three dimensions: attention to feelings, clarity and repair [
5]. It is considered that a good EI can have benefits for the person, through the measurement of these three dimensions, which contribute significantly to the development of psychological well-being [
16,
18]: (a) attention to feelings: the ability to feel and express emotions appropriately, (b) clarity: the perception an individual has of the understanding of their own emotional states and, (c) repair: the ability to regulate moods and repair negative emotional experience.
1.2. EI and Academic Performance
Research relating EI to academic achievement yields contradictory results, as we have already noted. In Spain, Fernández-Berrocal and Extremera [
20] study this relationship in students in compulsory secondary education (ESO, from its Spanish initials), and their results highlight the connections between these two constructs as a mediating effect that good emotional health exerts on performance. Pérez and Castejón [
10] analyse the specific contribution of EI in predicting academic performance. The results suggest the independent contribution (i.e., a direct effect) of EI to the prediction and/or explanation of academic performance in a sample of university students. López Fernández [
9] also concludes the predictive value of EI on academic performance in university nursing students. Barna and Brott, Brackett and Mayer, and Buenrostro [
12,
21,
22] also found positive and significant correlations between socioemotional development and academic performance. Gil-Olarte et al. [
23] reveal positive effects regarding the importance of EI in academic success and social development in a group of adolescents, as measured by final grades. However, other studies do not find the relationship between EI and academic achievement in university students to be significant [
11,
12,
13]. Newsome et al. [
24] also found no significant association between EI and students’ grades. Broc Cavero [
25] concludes that EI is overestimated, and does not have as much relevance and influence on academic performance as other studies have indicated.
1.3. Reading Comprehension and Its Importance in Education
One of the constructs most closely related to academic success is reading comprehension [
26,
27,
28]. Reading comprehension is a mental process, whereby meanings are generated from a text, which is integrated with the prior knowledge and personal experiences of the individuals involved [
29]. Thus, reading comprehension involves generating representations of the meaning of ideas in a text, beyond eliciting the meaning of words or sentences. In this process, readers must be able not only to read and comprehend, but also to interpret and analyse the information presented in the text.
It is therefore not surprising that various institutions, both at a national level, in Spain, (e.g., using the Indispensable Knowledge and Skills tests or Conocimientos y Destrezas Indispensables, CDI tests, from the acronym in Spanish) and at an international level (e.g., using the PISA tests) have focused on identifying students’ reading comprehension levels.
The PISA test was designed to establish what young people know and are able to do at the end of ESO in the areas of science, reading and mathematics. In the area of reading comprehension, the PISA test includes three types of texts: continuous texts, discontinuous texts and mixed texts, which can include different formats, such as literary, informational and opinion texts. In particular, the PISA test focuses on measuring students’ ability to locate relevant information, integrate literal information from different texts, generate inferences, assess the quality and credibility of information, reflect on text content and form, and detect and manage intertextual conflict [
30]. This test has been used in several studies [
31,
32], which is evidence of its impact on research on reading comprehension assessment.
1.4. Reading Meta-Comprehension Strategies and Reading Self-Efficacy
As mentioned above, one of the variables traditionally most related to reading comprehension is the ability to implement reading strategies aimed at the comprehension, analysis and evaluation of texts [
33,
34]. Numerous studies have been carried out to identify the reading comprehension strategies used by students [
35,
36,
37] and the positive results in text comprehension if they are adequately trained [
38].
The assessment of reading comprehension strategies is usually carried out through self-reports [
39], one of the best-known instruments being the Metacognitive Awareness of Reading Strategies Inventory (MARSI) [
40] and its shorter version, the MARSI-R [
7,
41]. This instrument focuses on three distinct but related types of strategies: global reading strategies, which focus on the overall comprehension of a text; problem solving strategies, which focus on strategies that are implemented when a text is difficult to read; and support reading strategies, which involve note-taking or the use of support materials.
Another relevant variable for academic success, in this case in reading comprehension, is students’ perceived self-efficacy in this area [
42]. Self-efficacy can be described as the extent to which people perceive themselves as effective in performing certain behaviours. However, this variable is not only related to academic success but, as Zimmerman [
43] points out, it is also related to students’ perceived control of strategies. According to this author, students’ control of learning strategies makes them perceive themselves as more competent, which leads to greater persistence in the task, resulting in a positive effect on performance [
44].
Thus, two relationships can be established and should be considered when talking about reading self-perception. On the one hand, the relationship between students’ self-perception and the control they perceive when applying strategies in a task and, on the other hand, the relationship that this self-perception has, in itself, with reading performance. Much work has been carried out to identify the relationship between reading self-perception and the use of reading comprehension strategies [
41]. Thus, Naseri and Zaferanieh and Li and Wang (2010) [
45,
46], using self-reported reading strategy questionnaires, identified significant positive relationships with reading self-perception. However, it should be noted that, although a strong relationship is observed between perceived self-efficacy and the self-reporting of reading strategies, these may also be accentuated by the self-report nature of both variables. On the other hand, Naseri and Zaferanieh [
45] have also identified relationships between reading self-perception and reading achievement, as have other authors [
32,
47].
1.5. Oral Communication Strategies
A variable closely related to reading comprehension, which affects academic performance is that of oral communication strategies. The development of communication and language skills (listening, speaking, reading and writing) is fundamental in all educational contexts and levels, and can therefore be considered part of what has been understood as academic achievement. Their value lies not only in the possibility they offer to demonstrate the knowledge acquired in the classroom, but also in the fact that they are very powerful psychological tools that enable knowledge to be elaborated and transformed, learning to be optimized, and reflective thinking to be developed, among other aspects [
48,
49,
50].
The teaching and learning of oral language should be a priority in formal educational contexts, as its mastery has personal, professional and social implications, in line with the Common European Framework of Reference for Languages [
51], which attaches fundamental importance to oral language competence.
Being competent in oral language enables learners to produce complex, coherent, cohesive and contextually appropriate oral texts, while at the same time equipping them with the ability to request information, argue, ask for clarification, refute, synthesise, summarise, reflect on their own language, etc., with different interlocutors and for different purposes [
52,
53,
54].
Among the few instruments that exist to assess knowledge and use of oral communication strategies, some of them link self-perceived oral communicative competence with aspects related to the anxiety of speaking a foreign language or a language that is not one’s own, in multilingual contexts [
55]. Studies have also been carried out on the language itself, suggesting the use of the Speech Skills Self-Efficacy Scale proposed by Demir [
56] and designed to assess the use of oral communication strategies.
An ambitious paper by Croucher et al. [
57] reviews the research carried out using the questionnaire that was published in 1988 by McCroskey and McCroskey, to self-assess communicative competence in university contexts. Croucher et al. [
57] indicate that, despite being designed for this educational context, it has been used at other levels, and conclude that only four studies have conducted a reliability analysis of the test since 2000 and that, although it has been used in 12 different countries, statistical analyses have shown little evidence of construct validity.
Author [
6] developed the TSOC, an instrument for self-perception of oral competence, which assesses five key dimensions: (1) Interaction Management; (2) Multimodality Prosody; (3) Textual Coherence and Cohesion; (4) Argumentative Strategies; (5) Lexicon and Terminology. In this validation study with compulsory secondary education students, adequate reliability (between 0.85 and 0.88) and good fit with the correlated five-factor structure were observed. In terms of evidence of construct validity in relation to other variables, it was observed that the TSOC correlated with reading meta-comprehension strategies, assessed using MARSI-R [
7]. Although these are different skills, these results reveal a connection between the dimensions that constitute communicative competence (i.e., oral language and reading), which has not gone unnoticed in the curricula of educational laws in various countries [
51,
58]. The link between oral proficiency and reading comprehension was highlighted by Clarke et al. [
59] in a study in which they found that listening comprehension is the main predictor of reading comprehension, and that this relationship increases with age.
1.6. EI in Relation to Oral Proficiency
Some studies link children’s EI and language proficiency, and, in some cases, the latter is included in the former [
60]. The authors consider that the multiple components that include both competences and their relationships have not been sufficiently studied. The results of their study with 10-year-old schoolchildren showed strong positive correlations between language competence and EI, ranging between r = 0.12 and r = 0.45. In particular, receptive vocabulary and literacy were closely related to emotional knowledge. A confirmatory factor analysis revealed that there is a common general ability factor for language competence and EI. While the authors make a strong case for explaining these correlations, it is important to note that none of the skills included in language competence focus on the pragmatic aspects of language.
Other authors have also focused on the relationships between these competencies at early ages [
61], and specifically on the extent to which families use emotional language and talk with children about their experiences, as a way of explaining how early language development may contribute to emotional regulation skills. According to the authors, the separate analysis of language and emotional development contributes to a better understanding of how children become aware of their emotions and regulatory strategies and develop effective and appropriate emotional self-regulation. The authors point to the need to investigate how adults use a child’s emerging language skills to help them self-regulate (e.g., asking what can be done about a problem, rather than simply calming or solving it).
Lindquist et al. [
62], from a constructivist approach, argue that language is a fundamental element in emotion, and is constitutive of both emotional experiences and perceptions. The authors review evidence from cognitive and developmental science, to reveal that language builds concept knowledge in humans, helping them to acquire abstract concepts such as emotional categories throughout their lifespan. The authors review different research at school age and also in adulthood, but do not present results on the adolescent stage on which the present study focuses.
Pasquier et al. [
63], also with primary school children aged 9 to 11, developed a Moral and Civic Education programme, focused on the identification and expression of emotions, aimed at their comprehension and oral production. The results indicate that, although vocabulary and comprehension levels improved in both the experimental and control groups, only those who had completed the educational programme on emotions showed significantly better skills in general oral production. It is highlighted that educational practices that encourage public speaking in natural situations contribute to the development of oral language, listening and empathy skills.
1.7. The Present Study
Based on the background information reviewed, the goal of this study is to evaluate the relationships between the above-mentioned variables by testing a PLS-SEM model. More concretely, we consider that the EI variable can affect both performance in reading comprehension and reading self-perception, as has been identified in previous works. However, these effects are not always clear, which could be due to the mediation of several variables. In this sense, we consider that self-reported reading and oral strategies may mediate the effect that EI has on reading comprehension and perceived self-efficacy in this domain, as can be seen in
Figure 1.
The model analyses the direct effects of EI (TMMS-24) on the two proposed outcomes: reading comprehension (a measure of efficacy obtained from the PISA test) and participants’ self-perception as readers (READER variable). In addition, the model allows us to evaluate the potential mediation effect of oral communication strategies (TSOC) and reading meta-comprehension strategies (MARSI-R) between TMMS-24 and the two outcomes analysed. For this purpose, the indirect and total effects of EI (TMMS-24) are also analysed. Note that, in the case of EI, the measure is operationalised as reflective, whereas the oral communication and meta-comprehension strategies are operationalised as formative. Additionally,
Figure 1 does not reflect the potential relationship between the READER and PISA variables, because (1) our main objective is to analyse the direct and indirect effects of TMMS-24, MARSI-R and TSOC on each of the two outcomes separately, and (2) it is currently unclear as to what it the directionality of the relationship between READER and PISA (i.e., self-perception as readers → efficacy. or vice versa). We expand this issue in the
Section 4.
To evaluate the model in
Figure 1, an approach based on Confirmatory Composite Analysis [
64,
65] has been followed. There is a strong tradition of reducing dimensionality by means of the common factor model (i.e., Exploratory Factor Analysis and Confirmatory Factor Analysis). In the common factor model, the researcher assumes that the indicators (i.e., items) are reflective in nature. That is, each observable variable is a manifestation of a latent variable, construct or underlying concept that is the (common) cause of all the indicators it is composed of. In the common factor model, observable variables are interchangeable, as they are theoretically conceptualised as similar in meaning or content. If any indicator is removed, the construct does not change its essential meaning. In contrast, a formative construct is a weighted linear combination of observable indicators (causal indicators) [
66], and therefore the indicators do not necessarily share a common cause.
The composite variables (composite or derived) are a type of variable formed by combining two or more other variables (formative or reflective), and are designed to try to capture the most important features of the data as efficiently as possible. In recent years, this approach to theoretical concepts based on composites is being increasingly legitimised by several authors, in various fields [
65].
4. Discussion
Models combining reflective and formative variables are common in educational psychology. However, many of these studies do not make a correct specification of the type of variables used, which may lead to incorrect interpretations. In the present study, a predictive model with a reflective psychological variable, IE measured using TMMS-24, which is assumed to influence or explain part of the variance in the knowledge of strategies for oral-reading performance, assessed by MARSI-R and TSOC, has been proposed and analysed. In turn, the mastery of these strategies has been proposed as related to self-perception in reading competence (READER variable), and the reading comprehension assessed employing a PISA test.
By means of the PLS-SEM model proposed, it has been possible to evaluate the relationships between the variables analysed, specifying the reflective or formative nature. Thus, although initially the correlation between all the variables was found to be statistically significant at
p < 0.001 (except the correlation between TMMS-24 and PISA; see
Table 5), the PLS-SEM model makes it possible to go into the nature of these relationships in more depth, and these are discussed below.
Regarding the measure of EI, the reflective variable for which the TMMS-24 was applied, it was possible to observe a significant effect size (i.e., a significant shared variance with self-perceived oral competence measured using TSOC, and reading meta-comprehension strategies measured using MARSI-R).
EI measured using TMMS-24 is found to influence oral competence, with a positive linear relationship indicating a strong effect size. From a theoretical point of view, Mayer and Salovey [
18] conceive EI as an ability to perceive emotions, and to access, generate and determine emotions; this, in addition, in a reflective way, makes it possible to regulate emotions that promote both emotional and intellectual growth.
Although these results are consistent with previous research [
60,
61,
62,
63,
64], there are few studies that relate EI to oral language skills or oral language competence, and those that do so focus on foreign language learning, emphasising elements linked to the stress or anxiety involved in speaking in a language in which one is not an expert [
75]. However, research focusing on the development of the skills to participate in a classroom conversation, to ask questions of teachers or peers, and to manage a conversation, even if it is in one’s own language, indicates that there is a strong link with emotional aspects. Probably, the reason why there are not many studies that try to identify these effects is that there is also not much literature exploring oral language competence beyond the oral presentation in groups or individual work, which usually involves prior preparation and rehearsal. On the other hand, all those activities that involve participating in a classroom discussion, managing a discussion autonomously, reflecting on the language, on one’s own oral competence, self-evaluating one’s own activity at the end of the discussion, and making decisions about what can be improved—among other aspects—involve knowledge about one’s own emotions and a degree of self-control that are closely linked to EI [
60]. In fact, some of the items in the TSOC instrument are linked to EI. For example, when the student is asked to rate the degree to which “Other people clearly interpret the emotions you express with your face”, an appeal is certainly being made to emotional aspects, both in relation to oneself and to others. The same can be observed when one is asked about whether “You are aware of how your tone of voice and volume may affect others” where, in essence, the student is being asked to value the effort involved in putting oneself in another person’s shoes (empathy) to assess the degree to which a certain tone of voice may affect others (emotionally). Something similar occurs with the item “You pause so that others can follow better what you want to say”, since it requires the student to put him/herself in the place of his/her peers and try to speak slowly, to allow the peers to follow him/her. This situation is consistent with a study by Pasquier et al. [
63], in which it was observed that primary school students who had completed an educational programme on emotions showed significantly better skills than the control group, in general oral production. The authors conclude that educational practices that encourage public speaking in natural situations contribute to the development of oral language and to listening and empathy skills.
However, in the present study, the relationship of EI with respect to reading level operationalized as self-perception of reading success (READER variable) and reading comprehension performance (PISA) is no longer significant, implying that there is a full mediation effect. This result could explain the discrepancies between studies on EI and its relationship with educational outcomes such as academic performance, since although it is possible to find a statistically significant relationship, the inclusion of other more directly related variables leads to a reduction in this effect [
76,
77].
An effect of EI on reading meta-comprehension has also been observed, although its effect size is smaller than that observed for oral communication. EI is related to students’ ability to self-regulate when carrying out certain tasks, which is associated with different degrees of academic success [
78]. Therefore, this fact may explain the relationship obtained between EI and the reading-comprehension metacognitive-strategies variable. Several studies have specifically identified that EI has a positive effect on reading comprehension in different languages [
79,
80]. However, the literature has not considered the effect of EI on the self-reporting of metacognitive reading-comprehension strategies or the first language. Therefore, the results obtained are novel, and encourage the development of research that can provide more knowledge about this relationship.
Another noteworthy result is the different effect of the reading-comprehension-strategies variable on self-perception of reading success (READER variable) and reading comprehension performance (PISA variable). As we have seen, self-knowledge of reading strategies has a significant positive effect on reading self-perception, as evidenced by several authors (e.g., [
45,
46]). However, this effect was not observed when reading comprehension performance was analysed. This may be explained by the fact that, while reading self-concept refers to beliefs, perceptions and evaluations that students have about their own skills, competences and worth in the school context, academic reading achievement refers to the level of knowledge demonstrated in the area or subject, taking into account age and academic level [
81]. Thus, if the self-reported knowledge of reading strategies is considered to have the same self-reported nature as reading self-perception, it is justifiable that the relationships between self-reported strategies have a greater effect on reading self-perception than on performance. However, our results showed a low–moderate significant correlation between READER and PISA (
r = 0.22; see
Table 5). This linear relationship between both variables is consistent with Pajares and Johnson [
82]. We re-analysed the data, including the direct effect READER → PISA, and respecifying the direct and indirect effects of the PLS-SEM model (see
Figure A1 in the
Appendix B, for illustrative purposes). The relationship between self-perception as readers and reading performance, although statistically significant, is attenuated (β = 0.14), since part of the PISA variability is explained by the effect of oral communication strategies (β = 0.18). This model can be interpreted in the same way as the model proposed in
Figure 2 (i.e., the values and the significance level of the parameters in both models are very similar). The same can be said if we re-specify the model, including the direct effect PISA → READER. Again, the relationship between self-perception as readers and reading performance is attenuated (β = 0.12), but, in this case, the reading-comprehension-strategies variable (MARSI-R) has a significant effect on the outcome, instead of the oral communication strategies (TSOC). Therefore, with the information available, it seems that both READER and PISA variables have low shared variance in the presence of other variables, not affecting the interpretation of the model proposed in this study, in
Figure 2.
It can be highlighted that one of the possible differences between self-reporting measures and performance seems to be related to the effect of EI [
83]. As seen in this paper, EI has a rather strong indirect effect on the performance of reading-comprehension tasks, which is compatible with the results of some papers that identified relationships between EI and academic success [
76]. However, some authors, such as Villavicencio and Bernardo [
84], have also observed that self-efficacy was more highly correlated with performance in the absence of negative emotions. Therefore, perhaps different EI scores may have affected the effect that MARSI-R (which is still a measure of self-efficacy) has on PISA performance. Thus, we propose the need for further studies to identify the interaction effect that EI may have on the relationship between self-reporting measures and actual performance.
In any case, EI is not the only relevant intermediate variable in the relationship between self-reported strategies and performance. In line with Bandura, Loecher et al. or Lee and Jonson-Reid [
42,
85,
86], comprehension success is also mediated by aspects such as motivation or task persistence, among others.
Technical Considerations and Limitations
This work applied PLS-SEM, a recommended analysis technique for applying Confirmatory Composite Analysis [
69,
87]. PLS-SEM (or composite-based SEM) models are more flexible for analysing models that combine reflective and formative measures, than SEM models. In SEM, it is possible to analyse formative constructs in combination with reflective constructs. This statistical technique is characterised by imposing strong restrictions on the models evaluated, which can result in models with identification problems, which are difficult to justify in theoretical terms [
4]. In this regard, Brown [
66] gives some examples, such as setting the disturbance of composite variables to zero or specifying unidirectional paths between endogenous constructs. PLS-SEM does not have these identification problems. Moreover, compared to SEM, PLS-SEM is useful when the analysis is concerned with testing a theoretical framework from a prediction perspective, and when the structural model is complex (i.e., includes many constructs, indicators and/or relationships, avoiding a lack of power), as is the case in this study.
A critical difference between SEM and PLS-SEM is that, in the first approximation, researchers have widely used goodness-of-fit measures to evaluate models such as chi-square (χ
2) or standard root-mean-square residual (SRMR) [
66]. Some fit measures have been studied in PLS-SEM approximation, such as SRMR, root-mean-square residual covariance (RMStheta), or the exact fit test. However, the development of these measures is limited, so more research is needed [
4]. On the other hand, the notion of fit is not fully transferable to PLS-SEM, since the objective in SEM is to minimize the differences between covariance matrices (fit indices are based on the difference between the observed and estimated covariance matrices), while the objective in PLS-SEM is to maximize the explained variance. In any case, we consider that the difficulty in assessing the fit of the PLS-SEM model proposed in this work, beyond its predictive and explanatory power (i.e., structure path and R
2), is a limitation that must be pointed out (see Henseler et al. [
74]), to delve into issues related to the validity of PLS-SEM models.
PLS-SEM should be understood as a complementary technique to classical SEM (CB-SEM) [
88], being an ideal technique when the models to be tested are tentative and the objective is not so much to validate a consolidated theoretical model (the stated objective of SEM), but rather to focus on the researcher’s interest in prediction. It is also particularly useful when combining reflective and formative variables in the analysis of complex relationships. The flexibility of PLS-SEM in these scenarios, common in educational psychology, allows the researcher to know in a relatively simple way how the different predictive variables implemented contribute in terms of explained variance, as well as their direct and indirect relationships with the criteria of interest, constituting an essential source for elaboration and theoretical development.
This study has not considered the convergent validity (i.e., redundancy analysis) of the formative measures. For this, the study design must include the collection of data on some variable that serves to reflect in a similar way what is intended to be collected, through the formative measures evaluated. In our view, it is not easy to define quantitative measures that can be used to analyse the convergent validity of the MARSI-R, and especially the TSOC. It would be preferable to obtain some kind of qualitative assessment by teachers of students’ reading and speaking skills. On the other hand, to evaluate the structural model, Hair et al. [
4] recommend applying PLS-predict to analyse the model’s out-of-sample predictive power. The use of PLS-predict consists in estimating the model on a subsample and assessing its predictive power on a different subsample. In this study, the total sample, although having sufficient statistical power to apply PLS-SEM, is not optimal for partitioning. Questions related to the convergent validity of the formative measures and the predictive power of the model should be referred to further studies.