Next Article in Journal
Entrepreneurship on Social Networking Sites: The Roles of Attitude and Perceived Usefulness
Previous Article in Journal
The Effect of the Secret Agent Society Group Program on Parent-Teacher Agreement Regarding Children’s Social Emotional Functioning
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Italian Validation of the Online Student Engagement Scale (OSE) in Higher Education

1
Department of Human Studies, University of Foggia, 71121 Foggia, Italy
2
School of Psychology, Faculty of Medicine, Health and Life Science, Swansea University, Swansea SA2 8PP, UK
3
Department of Economics, University of Studies ‘G. D’Annunzio’ Chieti and Pescara, 66100 Chieti, Italy
4
Department of Humanities, Pegaso Online University, 80143 Napoli, Italy
5
Department of Neuroscience, Imaging and Clinical Sciences, University of Studies ‘G. D’Annunzio’ Chieti and Pescara, 66100 Chieti, Italy
*
Author to whom correspondence should be addressed.
Behav. Sci. 2023, 13(4), 324; https://doi.org/10.3390/bs13040324
Submission received: 12 March 2023 / Revised: 5 April 2023 / Accepted: 7 April 2023 / Published: 10 April 2023

Abstract

:
During the COVID-19 pandemic, entire university courses were moved online. This represented a challenge for universities, who were required to move toward an entirely online learning environment without adequate time to manage the change from traditional courses to online courses. However, beyond the emergency of the pandemic, higher education does increasingly incorporate an online learning element, and such a provision does appear to reflect both the desires of modern-day students and university offerings. For this reason, assessing students’ online engagement is fundamental, not least because it has been seen to be related both to students’ satisfaction and their academic achievement. A validated measure of student online engagement does not exist in Italy. Therefore, this study aims to assess both the factor structure and the validity of the Online Student Engagement (OSE) Scale in the Italian context. A convenience sample of 299 undergraduate university students completed a series of online questionnaires. The Italian OSE scale presents good psychometric properties and represents a valuable instrument for both practitioners and researchers examining students’ engagement in online learning.

1. Introduction

For many decades, both researchers and academic institutions have invested much effort and attention into understanding students’ engagement with their studies, with Hu and Kuh [1], for example, suggesting that this may be the most important factor influencing both learning and personal development during the college years. Despite the importance of student engagement within the academic context, there is not yet consensus on the definition of the term. In his pioneering work, Astin [2] defines student engagement as the extent to which students participate in learning through “the investment of physical and psychological energy” (p. 519). Skinner and Belmont [3] describe engagement as the intensity and quality of behavioural and emotional involvement during learning activities, whilst Kuh [4,5,6] suggests that engagement can be defined as the amount of time and effort students invest into academic activities relating to learning outcomes. A common theme amongst these definitions is the requirement of time and investment of resources into academic related activities.
It is also possible to differentiate between different types of student engagement. Handelsman et al. [7] make distinctions between affective and behavioural components of engagement, identifying four dimensions: Skills Engagement (such as keeping up with required reading and putting forth effort); Emotional Engagement (aspects such as making the course personally interesting or applying it to one’s own life); Participation/Interaction Engagement (having fun in sessions, participating actively in small group discussions); and Performance Engagement (doing well on tests, obtaining a good grade).
The reasons for a continued focus on academic engagement include the link to a variety of positive educational outcomes [8] as well as its seemingly protective effect against the discontinuation of students’ studies. It is well-documented that the more engaged students are with their studies, the greater their desire is to persist with their academic career [9,10,11].
Traditionally, academic engagement has been considered in the context of ‘in person’ education; however, in recent years there has been an increase in the number of courses which involve education delivered online. Higher education increasingly incorporates an online learning element in student learning, and online learning does appear to reflect the desires of modern-day students and university offerings. There are various ways of defining online learning. For example, Moore and Kearsley [12], in their definition, focus on the geographical distance between the instructor and students and where the majority of the content is remotely engaged with. Meanwhile, Allen and Seaman [13] provide a more precise definition of online learning, stating that these are courses where at least 80% of the material is delivered online. Whilst elements of online learning have been incorporated over time into many higher education offerings, in response to the COVID-19 pandemic, education providers transferred into entirely online formats, often with little time to manage the changes.
In many countries, including Italy, some aspects of the higher education provisions that were introduced in response to the pandemic remain, resulting in blended learning formats where courses integrate both online and in-person aspects [14,15]. Italy also has a number of universities offering courses delivered entirely online (amounting to 11 out of 94 institutions) [16]. Further, Massive Open Online Courses (MOOCs) are widespread in Italy and have an increasing number of student enrolments. EduOpen, for example, was created by a consortium of Italian universities to enable the delivery of MOOCs and currently offers 380 courses attended by more than 125,000 students [17,18]. Such provision enables access to education for students who would not otherwise be able to enrol in traditional universities and is also said to improve retention rates and to permit a response to space constraints in traditional settings [19]. With widespread access to the Internet in modern life and the rapid growth of online education, it is imperative that higher education institutions seek to provide quality online learning experiences. Whilst there are many advantages to online learning, there is also the risk of low student engagement and a sense of isolation amongst students [20,21]. For these reasons, it is vital for institutions to have available to them a method of assessment to ascertain students’ levels of engagement with their online studies in order to inform teaching practice. One such measure is the Online Student Engagement Scale (OSE) [22], which characterises online engagement as a multifaceted construct of four dimensions using the affective-behavioural conceptualization of engagement outlined by Handelsman et al. [7]. The four dimensions of the OSE Scale are: Skills (e.g., taking good notes when carrying out reading), Emotional Engagement (e.g., finding ways to make the course interesting), Participation (e.g., participating actively in small group discussion forums) and Performance (e.g., obtaining a good grade). The OSE Scale [21] was initially developed following a review of existing measures of student engagement. From this review, it was identified that the Student Course Engagement Questionnaire (SCEQ) [7] offered the most appropriate measure to develop further, given that it assessed engagement across a variety of factors. A pool of 30 behaviours that would operationalise these factors was devised by five instructors following a focus group, with amendments made to items in the SCEQ to reflect online courses. These 30 items were then piloted amongst the students in an online course, with the results suggesting a high level of reliability (Cronbach’s alpha = 0.95). Finally, the OSE Scale was tested across a larger and more diverse group of students, where factor analysis identified the four anticipated factors of Skills, Emotional Engagement, Performance and Participation, formed of 19 items. When combined, these items reportedly had a strong level of reliability (Cronbach’s alpha = 0.95).
The OSE Scale is widely cited and there is some evidence of its use outside of the Western world, although to the best of our knowledge, there are no attempts to adapt or validate the scale in this context. A translated version of the OSE Scale is employed by Xinlei and Yiyang [23] who report a Cronbach alpha of 0.82 in their assessment of online learning engagement amongst nursing students.
The OSE is widely used in English speaking countries to good effect, ascertaining factors that may predict online engagement [24], including stress perception and personality traits [25]. In one study [25], the relationship between the aspects of the OSE Scale and personality traits and stress is examined during the COVID-19 emergency. It finds that the personality trait of extraversion predicts participation and performance whilst neuroticism predicts both emotional and skills engagement, as well as performance. Agreeableness predicts participation whilst openness to experience predicts emotional engagement. Meanwhile, conscientiousness predicts all factors of student online engagement, whilst stress perceived as a hindrance is seen to negatively predict performance.
Other relevant factors influencing student online engagement have also been considered using the OSE Scale. Bollinger and Halupa [26], for example, focus on the relationship between online student engagement and transactional distance, which is the “distance of understandings and perceptions, caused in part by the geographic distance, that has to be overcome by teachers, learners and educational organizations if effective, deliberate, planned learning is to occur” (p. 2). Their results suggest that transactional distance predicts student engagement, such that as transactional distance decreases, levels of student engagement increase, thus illustrating the utility of this scale for modern university settings to consider ways to support students’ online learning.
Others [27] have identified relationships between learning satisfaction and online engagement using the OSE Scale, for example, noting that greater levels of students’ perceived learning satisfaction is associated with an increased likelihood of online learning engagement, whilst years of experience working post-qualification is associated with decreased levels of learning engagement, illustrating the applicability of the use of this tool for evaluation of higher education practice.
Despite its great utility, to the authors’ knowledge, the OSE Scale has not been translated into or validated in the Italian language. In fact, no such measure exists in Italy, which means there is not a validated measure available to assess students’ levels of engagement with their online courses. This is especially important as cultural aspects are a vital consideration when evaluating students’ engagement. Research has demonstrated that academic engagement potentially varies as a result of the different educational processes and cultural traditions of the countries in which universities sit. In her cross-cultural analysis, Shcheglova [28] observes that the levels of university student engagement vary across countries. Drawing on the work of Hofstede [29], she notes how countries can be differentiated by their individualistic and collectivist cultures, and in terms of Power Distance Indexes, highlights that these factors can influence educational practices and subsequent student engagement. This demonstrates the importance of evaluating both student engagement and educational practices in the context of cultural factors and the need for context-relevant instruments for the investigation of students’ engagement in online education.
Finally, it is important to note the benefits of this specific tool for the assessment of online engagement. Dixson [22] highlights three primary benefits of this tool: to enable research into the design of courses, to enable feedback on student engagement in response to course design and to enable the evaluation of the effectiveness of teaching. There is a justifiable need to validate this measure for use in Italy; therefore, this study aims to assess both the factor structure and the validity of the OSE Scale in an Italian context. We anticipated that the Italian version will demonstrate the same good psychometric properties observed in the validations of the English version of the OSE Scale. To test the convergent validity of the Italian OSE Scale, we correlate its dimensions with the Utrecht Work Engagement Scale, Student version in its short form (UWES-S-9) [30], as it is, to the best of our knowledge, the only reliable measure of university students’ engagement that has an Italian adaptation and validation. The UWES-S includes three dimensions: Vigour, Dedication and Absorption. More specifically, Vigour considers mental resilience, high energy levels and persistence in the face of difficulties; Dedication refers to high levels of involvement in one’s work, which also incorporates positive states such as enthusiasm, pride, and inspiration; whilst Absorption is defined as a positive state of full immersion in one’s work. We expect positive correlations amongst the sub-dimensions of the Italian OSE Scale and the UWES-S-9, given that they share the common background of student engagement, particularly regarding the affective components of engagement. However, we also anticipate that these scales will be sufficiently distinct constructs (correlation index below 0.70) given the differences between online learning and traditional (offline) learning.

2. Materials and Methods

2.1. Participants and Procedure

This study was conducted according to the ethical principles defined by the Declaration of Helsinki. It involves a convenience sample of 310 undergraduate university students enrolled at the University of Modena and Reggio Emilia (Italy) (response rate 62.2%). Eleven students (1.1% of the respondents) declared that they attended the course mainly in person rather than online, and so were excluded from analysis. The retained sample comprised 299 participants. Of these participants, 289 were female, aged between 20 and 48 (M = 23.29; S.D. = 4.99). The participants were either 1st or 3rd year students enrolled in an Education bachelor’s degree and taking part in a blended Psychology course. Participants did not receive any reward for participation.
Between May and June 2022, students were invited to participate in the study via the course’s Moodle platform. Those that expressed interest in participation received information sheets about the voluntary nature of their participation, the study and their role within it and their right to withdraw from the study at any time. Participants provided informed written consent to proceed and independently completed a self-administered and structured online questionnaire, which took around 10 min to complete.
The first section of the questionnaire aimed to assess demographic characteristics (i.e., age, gender, type of course). Then, all participants completed the OSE Scale [22] translated into Italian, and the Italian translation of the Utrecht Work Engagement Scale, Student version, short form (UWES-S-9) [30]. On the completion of the survey, participants were thanked and debriefed.

2.2. Instruments

Online Student Engagement Scale (OSE) [21,22] Italian translation.
Respondents indicated their level of online engagement with their course by indicating the extent to which 19 statements were characteristic of their behaviours using a 5-point Likert scale, from 1 (“not at all characteristic of me”) to 5 (“very characteristic of me”). Four dimensions of engagement were assessed: Skills (e.g., “taking good notes over readings, PowerPoints, or video lectures”), Emotional Engagement (e.g., “really desiring to learn the material”), Participation (e.g., “participating actively in small-group discussion forums”) and Performance (e.g., “getting a good grade”).
Utrecht Work Engagement Scale, Student version, short form; Italian translation (UWES-S-9), [30].
Respondents indicated their levels of engagement with their studies in terms of Vigour (e.g., “When I get up in the morning, I feel like going to class”), Absorption (e.g., “I get carried away when I am studying”) and Dedication (e.g., “I am enthusiastic about my studies”), indicating the extent to which 9 statements applied to them on a 7-point Likert scale, from 0 (Never) to 6 (Always, Every day).

2.3. Translation Process

Permission for translation and validation was received from the author of the original OSE Scale via email. The procedure of translation included three steps. First, two native English speakers independently translated the items of the scale into Italian (forward translation). One translator additionally had a psychology background and was the assessor of the translation. Next, two bilingual researchers, blind to the original version of the scale, independently back-translated the scale into English. These two new English versions were translated into Italian by two independent psychology researchers with a certificated knowledge of the English language, and blind to the original version (backward translation).

2.4. Analytical Approach

A preliminary analysis of the Italian OSE Scale was carried out using IBM SPSS Statistics for Windows vers. 27 and involved the detection of multivariate outliers via the Mahalanobis distance test [31]. We also assessed means, standard deviations and normal distribution by examining the indices of skewness and kurtosis.
To test the factor structure of the OSE scale in the Italian language, a Confirmatory Factor Analysis (CFA) was carried out. This was performed using the statistical software R [32], specifically, the Lavaan package [33]. The Diagonally Weighted Least Squares (DWLS) estimator was used. This method was chosen since an estimator of Maximum Likelihood (ML) for non-continuous variables (such as Likert scales) is not currently supported in the Lavaan library. The number of model parameters was 100, while the number of observations was 297.
To evaluate the CFA models, the goodness of fit was estimated using the Root Mean Square Error of Approximation (RMSEA), the Standardized Root Mean Square Residual (SRMR), the Comparative fit index (CFI) and the Tucker-Lewis index (TLI).
RMSEA is an absolute-fit index, in that it assesses how far a hypothesized model is from a perfect model, where a lower RMSEA value is better. A value less than 0.05 is generally considered a good fit. For SRMR, a value less than 0.08 is generally considered to be a good fit. CFI and TLI are incremental-fit indices that compare the fit of a hypothesized model with that of a baseline model, that is, a model with the worst fit [34]. Higher CFI and TLI values are better: the fit is good for values greater than 0.9. In the CFA, we tested a 4-factor model, following the original structure of the scale: Skills, Emotional Engagement, Participation and Performance. The internal consistency of the Italian translation of the OSE Scale was evaluated by assessing the Composite Reliability (CR). CR can be regarded as “an indicator of the shared variance among the observed variables used as an indicator of a latent construct” and can be calculated for each construct [35] (p. 384). According to Saunders, Lewis and Thornhill [36], a value of 0.7 or greater indicates that the items within a scale can be regarded as measuring the same variable of interest. We additionally computed measures of Average Variance Extracted (AVE). AVE provides an indication of the amount of variance provided by a construct relative to the amount of variance resulting from measurement error. Values exceeding 0.50 are generally considered acceptable.
In the Italian validation of the UWES-S-9 [30], the three dimensions (i.e., Vigour, Absorption, Dedication) showed adequate fit indices: RMSEA: 0.08, CFI: 0.96, TLI: 0.95. The authors also correlated the residuals for items 8 (‘I am immersed in my study’) and 9 (‘I get carried away when I am studying’), and item 1 (‘When I’m doing my work as a student, I feel bursting with energy’) and 5 (‘When I get up in the morning, I feel like going to class’). All the factor loadings were high, ranging from 0.62 to 0.91. The Cronbach alpha for the three subscales were above the cut-off of 0.70 and showed good internal consistency: α = 0.82 (95% CI: 0.79–0.85) for Vigour; α = 0.88 (95% CI: 0.86–0.90) for Dedication; and α = 0.76, (95% CI: 0.72–0.79) for Absorption. In this validation, the scale showed the invariance of the structure between Italian males and females, as well as between students of different ages and years of study (i.e., Bachelor’s years compared to Master’s years) [30].

3. Results

3.1. Preliminary Analyses and Descriptive Statistics

The Mahalanobis distance test indicated that two participants could be considered multivariate outliers with a distance value of 34.66 (p < 0.001) and 26.99 (p < 0.001), respectively. For this reason, we deleted these two participants from the analyses. The retained sample included 297 participants (287 females, M age = 23.29, SD = 5.02).
Table 1 reports the descriptive statistics (Mean, Standard Deviation, Skewness and Kurtosis) for the items of the Italian OSE Scale. Inspection of skewness and kurtosis values indicated that items are normally distributed (the indices are between −0.96 and 0.68), so no variable transformations are necessary.

3.2. Factor Structure of the Italian Translation of the OSE Scale

The solution of CFA converged in 29 iterations, yielding a model statistic of 588.037 with 146 degrees of freedom. CFA demonstrates that a 4-factor solution is a good fit for the data: RMSEA is 0.10, SRMR is 0.08, and CFI and TLI values are 0.97 and 0.96. In Table 2, fit indices for User versus Baseline model are reported.
Examining the factor loadings, all items have loadings greater than 0.30 on the expected factor (Table 3). Specifically, the six items of the Skills factor show factor loadings between 0.64 and 0.79, the five items of the Emotional factor show factor loadings between 0.75 and 0.83 and the six items of Participation factor show factor loadings between 0.65 and 0.82. Finally, the two items of Performance factor show factor loadings of 0.69 and 0.82.
In short, the CFA indicates that the Italian version of the OSE Scale shows a structure comparable to the original, with excellent fit indices and factor loadings on the expected factor.
The latent variables are positively correlated with each other (Table 4). The strongest correlation is observed between the Skills and Emotional engagement factors (Estimate = 0.65, p < 0.001, whereas a weaker correlation is observed between the Participation and Emotional engagement factors (Estimate = 0.46; p < 0.001).
The CR for the four dimensions of the Italian translation of the OSE Scale (Skills, Emotional Engagement, Participation and Performance) are 0.86, 0.89, 0.88, 0.72, respectively, thus confirming good reliability. In addition, the four dimensions of the translated OSE Scale show satisfactory values of AVE: Skills: 0.53, Emotional Engagement: 0.62, Participation: 0.55 and Performance: 0.57.

3.3. Convergent Validity

Before examining the correlations between the factors of the translated OSE Scale and the dimensions of the Italian UWES-S-9, we checked the reliability of this last scale in our sample. The three dimensions of the UWES-S-9 show good indices of CR and AVE: For Vigour, these values are 0.85 and 0.66, respectively, 0.87 and 0.69 for Dedication and 0.79 and 0.55 for Absorption.
For convergent validity, as expected, the factors of the Italian translation of the OSE scale positively correlate with the dimensions of the Italian UWES-S-9, supporting the suggestion that the OSE Scale is an appropriate tool in evaluating students’ engagement (Table 5). Using Cohen’s interpretation of r-values, we observe that Skills correlate highly with Vigour (r = 0.47; p < 0.001) and Absorption (r = 0.42; p < 0.001), and moderately with Dedication (r = 0.29; p <.001). Emotional Engagement correlates highly with Absorption (r = 0.42; p < 0.001) and Dedication (r = 0.45; p < 0.001), and moderately with Absorption (r = 0.34; p < 0.001). Participation correlates highly with Vigour (r = 0.47; p < 0.001) and moderately with Absorption (r = 0.31; p < 0.001) and Dedication (r = 0.24; p < 0.001), whilst Performance correlates moderately with the three dimensions of the UWES: Vigour (r = 0.33; p < 0.001), Absorption (r = 0.30; p < 0.001) and Dedication (r = 0.31; p < 0.001).

4. Discussion

The aim of this study is to provide an Italian version of the OSE Scale and to test its structure and validity. The results confirm the reliability and validity of the Italian version of the OSE Scale [21,22] on a Western European university student population from Italy. The confirmatory factor analysis suggests the same four-factor solution for the Italian version: Skills (e.g., taking good notes), Emotional Engagement (e.g., finding ways to make the course interesting), Participation (e.g., participating actively in small group discussion forums) and Performance (e.g., obtaining a good grade). The four dimensions show good reliability and good internal consistency. Finally, as expected, the factors of the OSE Scale positively correlate with the dimensions of the UWES-S [30], demonstrating the appropriateness of these scales for evaluating student engagement, which as online learning increases, becomes more important.
Numerous studies have linked student engagement with a variety of positive educational outcomes, [8] such as academic achievement. Extensive empirical research on the relationship between student engagement and academic achievement exists, and whilst results are not always consistent, e.g., [37,38], a recent meta-analysis [39] analysing 69 independent samples found a moderately strong and positive correlation between overall student engagement and academic achievement. As such, measuring students’ engagement in a reliable way is vital. Blended or online-only courses provide instructors unique opportunities to monitor engagement by using the trace data collected by the learning environment. However, most higher education instructors are not educational researchers or data specialists. Thus, the challenges of accessing and using some types of data might be a barrier to successfully monitoring student engagement [40]. As such, there is a benefit to instructors in employing the OSE Scale. Whilst achievement is not the only goal of an education system, it is a common measure of its success. It is used to evaluate, together with other indices, both the performance of schools and universities and to measure changes in individual students’ level of achievement.
Student engagement has also been seen to have an influence on students’ satisfaction. For example, Rajabalee and Santally [41] find a significant positive relationship between satisfaction and engagement (measured with the OSE Scale) in a sample of 844 first year university students across disciplines, as well as a weak but positive significant correlation between satisfaction and engagement with students’ overall performance. Moreover, Baloran et al. [42] find that satisfaction with an online course is significantly correlated with online student engagement in a sample of 529 university Filipino students during the COVID-19 pandemic. Through structural equation modelling, it is further demonstrated that online course satisfaction is significantly related to students’ skills engagement, emotional engagement, participation and performance, demonstrating the value of assessment of online engagement as part of a range of considerations.

5. Limitations and Conclusions

The study has some limitations that need to be considered. Firstly, it is important to note that our results are based on a single convenience sample and are limited by the sociocultural characteristics of the university settings where the study was conducted. Secondly, the percentage of females in the sample is high (96%) and this does not allow us to verify any gender differences in the levels of our variables. Thirdly, engagement is measured amongst students enrolled in a single course, and this may prevent generalisation to students enrolled in other majors and the university student population in general. The presence of only two items assessing the performance dimension is another limitation; however, this limitation follows the original version of the scale [21,22]. Another limitation regards the use of self-report measures and the possibility that this format may be impacted by socially desirable responses and measurement bias [43].
Despite these limitations, this work provides a useful tool for Italian researchers and academics to assess students’ online engagement. Assessing student engagement in online learning is fundamental in analysing students’ learning processes, and the level of engagement has the potential to be an indicator of online learning effectiveness. Overall, we provide evidence for the good psychometric properties of the Italian translated OSE Scale and suggest that it is a useful instrument for researchers and practitioners in several domains.
Future studies should now investigate university students’ online engagement in a more diverse sample to verify any potential differences. This Italian version of the OSE Scale will enable cross-cultural comparisons as well as comparative studies in general. The evidence of cultural differences in students’ engagement [28] acts to further justify the validation of this measure in the Italian language. Hofstede [29] previously reported a medium Power Distance Index amongst the Italian population. Studies using the OSE Scale can now assess this in a population of university students and compare students from different parts of the country (i.e., North vs. South vs. Isles). Future research is now able to consider Italian students’ online engagement as well as other variables linked to both their learning satisfaction and academic achievement to improve student experience. In short, this Italian OSE scale can be used to investigate, evaluate and further improve the higher educational landscape.

Author Contributions

Conceptualization, F.S. and A.A.; methodology, F.S., R.H., A.T. and A.A.; validation, F.S., R.H., P.L. and A.A.; formal analysis, A.T. and A.A.; investigation, F.S.; data curation, F.S. and A.T.; writing—original draft preparation, F.S.; writing—review and editing, F.S., R.H. and A.A.; all authors contributed to visualization; supervision, P.L.; project administration, F.S.; All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, following the general research principles and the ethical rules of the Italian Psychological Association (AIP).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy reasons.

Acknowledgments

The authors are very grateful to the students who participated in this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hu, S.; Kuh, G.D. Being (dis)engaged in educationally purposeful activities: The influences of student and institutional characteristics. Res. High. Educ. 2002, 43, 555–575. [Google Scholar] [CrossRef]
  2. Astin, A.W. Student involvement: A developmental theory for higher education. J. Coll. Stud. Pers. 1984, 25, 297–308. [Google Scholar]
  3. Skinner, E.A.; Belmont, M.J. Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. J. Educ. Psychol. 1993, 85, 571–581. [Google Scholar] [CrossRef]
  4. Kuh, G.D. Assessing what really matters to student learning inside the national survey of student engagement. Change 2001, 33, 10–17. [Google Scholar] [CrossRef]
  5. Kuh, G.D. What we’re learning about student engagement from NSSE: Benchmarks for effective educational practices. Change 2003, 35, 24–32. [Google Scholar] [CrossRef]
  6. Kuh, G.D. The national survey of student engagement: Conceptual and empirical foundations. New Dir. Inst. Res. 2009, 141, 5–20. [Google Scholar] [CrossRef]
  7. Handelsman, M.M.; Briggs, W.L.; Sullivan, N.; Towler, A. A measure of college student course engagement. J. Educ. Res. 2005, 98, 184–191. [Google Scholar] [CrossRef]
  8. Vytasek, J.M.; Patzak, A.; Winne, P.H. Analytics for student engagement. In Intelligent Systems Reference Library: Machine Learning Paradigms; Virvou, M., Alepis, E., Tsihrintzis, G., Jain, L., Eds.; Springer: Cham, Switzerland, 2020; Volume 158, pp. 23–48. [Google Scholar] [CrossRef]
  9. Carini, R.M.; Kuh, G.D.; Klein, S.P. Student engagement and student learning: Testing the linkages. Res. High. Educ. 2006, 47, 1–32. [Google Scholar] [CrossRef] [Green Version]
  10. Fantinelli, S.; Esposito, C.; Carlucci, L.; Limone, P.; Sulla, F. The Influence of Individual and Contextual Factors on the Vocational Choices of Adolescents and Their Impact on Well-Being. Behav. Sci. 2023, 13, 233. [Google Scholar] [CrossRef]
  11. Junco, R. The relationship between frequency of Facebook use, participation in Facebook activities, and student engagement. Comput. Educ. 2012, 58, 162–171. [Google Scholar] [CrossRef]
  12. Moore, M.G.; Kearsley, G. Distance Education: A Systems View of Online Learning, 3rd ed.; Wadsworth, Cengage Learning: Belmont, CA, USA, 2012. [Google Scholar]
  13. Allen, I.E.; Seaman, J. Going the Distance: Online Education in the United States; The Online Learning Consortium. 2011. Available online: http://sloanconsortium.org/publications/survey/going_distance_2011 (accessed on 10 March 2023).
  14. Limone, P.; Di Fuccio, R. teleXbe 2021-The role of technologies in education and new trajectories of blended learning. In Proceedings of the Technology Enhanced Learning Environments for Blended Education—The Italian e-Learning Conference 2021, Foggia, Italy, 5–6 October 2021; pp. 1–6. Available online: https://www.semanticscholar.org/paper/teleXbe-2021-The-role-of-technologies-in-education-Limone-Fuccio/11781816fe534e86cec3ea2e48821a5804792e83 (accessed on 16 January 2023).
  15. Limone, P. Towards a hybrid ecosystem of blended learning within university contexts. In Proceedings of the Technology Enhanced Learning Environments for Blended Education—The Italian e-Learning Conference 2021, Foggia, Italy, 5–6 October 2021; pp. 1–7. Available online: https://www.semanticscholar.org/paper/Towards-a-hybrid-ecosystem-of-blended-learning-Limone/cbf475bdf39356b1027f22ae7bdd76376346247d (accessed on 16 January 2023).
  16. Limone, P.; Toto, G.A. Manuale T.I.C. Per una Didattica Inclusiva; [ICT Textbook for an Inclusive Education]; McGraw Hill: Milano, Italy, 2022. [Google Scholar]
  17. Limone, P. EduOpen network in Italy. In How to Design for Persistence and Retention in MOOCs? Brasher, A., Weller, M., McAndrew, P., Eds.; EADTU: Maastricht, The Netherlands, 2016; pp. 19–24. [Google Scholar]
  18. Limone, P.; Pati, S.; Toto, G.A.; Di Fuccio, R.; Baiano, A.; Lopriore, G. Literature Review on MOOCs on Sensory (Olfactory) Learning. Computers 2022, 11, 32. [Google Scholar] [CrossRef]
  19. Bacow, L.S.; Bowen, W.G.; Guthrie, K.M.; Lack, K.A.; Long, P.L. Barriers to Adoption of Online Learning Systems in U.S. Higher Education; Ithaka: New York, NY, USA, 2012; pp. 39–51. Available online: http://www.sr.ithaka.org/research-publications/barriers-adoption-online-learningsystems-us-higher-education (accessed on 10 March 2023).
  20. Sulla, F.; Camia, M.; Scorza, M.; Giovagnoli, S.; Padovani, R.; Benassi, E. The Moderator Effect of Subthreshold Autistic Traits on the Relationship between Quality of Life and Internet Addiction. Healthcare 2023, 11, 186. [Google Scholar] [CrossRef] [PubMed]
  21. Dixson, M.D. Creating effective student engagement in online courses: What do students find engaging? J. Scholarsh. Teach. Learn. 2012, 10, 1–13. Available online: https://scholarworks.iu.edu/journals/index.php/josotl/article/view/1744 (accessed on 10 March 2023).
  22. Dixson, M.D. Measuring Student Engagement in the Online Course: The Online Student Engagement Scale (OSE). Online Learn. 2015, 19, n4. [Google Scholar] [CrossRef] [Green Version]
  23. Xinlei, C.; Yiyang, L. Online Learning Engagement in International Collaborative Nursing Students: A Questionnaire Study. Bachelor’s Thesis, Lishui University, Lishui, China, 2022. [Google Scholar]
  24. Cole, A.W.; Lennon, L.; Weber, N.L. Student perceptions of online active learning practices and online learning climate predict online course engagement. Interact. Learn. Environ. 2021, 29, 866–880. [Google Scholar] [CrossRef]
  25. Quigley, M.; Bradley, A.; Playfoot, D.; Harrad, R. Personality traits and stress perception as predictors of students’ online engagement during the COVID-19 pandemic. Pers. Individ. Dif. 2022, 194, 111645. [Google Scholar] [CrossRef] [PubMed]
  26. Bolliger, D.U.; Halupa, C. Online student perceptions of engagement, transactional distance, and outcomes. Distance Educ. 2018, 39, 299–316. [Google Scholar] [CrossRef]
  27. Chan, S.L.; Lin, C.C.; Chau, P.H.; Takemura, N.; Fung JT, C. Evaluating online learning engagement of nursing students. Nurse Educ. Today 2021, 104, 104985. [Google Scholar] [CrossRef]
  28. Shcheglova, I.A. A cross-cultural comparison of the academic engagement of students. Sov. Educ. 2018, 60, 665–681. [Google Scholar] [CrossRef]
  29. Hofstede, G. Cultural differences in teaching and learning. Int. J. Intercult. Relat. 1986, 10, 301–320. [Google Scholar] [CrossRef]
  30. Loscalzo, Y.; Giannini, M. Study Engagement in Italian University Students: A Confirmatory Factor Analysis of the Utrecht Work Engagement Scale—Student Version. Soc. Indic. Res. 2019, 142, 845–854. [Google Scholar] [CrossRef]
  31. Ghorbani, H. Mahalanobis Distance and Its Application for detecting multivariate outliers. Facta Univ. Ser. Math. Inform. 2019, 34, 583–595. [Google Scholar] [CrossRef]
  32. Team R Core. A Language and Environment for Statistical Computing (Version 3.4.2); [Computer Software]; R Foundation for Statistical Computing: Vienna, Austria, 2017. [Google Scholar]
  33. Rosseel, Y. Lavaan: An R package for structural equation modeling. J. Stat. Softw. 2012, 48, 1–36. [Google Scholar] [CrossRef] [Green Version]
  34. Xia, Y.; Yang, Y. RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behav. Res. 2019, 51, 409–428. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Fornell, C.; Larcker, D.F. Structural equation models with unobservable variables and measurement error: Algebra and statistics. J. Mark. Res. 1981, 18, 382–388. [Google Scholar] [CrossRef]
  36. Saunders, M.; Lewis, P.; Thornhill, A. Research Methods for Business Students; Pearson Education: Harlow, UK, 2009; ISBN 978-0-273-71686-0. [Google Scholar]
  37. Crossan, B.; Field, J.; Gallacher, J.; Merrill, B. Understanding participation in learning for nontraditional adult learners: Learning careers and the construction of learning identities. Br. J. Sociol. Educ. 2003, 24, 55–67. [Google Scholar] [CrossRef]
  38. Shernoff, D.J.; Schmidt, J.A. Further evidence of an engagement–achievement paradox among U.S. high school students. J. Youth Adolesc. 2008, 37, 564–580. [Google Scholar] [CrossRef]
  39. Lei, H.; Cui, Y.; Zhou, W. Relationships between student engagement and academic achievement: A meta-analysis. Self Identity 2018, 46, 517–528. [Google Scholar] [CrossRef]
  40. Pardo, A. Designing learning analytics experiences. In Learning Analytics; Larusson, J., White, B., Eds.; Springer: Cham, Switzerland, 2014; pp. 15–38. [Google Scholar] [CrossRef]
  41. Rajabalee, Y.B.; Santally, M.I. Learner satisfaction, engagement and performances in an online module: Implications for institutional e-learning policy. Educ. Inf. Technol. 2021, 26, 2623–2656. [Google Scholar] [CrossRef]
  42. Baloran, E.T.; Hernan, J.T.; Taoy, J.S. Course satisfaction and student engagement in online learning amid COVID-19 pandemic: A structural equation model. Turk. Online J. Distance Educ. 2021, 22, 1–12. [Google Scholar] [CrossRef]
  43. Lavidas, K.; Papadakis, S.; Manesis, D.; Grigoriadou, A.S.; Gialamas, V. The Effects of Social Desirability on Students’ Self-Reports in Two Social Contexts: Lectures vs. Lectures and Lab Classes. Information 2022, 13, 491. [Google Scholar] [CrossRef]
Table 1. Descriptive statistics of the Italian OSE Scale items.
Table 1. Descriptive statistics of the Italian OSE Scale items.
ItemsMeanSDSkewnessKurtosis
S1: Assicurarsi di studiare con regolarità (Making sure to study on a regular basis)3.610.91−0.470.14
S2: Stare al passo con le lezioni (Staying up on the readings)3.551.11−0.38−0.63
S3: Controllare gli appunti prima della lezione per essere certo di comprendere il materiale del corso (Looking over class notes)3.071.12−0.11−0.74
S4: Essere organizzatə (Being organized)3.990.99−0.840.28
S5: Prendere degli appunti accurati sulle lezioni, in PowerPoint o le video lezioni
(Taking good notes over readings, PowerPoints or, video lectures)
3.811.06−0.71−0.12
S6: Ascoltare/leggere attentamente (Listening/reading carefully)4.080.78−0.680.68
E1: Metterci molto impegno (Putting forth effort)4.180.76−0.760.67
E2: Trovare modi per rendere il materiale del corso pertinente con la mia vita (Finding ways to make the course materials relevant to my life)3.840.96−0.49−0.27
E3: Applicare i contenuti del corso alla mia vita (Applying course material to my life)3.950.91−0.610.01
E4: Trovare modi per rendere il corso interessante per me (Finding ways to make the course interesting to me)3.840.83−0.33−0.27
E5: Avere davvero voglia di imparare il materiale del corso (Really desiring to to learn the material)4.150.77−0.620.19
PA1: Divertirsi nelle chat online, discussioni o via email con il docente o gli altri studenti (Having fun in online chats)2.751.120.15−0.72
PA2: Partecipare in maniera attiva in discussioni in piccolo gruppo su forum (ad es. whatsapp, etc.) (Participating actively in small-group discussion forums)3.031.080.04−0.57
PA3: Aiutare ɜ colleghɜ (Helping fellow students)3.910.87−0.620.17
PA4: Partecipare a conversazioni online (chat, discussioni, email) sul corso (Engage in conversations online chat, discussions, email)2.991.090.07−0.58
PA5: Postare sul forum della piattaforma online (es. dolly, Teams) con regolarità (Posting in the discussion forum regularly)2.571.230.37−0.87
PA6: Riuscire a conoscere altr * collegh * che frequentano il corso (Getting to know other students in the class)3.471.12−0.45−0.41
PER1: Voler ottenere un buon voto (Getting a good grade)4.430.69−0.960.13
PER2: Avere buoni risultati nei [in eventuali] test/quiz [in itinere] (es. Kahoot) (Doing well in the tests/quizzes)3.910.97−0.710.17
Table 2. Fit indices of User Model (Standard) versus Baseline (Standard) Model.
Table 2. Fit indices of User Model (Standard) versus Baseline (Standard) Model.
Fit IndexStandard Robust
CFI0.970.89
TLI0.960.88
RMSEA0.100.11
RMSEA 90% Confidence Interval-Lower0.090.10
RMSEA 90% Confidence Interval-Upper0.110.12
p Value RMSEA<0.001<0.001
SRMR0.080.08
Table 3. Results of CFA: Estimates (EST), Standard Error (SE), z-value and significance (p) values are reported for each item of the OSE scale (the original item is reported in brackets).
Table 3. Results of CFA: Estimates (EST), Standard Error (SE), z-value and significance (p) values are reported for each item of the OSE scale (the original item is reported in brackets).
ESTSEz-Valuep
Skills =~
S1: Assicurarsi di studiare con regolarità (Making sure to study on a regular basis) 0.740.0323.86<0.001
S2: Stare al passo con le lezioni (Staying up on the readings)0.640.0317.04<0.001
S3: Controllare gli appunti prima della lezione per essere certo di comprendere il materiale del corso (Looking over class notes…)0.740.0324.73<0.001
S4: Essere organizzatǝ (Being organized)0.690.0418.98<0.001
S5: Prendere degli appunti accurati sulle lezioni, in PowerPoint o le video lezioni
(Taking good notes over readings, PowerPoints or, video lectures)
0.750.0325.63<0.001
S6: Ascoltare/leggere attentamente (Listening/reading carefully)0.790.0326.06<0.001
Emotional =~
E1: Metterci molto impegno (Putting forth effort)0.750.0418.79<0.001
E2: Trovare modi per rendere il materiale del corso pertinente con la mia vita (Finding ways to make the course materials relevant to my life)0.830.0331.97<0.001
E3: Applicare i contenuti del corso alla mia vita (Applying course material to my life)0.780.0328.54<0.001
E4: Trovare modi per rendere il corso interessante per me (Finding ways to make the course interesting to me)0.830.0331.28<0.001
E5: Avere davvero voglia di imparare il materiale del corso (Really desiring to to learn the material)0.750.0324.50<0.001
Participation =~
PA1: Divertirsi nelle chat online, discussioni o via email con il docente o gli altri studenti (Having fun in online chats…)0.760.0327.38<0.001
PA2: Partecipare in maniera attiva in discussioni in piccolo gruppo su forum (ad es. whatsapp, etc.) (Participating actively in small-group discussion forums)0.790.0330.54<0.001
PA3: Aiutare ɜ colleghɜ (Helping fellow students)0.650.0514.35<0.001
PA4: Partecipare a conversazioni online (chat, discussioni, email) sul corso (Engaging in conversations online (chat, discussions, email))0.820.0233.82<0.001
PA5: Postare sul forum della piattaforma online (es. dolly, Teams) con regolarità (Posting in the discussion forum regularly)0.710.0322.01<0.001
PA6: Riuscire a conoscere altrɜ colleghɜ che frequentano il corso (Getting to know other students in the class)0.690.0418.29<0.001
Performance =~
PER1: Voler ottenere un buon voto (Getting a good grade)0.690.0513.97<0.001
PER2: Avere buoni risultati nei (IN EVENTUALI) test/quiz (IN ITINERE) (es. Kahoot) (Doing well in the tests/quizzes)0.820.0516.14<0.001
Table 4. Zero-order correlations among the latent variables (Oblimin Rotation): Estimates (EST), Standard Error (SE), z-value and significance (p) values.
Table 4. Zero-order correlations among the latent variables (Oblimin Rotation): Estimates (EST), Standard Error (SE), z-value and significance (p) values.
ESTSEz-Valuep
Skills ~~
Emotional0.650.0318.51<0.001
Participation0.590.0413.47<0.001
Performance0.590.0510.87<0.001
Emotional ~~
Participation0.470.0510.23<0.001
Performance0.460.067.38<0.001
Participation ~~
Performance0.590.0610.41<0.001
Table 5. Zero-order Correlations between the Italian OSE Scale and the Italian UWES-S-9 dimensions.
Table 5. Zero-order Correlations between the Italian OSE Scale and the Italian UWES-S-9 dimensions.
VigorAbsorptionDedication
Skills0.47 ***0.42 ***0.29 ***
Emotional0.34 ***0.42 ***0.45 ***
Participation0.47 ***0.31 ***0.24 ***
*** p < 0.001.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Sulla, F.; Harrad, R.; Tontodimamma, A.; Limone, P.; Aquino, A. Italian Validation of the Online Student Engagement Scale (OSE) in Higher Education. Behav. Sci. 2023, 13, 324. https://doi.org/10.3390/bs13040324

AMA Style

Sulla F, Harrad R, Tontodimamma A, Limone P, Aquino A. Italian Validation of the Online Student Engagement Scale (OSE) in Higher Education. Behavioral Sciences. 2023; 13(4):324. https://doi.org/10.3390/bs13040324

Chicago/Turabian Style

Sulla, Francesco, Rachel Harrad, Alice Tontodimamma, Pierpaolo Limone, and Antonio Aquino. 2023. "Italian Validation of the Online Student Engagement Scale (OSE) in Higher Education" Behavioral Sciences 13, no. 4: 324. https://doi.org/10.3390/bs13040324

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop