Next Article in Journal
Correction: Park, H.; Lee, M.-Y. The Two-Sided Effect of the COVID-19 Pandemic on Online Apparel Renting. Sustainability 2022, 14, 16771
Previous Article in Journal
Intergenerational Transmission of Collective Memory in Public Spaces: A Case Study of Menghe, a Historic and Cultural Town
Previous Article in Special Issue
Bringing the Studio Home: Fostering Socially Engaged Arts Education and Sustainability in Online Learning
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 17
Issue 19
10.3390/su17198597
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Internet Skills Scale (ISS) in University Students from Chile: Factorial Structure, Reliability, Validity and Measurement Invariance of the Chilean Version

by
Miguel Galván-Cabello
1,2,
Julio Tereucan-Angulo
1,*,
Gustavo Troncoso-Tejada
3,
David Arellano-Silva
4,
Víctor Sánchez-Gallegos
5 and
Isidora Nogués-Solano
1
1
Department of Social Work, Universidad de La Frontera, Temuco 4780000, Chile
2
Millennium Nucleus on Digital Inequalities and Opportunities (NUDOS), Santiago 8320155, Chile
3
Department of Education, Universidad de La Frontera, Temuco 4780000, Chile
4
Graduate Academic Administration, Universidad de La Frontera, Temuco 4780000, Chile
5
Faculty of Law and Social Sciences, Universidad de Monterrey, Monterrey 66238, Mexico
*
Author to whom correspondence should be addressed.
Sustainability 2025, 17(19), 8597; https://doi.org/10.3390/su17198597
Submission received: 23 June 2025 / Revised: 3 August 2025 / Accepted: 4 August 2025 / Published: 25 September 2025
(This article belongs to the Special Issue Efficiency and Effectiveness of Universities in Achieving Sustainable Development Goals (SDGs))
Browse Figure
Versions Notes

Abstract

Within the framework of the 2030 Agenda, universities are key institutions in promoting digital competencies aligned with Sustainable Development Goals (SDGs), particularly SDG 4 (Quality Education) and SDG 10 (Reduced Inequalities). This study evaluates the psychometric properties of the Internet Skills Scale (ISS), adapted for Chilean university students, as a tool to assess how effectively higher education fosters digital skills that enable critical participation and social inclusion. Using a sample of 906 students from nine public universities across Chile, the ISS was linguistically and culturally adapted, and its factorial structure, reliability, validity, and measurement invariance were tested. The results support a four-factor model—operational, navigation, social, and creative skills—under a second-order structure, with strong fit indices (CFI = 0.987; RMSEA = 0.055) and high internal consistency (α > 0.83). The ISS also demonstrated gender-based measurement invariance and convergent validity with digital citizenship. These findings underscore the ISS as a valid instrument for monitoring the effectiveness and equity of digital education policies in universities. Its application contributes to diagnosing institutional performance regarding the integration of digital competencies into curricula, thus guiding improvements in educational strategies toward socially just, inclusive, and sustainable digital participation.

1. Introduction

Within the framework of the 2030 Agenda for Sustainable Development, the Sustainable Development Goals (SDGs) propose a global commitment for achieving equitable, just, and environmentally sustainable development [1]. Among these goals, SDG 4 emphasizes the importance of ensuring an Education for Sustainable Development (ESD), which is centered on three core aspects—environmental integrity, economic viability, and a just society—that promote lifelong learning opportunities, including the development of essential digital skills for an active and informed participation in contemporary society [1,2]. In this sense, higher education positions itself as a key element in shaping digital citizens that are critical, responsible, and committed with regard to the social and environmental challenges facing future generations [3,4,5,6]. Digital skills thus emerge as an element that enables university students to become agents of change in the social and political dimensions of society [7,8].
From this perspective, access to and competent use of the Internet has become a fundamental skill for university students to not only access information but also actively participate in processes of social and political transformation, aligned with the principles and goals of the SDGs [9,10]. In this regard, some studies show that the development of the ability to navigate, critically evaluate, and effectively use digital tools enables young university students to become agents of change, capable of exercising an informed and responsible digital activism that contributes to social justice, diversity, inclusion, and environmental sustainability [11,12].
Despite global recognition of the importance of digital skills for the education of young critical citizens responsible with their social and natural environment, various investigations indicate that university education in this area presents significant gaps and challenges [13,14,15]. In Chile, as in other contexts, it is important to continue advancing in the development of appropriate instruments for the evaluation of digital competencies in university settings [16,17]. In that sense, accurately measuring specific digital competencies related to the critical and active use of the internet among university students presents a challenge for ensuring and enhancing sustainable impacts [18], as well as for strengthening educational programs in higher education institutions aimed at developing the SDGs.
In relation to the above, Ref. [19] reflect on the controversy surrounding the incorporation of the sustainable development goals into Chilean education. The authors warn that the epistemological foundations of education for sustainability become strained when questioning to what extent it promotes an active and critical citizenship, capable of boosting the transformation of new generations into committed eco-citizens. This tension deepens when sustainability is framed within a model of hyper-consumerism aimed at a sustainable society only at the individual level, rather than collectively [19].
From this perspective, we consider that this epistemological problem present in the education for sustainability in Chile could widen the existing gap between the digital skills students possess and their actual critical capacity to consciously engage in the promotion of the Sustainable Development Goals (SDGs). This disconnection is particularly relevant in educational spaces that, among their objectives, claim to achieve greater social justice through the encouragement of sociopolitical participation and digital activism. In this regard, it would seem that technical mastery of digital tools is not enough; rather, it is also fundamental to pair this with an integral education that fosters critical thinking and civic commitment, as reflected upon by [19].
For the reasons mentioned above, it is crucial to have reliable and validated tools, such as the Chilean adaptation of the Internet Skills Scale (ISS), that make it possible to identify the level of internet skills among university students and their relationship with the commitment to the SDGs, especially in increasingly digitalized contexts. In this way, the aim is to offer new insights into how to strengthen the education of critical and engaged digital citizens, aligning higher education with both global and local challenges of sustainable development, capable of overcoming the individualistic logics inherent in the neoliberal economic model.
In this context, the acceleration of digital transformation in universities following the pandemic has intensified the demand for advanced digital skills among university students, who must adapt to an educational environment characterized by technological omnipresence and the need for competencies in areas such as cybersecurity, digital content creation, and online information verification [20,21,22]. However, the comprehensive and contextualized measurement of these digital skills remains limited in Chile, which could limit the implementation of educational policies based on evidence and adapted to the real needs of students [23]. In this regard, having an intrinsic characterization of university students is essential for formulating and applying educational policies with more efficiency [18].
From this scenario, the development of digital skills in higher education would imply not only being restricted to the development of technical abilities, which could jeopardize the formation of a socially critical professional, but also the consideration of an ethical and critical understanding of the digital environment, with the capacity to manage information and adapt to emerging technologies [24,25]. In this sense, research indicates that university digital literacy should be oriented toward transcending instrumental mastery in order to incorporate competencies that enable students to navigate, evaluate, and use new technological tools critically and responsibly, integrating an interdisciplinary and socially committed approach [26,27].
However, incorporating the above considerations into educational policies, and subsequently achieving effective implementation, presents structural challenges. In this regard, the persistent digital divide in terms of access, use, and prior skills demands that the Chilean education system improve its specific strategies at the school level to ensure greater equity during university education [28]. Likewise, the ongoing evaluation of the effectiveness of these policies must take into account contextual variables specific to the student population, such as the socioeconomic status and the learning modalities, in order to design relevant and sustainable interventions [23,29].
In this scenario, the validation of instruments such as the Internet Skills Scale (ISS) within the Chilean context becomes fundamental for accurately diagnosing the digital skills of Chilean university students. Offering an appropriate instrument allows for increasing the quality of feedback and the impact of educational policy design and contributes to the formation of digital citizens capable of leading processes of innovation, inclusion, and sustainability in Chilean society and the global context [30,31].

Digital Divide and Sociodigital Inequalities

As the Internet and its use expanded throughout the world, the debate on inequalities began to incorporate the digital dimension. Thus, definitions of digital inequalities initially described the differences between people who had and did not had access to computers and the Internet. In this context, during the 1990s, the term ‘digital divide’ became popularized [32]. This access-based perspective evolved as the Internet consolidated itself as a necessary resource for participating in society [33,34].
Afterwards, and during the 1990s, the line of research by [35,36,37,38] among others, contributed to understanding the digital divide from a logic that transcends the dichotomy between those who have and those who do not have internet access [35,36,39,40,41]. This perspective incorporates dimensions such as the quality of internet use [42], the digital skills of citizens [40,43], and the diverse forms in which ICTs are used [44]. More operationally, Ref. [37] distinguish three levels of the digital divide: (1) the access gap, (2) the gap in individual capacities and skills for using ICTs, and (3) the gap in outcomes and benefits obtained through digital means.
In this way, the conceptualization of digital inequalities has followed a path similar to that of traditional social inequalities, moving from more operational definitions toward increasingly complex ones that involve a greater quantity of elements from the social and digital life of contemporary societies [45]. Hence, the debate on digital equality has shifted from the concept of digital divide (under the access/no access dichotomy) towards that of digital exclusion, expressed through progressively complex skills, motivation, and levels of participation.
In this regard, Ref. [38] coins the term ‘sociodigital inequalities’, as the systematic differences between individuals from different backgrounds in terms of their opportunities and abilities to translate digital engagement into benefits. This notion of inequality suggests that the specific spheres of digital and social exclusion influence one another [34], relating mainly through similar fields of resources (economic, cultural, social, personal). This perspective also proposes that the influence of offline exclusion domains on digital exclusion domains is mediated by access, skills, and attitudinal and motivational factors. The above is fundamental, as it highlights the distinction between internalized resources (internet skills) and externalized ones (access) [37,46], while also emphasizing their close connection. Therefore, from this position, when studying internet skills as a subset of digital skills, we understand them as aspects derived from capital. That is, internet skills related to creation and programming are not independent digital capital, but rather capital that stems from other capitals, such as economic capital, which ensures access to quality internet, or cultural capital, which socializes the importance of these types of skills from an early age [38].
Building on the capital-based perspective of sociodigital inequalities, ref. [32] reconceptualized digital inequality as a multi-stage appropriation process, spanning motivation, access, skills, and actual use, and thus shifted the emphasis from a mere “access gap” to a nuanced hierarchy of competencies. Ref. [43] then refined this framework into six discrete skill domains: operational (interface handling), formal (structure navigation), information (search and evaluation), strategic (goal-oriented application), communication (online interaction), and content creation (digital production), each reflecting a distinct field of resources.
At the measurement level, the Internet Skills Scale [47] stands out as an instrument that integrates a critical framework of sociodigital inequalities with the taxonomy of digital competencies based digital divide proposal [32,33]. Originally developed for UK and Dutch populations, the ISS was subsequently adapted in Italy [48] where a five-factor structure was confirmed, and in Cyprus [49], retaining its core dimensions among Generation Z undergraduates. In Latin America, a short version of the scale was translated into Spanish for a sample of university students in Argentina [50]. In that version, the scale presented a factorial structure of three dimensions. More recently, in 2021, a version of the scale was adapted and validated in Slovenia [51]; this version consists of 17 items distributed across 4 factors. Thus, following this trajectory, the aim of this article is to evaluate the psychometric properties of the Internet Skills Scale in its version adapted for Chilean university students.
Thus, this study poses the following research question: Is the ISS adaptation valid, reliable, and invariant across genders for university students in Chile?
In order to answer this question, three research hypotheses are proposed:
H1. 
The ISS scores, translated and adapted to the Chilean population, will present a factorial structure consistent with the theoretical/empirical framework of the instrument, showing evidence of reliability and validity.
H2. 
The ISS scores, translated and adapted to the Chilean population, will show evidence of reliability.
H3. 
The ISS scores, translated and adapted to the Chilean population, will present factorial structures that are invariant according to sex.

2. Methods

2.1. Participants

The instruments were applied to university students in Chile between the ages of 18 and 26 (M = 22; SD = 3.89), of whom 59.8% were women. The considered inclusion criterion was to be an actively enrolled student at a Chilean Public University. Data collection was divided across four geographical regions of the country—North, Central, South, and Southern Austral; in each region, State Public Universities were invited to participate in the project. Of the universities invited, 9 agreed to participate in the study, these being the ones with the highest concentration of students in each territory. This included five universities in the central region of the country, three in the city of Santiago, and two in the city of Valparaiso, as well as one university in the northern region, in the city of Iquique. In the southern and southwestern regions, one university was considered for each region, based in Temuco and Coyhaique, respectively. This was done in order to respect quotas in proportion to higher education enrollment at the national level. The sample consisted of 1116 participants, with a data matrix comprising 906 cases, as 19% did not agree to participate in the study or did not answer the questionnaire in its entirety. The sample satisfies the required conditions for conducting factor analysis, including at least 10 cases per item [52], five observations per estimated factor [53,54], and a minimum sample size of 100 participants [55,56]. Table 1 presents the characteristics of the sample.

2.2. Procedure

The study was conducted in three phases. (1) A team of four translators, with training in Internet Skills and Psychometrics, translated the original items from the Internet Skills Scale [57]. The translations were carried out independently and in accordance with the guidelines recommended for adapting psychometric tests from one culture to another [58]; subsequently, four native English-speaking professors back-translated the items into the original language. In the case of the five items associated with the navigation skills factor, it was decided to incorporate a positively worded version for them (see Table 2). While in the original version the statements for this factor began with the phrase “I find it difficult…”, in the version for Chile they begin with the phrase “It is easy for me…”. Another decision made during the translation process was not to include the mobile skills factor, given that, currently, the skills included in the other four factors can be performed on laptops, desktop computers, and cell phones. This decision is consistent with the Slovenian version [51]. To ensure that existing inequalities in the Chilean context were represented, the translation and adaptation of the scale took into account the marked disparities in digital infrastructure in different territories. It also sought to capture the wide variety of contexts from which students at public universities in Chile come. In this regard, the review of the original items took into account low-bandwidth scenarios and offline planning strategies common in communities with medium-low development indices, areas that often lack high-speed connectivity and up-to-date devices (cell phones, computers, tablets).
Additionally, during this stage of translation and linguistic adaptation, the version of the scale made by [50] for the Argentine population was incorporated as an element. Thus, through a mixed team of researchers and translators, the final wording of the scale was developed, translated, and adapted into Spanish, especially made for Chilean university students. At this point, the particularities of the cultural Chilean context for adapting the ISS were organized along two main lines. First, the scale’s characteristics and items were reviewed and adjusted to capture the broad heterogeneity of digital competencies among students at Chilean public universities, a necessity stemming from the structural inequalities that shape access to higher education in Chile. Second, the terminology was carefully selected to avoid excessive technicality while still ensuring faithful coverage of the four key Internet-skill dimensions. Accordingly, the translation team selected the 20 items deemed most culturally appropriate, 5 per factor (see Table 2), to preserve the original factor structure, foregoing any additional items that might have extended the scale. For a qualitative insight, the initial translation was piloted with 15 Chilean students who met the study’s inclusion criteria; upon completing the questionnaire, they participated in a focus group to share their feedback and observations on the instrument. In that session, we evaluated each item according to four criteria—(1) coherence, (2) clarity, (3) sufficiency, and (4) relevance—and discussed these dimensions in detail with all participants. We also presented the wording of the negatively phrased items for the Navigation Skills factor to gauge comprehension. The discussion revealed strong endorsement of the positively framed language chosen for each statement and confirmed the decision to retain all Navigation Skills items in their positive form. In this instance, the positive wording of the instruments was valued, and the familiarity of the skills covered in the ISS version was recognized by the participants.
(2) To recruit the sample, nine State Universities in Chile were contacted. Once authorization was obtained from their administrators, the application of the instruments was coordinated. Students accessed the questionnaire through the QuestionPro platform. The dissemination of this instance was carried out through institutional emails and social media. The participants who accessed the platform signed an informed consent form through which the ethical principles of the study are safeguarded; this protocol was approved by the Scientific Ethics Committee of the Universidad de La Frontera (protocol code 140_23, approved on 22 September 2023). The instruments were answered by the participants in approximately 14 min.
(3) The psychometric properties, reliability, validity, and gender invariance of the Internet Skills Scale were evaluated through exploratory and confirmatory factor analyses.

2.3. Instruments

2.3.1. Sociodemographic Questionnaire

This instrument included five closed-ended questions relating to sex, age, area of residence (urban/rural), internet access, belonging to Indigenous peoples, and self-perceived socioeconomic status.

2.3.2. Internet Skills Scale

This instrument measures a broad range of the skills individuals have in their internet use. The range spans from basic operational and navigation skills to skills related to online creativity. The scale was originally developed by [47] and is composed of 35 items distributed across 5 factors: (1) Operational Skills, (2) Navigation Skills, (3) Social Skills, (4) Creative Skills, and (5) Mobile Skills (cell phone use). For the present study, and consistent with the Slovenian version [51], the research team followed the recommendation of the authors [47] to prioritize short versions, with five items for each factor, except for the incorporation of statements related to mobile skills into the Operational and Navigation Skills factors. Hence, this short version solely considers the first four factors of the original scale.

2.3.3. Digital Citizenship Scale

The Digital Citizenship Scale measures the skills, perceptions, and levels of participation of young adults in internet-based communities. For this study, the Mexican version, Ref. [59] adapted to the context of university students, was used. The instrument is composed of 26 items distributed across 5 factors: Internet Political Activism (9 items), Technical Skills (5 items), Local/Global Awareness (3 items), Critical Perspective (4 items), and Network Agency (5 items). Response options are provided on a Likert-type scale with a 5-point response range, ranging from 1 (strongly disagree) to 5 (strongly agree). To interpret the results, scores from the 5 factors or subscales as well as the total scale are taken into account.

2.3.4. University Political Action Tendencies Scale

The University Political Action Tendencies Scale was designed in Chile by [60] and has its theoretical bases in [61], who identifies different types of collective action as well as various definitions of conventional and unconventional politics. The scale is composed of 12 items distributed across 3 factors: (1) Normative Political Action, (2) Non-Normative Political Action, and (3) Organized Political Action. The Cronbach’s alpha of the instrument in its original evaluation was of 0.82. A Likert-type scale of 5 points was used, ranging from 1 (strongly disagree) to 5 (strongly agree).

2.4. Data Analysis

A database was created using the SPSS v22 software. The data was divided into two equal parts to obtain an estimation subsample and a validation subsample [62]. An exploratory factor analysis was conducted with the estimation subsample (n = 453); subsequently, with the validation subsample (n = 453), a confirmatory factor analysis and an internal consistency analysis were developed using ordinal alpha [63]. These analyses aimed to evaluate whether the data fit the factorial structure obtained in the exploration or if it showed better goodness of fit indices for other factorial structures that the scale has taken in different versions. Although the original version of the instrument was evaluated using Cronbach’s alpha, there is now evidence that the ordinal alpha is more precise when analyzing ordinal data [63].
Due to the Internet Skills Scale being composed of items with ordinal responses, a polychoric correlation matrix [64] was used in the Exploratory Factor Analysis (EFA), applying the Unweighted Least Squares (ULS) estimation method with PROMIN rotation, including Horn’s parallel analysis [65] to determine the number of factors to retain. These analyses were made with the FACTOR program, version 10.3.1 [66]. With the validation subsample, a Confirmatory Factor Analysis (CFA) was carried out through structural equation modeling on three measurement models to verify whether the factorial structure obtained from the EFA provided the most appropriate fit indices compared to other models that have resulted in versions of the scale in different countries under similar conditions. The models analyzed included the following: (a) a three correlated factors model developed in Argentina [50], (b) a four-factor model developed in Slovenia by [51], (c) the original five-factor model [57], and (d) a four-factor model with a second-order factor [51].
The CFA was performed with the R programming language [67], through the Lavaan package [68], and applied to the polychoric correlation matrix. Parameters were estimated using the Unweighted Least Squares (ULS) method. The ULS estimator minimizes the sum of squared differences between the observed and reproduced correlation matrices to work with small samples and few factors to retain, while still obtaining appropriate parameter estimates and their level of error [62,69]. The CFA models were evaluated using the following global fit indices: [70] Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Standardized Root Mean Square Residual (SRMR), and Root Mean Square Error of Approximation (RMSEA). For the CFI and TLI indices, values greater than or equal to 0.95 are considered reasonable model adjustments; for SRMR, values below 0.08; and for RMSEA, values below 0.06 [71].
For the reliability evaluation of the scale, the ordinal alpha coefficient [63] was used for each factor and total. Additionally, evidence of discriminant and convergent validity was provided through correlation analyses between the Internet Skills Scale and the political action tendency scales, and the Digital Citizenship Scale, respectively.
Subsequently, a factorial invariance analysis was conducted with the total sample to evaluate whether the model with the best global fit indices adequately fits the data for both men and women. The following levels of invariance were considered: (1) configural invariance, same number of factors and same item distribution across both groups; (2) metric invariance, equality of factor loadings; (3) scalar invariance, equality of item means; (4) strict invariance, equality of residuals. The comparison between the two groups (men/women) at each level of invariance was conducted by considering the Satorra–Bentler chi-square difference [72], the criterion of change <0.01 in CFI [73], the criterion of change in RMSEA < 0.015, and, in SRMR, < 0.030 for metric invariance and 0.015 for scalar and strict invariance [74].
Sex-based differences in each Internet Skills factor were assessed using Mann–Whitney U tests, whereas study-area differences in the overall ISS total score were examined via one-way ANOVA.

3. Results

3.1. Exploratory Factor Analysis

Horn’s parallel analysis suggested retaining four factors with real eigenvalues greater than the random eigenvalues. The fit indices indicated viability for conducting a factor analysis: KMO test (0.937); Bartlett’s test of sphericity (χ2 [df = 190] = 10,321.3; p < 0.001); and goodness-of-fit index (GFI = 0.99). Regarding the factor structure, the presence of four correlated factors was observed, which together explained 76% of the variance. The first factor, labeled Operational Skills, explained 50% of the variance; the second factor, Navigation Skills, explained 11%; the third factor, Social Skills, explained 9%; and finally, the fourth factor, Creative Skills, explained 6% of the variance. This structure aligns with the proposal of [51], validated in the Slovenian population. The factor loadings of the items ranged from 0.421 to 0.976 (Table 2), suitable values that suggest the retention of all items [75].

3.2. Confirmatory Factor Analysis

With the confirmation sample (n = 453), a CFA was made through structural equation modeling with a measurement model to evaluate three factorial models (Table 3): (a) a three correlated factors model developed in Argentina [50]; (b) a four first-order factors model (resulting from the EFA in the present study); and (c) a four-factor model with a second-order factor [51]. As shown in Table 4, the ISS adaptation adopting the Slovenian four-factor second-order structure achieved the strongest fit indices among all tested models, marginally outperforming the EFA-derived version. Given the chi-square statistic’s known sensitivity to sample size (n = 453) and model complexity, we prioritized the relative fit indices (CFI, RMSEA, SRMR) over χ2. Additionally, none of the modification indices reached a threshold sufficient to justify freeing an error term, and in order to preserve the theoretical integrity, we retained the second-order model specification. The factor loadings of the 20 retained items ranged between highly significant saturation levels p ≤ 0.001.
Figure 1 presents the details of the four-correlated factors model with a second-order factor. It includes the values of the correlations between the factors and the second-order factor, the standardized saturations of the items. The item factor loadings ranged from 0.53 to 0.96.

3.3. Evidence of Reliability, Discriminant and Convergent Validity

Regarding the internal consistency of the instrument, adequate ordinal alpha coefficients were found for each factor: 0.95 for the Operational Skills factor, 0.90 for the Navigation Skills factor, 0.90 for the Social Skills factor, and 0.83 for the Creative Skills factor.
Spearman correlation results show that the ISS is not associated in a statistically significant way with the Political Action Tendencies Scale, thus providing evidence of discriminant validity. The result was ρ = 0.050 (p = 0.130). On the other hand, Spearman correlation results show that the ISS is statistically significant in relation to the Digital Citizenship Scale (ρ = 0.266; p = < 0.001). The above shows evidence of convergent validity, based on what is expected from the composition of the factors of both scales [57,59]. To complement our convergent validity assessment, we computed the average variance extracted (AVE) for each latent factor, using the criterion that AVE ≥ 0.50 indicates a factor explains more than half of its items’ variance [76]. The AVE values were Operational = 0.812; Navigation = 0.643; Social = 0.620; and Creative = 0.554, confirming that all four factors demonstrate strong convergent validity.

3.4. Measurement Invariance of the Instrument

The results of the measurement invariance assessment by gender for the ISS showed an adequate fit at the configural level in both groups (men n = 374 and women n = 532): χ2 S−B = 2130 (df = 578); CFI = 0.969; SRMR = 0.083; and RMSEA = 0.077 (CI 90% [0.073–0.081]). The fit indices indicate that the factorial configuration remains consistent across both men and women, and thus this model was used as the baseline for the subsequent invariance tests.
Subsequently, metric, scalar, and strict equivalence constraints were sequentially applied, each model being evaluated against the configural model through the χ2 S−B test and the analysis of the variations in CFI, RMSEA, and SRMR. As shown in Table 4, the differences in CFI, RMSEA, and SRMR for each level of invariance (metric, scalar, and strict) relative to the configural invariance level are below the change criteria proposed by [73,74]. These findings support the comparability of scores between men and women and allow for valid mean comparisons [77].

3.5. Comparisons of the Internet Skills Scale Among University Students in Chile According to Sex

Once the non-normality of the data was verified, the Mann–Whitney U test was used to compare the scores of the ISS between men and women. Statistically significant sex differences were found in both the total ISS score and its four dimensions—operational skills, navigation skills, information evaluation, and online communication—with higher scores observed in women. Table 5 presents the contrast statistics by sex, as well as the means and standard deviations. As shown in the comparisons, there are no statistically significant differences between men and women in the scale’s factors, except for the Creative Skills factor. This factor refers to the most complex skills, measured by the scale, associated with the creation of digital content. In this case, male participants obtained higher scores with statistically significant difference compared to the scores obtained by women.
The results of the one-way ANOVA revealed no significant differences in total ISS scores by study area: F(7, 897) = 1.44; p = 0.187; η2 = 0.011. This result was confirmed by Welch’s test: F(7, 197.77) = 1.49; p = 0.173. Mean ISS scores were consistently high across disciplines, ranging from 3.99 (SD = 0.61) in arts and architecture to 4.22 (SD = 0.44) in agriculture, with coefficients of variation between 10.5% and 15.2%. These findings indicate a broadly uniform level of Internet skills among students, regardless of their field of study (Table 6).

4. Discussion

Research on internet skills gains relevance within the framework of Sustainable Development Goal 4 (SDG 4) of the 2030 Agenda, which promotes inclusive, equitable, and quality education and lifelong learning opportunities for all. In this regard, the acquisition of digital competencies by Chilean university students is key to achieving target 4.4, which aims to equip young people with the technical and professional skills necessary for their personal development and integration into the labor market.
In relation to the above, the results offered for the Chilean adaptation of the ISS acquire fundamental importance for advancing an accurate evaluation of digital competencies in university students for the development of a critical and committed digital citizenship [2,3]. In this regard, higher education faces the challenge of going beyond mere technical mastery of digital tools, marked by the interference of the neoliberal model in professional education, and promoting an integral education that fosters critical thinking, digital ethics, and sociopolitical engagement aligned with the principles of social justice, inclusion, and environmental sustainability [11,19].
Thus, the present study aimed to evaluate the psychometric properties of the Internet Skills Scale, adapted for university students in Chile. The trajectory of this scale traces back to the original proposal by [47], which was designed for a wide age range of populations, specifically in the United Kingdom and the Netherlands. This measurement was later adapted and validated in Italy [48] and in Cyprus [49]. In Latin America, a shortened version applied to Argentine university students was translated into Spanish, which revealed a three-dimensional structure. Finally, in 2021, a new 17-item adaptation organized into four factors was published in Slovenia. This latest version of the scale served as the reference for the adaptation and translation of the Chilean version.
For the purpose of evaluating the psychometric properties, an exploratory factor analysis and a confirmatory factor analysis were conducted. The results of the exploratory analysis yielded a factorial structure composed of 20 items divided into four factors. Subsequently, the confirmatory factor analysis comparatively evaluated three different models of the scale. The model that presented the best goodness-of-fit indicators was the one proposed by [51], which considers a second-order model with four correlated factors. This result highlights the close relationship between the four dimensions of the scale and particularly enables its use considering a general second-order factor (Internet Skills) that encompasses the sum of the scores obtained in each dimension. Regarding the reliability and validity of the instrument, the results show appropriate indicators to consider its use in the university population of Chile, since reliability values greater than 0.80 are obtained for the total scale, and for each of the dimensions. As for convergent and discriminant validity, the results are consistent with specialized literature.
In this regard, the psychometric robustness of the ISS and the confirmation of its four-dimensional factorial structure make it possible not only to identify levels of technical mastery, but also to broaden the scope of observation on existing gaps in more complex skills, such as digital content creation and critical internet navigation. These aspects are especially relevant in the face of the acceleration of digital transformation within the university environment [20,21,22]. Thus, the findings align with the literature that warns about the persistence of sociodigital inequalities, which are not resolved merely by internet access, but require the development of advanced competencies to enable an active and transformative participation in the digital society, in line with the principles of the Sustainable Development Goals [9,38].
Also, the findings from the measurement invariance evaluation reinforce the robustness of the ISS by demonstrating that its four-factor correlated factorial structure remains stable across men and women. In particular, the absence of significant differences at the configural, metric, scalar, and strict levels confirms that the factor loadings, item means, and error terms operate equivalently by gender, thereby validating score comparisons between both groups. This level of strict invariance not only supports the internal validity of the instrument but also enables the detection of true gender differences in digital competencies without measurement biases.
The validation of the ISS in the Chilean context, insofar as it provides a tool to observe the scope of the SDG 4, presents itself as a scale that allows to discriminate between different levels of skills and their invariance according to gender. Likewise, it reinforces its usefulness in contributing to the literature within the framework of designing inclusive educational policies that consider the contextual particularities of Chilean university students, such as sociodigital inequality [23,29], as well as for evaluating the impact of current educational policies.
In line with the above, as in the theory of corresponding fields stated by [38], overcoming sociodigital inequalities requires interventions that address both the structural level (access, institutional policies) and the level of individual agency (motivations, critical and creative uses of the internet). In this sense, the ISS can be a fundamental instrument for designing education strategies at the meso level—that is, within university learning environments where social structures and individual trajectories intersect—thus enabling progress towards an idea of higher education or of a university model that not only responds to the mechanistic demands of the labor market but also educates digital citizens capable of leading processes of innovation, inclusion, and sustainability [30,31].
Regarding the limitations of this study, it is worth noting the predominance of social science students in the sample (39.7%), a factor that should be taken into account when generalizing these results to cohorts from other study areas with different Internet use profiles as students from technology and natural science fields. Moreover, we recommend conducting an adaptation of the Internet Skills Scale for students at Chilean private universities, particularly because the ISS addresses competencies that can be influenced by socioeconomic contexts and educational backgrounds prior to higher education.
Furthermore, given the quantitative nature of the study, it would be advisable to complement these results with qualitative approaches —for example, in-depth interviews or focus groups—that allow for exploration of the motivations, perceptions, and meanings that students attribute to their online skills, thereby enriching the interpretation of the observed differences and similarities in the participants’ internet abilities. Lastly, while the Internet Skills Scale effectively assesses users’ ability to navigate, create, and interact online, it does not include items targeting competencies in artificial intelligence, a domain that is rapidly reshaping Internet. Recent validated AI literacy instruments such as [78,79,80,81] highlight the importance of evaluating technical understanding, ethical considerations, and practical applications of AI. To remain comprehensive and relevant in today’s digital landscape, future versions of the ISS should integrate an AI-focused factor that captures these critical dimensions. Accordingly, we propose to operationalize this new factor by defining three subdomains, prompt engineering, ethical awareness, and applied integration, and drafting nine items (three per subdomain) grounded in existing AI-literacy instruments. These items will first be vetted by a panel of experts, then refined through student focus groups, pilot-tested on a diverse undergraduate sample, and finally subjected to EFA and CFA (including invariance testing) to confirm the theoretical coherence and psychometric robustness of the AI-skills dimension.

5. Conclusions

This study presents the evaluation of the psychometric properties of the Internet Skills Scale (ISS), adapted for the university population in Chile, confirming its four-dimensional factorial structure under a second-order model with excellent fit indices (CFI = 0.987; RMSEA = 0.055), confirming H1, and ordinal reliability coefficients above 0.83 for each factor, confirming H2. This factorial structure strengthens the evidence for supporting a common factor that encompasses internet skills, along with four interrelated factors, as in the case of the proposal of [51]. The study also provides evidence of convergent validity (ρ = 0.266 with the Digital Citizenship Scale) and discriminant validity (ρ = 0.050 with the Political Action Tendencies Scale), along with the strict invariance by gender, which supports measurement equivalence and enables robust comparisons of internet skills between men and women, which confirms H3. The findings offer a rigorous instrument aligned with SDG 4 (Quality Education), as it allows for the diagnosis of strengths and weaknesses in university students’ internet skills and guides the design of educational programs adapted to their real needs. Furthermore, by highlighting sociodigital inequalities linked to economic and cultural capital, the ISS contributes to achieving SDG 10 (Reduced Inequalities), providing a foundation for more equitable policies and intervention strategies, especially in university contexts. Thus, the ISS stands as a reliable tool for studies examining internet skills in university contexts, contributing to both theoretical and practical advancement for assessing the achievements of the Sustainable Development Goals.

Author Contributions

Conceptualization, M.G.-C., G.T.-T., and D.A.-S.; methodology, M.G.-C. and J.T.-A.; software, M.G.-C.; validation, M.G.-C., J.T.-A., G.T.-T., and D.A.-S.; formal analysis, M.G.-C.; investigation, M.G.-C., J.T.-A., and I.N.-S.; resources, M.G.-C. and J.T.-A.; data curation, M.G.-C.; writing—original draft preparation, M.G.-C., G.T.-T., and D.A.-S.; writing—review and editing G.T.-T. and D.A.-S.; visualization, V.S.-G. and I.N.-S.; supervision, J.T.-A.; project administration V.S.-G. and I.N.-S.; funding acquisition, M.G.-C. and J.T.-A. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Universidad de La Frontera, Project DI23-0032 and DI25-0047 and the APC was funded by Universidad de La Frontera.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Universidad de La Frontera (protocol code 140_23 and approved on 22 September 2023).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author due to ethical reasons.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. United Nations General Assembly. Resolution Adopted by the General Assembly on 11 September 2015 (A/RES/70/1); United Nations General Assembly: New York, NY, USA, 2015. [Google Scholar]
  2. Lozano-Díaz, A.; Fernández-Prados, J.S. Educating digital citizens: An opportunity to critical and activist perspective of sustainable development goals. Sustainability 2020, 12, 7260. [Google Scholar] [CrossRef]
  3. Cañabate, J.P.M.; López, C. Claves para el desarrollo de la docencia de la comunicación corporativa en el ámbito de la educación líquida [Keys to developing corporate communication teaching in the field of liquid education]. Rev. Pensam. Sobre Comun. Tecnol. Soc. 2015, 101, 62. [Google Scholar]
  4. Jácome, W.L.C.; Arizaga, L.d.C.B. Educar para la ciudadanía digital en el contexto universitario [Educating for digital citizenship in the university context]. Código Científico Rev. Investig. 2024, 5, 1465–1486. [Google Scholar] [CrossRef]
  5. Prados, J.S.F.; Díaz, A.L. El reto de la ciudadanía digital activa en la educación superior europea: Análisis del ciberactivismo entre los estudiantes universitarios [The challenge of active digital citizenship in European higher education: An analysis of cyberactivism among university students]. Edmetic 2021, 10, 118–134. [Google Scholar] [CrossRef]
  6. Rodríguez García, A.M.; Martínez Heredia, N.; Raso Sánchez, F.M. La Formación Del Profesorado en Competencia Digital: Clave Para la Educación Del Siglo XXI [Teacher Education in Digital Competence: Key to XXI Century Education]. 2017. Available online: http://hdl.handle.net/10481/61748 (accessed on 22 June 2025).
  7. López, J.C.C. Competencias Digitales en la Educación Superior [Digital Skills in Higher Education]. 2023. Available online: https://repositorio.cidecuador.org/jspui/bitstream/123456789/2705/1/Articulo_37_Horizontes_N29V7.pdf (accessed on 22 June 2025).
  8. Oliva, M.A.; Coronas, T.T.; Luna, J.C.Y. El desarrollo de competencias digitales en la educación superior [The development of digital skills in higher education]. Hist. Comun. Soc. 2014, 19, 355–366. [Google Scholar] [CrossRef]
  9. Gaibazzi, M.F.; Berizzo, M.L.; Trottini, A.M. Demanda de competencias digitales al Contador Público. Una mirada desde la Educación Superior [Demand for Digital Competencies in Public Accountants: A Perspective from Higher Education]. SaberEs 2021, 13, 73–90. Available online: https://www.scielo.org.ar/scielo.php?pid=S1852-42222021000100073&script=sci_arttext (accessed on 22 June 2025).
  10. Reyes, C.E.G.; Martínez, R.A. Competencias digitales para la práctica docente en pregrado en dos universidades latinoamericanas [Digital skills for undergraduate teaching practice at two Latin American universities]. Edmetic 2021, 10, 1–19. [Google Scholar]
  11. Basulto, Ó.; Amigo, A. Facebook como herramienta discursiva del movimiento feminista chileno del 2018 [Facebook as a discursive tool of the Chilean feminist movement of 2018]. Perspect. Comun. 2024, 17, 1–29. [Google Scholar] [CrossRef]
  12. Lapierre Acevedo, M. Activismo digital de mujeres con discapacidad en Instagram: Análisis de tres casos en Chile [Digital activism of women with disabilities on Instagram: Analysis of three cases in Chile]. Cuad. Info 2023, 54, 225–246. [Google Scholar] [CrossRef]
  13. Caicedo, S.S.G.; Zambrano, Á.A.S.; Aroca, I.A.A. Brecha Digital en Universidades del Ecuador [Digital Divide in Ecuadorian Universities]. Rev. Científica Sinapsis 2021, 1, 1–7. [Google Scholar] [CrossRef]
  14. Ciriaco, L.A.G. Desafíos y estrategias para superar la brecha digital en entornos universitarios: Una revisión sistemática [Challenges and strategies for overcoming the digital divide in university environments: A systematic review]. Rev. Multidiscip. Voces América Caribe 2024, 1, 217–243. [Google Scholar] [CrossRef]
  15. Mellado Gatica, A. La organización política ciudadana asistida por TIC, una aproximación sobre la influencia del “efecto red”, la brecha digital y la brecha participativa en el contexto chileno [ICT-assisted citizen political organization, an approach to the influence of the “network effect,” the digital divide, and the participatory gap in the Chilean context]. Cult.-Hombre-Soc. 2018, 28, 67–91. [Google Scholar] [CrossRef]
  16. Cerda-González, C.; León-Herrera, M.; Saiz-Vidallet, J.L.; Villegas-Medrano, L. Propósitos de uso de tecnologías digitales en estudiantes de pedagogía chilenos: Construcción de una escala basada en competencias digitales [Purposes of using digital technologies in Chilean pedagogy students: Construction of a scale based on digital competencies]. Pixel-Bit. Rev. Medios Educ. 2022, 64, 7–25. [Google Scholar] [CrossRef]
  17. Silva-Quiroz, J.-E.; Abricot-Marchant, N.; Aranda-Faúndez, G.; Rioseco-País, M. Diseño y Validación de un instrumento para evaluar competencia digital en estudiantes de primer año de las carreras de educación de tres universidades públicas de Chile [Design and Validation of an instrument to assess digital competence in first-year education students at three public universities in Chile]. Edutec Rev. Electrónica Tecnol. Educ. 2022, 79, 319–335. [Google Scholar] [CrossRef]
  18. Oliva-Albornoz, C.; Severino-González, P.; Sarmiento-Peralta, G.; Ramírez-Molina, R. Habilidades digitales y post pandemia: El caso de los estudiantes universitarios en Chile [Digital skills and post-pandemic: The case of university students in Chile]. Form. Univ. 2024, 17, 79–88. [Google Scholar] [CrossRef]
  19. Berríos Villarroel, A.; González Gamboa, J. Educación para el desarrollo sustentable en Chile: Deconstrucción pedagógica para una ciudadanía activa [Education for Sustainable Development in Chile: Pedagogical deconstruction for active citizenship]. Actual. Investig. Educ. 2020, 20, 570–600. [Google Scholar] [CrossRef]
  20. Herrero-Diz, P.; Pérez-Escolar, M.; Varona Aramburu, D. Competencias de verificación de contenidos: Una propuesta para los estudios de Comunicación [Content verification skills: A proposal for communication studies]. Rev. Comun. 2022, 21, 231–249. [Google Scholar]
  21. Santamaría-Ruiz, M.; Ramírez-Molina, R.; Antaquera-Amaris, R.; Lay-Raby, N. Types of competencies of human talent supported by ICT: Definitions, elements, and contributions. Procedia Comput. Sci. 2022, 210, 368–372. [Google Scholar] [CrossRef]
  22. Van De Werfhorst, H.G.; Kessenich, E.; Geven, S. The digital divide in online education: Inequality in digital readiness of students and schools. Comput. Educ. Open 2022, 3, 100100. [Google Scholar] [CrossRef]
  23. Silva-Quiroz, J.; Morales-Morgado, E.M. Assessing digital competence and its relationship with the socioeconomic level of Chilean university students. Int. J. Educ. Technol. High. Educ. 2022, 19, 46. [Google Scholar] [CrossRef]
  24. Livingstone, S.; Mascheroni, G.; Stoilova, M. The outcomes of gaining digital skills for young people’s lives and wellbeing: A systematic evidence review. New Media Soc. 2023, 25, 1176–1202. [Google Scholar] [CrossRef]
  25. Velasco, A.F.P. Educación para la ciudadanía digital, y no una ciudadanía cero. Retos y amenazas desde una mirada crítica-social [Education for digital citizenship, not zero citizenship. Challenges and threats from a critical social perspective]. Boletín Redipe 2023, 12, 36–64. [Google Scholar] [CrossRef]
  26. Bernacki, M.L.; Cogliano, M.C.; Kuhlmann, S.L.; Utz, J.; Strong, C.; Hilpert, J.C.; Greene, J.A. Relations between undergraduates’ self-regulated learning skill mastery during digital training and biology performance. Metacognition Learn. 2023, 18, 711–747. [Google Scholar] [CrossRef]
  27. Molina, R.I.R.; Castro, L.C.M.; Antúnez, J.V.V.; Raby, N.D.L.; Severino-González, P.; Palencia, D.A.B. Human management by competencies in competitive and complex scenarios: A reflective theoretical approach. Procedia Comput. Sci. 2022, 203, 678–682. [Google Scholar] [CrossRef]
  28. Bolaños, F.; Pilerot, O. Where are digital abilities within Chile’s State technical formation centres? A discourse analysis of public policy. J. Vocat. Educ. Train. 2024, 76, 1165–1184. [Google Scholar] [CrossRef]
  29. Almenara, J.C.; Pérez, S.M.; Castillo, J.J.G.; Rodríguez, A.P. University students’ perceptions of the use of technologies in educational activities and mental effort invested. RIED-Rev. Iberoam. Educ. Dist. 2022, 25, 305–322. [Google Scholar] [CrossRef]
  30. Breen, M.; Waters, J.; O’Shea, L. Taking a lead on digital literacy for students—A case study from the library at the University of Limerick. New Rev. Acad. Librariansh. 2023, 29, 11–32. [Google Scholar] [CrossRef]
  31. Webb, A.; Layton, J. Digital Skills for Performance: A framework for assessing current and future digital skills needs in the performing arts sector. Arts Mark. 2023, 13, 33–47. [Google Scholar] [CrossRef]
  32. Van Dijk, J.A.G.M. The Deepening Divide: Inequality in the Information Society; Sage Publications: London, UK, 2005. [Google Scholar]
  33. Hargittai, E.; Walejko, G. The participation divide: Content creation and sharing in the digital age. Inf. Community Soc. 2008, 11, 239–256. [Google Scholar]
  34. Helsper, E. A corresponding fields model for the links between social and digital exclusion. Commun. Theory 2012, 22, 403–426. [Google Scholar] [CrossRef]
  35. Norris, P. Digital Divide: Civic Engagement, Information Poverty, and the Internet Worldwide; Cambridge University Press: Cambridge, UK, 2001. [Google Scholar]
  36. DiMaggio, P.; Hargittai, E.; Neuman, W.R.; Robinson, J.P. Social implications of the Internet. Annu. Rev. Sociol. 2001, 27, 307–336. [Google Scholar]
  37. Ragnedda, M.; Muschert, G.W. (Eds.) Theorizing Digital Divides; Routledge: London, UK, 2017. [Google Scholar]
  38. Helsper, H. The Digital Disconnect: The Social Causes and Consequences of Digital Inequalities; Sage Publications: London, UK, 2022. [Google Scholar] [CrossRef]
  39. Hargittai, E. Second-level digital divide: Mapping differences in people’s online skills. arXiv 2001, arXiv:cs/0109068. [Google Scholar]
  40. Hargittai, E. The digital divide and what to do about it. In New Economy Handbook; Academic Press: San Diego, CA, USA, 2003; pp. 821–839. [Google Scholar]
  41. Hargittai, E. Connected in Isolation: Digital Privilege in Unsettled Times; MIT Press: Cambridge, MA, USA, 2022. [Google Scholar]
  42. Benkler, Y. The Wealth of Networks: How Social Production Transforms Markets and Freedom; Yale University Press: New Haven, CT, USA; London, UK, 2006; Volume 26, pp. 259–261. [Google Scholar]
  43. Van Deursen, A.; Van Dijk, J. Internet Skills and the digital divide. New Media Soc. 2010, 13, 893–911. [Google Scholar]
  44. Hargittai, E.; Hsieh, Y. Predictors and consequences of differentiated practices on social network sites. Inf. Commun. Soc. 2010, 13, 515–536. [Google Scholar]
  45. Reygadas, L. Las redes de la desigualdad: Un enfoque multidimensional. Política Cult. 2004, 22, 7–25. [Google Scholar]
  46. Kvasny, L. The role of the habitus in shaping discourses about the digital divide. J. Comput.-Mediat. Commun. 2005, 10, JCMC1025. [Google Scholar]
  47. Van Deursen, A.J.; Helsper, E.; Eynon, R.; Van Dijk, J.A. The compoundness and sequentiality of digital inequality. Int. J. Commun. 2017, 11, 452–473. [Google Scholar]
  48. Surian, A.; Sciandra, A. Digital divide: Addressing Internet skills. Educational implications in the validation of a scale. Res. Learn. Technol. 2019, 27, 1–12. [Google Scholar] [CrossRef]
  49. Miliou, O.; Angeli, C. Measuring the internet skills of Gen Z students in higher education: Validation of the internet skills scale in university settings. In Proceedings of the 7th International Conference on Higher Education Advances (HEAd’21), Valencia, Spain, 22 June 2021; Editorial Universitat Politècnica de València: Valencia, Spain, 2021; pp. 1359–1368. [Google Scholar] [CrossRef]
  50. González, F.M.; Burin, D. Habilidades de Internet: Relación con la experiencia en Internet y la aptitud verbal. Rev. Argent. Cienc. Comport. 2017, 9, 44–59. [Google Scholar] [CrossRef]
  51. Grošelj, D.; van Deursen, A.J.M.; Dolničar, V.; Burnik, T.; Petrovčič, A. Measuring internet skills in a general population: A large-scale validation of the short Internet Skills Scale in Slovenia. Inf. Soc. 2020, 37, 63–81. [Google Scholar] [CrossRef]
  52. Nunnally, J.; Bernstein, I. Teoría Psicométrica; Mc Graw-Hill: New York, NY, USA, 1995. [Google Scholar]
  53. Bentler, P.; Chou, C. Practical issues in structural modelling. Sociol. Methods Res. 1984, 16, 78–117. [Google Scholar] [CrossRef]
  54. Bollen, K. Structural Equations with Latent Variables; Wiley: Hoboken, NJ, USA, 1989. [Google Scholar] [CrossRef]
  55. Broomsma, A. Robustness of LISREL against small sample sizes in factor analysis models. In Systems Under Indirect Observation: Causality, Structure, Prediction (Part I); Jorekog, E.K.G., Wold, H., Eds.; North Holland: Amsterdam, The Netherlands, 1982; pp. 149–173. [Google Scholar]
  56. Myers, N.; Ahn, S.; Jin, Y. Sample size and power estimates for a confirmatory factor analytic model in exercise and sport: A Monte Carlo approach. Res. Q. Exerc. Sport 2011, 82, 412–423. [Google Scholar] [CrossRef]
  57. Van Deursen, A.J.A.M.; Helsper, E.J.; Eynon, R. Development and validation of the Internet Skills Scale (ISS). Inf. Commun. Soc. 2016, 19, 804–823. [Google Scholar]
  58. Muñiz, J.; Elosua, P.; Hambleton, R. Directrices para la traducción y adaptación de los tests: Segunda edición. Psicothema 2013, 25, 151–157. Available online: https://www.redalyc.org/articulo.oa?id=72726347014 (accessed on 22 June 2025).
  59. Galván, M.; Salazar, A.; Tereucan, J. Nativos/as digitales en México: Evaluación de las Propiedades Psicométricas de la Escala de Ciudadanía Digital en estudiantes universitarios/as. Edutec Rev. Electrónica Tecnol. Educ. 2021, 82, 119–135. [Google Scholar]
  60. Sandoval, J.; Hatibovic, F.; Bobowik, M. Validación de la Escala de Tendencias de Acción Política en una muestra de estudiantes universitarios chilenos. Psykhe 2018, 27, 1–17. [Google Scholar] [CrossRef]
  61. Tausch, N.; Becker, J.; Spears, R.; Christ, O.; Saab, R.; Singh, P.; Siddiqui, R.N. Explaining radical group behavior: Developing emotion and efficacy routes to normative and 37 nonnormative collective action. J. Personal. Soc. Psychol. 2011, 101, 129–148. [Google Scholar] [CrossRef]
  62. Lloret-Segura, S.; Ferreres-Traver, A.; Hernández-Baeza, A.; Tomás-Marco, I. El Análisis Factorial Exploratorio de los Ítems: Una guía práctica, revisada y actualizada. An. Psicol. 2014, 30, 1151–1169. [Google Scholar] [CrossRef]
  63. Elosua, P.; Zumbo, B. Coeficientes de Fiabilidad para escalas de respuesta categórica ordenada. Psicothema 2008, 20, 896–901. Available online: https://reunido.uniovi.es/index.php/PST/article/view/8747 (accessed on 22 June 2025).
  64. Freiberg, D.; Stover, J.; Fernández-Liporace, M.; García-Cueto, E.; Muñiz, J. Correlaciones policóricas y tetracóricas en estudi-os factoriales con variables ordinales. Psicothema 2013, 25, 7–14. [Google Scholar]
  65. Timmerman, M.; Lorenzo-Seva, U. Dimensionality Assessment of Ordered Polytomous Items with Parallel Analysis. Psychol. Methods 2011, 16, 209–220. [Google Scholar] [CrossRef]
  66. Lorenzo-Seva, U.; Ferrando, P. FACTOR: A computer program to fit the exploratory factor analysis model. Behav. Res. Methods 2022, 38, 88–91. [Google Scholar] [CrossRef]
  67. R Core Team. R: A Language and Environment for Statistical Computing, version 4.1; R Foundation for Statistical Computing: Vienna, Austria, 2021; Available online: https://cran.r-project.org (accessed on 22 June 2025).
  68. Rosseel, Y. lavaan: Latent Variable Analysis. [R Package]. 2018. Available online: https://cran.r-project.org/package=lavaan (accessed on 22 June 2025).
  69. Flora, D.; Curran, P. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychol. Methods 2004, 9, 466–491. [Google Scholar] [CrossRef]
  70. Satorra, A.; Bentler, P. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika 2001, 66, 507–514. [Google Scholar] [CrossRef]
  71. Brown, T.A.; Moore, M.T. Confirmatory factor analysis. In Handbook of Structural Equation Modeling; Guilford Publications: New York, NY, USA, 2012; Volume 361, p. 379. [Google Scholar]
  72. Muthén, B.; Asparouhov, T. BSEM measurement invariance analysis. Mplus Web Notes 2013, 17, 1–48. [Google Scholar]
  73. Cheung, G.; Rensvold, R. Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Struct. Equ. Model A Multidiscip. J. 2002, 9, 233–255. [Google Scholar] [CrossRef]
  74. Chen, F. Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct. Equ. Model A Multidiscip. J. 2007, 14, 464–504. [Google Scholar] [CrossRef]
  75. Kline, P. Handbook of Psychological Testing, 2nd ed.; Routledge: New York, NY, USA, 2000. [Google Scholar]
  76. dos Santos, P.M.; Cirillo, M.Â. Construcción del índice de varianza media extraída para la validación de constructos en modelos de ecuaciones estructurales con regresiones adaptativas. Commun. Stat.-Simul. Comput. 2021, 52, 1639–1650. [Google Scholar] [CrossRef]
  77. Putnick, D.L.; Bornstein, M.H. Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev. Rev. 2016, 41, 71–90. [Google Scholar]
  78. Hornberger, M.; Bewersdorff, A.; Nerdel, C. What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Comput. Educ. Artif. Intell. 2023, 5, 100165. [Google Scholar] [CrossRef]
  79. Ma, S.; Chen, Z. The development and validation of the artificial intelligence literacy scale for Chinese college students (AILS-CCS). IEEE Access 2024, 12, 146419–146429. [Google Scholar] [CrossRef]
  80. Laupichler, M.C.; Aster, A.; Haverkamp, N.; Raupach, T. Development of the “Scale for the assessment of non-experts’ AI literacy”–An exploratory factor analysis. Comput. Hum. Behav. Rep. 2023, 12, 100338. [Google Scholar] [CrossRef]
  81. Mansoor, H.M.; Bawazir, A.; Alsabri, M.; Alharbi, A.; Okela, A. Artificial intelligence literacy among university students—A comparative transnational survey. Front. Commun. 2024, 9, 1478476. [Google Scholar] [CrossRef]
Sustainability 17 08597 g001
Figure 1. Standardized estimated parameters of the ISS with second-order model with four correlated factors.
Figure 1. Standardized estimated parameters of the ISS with second-order model with four correlated factors.
Sustainability 17 08597 g001
Table 1. Sociodemographic characteristics of the sample (n = 906).
Table 1. Sociodemographic characteristics of the sample (n = 906).
Characteristic%
Sex
Men41.3
Women59.7
Zone
North22.5
Center55.6
South22.2
Austral South3.2
Internet access
Yes96.7
No3.3
Years using the Internet (M)10.7
Residential Area, (%)
Rural11.10%
Urban88.90%
Belonging to Indigenous Peoples (%)
No belonging79.90%
Belonging20.10%
Socioeconomic Level (%)
Low2.60%
Medium-low21.90%
Medium62.90%
Medium-high10.90%
High1.70%
Study Area
Technologies16%
Health1.8%
Humanities1.3%
Education13.4%
Law 11.9%
Social Sciences39.7%
Natural Sciences1.4%
Art and Architecture6.9%
Agriculture, Forestry, and Fishing2.1%
Administration and Business 5.3%
Table 2. Factor loadings of the exploratory factor analysis by factor (n = 453).
Table 2. Factor loadings of the exploratory factor analysis by factor (n = 453).
ItemsScale FactorsCom
OPNAVSOCRE
1. I know how to open downloaded files from the Internet.0.9630.1130.0600.0960.922
2. I know how to fill out online forms.0.9640.0310.0380.0860.926
3. I know how to upload files to the Internet.0.9760.0390.0710.0590.850
4. I know how to adjust the privacy settings.0.5320.0540.0470.2100.595
5. I know how to connect to a WIFI network.0.6340.1220.0080.1450.881
6. It is easy for me to decide which are the best keywords for internet searches.0.0540.6270.1460.1080.594
7. It is easy for me to find a website that I have previously visited.0.0040.7110.0920.0500.719
8. I always know how I got to the websites I browse.0.0040.7170.1590.1480.631
9. The different web page designs make Internet work easier for me.0.0610.6630.2160.0480.582
10. I find it easy to verify the information I get from the Internet.0.1780.7420.2040.2770.528
11. I know what information I should and should not share on the Internet.0.0160.0080.9350.1100.825
12. I know when I should and should not share information on the Internet.0.0020.0120.9040.0920.869
13. I am careful that my comments and behavior are appropriate to the situation I find myself in online.0.0840.0230.5320.1100.469
14. I know how to change who I share content with (for example, friends, friends of friends, or public).0.2100.2470.4550.0950.659
15. I know how to eliminate friends from my contact list.0.1410.2880.4210.2120.819
16. I know how to create something new out of already existing images, music, or video online.0.0280.0930.0220.5560.691
17. I know how to make basic changes to content that others have produced.0.0150.0410.1120.5650.701
18. I know how to design a web site.0.0120.1200.1810.9290.592
19. I know what types of licenses apply to online content.0.0690.0920.0730.7710.907
20. I know how to identify which apps/software are safe for download.0.2400.2030.0780.5770.544
OP: operational skills; NAV: navigation skills; SO: social skills; CRE: creative skills; Com: communalities.
Table 3. Fit indices for the factorial models with the confirmation sample (n = 453).
Table 3. Fit indices for the factorial models with the confirmation sample (n = 453).
Modelχ2 S−Bdfχ2/dfTLICFISRMRRMSEA
Three correlated factors [50]295.009417.190.9080.9230.0600.083
Four correlated factors (EFA results)946.0981595.940.9600.9610.0600.079
Second-order model with four correlated factors [51] 1172.2131607.320.9820.9870.0550.074
χ2 S−B = Satorra–Bentler chi-square; df = degrees of freedom; TLI = Tucker–Lewis Index; CFI = Comparative Fit Index; SRMR = Standardized Root Mean Square Residual; RMSEA = Root Mean Square Error of Approximation. The fit indices of the selected model are indicated in bold.
Table 4. Fit indicator models according to levels of invariance for variable sex.
Table 4. Fit indicator models according to levels of invariance for variable sex.
ModelCFIRMSEASRMRΔCFIΔRMSEAΔSRMR
Configural0.9940.0680.073---
Metric0.9920.0700.0770.0020.0020.004
Scalar0.9910.0710.0790.0010.0010.002
Strict0.9910.0790.08000.0080.001
Table 5. Mann–Whitney U test results for ISS scores according to sex.
Table 5. Mann–Whitney U test results for ISS scores according to sex.
M(SD)Mann–Whitney U Test
Z (p Value)
Women
n = 532		Men
n = 374
Operational Skills4.59 (0.57)4.51 (0.75)97,375 (0.899)
Navigational Skills4.16 (0.73)4.22 (0.71)101,709 (0.207)
Social Skills4.47 (0.56)4.38 (0.65)90,300 (0.077)
Creative Skills3.16 (0.84)3.41 (0.91)113,081 (<0.001)
Total ISS4.09 (0.55)4.13 (0.60)103,168 (0.383)
Table 6. ISS mean scores according to study area.
Table 6. ISS mean scores according to study area.
12345678910
Total ISS4.14 (0.58)4.08 (0.45)4.20 (0.54)4.14 (0.59)4.18 (0.44)4.06 (0.60)4.15 (0.50)3.99 (0.60)4.21 (0.44)4.16 (0.44)
1 = technologies; 2 = health; 3 = humanities; 4 = education; 5 = law; 6 = social sciences; 7 = natural sciences; 8 = art and architecture; 9 = agriculture, forestry, and fishing; 10 = administration and business.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Galván-Cabello, M.; Tereucan-Angulo, J.; Troncoso-Tejada, G.; Arellano-Silva, D.; Sánchez-Gallegos, V.; Nogués-Solano, I. Internet Skills Scale (ISS) in University Students from Chile: Factorial Structure, Reliability, Validity and Measurement Invariance of the Chilean Version. Sustainability 2025, 17, 8597. https://doi.org/10.3390/su17198597

AMA Style

Galván-Cabello M, Tereucan-Angulo J, Troncoso-Tejada G, Arellano-Silva D, Sánchez-Gallegos V, Nogués-Solano I. Internet Skills Scale (ISS) in University Students from Chile: Factorial Structure, Reliability, Validity and Measurement Invariance of the Chilean Version. Sustainability. 2025; 17(19):8597. https://doi.org/10.3390/su17198597

Chicago/Turabian Style

Galván-Cabello, Miguel, Julio Tereucan-Angulo, Gustavo Troncoso-Tejada, David Arellano-Silva, Víctor Sánchez-Gallegos, and Isidora Nogués-Solano. 2025. "Internet Skills Scale (ISS) in University Students from Chile: Factorial Structure, Reliability, Validity and Measurement Invariance of the Chilean Version" Sustainability 17, no. 19: 8597. https://doi.org/10.3390/su17198597

APA Style

Galván-Cabello, M., Tereucan-Angulo, J., Troncoso-Tejada, G., Arellano-Silva, D., Sánchez-Gallegos, V., & Nogués-Solano, I. (2025). Internet Skills Scale (ISS) in University Students from Chile: Factorial Structure, Reliability, Validity and Measurement Invariance of the Chilean Version. Sustainability, 17(19), 8597. https://doi.org/10.3390/su17198597

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop