Measuring Digital Citizenship: A Comparative Analysis

Abstract

This paper aims at showing a state of the art about digital citizenship from the methodological point of view when it comes to measuring this construct. The review of the scientific literature offers at least ten definitions and nine different scales of measurement. The comparative and diachronic analysis of the content of the definitions shows us two conceptions of digital citizenship, some more focused on digital competences and others on critical and activist aspects. This paper replicates and compares three scales of measurement of digital citizenship selected for their relevance and administered in a sample of 366 university students, to analyze their psychometric properties and the existing coincidences and divergences between the three. The most outstanding conclusion is that not all of them seem to measure the same construct, due to its diversity of dimensions. An online activism dimension needs to be incorporated if digital citizenship is to be measured. There is an urgent need to agree internationally on a definition of digital citizenship with its corresponding dimensions to elaborate a reliable and valid measuring instrument.

1. Introduction

Academic publications, and particularly international educational practices and programs, have shaped different ideas and approaches to digital citizenship since its start [1]. Authors have gone even further by adding other epithets such as democratic digital citizenship [2], global digital citizenship [3], radical digital citizenship [4], etc. This process has been reflected in an evolution that, in the words of Heath [5], has shifted from conservative and more technologically constrained positions to more critical and active ones (see Table 1).

Thus, the concept of digital citizenship has evolved from one of the first definitions of digital citizenship written by Ribble and Bailey [6], which focused on technological aspects and digital competencies, to the one proposed by Emejulu and McGregor [4], which highlights the commitment to social justice and to emancipatory and alternative technology. In the same vein, Pangrazio and Sefton-Green [7] point out that early conceptions of digital citizenship were concerned with the individual’s right to access and participate online to bridge the digital divide. Currently, the relationship of citizenship with the digital world has become much more complex around collective identities and in the context of social networks with ample possibilities.

This recent trend and conceptual process of the category of digital citizenship shows the diversity of approaches and perspectives, whether they are educational, socio-political, etc. For example, Law, Chow and Fu [8] consider up to three curricular or pedagogical perspectives of digital citizenship, such as digital competence, as an integral part of the culture of information and a final perspective related to the preparation for civic participation and politics. Likewise, Ryland [9] places the different approaches and definitions of digital citizenship into four groups: digital literacy, digital access, digital character, and civic engagement. It is not surprising that the most recent publications call for the urgent redesign and redefinition of digital citizenship to include the latest contributions of the most advanced, critical, and global concepts, as well as to consider the broader contexts and the latest educational practices [10,11].

Table 1. Scales for measuring digital citizenship and psychometric properties.

Source	Definition
Ribble and Bailey, 2007 [6] (p. 10)	“norms of appropriate, responsible behavior with regard to technology use. Digital citizenship Is a concept which helps teachers, technology leaders, and parents to understand how use technology appropriately”
International Society for Technology in Education, 2008 [12] (p. 1)	“advocate and practice safe, legal, and responsible use of information and technology; exhibit a positive attitude toward using technology that supports collaboration, learning, and productivity; demonstrate personal responsibility for lifelong learning; exhibit leadership for digital citizenship.”
Mossberger, Tolbert, and McNeal 2008 [13] (p. 1–2)	“those who use the internet regularly and effectively- that is, on a daily basis […] digital citizens are those who use technology frequently, who use technology for political information to fulfill their civic duty, and who use technology at work for economic gain”
Robles, 2009 [14] (p. 55)	“that individual, citizen or not of another community or State, who exercises all or part of his political or social rights through the Internet, independently or through his membership in a virtual community”
Ohler, 2010 [15] (p. 187)	“I can make the topic much more accessible if I refer to digital citizenship as “character education for the Digital Age.”
Richards, 2010 [16] (p. 518)	“practices conscientious use of technology, demonstrates responsible use of information, and maintains a good attitude for learning with technology”
Choi, 2016 [2] (p. 565)	“4 major categories that construct digital citizenship: Ethics, Media and Information Literacy, Participation/Engagement, and Critical Resistance.”
eTwinning, 2016 [17] (p. 11)	“Three main pillars come to mind when trying to define digital citizenship: belonging, engagement, and protection. Digital citizens belong to the digital society. They use technology to actively engage in and with society. Digital citizenship empowers people to reap the benefits of digital technology in a safe and effective way.”
Council of Europe, 2017 [18] (p. 10)	“Digital Citizenship may be said to refer to the competent and positive engagement with digital technologies and data (creating, publishing, working, sharing, socializing, investigating, playing, communicating and learning); participating actively and responsibly (values, skills, attitudes, knowledge and critical understanding) in communities (local, national, global) at all levels (political, economic, social, cultural and intercultural); being involved in a double process of lifelong learning (in formal, informal, non-formal settings) and continuously defending human dignity and all attendant human rights”
Emejulu and McGregor, 2019 [4] (p. 140)	“as a process by which individuals and groups committed to social justice deliberate and take action to build alternative and emancipatory technologies and technological practices”

Source: Prepared by authors.

Table 1. Scales for measuring digital citizenship and psychometric properties.

Source	Definition
Ribble and Bailey, 2007 [6] (p. 10)	“norms of appropriate, responsible behavior with regard to technology use. Digital citizenship Is a concept which helps teachers, technology leaders, and parents to understand how use technology appropriately”
International Society for Technology in Education, 2008 [12] (p. 1)	“advocate and practice safe, legal, and responsible use of information and technology; exhibit a positive attitude toward using technology that supports collaboration, learning, and productivity; demonstrate personal responsibility for lifelong learning; exhibit leadership for digital citizenship.”
Mossberger, Tolbert, and McNeal 2008 [13] (p. 1–2)	“those who use the internet regularly and effectively- that is, on a daily basis […] digital citizens are those who use technology frequently, who use technology for political information to fulfill their civic duty, and who use technology at work for economic gain”
Robles, 2009 [14] (p. 55)	“that individual, citizen or not of another community or State, who exercises all or part of his political or social rights through the Internet, independently or through his membership in a virtual community”
Ohler, 2010 [15] (p. 187)	“I can make the topic much more accessible if I refer to digital citizenship as “character education for the Digital Age.”
Richards, 2010 [16] (p. 518)	“practices conscientious use of technology, demonstrates responsible use of information, and maintains a good attitude for learning with technology”
Choi, 2016 [2] (p. 565)	“4 major categories that construct digital citizenship: Ethics, Media and Information Literacy, Participation/Engagement, and Critical Resistance.”
eTwinning, 2016 [17] (p. 11)	“Three main pillars come to mind when trying to define digital citizenship: belonging, engagement, and protection. Digital citizens belong to the digital society. They use technology to actively engage in and with society. Digital citizenship empowers people to reap the benefits of digital technology in a safe and effective way.”
Council of Europe, 2017 [18] (p. 10)	“Digital Citizenship may be said to refer to the competent and positive engagement with digital technologies and data (creating, publishing, working, sharing, socializing, investigating, playing, communicating and learning); participating actively and responsibly (values, skills, attitudes, knowledge and critical understanding) in communities (local, national, global) at all levels (political, economic, social, cultural and intercultural); being involved in a double process of lifelong learning (in formal, informal, non-formal settings) and continuously defending human dignity and all attendant human rights”
Emejulu and McGregor, 2019 [4] (p. 140)	“as a process by which individuals and groups committed to social justice deliberate and take action to build alternative and emancipatory technologies and technological practices”

Source: Prepared by authors.

A review of literature using Google Scholar and Scopus database provides at least ten research projects that have had, among other results, the construction of a scale or instrument to measure digital citizenship (“Digital citizenship scale”;

Table 2. Scales for measuring digital citizenship and psychometric properties.

Authors, Year—Country [Source] (Citations Google Scholar/Scopus)	Sample (Items—Options) Cronbach’s Alpha	Dimensions
Isman and Gungoren, 2014—Saudi Arabia [21] (81/16) *	229 college students (34 items—5 opt.) α = 0.85	1. Digital Literacy
		2. Digital Law
		3. Digital Rights and Responsibility
		4. Digital Communication
		5. Security Digital
		6. Digital Commerce
		7. Digital Access
		8. Digital Etiquette
		9. Digital Health and Wellness
Al-Zahrani, 2015—Saudi Arabia [22] (50/14) *	174 college students (46 items—5 opt.) α = 0.92	1. Respect Yourself/Others
		2. Educate Yourself/Others
		3. Protect Yourself/Others
Nordin et al., 2016—Malaysia [23] (23/4) *	391 college students (17 items—5 opt.) α = 0.78 a 0.86	1. Etiquette
		2. Responsibility
		3. Wellbeing/Health
		4. Commerce
		5. Security
Jones and Mitchell, 2016—USA [19] (151/48) *	979 high school students (11 items—5 opt.) α = 0.70 and α = 0.92	1. Online Respect
		2. Online Civic Engagement
Choi, Glassman, and Cristol, 2017—USA [20] (90/27) *	508 college students (26 items—7 opt.) α = 0.88	1. Internet Political Activism
		2. Technical Skills
		3. Local/Global Awareness
		4. Critical Perspective
		5. Networking Agency
Torrent-Sellens and Martínez-Cerdá, 2017—Spain [24] (8/2) *	544 college students (8 items—5 opt.) α = 0.94	1. “Ciudadanía activa” (5 items, α = 0.93)
		2. “Uso diversificado de los medios” (3 items, α = 0.95)
Hui and Campbell, 2018—Canada [25] (12/2) *	26 college students (40 items—7 opt.) α = 0.80	Idem… [20]
Mata-Domingo and Guerrero, 2018—Oman [26] (3)	200 high school students (9 items—4 opt.)	Idem… [21]
Jwaifell, 2018—Jordan [27] (6)	263 college students (9 items—2 opt.)	Idem… [20]
Peart, Gutiérrez-Esteban, and Cubo-Delgado, 2020—Spain [28] (0/0) *	205 secondary schools, universities and NGOs ** (59 items—5 opt.) α = 0.91 and α = 0.90	1. Digital Skills
		2. Socio-civic Skills

Source: Prepared by authors, based on Lozano-Díaz and Fernández-Prados [15] (p. 85). * Indexing in Scopus. ** Non-Governmental Organization.

). The publications come from a diverse range of countries, although there are more from Asia and the Middle East (China, Malaysia, South Korea, Jordan, Saudi Arabia, Oman, Jordan, and Turkey) than from the West (USA, Canada, and Spain). A search in the databases also provides the number of quotes of each category that has any of the ten scales found, with 151 mentions in Google Scholar and 58 in Scopus by Jones and Mitchell [19] followed with 90 and 27 quotes, respectively, by Choi, Glassman, and Cristol [20].

The next column of Table 2 provides information on the recipients to whom the instrument was administered in its first application, mostly to university students or recent graduates [21,22,23], to adolescent or secondary school students [19,26,28], and to teachers [25,27]. Some technical and psychometric properties of the scales are also shown, such as the samples, small and random in all cases; characteristics of the instruments, between 8 and 59 items and between 2 and 7 response options; and the reliability of the scales using Cronbach’s alpha, always above 0.70.

Finally, the third column contains the list of dimensions of each scale, between two and nine, resulting chiefly from the application of a theoretical model or concept such as the three mentioned above [21,25,27], which literally apply the approach of Ribble and Bailey’s nine elements [21], without performing a critical analysis on the theoretical corpus to date or considering the creation and selection procedure of items by judges and experts [2,20]. Only some scales apply an exploratory and confirmatory factor analysis to obtain their dimensions [23,24].

Only five scales have been replicated in the last few years by diverse authors (see

Table 3. Replications or adaptations of main digital citizenship scales.

Authors, Year [Source]	Authors, Year—Country [Source]	Sample (Items—Options)
		Cronbach’s Alpha (Dimensions)
Isman and Gungoren, 2014 [21]	Aladag and Ciftci, 2017—Turkey [29]	346 teachers (34 items—5 opt.) (9 dimensions)
	Elcicek, Erdemci, and Karal, 2018—Turkey [30]	143 college students (34 items—5 opt.)
		α = 0.79 (9 dimensions)
	Elmali, Tekin, and Polat, 2020—Turkey [31]	80 teacher candidates (34 items—5 opt.)
		α = 0.85 (9 dimensions)
Al-Zahrani, 2015 [22]	Alqahtani, Alqahtani, and Alqurashi, 2017—USA [32]	51 college students (46 items—5 opt.)
		α = 0.91 (3 dimensions)
	Alqahtani, 2017—Saudi Arabia [33]	361 teachers (46 items—5 opt.)
		α = 0.89 (3 dimensions)
	Ke and Xu, 2017—China [34]	115 college students (46 items—5 opt.)
		(3 dimensions)
	Xu, Yang, MacLeod, and Zhu, 2019—China [35,36]	712 college students (46 items—5 opt.)
		α = 0.89 (3 dimensions)
Nordin et al., 2016 [23]	Jwaifell, Aljazi, and Gasaymeh, 2019—Jordan [37]	189 high school students (16 items—5 opt.)
Choi, Glassman, and Cristol, 2017 [20]	Kim and Choi, 2018—South Korea [38]	200 teachers (18 items—7 opt.)
		α = 0.75 (5 dimensions)
	Kara, 2018—Turkey [39]	435 college students (26 items—7 opt.)
		α = 0.89 (5 dimensions)
	Choi, Cristol, and Gimbert, 2018—USA [40]	348 teachers (26 items—7 opt.)
		α = 0.79 a 0.89 (5 dimensions)
	Erdem and Koçyiğit, 2019—Turkey [41]	272 college students (26 items—7 opt.)
		α = 0.87 (5 dimensions)
	Yoon, 2019—South Korea [42]	283 college students (23 items—5 opt.)
		α = 0.88 (5 dimensions)
	Lozano-Díaz and Fernández-Prados, 2020—Spain [43,44]	302 college students (26 items—7 opt.)
		α = 0.89 (5 dimensions)

Source: Prepared by authors.

). Thus, the most replicated scale, six times, has so far been that of Choi Glassman and Cristol, while Nordin et al. have replicated it only once. In addition to this diversity in the repercussion and impact on the scientific community, the authors of the replications have made small variations in the number of questions or response options. These studies have also expanded the number of countries (China, Turkey, South Korea, etc.) and target groups (teachers, college students, high school, etc.). In any case, they have ratified and validated the instruments in other cultural contexts and have expanded the scientific and experimental corpus.

Given this amalgam of definitions and measurement instruments of digital citizenship, this main aim of this paper is to replicate and compare the most important scales and to discuss the weaknesses and strengths of each. The conclusions obtained can be used to make headway in this recent field of research [45] with an ever-growing interest in education and social sciences around the world [46].

2. Materials and Methods

2.1. Materials and Scales

First, three of the ten scales found in the review were selected. The selection criteria fundamentally responded to the impact on academic literature and citations in Scopus. As a result, the article by Jones and Mitchell scale (JM) [19] was the most quoted, with 48 publications on Scopus; the Choi, Glassman, and Cristol (CGC) [20], which was the second most quoted, had 27 references; and finally, Al-Zahrani’s scale [22] was selected despite having a somewhat smaller number of quotations than Isman’s [21] because it is more reliable both in its scale and in the numerous replications.

An expert translated digital citizenship scales from English to Spanish (see Appendix A). Prior to their application with the sample participants, a test was conducted with ten students to detect errors of comprehension or of another type, thereby making it possible to carry out the necessary adjustments. The questionnaire with the scales was administered over the internet using the open source LimeSurvey, an application for online surveys (for more information, see the LimeSurvey website: the online survey tool open-source surveys. URL: https://www.limesurvey.org, accessed on 24 January 2021). It was sent to the participants of the sample by email, while a brief explanation was given in class beforehand on how to respond and self-administer said scale.

2.2. Sample

The sample was selected among students from one university, like most of the studies referenced in Table 2, during the 2019–2020 academic year. It involved a sample of 366 students out of a total of 12,000 at the university, belonging to various schools such as the School of Education (Degree in Early Childhood Education, Primary Education and Social Education), the School of Law and the School of Social Work [47]. Thus, for this population and sample as well as for a confidence level of 95%, the sampling error was ±5%. The sample profile by sex was 78.7% female and 21.3% male, and by age, the majority was between 18 and 22 years old (79.5%).

2.3. Analysis

The resulting data analyses set out to compare the psychometric properties of each scale, which is why we performed a reliability analysis with Cronbach’s alpha statistic, and we performed an exploratory factor analysis by extracting the principal components to obtain the dimensions and the total explained variance. It also set out to establish the possible relationship among the three scales and their dimensions through correlation with Spearman’s rank correlation coefficient. Likewise, after categorizing the scales into high, medium, and low digital citizenship by dividing the distribution into terciles, they were compared in order to estimate the coincidences and the chi-squared statistic. In summary:

• Analysis of the psychometric properties of each scale
• Correlational analysis between the scales and their dimensions
• Comparative analysis of the digital citizenship level classifications of the scales and their coincidences

2.4. Limitations

The first limitation of this study is the sample drawn from a university context. All the studies reviewed, like this one, selected their sample randomly or incidentally from an educational group. This decision configures samples that are excessively homogeneous by age and educational level.

The second limitation is the difficulty of administering questionnaires developed by other researchers in different countries. Both the translation and the cultural contexts of different countries can lead to errors and make comparisons difficult.

In addition to the methodological limitations mentioned above and others, the diversity of definitions, the different theoretical approaches, and the evolution of the term “digital citizenship” also make it difficult to find coincidences among measurement instruments.

3. Results

3.1. Analysis of the Psychometric Properties of Each Scale

The result of the main psychometric properties of the three scales achieved more than sufficient scores about the reliability of the measurement instruments (see

Table 4. Psychometric properties of the three digital citizenship scales (DCS).

DCS	Reliability Analysis 1	KMO and Bartlett’s Test 2	Factor Analysis and Principal Component Analysis 3
AZ scale	0.89 (8)	0.80 (p < 0.001)	12 components and 62.73%
JM scale	0.83 (0)	0.84 (p < 0.001)	2 components and 54.18%
CGC scale	0.90 (4)	0.84 (p < 0.001)	6 components and 69.85%

¹ Cronbach’s alpha (numbers of items with corrected item-total correlation < 0.30). ² Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy (Bartlett’s test of sphericity and p-values). ³ Numbers of components and total variance explained (fixed number of factors by original scale and total variance explained). AZ, Al-Zahrani scale; JM, Jones and Mitchell scale; CGC, Choi, Glassman, and Cristol scale. Source: Prepared by authors.

). Cronbach’s alpha coefficient exceeded 0.80 in all three cases, although the analysis of the correlation of each item with the total of its scale would question its presence, as it was below 0.30 for eight items on the Al-Zahrani (AZ) scale and four items on the Choi, Glassman, and Cristol (CGC) scale.

Meanwhile, the Kaiser–Meyer–Olkin (KMO) measure was high and significant enough with Bartlett’s test of sphericity to propose, in the three scales, an exploratory factor analysis to extract the main components. Therefore, from the principal components we extracted all the factors whose eigenvalues exceeded one to compare them with the fixed number that the corresponding authors proposed in their respective papers.

The AZ scale was far from the number of components extracted; in fact, if the factor analysis is attached to the three proposed components, it loses a great deal of the explanatory power of the variance. Meanwhile, in the JM scale the number of components matched a total explained variance of more than 50%. Finally, in the CGC scale only one more component appeared, yet it was the instrument that explained the most variance (66%). In summary, the AZ digital citizenship scale, despite its high reliability, raised severe doubts about the constitution of its components based only on the exploratory factor analysis, while the JM and CGC scales, which were both very reliable, showed a better match of their components and obtained a high explained variance, especially the CGC.

3.2. Correlational Analysis between the Scales and Their Dimensions

Table 5. Correlations among the different dimensions of the CD scales.

DCS, Dimensions. (Cronbach’s Alpha)	AZ Scale	JM Scale	CGC Scale
AZ scale	1.000	0.295 ***	0.093
Dimension 1. Respect Yourself/Others (0.87)	0.853 ***	0.274 ***	0.023
Dimension 2. Educate Yourself/Others (0.72)	0.647 ***	0.263 ***	0.348 ***
Dimension 3. Protect Yourself/Others (0.86)	0.768 ***	0.206**	0.042
JM scale	0.295 ***	1.000	0.233 ***
Dimension 1. Online Respect (0.85)	0.249 **	0.905 ***	0.100
Dimension 2. Online Civic Engagement (0.67)	0.248 **	0.804 ***	0.347 ***
CGC scale	0.093	0.233 ***	1.000
Dimension 1. Internet Political Activism (0.91)	0.158 *	0.154 *	0.820 ***
Dimension 2. Technical Skills (0.92)	−0.025	0.167 **	0.207 ***
Dimension 3. Local/Global Awareness (0.79)	0.010	0.153 *	0.488 ***
Dimension 4. (0.84): Critical Perspective	0.035	0.203 **	0.863 ***
Dimension 5. Networking Agency (0.64):	0.043	0.141 *	0.704 ***

* p < 0.05. ** p < 0.01. *** p < 0.001. Source: Prepared by authors.

shows, in the first column, the relationship of the various components of the scales as defined by the authors. Therefore, the three dimensions of the AZ scale exceeded the minimum reliability of 0.70, while one of the two components of the JM scale did not reach the indicated threshold, nor did one of the five of the CGC scale. At this point, we should recall that the AZ instrument was the one that included the greatest number of items, which helps obtain better coefficients.

The Spearman rank correlation coefficient and the significance level of all dimensions with the three digital citizenship scales are also recorded in the following columns. As expected, the dimensions of each scale correlated highly with the total of its questionnaire (with a significance level of p < 0.001). One dimension stood out in each one of them: in the AZ scale, the component “Respecting oneself and others”; in the JM scale, “Online respect”; and in the CGC scale, two dimensions, “Internet political activism” and “Critical approach”, which, to a certain extent, defined the emphasis or focus of the instrument.

The two scales that most correlated with each other in their dimensions and in the total were the AZ scale with the JM scale, doing so at a level of 0.30, which was significant (p < 0.001), as can be seen in the last three rows of Table 4. Meanwhile, the AZ and CGC scales did not correlate as a whole or by dimensions, except one of AZ with the CGC scale. Again, it was the JM scale that correlated with an index of 0.23 (p < 0.001) with the CGC scale.

3.3. Comparative Analysis of the Digital Citizenship Scales

Finally, the three scales were compared according to coincidences in classifying distinct levels of digital citizenship. Each scale was categorized into terciles of their respective distributions—in other words, three levels of high, medium, or low digital citizenship. Therefore, Figure 1 shows that the three scales coincided in classifying 22% of respondents in the same category or level. In addition, the Venn diagram shows the percentage of respondents classified in the same category by pairs of scales; for example, those that shared the most, the JM and AZ scales, were the ones that coincided the most (22.2% + 29.4% = 51.6%). The scale that shared the least with the other two was CGC (with only 61.1%), and the scale that shared the most with the other two was JM (with 73.8%).

Likewise, the three scales organized in the three categories or levels were crossed with each other to calculate the chi-square statistic (see

Table 6. Percentage of the sample in the same terciles [chi-square test].

DCS in Terciles	JM Scale	CGC Scale
AZ scale	(χ2 (4, n = 153) = 28.12 p < 0.001) ***	(χ2 (4, n = 153) = 4.45 p = 0.346)
JM scale	-	(χ2 (4, n = 153) = 15.06 p < 0.01) **

** p < 0.01. *** p < 0.001. Source: Prepared by authors.

). It can be seen how the Jones and Mitchell (JM) with the Al-Zahrani scale (AZ) obtained a statistically highly significant chi-square (p < 0.001), while the AZ and CGC scales were far from being statistically significant.

4. Discussion

The introduction already warned of the diversity of definitions and the variety of instruments for measuring digital citizenship that required a methodological approach with a comparative perspective, replicating some of the most prominent scales. The most prominent results point to methodological deficiencies in the Al-Zahrani scale when specifying the components of its instrument and not providing a factor analysis, neither exploratory nor confirmatory, to support them statistically. Likewise, correlation and statistical analyses such as the chi-square statistic conclusively showed the scarce relationship between the scale with the most items and impact, that of Al-Zahrani, and the most replicated scale, that of Choi, Glassman and Cristol. In summary, the review, replication, and comparison lead us to two underlying issues or problems related to these findings—one of a conceptual and theoretical nature, and the other of a methodological nature.

The conceptual debate was not only triggered by a wide list of definitions on digital citizenship but by the various theoretical approaches described by different authors [7,8,9], which somehow recall the disquisitions of the theory policy around the two existing concepts of citizenship, liberal citizenship, and republican citizenship [14], and which required a study in itself. In any case, at least two questions should be raised for the re-elaboration or reconceptualization of digital citizenship and its application in instruments for measuring it.

First, it is necessary to eliminate the component related to digital literacy or skills, which, in addition to having its own scientific literature and differentiated foundation [38], is less and less important in a highly literate generational context and which, as seen in the Choi scale, is the component with the lowest correlation with its own scale. Secondly, it is necessary to incorporate the activist and critical component as part of the essence of digital citizenship, following not just in the wake of the latest definitions but also educational practices across the world in education for global citizenship [10,48,49]. Certainly, these components and their emphasis on involvement and participation shift the Choi, Glassman, and Cristol scale further away from the Al-Zahrani scale, which accentuates more components of protection and respect or, in other words, a more “defensive” and “negative” version of digital citizenship, inspired by the definition of Ribble and Bailey [6].

The methodological aspects and their restraints appear on a recurring basis in the creation of new instruments to study or evaluate emerging realities such as digital citizenship. The methodological problems focus on aspects such as (i) the elaboration of instruments based on definitions of specific authors without a thorough critical review; (ii) the incorporation of items with no other criteria than the own intuition of the researcher or researchers; (iii) the fact of ignoring the validation of content and of the construct of the questionnaires; (iv) the use of random samples of little variability; and (v) and the failure to give an option to access the data matrices, etc.

5. Conclusions

As a general conclusion and future research proposal, we should highlight the fact that more replications, comparatives, and studies are needed in the use of digital citizenship assessment tools to reach a consensus. In them, it is necessary to (i) clarify the citizenship theory model under consideration; (ii) explain which correlates can be found with instruments that claim to assess the same construct; (iii) expand the study sample as far as possible in the general population; and (iv) observe the necessary methodological rigor. At the same time, theoretical debate on the construct of digital citizenship is also urgently needed to reach a consensus around it. In a context of international and transparent studies with widely tested questionnaires, it is necessary to move toward the design of a rigorous, reliable, and validated digital citizenship measurement instrument, one which is also administered in equiprobable and representative samples. In this sense, the first steps taken by international organizations such as the European Union, UNICEF, and UNESCO are encouraging [50,51,52].

This theoretical–conceptual and methodological–instrumental consensus would be particularly useful to assess not only different dimensions of students’ digital competencies [53] but also other aspects and social contexts. Thus, whether for addressing challenges in small rural communities [54] or for major sustainable development goals [55], assessing the participation and digital citizenship of the population is crucial in the information society.

Future research should focus on what is proper to the concept of digital citizenship: “critical awareness” and “digital activism”, since the other dimensions are addressed by other conceptual categories and measurement tools such as “digital literacy” or “digital competence”. The first dimension of digital citizenship “Critical Awareness” should combine the dimensions “Critical Perspective” and “Local/Global Awareness” of the CGC scale with some items of the other scales. Likewise, the second dimension of digital citizenship “Digital Activism” should combine the dimensions “Internet Political Activism” of the CGC scale with the “Online Civic Engagement” of the JM scale, incorporating the new forms of digital participation.