The Implementation of a Sustainable Online Course for the Development of Digital Citizenship Skills in Higher Education

Abstract

This study aims to develop and evaluate the effectiveness of a digital citizenship curriculum developed using online teaching for undergraduate students. For this purpose, answers were sought to the following general research questions: “How are students performing their digital citizenship skills?”, “What is the efficiency level of the developed digital citizenship curriculum?”, “What are the students’ self-evaluations during the implementation process of the developed digital citizenship curriculum?” and “What are student opinions about the implemented digital citizenship course?”. A “parallel-simultaneous design”, one of the mixed research methods, was used in this research. The content of the curriculum was determined within the framework of a needs analysis consisting of university students and field experts, and a document review. A quasi-experimental design method was used to implement the programme. A total of 39 teacher candidates, registered in the spring semester of the 2022 academic year, participated in the experimental process. A Moodle learning management system was used as the teaching environment. Participants’ digital citizenship behaviours and attitude levels were measured before and after the experimental application. It was observed that there was a significant (positive) difference between the pre-test and post-test average scores in favour of the post-test scores, and the effect value of this difference was high. It turned out that the teacher candidates’ opinions on the effectiveness of the online digital citizenship programme were positive. It is understood that the digital citizenship teaching curriculum developed for higher education can be used sustainably for similar university student groups as in this study. However, it is anticipated that the developed programme can be updated along with technological evolutions.

1. Introduction

From antiquity to the present, civilisations have occupied the “Fertile Crescent” region of the Middle East where socio-economic and technological phenomena were first formed, and this has laid the initial foundations for the concepts of homelands and citizens used today [1,2].

It is thought that the concept of citizenship was first used in the Ancient Greece to define the members of a city-state and those who were free [3,4]. Rome, which ruled over a large amount of territory for many years, first recorded the right to the concept of citizenship, and the rules and regulations of being a citizen either through birth or later in life. The modern view of citizenship has resulted in many different problems [5]. People now have social rights, such as the right to education, health, a clean environment and social security, for citizens to live humanely according to certain standards, and these rights are mostly realized by increasing the level of welfare. This shows that citizenship can develop and change [6,7].

The technological developments in the contemporary age of information have also changed the lifestyles of individuals. One of these technological developments, the internet, has rapidly increased its impact on the lives of individuals since the 1990s, and is now influential in many areas such as education, communication, health, politics, industry, media and public citizenship transactions. These changes have triggered a “change of the social structure, the social relationships web that forms the social structure and the social establishments that determine these relationships” [8]. According to Mota and Cilento [9], the internet may be considered one of the most important means of communication and information in contemporary society. Access to this technology has been growing year by year.

In modern society, where using technology and the internet has changed from being a luxury to a necessity, people need to gain knowledge, skills, methods and habits in order to adapt to the ever-changing and developing technology, to understand this technology and to benefit from the opportunities it provides [10]. Given the importance of the internet in our lives, possessing the ability to use these tools effectively and being literate in the information and media of the digital age are crucial skills. Thus, being an active and productive digital citizen will be possible in the 21st century [11]. It is crucial for teachers and administrators to remain up-to-date with the emerging skills and technologies of the 21st century, while not ignoring the need to teach students to be responsible citizens [12]. There have been many changes and innovations in our 21st century habits. Accordingly, 21st century skills tell us to be competent in the field of information, media and technology, and to exhibit responsible behaviour in this context. This concept is referred to as digital citizenship, which has gained importance with the widespread use of the internet.

The responsibilities, rights and obligations of regular citizenship should continue in digital environments [13]. The aim of digital citizenship is to apply these responsibilities and the norms of the people to a digital platform [14]. Digital citizenship means interacting positively, critically and competently on a digital platform, as well having the skills to apply forms of participation that respect human rights and human dignity through the responsible use of technology [15]. Additionally, a digital citizen has been defined as someone who knows technology, who understands and can create digital content, who can find correct and reliable resources, who does not neglect ethical values and who can conduct communication through digital platforms via the internet [16]. Digital citizens are a part of the digital community. In this regard, Prensky [17], defined digital citizenship as a metaphor for country citizenship and those who are users of digital tools as the citizens of this country. According to Mossberger, Tolbert and McNeal [18], the reasons that digital citizenship has become more common in recent years are as follows:

• The positive effects of technology on society and the economy,
• The opportunity to provide equal economic opportunities through the internet,
• Active participation of citizens,
• Unequal participation in some regions and communities.

Digital technologies make it easier for universities to achieve their aim of being institutions that learn and develop [19]. Ribble and Bailey [20] referred to the necessity for digital citizenship to become part of educational culture and the need to create digital citizenship programmes; in this context, parents, schools and the community play an important role in supporting suitable online behaviour. Schools in particular play a critical role in evaluating and implementing the concept of digital citizenship [21]. The ISTE standards were developed by the International Society for Technology in Education as a result of the increased importance of the concept of digital citizenship [14]. As a result of devices that provide easy access, such as smartphones and smart devices, as well as the increase in the variety of social media platforms, it is seen that people have a tendency to share their private and personal information in an uncontrolled manner. As Arıcak, Kınay and Tanrıkulu state [22], “In some countries, alongside traditional crimes, cyber-crimes are also increasing. Students must be trained on how to be a responsible, aware and ethical user and must be aided to become a digital citizen” [23]. The ISTE established the International Education Technologies Standards (NETSs) and prepared a guide that includes the points that all schools must include in their educational technology curricula [24]. Digital citizenship skills have been separated into five competencies by Oyedemi [25] and nine by Ribble [26].

Oyedemi [25] listed these as: (1) Human and citizenship rights, (2) Communication and access through technology, (3) Communication through technology, (4) Regular technology use skills, and (5) Policies that enable these.

Ribble and Park [27] examined digital citizenship through nine topics. These have been reflected in the world of education and are often used in the teaching of digital citizenship. These are: (1) Digital Health, (2) Digital Ethics, (3) Digital Rights and Responsibilities, (4) Digital Communication, (5) Digital Security, (6) Digital Law, (7) Digital Access, (8) Digital Literacy, and (9) Digital Business. While this list proposed by Ripple is recognised globally, it is seen that there is no curriculum study in higher education in the literature that includes these sub-dimensions.

It is found that in developed countries, digital citizenship education is added to teaching programmes in both formal and informal structures, whereas studies show that this concept is not sufficiently reflected in education in less developed countries [28]. It is particularly important for developing countries to develop digital citizenship teaching policies at the higher education level [29]. According to Tatlı [30], determining students’ digital citizenship levels from primary school through to higher education is not up to the schools themselves. It is stated that digital citizenship should be included in higher education curricula as a compulsory subject at the undergraduate level [31]. It is claimed that university-level students need an in-depth digital citizenship education [32]. This is because after graduating from university, students will enter their professional lives. Before going on to join their profession, individuals have a tendency to become more active participants in technology [33].

Digital citizenship skills in higher education are imperative for a variety of interconnected reasons arising from the evolving technology and educational environment. A digital citizenship education in higher education should ensure information literacy and critical thinking, the effective use of information, online collaboration and communication, the awareness of privacy and security issues, preparation for future careers and global citizenship. Digital citizenship skills are essential to prepare students not only for academic success but also for responsible and effective participation in the digital society [34].

It has been demonstrated by previous research that digitalisation is especially important in higher education. Especially with COVID-19, with the intensive use of the internet and computer technologies, digital transformation has become a current issue in higher education [35]. At this point, digital citizenship education is necessary in order to have digital leaders in the modern era. The increasing complexity and dependence of the university curriculum on information and communication technologies such as the internet and mobile telephony call for a systematic integration of digital citizenship into academia [36].

In this regard, the need for the digital citizenship training of undergraduate-level students must be determined and a digital citizenship education curriculum must be developed.

The aim of this study is to develop a digital citizenship course-teaching programme and to evaluate its efficiency in higher education. The research questions designed to achieve this general aim are as follows:

(1)

What are the needs for the digital citizenship curriculum to be developed?

(1.1)

How are students performing their digital citizenship skills?

(1.2)

What are the field expert opinions on why and how the digital citizenship curriculum should be developed?

(1.3)

What are the lecturers’ opinions on why and how the digital citizenship curriculum should be developed?

(1.4)

What do the findings of the document analysis suggest for improving the content of the digital citizenship curriculum?

(2)

What is the efficiency level of the developed digital citizenship curriculum?

(2.1)

What are the students’ digital citizenship attitude levels before and after the experimental application?

(2.2)

What are the students’ digital citizenship behaviour levels before and after the experimental application?

(3)

What are the students’ self-evaluations during the implementation process of the developed digital citizenship curriculum?

(4)

What are student opinions about the implemented digital citizenship course?

2. Methods

In this section, the research model, study sample, study scope, learning activities and course plans, data collection tools, statistical methods and techniques used to collect and analyse the data are all presented under different sub-headings.

In this study, a mixed research model, in which quantitative and qualitative research methods are used together, was used to investigate the development of the digital citizenship course curriculum and to evaluate its effectiveness. It is possible to reduce the limitations of using just one method, to obtain more comprehensive data, and to strengthen the findings through the use of a mixed research model [37]. In mixed study methods, there are research models such as exploratory sequential, explanatory sequential, parallel-simultaneous and transformative [38].

In this study, the “parallel-simultaneous design” mixed method was used. In this design, the qualitative and quantitative research methods are conducted simultaneously, the data are collected either simultaneously or asynchronously, and the quantitative and qualitative data that are collected either from the same sample or different samples are analysed separately [39].

In the quantitative part of the study, scanning and experimental research techniques were used, and the changes in attitudes and behaviours before and after the application were revealed in the experimental study. In studies using a descriptive-relational survey model, a situation or event is described as it is, and the relationships between as well as the effect and degrees of the variables that cause this situation are determined [40]. Experimental studies try to determine if a procedure has an effect on the result. The researcher intervenes with at least one of the application groups [41].

Experimental designs are divided into three sub-models: true experimental designs, quasi-experimental designs and single-subject experimental designs. Quasi-experimental designs were used in the experimental part of this research. In quasi-experimental designs, it is possible to define the distribution between groups as not being partially or fully randomly assigned [42]. Since it was difficult to access the control group variable, a quasi-experimental design without a control group was chosen. The dependent and independent variables of experimental research need to be determined [43]. Generally, the dependent variable refers to the result after the application, and the independent variable is shown as the training programme [44]. The dependent variable in this research is the students’ digital citizenship behaviours and attitudes. The independent variable is the digital citizenship curriculum created.

In the qualitative part of our study, the case study pattern was used. Comprehensive and longitudinal reporting of what is happening in the environment over time provides an interpretation of the findings regarding the change in practice [43]. In light of this design, interviews, compositions and document examinations were conducted for the needs analysis. Again, in the qualitative part, practical opinions were collected after the implementation and self-evaluations were collected after the implementation activities.

In a document review, reports and all other written documents are treated as an empirical data source for case studies [45]. This is also used in studies to investigate only one phenomenon, situation, organisation or programme [46]. The research problem or problems provide the researcher with a general framework for what kind of documents are needed [47]. So, earlier digital citizenship studies were examined during the needs analysis phase of the present study. A document analysis method was used in this process. The documents obtained would then go on to guide the determination of the content of the new higher education digital citizenship education programme to be developed.

The interview technique is when two people meet face-to-face on an online platform to obtain knowledge for a specific aim [48]. In this context, students’ opinions and self-evaluations were obtained both during the needs analysis phase and after the implementation, to determine the positive and negative aspects of the curriculum and to determine the overall contributions of the programme.

2.1. Study Group

This study was conducted in the fall/spring academic semester of 2021–2022 at a faculty of education. In the relevant year, the selection of the study group was conducted in accordance with the research method. The purposive sampling method was used in the qualitative phase of the research. This method provides the opportunity to conduct in-depth analyses of situations that are thought to include rich information [49]. The criterion sampling technique from the purposeful sampling methods was chosen. In this sampling method, the criteria that are thought to be important for selection are specified. It is thought that each sample selected according to these criteria represents all the specified aspects [50]. Because a document review was conducted in the needs analysis, we decided to access previous resources regarding the “digital citizenship” topic for our source of data. For this purpose, the documents that would be subjected to document analysis were selected via criterion sampling, one of the purposeful sampling methods. In the document search process, only “digital citizenship-dimensions-criteria and education” was used as a keyword. In this regard, a total of 28 documents, of which the oldest was published in 2016 and the most recent in 2021, were selected.

We planned to ask the students to write compositions in order to specify their level of readiness for digital citizenship in the needs analysis process. In this process, the criterion sampling method was used. The students who had “previously had basic computer and information technologies courses” were selected. At this stage, a total of 30 teacher candidates were reached.

In the needs analysis, in order to find an answer to the question, “What needs to be taught and how?”, field experts were interviewed. For this reason, the criteria determined in the process of including field experts in the sample included teachers and academicians who taught “technology courses, courses with digital content, courses related to instructional technologies, computer courses”. In this process, opinions were received from 10 participants.

Since an experimental study was conducted in the quantitative phase of the research, the participants were selected by using the convenient sampling method. Due to the limitations of time, finance and labour, this method was used to select the sample from easily accessible departments where the application could be conducted [43]. Resultantly, the experimental study group consisted of 39 teacher candidates who were studying during the spring semester of the 2021–2022 academic year. Students who attended the faculty course opened in the relevant period were the natural experiment group.

The field experts who gave their opinions and the participants who participated in the experiment live in north Cyprus. The distribution of all participants’ demographic information is given in

Table 1. Distribution of demographic information of participants.

			(f)	(%)
Experimental Study Group	Distribution of participants by gender	Female	24	61.53
		Male	15	38.46
	Distribution of participants by age	21–22	20	51.28
		23–25	13	33.33
		26–28	5	12.82
		29>	1	02.05
	Students’ grade levels at university	2	12	30.76
		3	16	41.02
		4	11	28.20
Field Experts Working Group	Distribution of participants by gender	Female	5	50.0
		Male	5	50.0
	Distribution of participants by seniority in their occupation	<5	2	25.0
		6–11	5	41.7
		12–17	2	25.0
		18>	1	8.3
	Distribution of participants by seniority in their occupation	5	2	20.0
		6	1	10.0
		7	2	20.0
		10	2	20.0
		11	1	10.0
		19	2	20.0
	Distribution of participants according to their studies on digital citizenship	Taught the same/similar course content	10	100.0
		Made a presentation at a conference	1	10.0
		Wrote a book/chapter	1	10.0
		Printed a thesis	1	10.0
		Wrote a thesis	1	10.0
		Made a scientific publication	3	30.0
	Distribution of participants by title	Associate Professor	1	10.0
		Assistant Professor	3	30.0
		Doctor	4	40.0
		Instructor	2	20.0

.

2.2. Experimental Working Environment

The open-source Moodle learning management system provided by the Near East University Distance Learning Centre as a distance education opportunity was used as the learning environment throughout this study. Moodle stands for “Modular Object-Oriented Dynamic Learning Environment”. The software functions on MySQL 8.0 and PostgreSQL 13 database systems and in any environment that supports the PHP (8.0.0) language. Moodle 4.3 was chosen because it is the most commonly used open-source learning management system. This system is powerful, dynamic and eye-catching. It has an easy-to-use interface and allows the creation of online courses and experiences [51].

A Moodle system was established to implement this educational programme. Lessons and related contents were uploaded to the Moodle system. Lessons were held online weekly. All course materials and course registration information were shared over the system on a weekly basis with the students. All of the evaluation tools for the course were provided via the Moodle system online and the results were delivered from the students in the Google Drive cloud system. Permission was obtained from the Near East University Ethics Board for the use of this platform and the evaluation tools. The content of the course and its creation process are explained under the next heading.

2.3. Teaching Activities Programme

Course plans were formed to develop teaching activities for digital citizenship. The digital citizenship curriculum was prepared with the modular approach. When we take modular teaching as a programme development approach, it is inevitable that it will touch on certain philosophical views. It would be correct to state the principles of the progressivism movement, which is the reflection of the pragmatist philosophy that focuses on student-centred learning, are taken into account in modular teaching. It is stated that modular education involves the planning, implementation and evaluation of behaviours to be gained in modular units [52].

The modular curriculum was created through a needs analysis of students and interviews with field experts. The findings of these were also compared with the findings of the document review, and a programme content supported by the literature was created. In the process of developing the digital citizenship curriculum after the needs analysis, the teaching experts who would give their opinions on the programme were selected based on the criterion sampling method. Accordingly, the curriculum development science committee consisted of four experts: a scientific research methods specialist, a programme development specialist, a measurement and evaluation specialist, and an information technology teaching specialist.

The module education is unbelievably valuable to the course, as it makes students’ learning experience well-stimulated [53]. In this context, the module course plans were prepared with the headings of “1. Digital Health, 2. Digital Ethics, 3. Digital Rights and Responsibilities 4. Digital Communication, 5. Digital Security, 6. Digital Law, 7. Digital Access, 8. Digital Literacy, 9. Digital Business” and units were prepared under the same headings. The outcomes of the course were cognitive and affective. Information on the topics, outcomes and teaching times can be seen in

Table 2. Subjects, achievements and learning period.

Dimension	Gains	Duration
Introduction to Digital Citizenship	Explains the concept of digital citizenship.	2 h
	Explains the sub-dimensions of digital citizenship.
Digital Access	Can access their desired information in digital environments.	2 h
	Can provide equal opportunities for themself and others.
	Explains the concept of digital access.
Digital Commerce	Can make secure purchases in digital environments.	2 h
	Explains the concept of digital commerce.
	Knows the risks and problems related to e-commerce.
Digital Communication	Explains the concept of digital communication.	2 h
	Can communicate correctly and safely.
	Can choose the means of communication suitable for their purpose.
Digital Literacy	Explains the concept of digital literacy.	2 h
	Can access their desired information in digital environments.
	Can use software and hardware consciously.
	Can reflect their digital access abilities in their academic success.
	Can produce information in digital environments.
Digital Ethics	Explains the concept of digital ethics.	2 h
	Uses digital devices and environments with an awareness of ethical responsibilities.
	Distinguishes between ethical and unethical behaviours or content in digital spaces.
Digital Law	Explains the concept of digital law.	2 h
	Knows that they and others have rights in digital environments.
	Knows that situations and behaviours in the digital environment are subject to the law.
	Explains the law of informatics.
Digital Rights and Responsibilities	Explains the concept of digital rights and responsibilities. Knows that they and society have rights and responsibilities in the digital environment. Defines private, personal and public rights and responsibilities.	2 h
Digital Health	Explains the concept of digital health.	2 h
	Knows that the software–hardware they use can impair their physical and psychological health.
	Can use software and hardware in accordance with their physical and psychological health.
Digital Security	Explains the concept of digital security.	2 h
	Can protect personal information, data and devices.
	Knows the software required to stay safe within digital applications.

.

The “scenario-based learning” model was used as the teaching approach in this study. The aim of scenario-based learning is for students to learn information through their own lifestyles [54]. Expert views were taken into account when preparing these scenarios. It was deemed appropriate to use experimental studies during the application. The implementation of this developed programme took 10 weeks. Each unit/course plan was conducted in two-hour course periods. During the first hour of the course, the subject was explained by the researcher using a PowerPoint presentation, and during the second hour, scenarios were given/read to students, and they were asked to discuss which situations exhibited suitable behaviour and which were unsuitable. Then, the suitable and unsuitable situations were reinforced. At the end of each course, the students were given self-evaluation forms to complete. Self-evaluation helps gather information about what the teaching candidates learnt as well as their cognitive and affective situation before and after the course [55].

2.4. Scenario Creation Process

Scenarios used in a planned manner to achieve learning outcomes should be developed with a pedagogical approach [56]. The scenarios developed in this research, as stated by Çubukçu [57], were created in accordance with problem-solving strategies, allowing students to think about a problem, mobilise their knowledge, adapt it to similar situations, realise their knowledge deficiencies and conduct research to eliminate these deficiencies. A scenario has been prepared for the weeks specified in Table 2. The relevant agenda was followed in creating the scenarios, and current and real news from online newspapers, blogs and forums were used. Expert opinions were taken into account on the developed scenarios from two subject experts and two programme development experts. Each of the developed scenarios was implemented during the relevant course week. Students were directed to a thought process in which they could use their knowledge and skills regarding appropriate and inappropriate situations in the scenarios.

2.5. Data Collection Tools

2.5.1. Quantitative Data Collection Tools

In order to collect quantitative data in this study, a personal information form, a digital citizenship scale and a digital citizenship attitude scale were used. These tools have been described below. Since concepts cannot be measured directly, the relationship between concepts in social sciences is determined through scales formed by transforming the behaviours and attitudes that define these concepts into expressions [58].

Our reason for choosing these scales is to obtain the change in behaviours and attitudes achieved with the programme we propose to implement. Thus, with these scales, the post-test results will be examined after the programme is implemented. The developed programme includes teaching digital citizenship skills. The aim of teaching programmes is to reveal changes at the cognition, attitude and behaviour levels of individuals [59].

2.5.2. Personal Information Form

The personal information form was developed in order to obtain the demographic information of the participants and includes information that is thought to affect the digital citizenship opinions, attitudes and behaviour levels of the participants. This information includes their age, gender, university classroom level, occupation, title and any studies that they have conducted.

2.5.3. Digital Citizenship Scale

The digital citizenship scale designed by Akcil [60] as part of their doctoral thesis study was used after obtaining the necessary permissions. This digital citizenship scale consists of cognitive questions. It comprises 39 items and 9 factors. These factors are related to: 1. Digital Health, 2. Digital Ethics, 3. Digital Rights and Responsibilities 4. Digital Communication, 5. Digital Security, 6. Digital Law, 7. Digital Access, 8. Digital Literacy, and 9. Digital Business. The Cronbach alpha coefficient of the scale developed by Akcil was calculated as 0.87. Confirmatory and exploratory factor analyses were performed on the scale. It can be seen that each dimension was created with at least three and at most six items. The Cronbach’s alpha coefficient of the scale used in our study was found to be 0.76. The scale was created as a six-point Likert-type model, and the options were arranged as 1: Always Agree, 2: Mostly Agree, 3: Slightly Agree, 4: Disagree, 5: Strongly Disagree, 6: Do not know.

2.5.4. Digital Citizenship Attitude Scale

The digital citizenship attitude scale developed by Karaduman [3] in their doctoral thesis was used after obtaining the necessary permissions. It consists of 32 items and 9 factors, and the Cronbach alpha reliability coefficient of their scale was found to be 0.74. Within the scope of the validity studies of the scale, a factor analysis was performed for construct validity and item discrimination analysis was performed for criterion validity. It was created with at least three and at most six items in each factor. These factors are related to: (1) Digital Health, (2) Digital Ethics, (3) Digital Rights and Responsibilities (4) Digital Communication, (5) Digital Security, (6) Digital Law, (7) Digital Access, (8) Digital Literacy, and (9) Digital Business. In our study, the Cronbach’s alpha coefficient was determined as 0.78. The scale was created as a five-point Likert-type model and the options were arranged as: (1) Strongly Disagree; (2) Disagree; (3) Partially Agree; (4) Agree; (5) Strongly Agree.

2.5.5. Qualitative Data Collection Tools

A self-evaluation form, a teaching experts interview form, digital citizenship compositions and document analysis methods were used to collect the qualitative data for this study.

2.5.6. Self-Evaluation Form

The self-evaluation form was prepared for the prospective teachers participating in the research so that they could evaluate their own learning. The data obtained were used in supporting the obtained quantitative data. A self-evaluation method enables learners to evaluate their own learning level and to take on more responsibility [61], while also making students more active and focused [62]. Self-evaluation forms help to determine what the teacher candidates learnt and their cognitive and affective situation before and after the course [55]. After a literature review was conducted, weekly questions were chosen for our self-evaluation forms. The students were informed about the criteria that were to be used in their self-assessment and evaluation, and how to evaluate themselves according to these criteria. The teacher candidates completed the self-evaluation form, taking into account their cognitive and affective states. In the first week, the question in the self-evaluation form was an open-ended question about digital citizenship. This week’s course topic included explaining the general framework of digital citizenship. The second week’s question and the questions thereafter were open-ended questions about the sub-dimensions of digital citizenship. Starting from the second week, a different digital citizenship sub-dimension was discussed each week. The recurring question was, “What did you learn that you did not know after the digital citizenship covered this week?”.

2.5.7. Education Specialist Interview Form

In order to obtain the views of education specialists in the needs analysis stage of this study, a digital citizenship interview form was used. The questions in this form were prepared by the researchers and necessary modifications were made and implemented based on the opinions of four experts. This expert team consisted of two field experts and two programme development experts. There are eight questions in this interview form. These questions covered: “General views on the importance of the concept of digital citizenship today, “Opinions on whether there is a need to develop an undergraduate teaching programme on digital citizenship”, “Views on the possible impact of an undergraduate programme on digital citizenship”, “Opinions on what should be the achievements of the curriculum”, “Opinions on what the learning–teaching strategy for the curriculum should be”, “Opinions on what teaching activities should be”, “Opinions on the learning–teaching material suggestions that can be used in transferring the content” and “Opinions on the student evaluation process”. At the beginning of the interview form, preliminary questions were included to obtain the demographic information of the experts who participated in filling out the form.

2.5.8. Students’ Compositions

In the needs analysis stage of our study, a digital citizenship composition study was conducted to determine the cognitive and affective situation of the teaching candidates according to the digital citizenship and digital citizenship sub-dimensions. The teacher candidates were asked two open-ended questions for the composition study. The first question was, “What do you think digital citizenship is?”; the second question was, “What skills are required to be a digital citizen? and can you give an example?”. Expert opinions were also consulted in preparing these questions.

2.5.9. Digital Citizenship Document Analysis

The document analysis method was used to examine other digital citizenship studies in the literature during the needs analysis stage of the study. The scanning process was limited to articles, reports and books. Two keywords were determined to find the relevant documents: “Digital citizenship dimensions” and “Digital citizenship education”. After these documents were obtained, they were recorded within a certain systematic framework. A total of 28 documents were found to be worthy of review. When these documents are examined in more detail: five documents were found between 2007 and 2009, and between 2010 and 2012; nine documents were found between 2013 and 2016; nine documents were found between 2019 and 2021; thirteen documents were of a general report type; seven documents were books; five documents were academic articles; one document was a manual type (hand book); one documents were of a thesis type; and one document was an agreement type.

2.6. Analysis and Interpretation of Data

2.6.1. Analysis of Quantitative Data

The analysis of the data collected with the digital citizenship scale and the digital citizenship attitude scale was conducted firstly with normality tests. Skewness, kurtosis and graphical normality plots were conducted to evaluate the normal distribution. Reliability calculations were made, and the Cronbach alpha coefficient was found to be at a reliable level. As the results indicated that the data were not distributed normally, it was decided that parametric tests would be used. Parametric statistics are based on the assumption of normality. Recent findings suggest that type-I error and power can be adversely affected when data are non-normal [63].

The research data were transferred to the IBM SPSS Statistics 25.0 programme and analysed. Frequency distributions and percentage calculations were used for the personal characteristics of the participants and other categorical variables.

The paired sample t-test was used to find the differences before and after the experimental study. With this method, the difference between each student’s attitude and behaviour level before the application and their level after the application was revealed. The corresponding case groups or individuals, or the average measurement from only one group is compared [64]. When comparing the pre-test and post-test mean scores of the experimental group, a significance level of 0.05 was found. Büyüköztürk [65] considers it sufficient to set the statistical significance level to 0.05 in social sciences. Cohen’s d value was calculated for the effect level of the significant difference revealed in the paired sample t-test analysis. The effect size (Cohen’s d) was calculated as the difference between the means of two groups divided by the weighted pooled standard deviations of these groups [66]. An effect size above 0.8 is considered high. Cohen’s d potency was calculated with G-Power 3.1 software, a free software developed by Faul et al. [67]. All analysed data are presented in the Findings section in the form of tables.

2.6.2. Document Analysis

The data obtained from the document review of this study were analysed with the descriptive analysis method. The descriptive analysis method aims to determine the tendencies about a topic [68]. Descriptive analysis is used to process data that do not require in-depth analysis and reveals introductory findings. It is frequently used in reviewing documents [48]. We staged our document analysis as follows [69]: (1) finding appropriate documents, (2) checking the originality of documents, (3) creating a systematic for coding and cataloguing the documents, and (4) performing the data analysis. In this research, within the scope of this systematic process, documents were recorded in a computer environment. Then, each document was examined in detail. It was decided that 28 documents could be used within the framework of this research. Then, the coding process started. This process was carried out using the Microsoft Excel programme. The documents were described according to who they were reported by, on what dates, and what their target group was. In accordance with the descriptive analysis process, documents were analysed according to predetermined codes and themes. The data obtained were noted and presented in the framework of the digital citizenship and digital citizenship subheading themes defined by Ribble and Park [27].

2.6.3. Analysis of Qualitative Data

In this research, the compositions prepared by the students, the interviews with the experts, the students’ self-evaluations and the views on the programme after the application were all analysed via content analysis. The content analysis method is used to conceptualise and organise data obtained in qualitative research and to determine the themes that explain these data [48]. In content analyses, firstly, the collected data are organized (a computer environment can be used in this process); separation into codes and/or themes is performed; and finally, the findings are explained and reported [70]. To ensure reliability, the two researchers involved in this study coded the collected data separately. The researchers then compared the resulting codes. As a result of the evaluation, Miles and Huberman’s [71] security level formula was used to determine the reliability of the codes (reliability = agreement/consensus + disagreement). The result was determined as 90%. A result of at least 70% is reported to be the reliability level [48]. It was hence concluded that the codes of this study were reliable. The findings were reported by giving the frequency (f) and percentage (%) of code repetition.

2.6.4. Validity and Reliability Study in Qualitative Analysis

Validity and reliability are especially important in qualitative research [72]. For the research to be credible, internal validity, external validity, reliability and objectivity criteria must be met [73]. In this context, the processing steps shown in

Table 3. Reliability and validity process.

Factor	Process
Internal validity	Long-term interaction with the data is provided. Confirmation of the data was obtained from the participants. Consistency is ensured. Themes were determined according to the conceptual framework. The findings are given under these themes.
External validity	The transferability of the findings is ensured by direct quotations. Exhibitor information is given in detail. With the purposeful sampling method, the participants were selected according to the phenomenon.
Reliability	Two researchers conducted the analysis at various times and places. The findings were reviewed by an impartial person after they were made available. The research method was explained in detail.
Impartiality	In the analysis and interpretation process, the researcher was free from prejudices. This was confirmed by an impartial person. An unbiased expert compared the results and comments. The researcher explained their role in the process.

were applied.

3. Findings

3.1. Needs for the Development of Digital Citizenship Curriculum

Under this heading, the students’ digital citizenship skills, expert opinions on the needs of the digital citizenship curriculum, and document analysis findings to support the content of the developed digital citizenship curriculum are included.

3.2. Students’ Digital Citizenship Skills

The findings regarding the question, “How are students performing their digital citizenship skills?” are given below. Content analysis was conducted on the information collected for this research question. After the content analysis, main categories were created.

When

Table 4. Student composition analysis on digital citizenship and its features.

Category	(f)	(%)
Ability to use technology (technological tools)	8	26.66
Ability to use the internet and technology	5	16.66
Ability to use technology and social media	5	16.66
Ability to use the internet	4	13.33
I do not know anything	3	9.99
Safe internet use	2	6.66
Awareness of rights and responsibilities	2	6.66
Ethics and safety	1	3.33

is examined, it is seen that the general knowledge that the student candidates possess about digital citizenship and its characteristics is weak. According to the literature, digital citizenship is examined under nine headings. When students’ compositions are evaluated within the scope of the literature, it is understood from the compositions that few students touched on the issues of Digital Security, Rights and Responsibilities (6.66%) and Ethics (3.33%), and that knowledge in these areas was superficial. In the compositions written by the students, it is seen that the ability to use technology (26.66%) was mentioned the most. In this context, the weaknesses of students regarding their attitude and behaviours towards digital citizenship concepts were revealed. Some direct quotes are shown below.

(S1) “We can give examples of people called YouTubers today as digital citizens.”

(S3) “We call people who can actively use technological devices digital citizens. In my opinion, being a digital citizen requires having a technological device.”

(S7) “Digital citizenship means acting in social media channels in accordance with general citizenship rules, without harming other people.”

(S23) “These people have high knowledge of digital hardware. Be aware of digital rights and responsibilities, know and understand digital laws. We can all be digital citizens.”

3.3. Opinions of Field Experts on the Needs of the Digital Citizenship Curriculum

When

Table 5. Analysis of field experts’ needs for the digital citizenship curriculum.

Category	Codes	(f)	(%)
General views on the importance of the concept of digital citizenship today	Ability to use technology	4	33.3
	Adapting to digital transformation	2	16.7
	Current social challenges (the pandemic, disabilities, immigration)	2	16.7
	Professional job skills	2	16.7
	Awareness of rights and responsibilities	1	8.3
	Security needs	1	8.3
Opinions on whether there is a need to develop an undergraduate teaching programme on digital citizenship	Digital citizenship education is needed	12	100.0
	There is no need for digital citizenship education	0	00.0
Views on the possible impact of an undergraduate programme on digital citizenship	Affects job/occupational skills	5	41.7
	Affects the correct use of technology	4	33.3
	Affects lifelong learning	2	16.7
	Provides future-oriented development	1	8.3
Opinions on what the achievements of the curriculum should be	Improvements across all sub-dimensions	12	100
	Outcomes in the context of additional topics containing sub-dimensions	5	41.7
	Technology use skill gains	3	25.0
Opinions on what the learning–teaching strategy for the curriculum should be	Research study path (group study)	5	41.7
	Presentation path (project-based)	6	50.0
	Invention discovery (learning by doing and experiencing)	1	8.3
	Invention discovery (scenario-based)	1	8.3
Opinions on what the teaching activities should be	Project	2	16.7
	Scenario	2	16.7
	Case study	3	25.0
	Group work/collaborative studies	5	41.7
Opinions on the learning–teaching material suggestions that can be used in transferring the content	Technological devices	6	50.0
	Digital content	4	33.3
Opinions on the student evaluation process	Process-oriented evaluation	1	8.3
	Exam and process evaluation	3	25.0
	Project and presentation performance evaluation	4	33.3
	Evaluating qualitative and quantitative outcomes	4	33.3

, which contains the general views of experts about the importance of the digital citizenship concept, is examined, one can see that it is important for technological and digital transformation to use technology in a more conscious manner.

Their general views on the necessary behaviours towards the concepts specified in order to become a digital citizen were expressed as the ability to use technology and adaptability in the context of digital transformation. They stated their views on the fact that responsibility and awareness towards digital technology and its sufficient usage can be achieved through technology usage skills and adaptation to the digital age.

Their opinions on whether their students exhibit behaviours of digital citizenship indicated insufficient skills in the context of technology use and misuse in relation to rights and responsibilities. The views of experts on whether there is a need to develop the digital citizenship teaching programme at an undergraduate level showed that there is a need on a public basis and that the use of technology must become more common. Experts stated that they believed that an undergraduate-level course on digital citizenship would positively affect the professional competencies of students.

When the opinions of the experts on which digital citizenship sub-dimensions should be included in the curriculum to be developed at the higher education (university) level were examined, they agreed that all sub-dimensions should be included in the content.

When the opinions of the experts on what the outcomes of the teaching programme should be were examined, it was found that the skills required within the framework of the sub-dimensions of digital citizenship in the literature, the skills needed to produce information in the context of digital transformation, the necessary skills in the context of technology use, the skills to process information, and the skills pertaining to explaining rights and responsibilities must be included.

When their views on how the teaching–learning strategy should be developed were examined, they stated that the presentation strategy should include project-based learning, the research/investigation strategy should include group studies and that the discovery strategy should include scenarios and application.

When their views on teaching activities were examined, the need for group studies and sample scenarios was stated.

When their views on learning–teaching material suggestions were obtained, it was stated that portable laptops, tablets and other devices must be used, while animations, slideshows and video materials must also be used as visuals.

When their opinions about the student evaluation process were examined, it was stated that students’ achievements should be measured with qualitative and quantitative methods.

3.4. Findings Regarding the Views of Lecturers on the Digital Citizenship Curriculum

The findings regarding the question, “What are the lecturers’ opinions on why and how the digital citizenship curriculum should be developed?” are given below. A content analysis was conducted on the information collected for this research question. After the content analysis, the main categories were created.

When

Table 6. Analysis of the views of lecturers on the digital citizenship curriculum.

Category	Codes	(f)	(%)
Distribution of the answers of the lecturers on the importance of the concept of digital citizenship today	Correct behaviour in digital environments	3	30.0
	Adapting to the digital age	3	30.0
	Forward thinking	2	20.0
	Professional business skills	2	20.0
Distribution of the responses of the academic staff on the status of university students having digital citizenship behaviours	They do not have digital citizenship behaviours	9	90.0
	They have digital citizenship behaviours	1	10.0
Distribution of the answers to the question, “Is there a need to develop an undergraduate-level curriculum on digital citizenship for faculty members?”	Digital citizenship education is needed	10	100.0
	There is no need for digital citizenship education	0	00.0
Distribution of the answers to the question, “When you look at the digital citizenship sub-dimensions given by academic staff, which should be included in the curriculum?”	It is appropriate to include all of them in the order given	8	80.0
	Digital Health, Digital Security, Digital Literacy, Digital Communication, Digital Access, Digital Commerce, Digital Law, Digital Rights and Responsibilities	1	10.0
	Digital Access, Digital Commerce, Digital Communication, Digital Literacy, Digital Health	1	10.0

, which contains the general views of lecturers about the importance of the concept of digital citizenship, is examined, one can see that the digital citizenship concept is perceived by lecturers to mean learning the correct behaviours in digital platforms and adapting to the digital age. When examining the answers of university students to the question of whether or not they exhibited behaviours of digital citizenship, they stated that they did not have digital citizenship behaviours. When examining their answers to the question, “Is there a need for a digital citizenship curriculum to be developed?”, it was found that they stated that there is a need for a digital teaching programme to be developed. When their replies to the question, “Which of the subgroups of digital citizenship do you think should be included in this teaching programme?”, were examined, the majority held the view that all of them should be included.

3.5. Findings Regarding the Document Analysis for the Digital Citizenship Curriculum

The findings regarding the question, “What do the findings of the document analysis suggest for improving the content of the digital citizenship curriculum?” are given below. A descriptive analysis was conducted on the information collected for this research question. After the content analysis, the main categories were created. The tables were examined by keeping in mind the questions and their order in the document analysis process.

When

Table 7. Document analysis for improving the content of the digital citizenship curriculum.

Document Code	Digital Access	Digital Commerce	Digital Communication	Digital Literacy	Digital Ethics	Digital Law	Digital Rights and Responsibilities	Digital Health	Digital Security
D1	x		x	x	x		x	x	x
D2	x	x	x	x	x	x	x	x	x
D3	x	x	x	x	x	x	x	x	x
D4	x						x		x
D5	x	x	x	x	x	x	x	x	x
D6	x		x	x	x		x	x	x
D7	x	x	x	x	x	x	x	x	x
D8	x			x
D9	x			x
D10	x		x	x	x	x	x	x	x
D11	x	x	x	x	x	x	x	x	x
D12	x
D13	x		x				x
D14	x	x	x	x	x	x	x	x	x
D15	x	x	x	x	x	x	x	x	x
D16	x	x	x	x	x	x	x	x	x
D17	x	x	x	x	x	x	x	x	x
D18	x	x	x	x	x	x	x	x	x
D19	x	x	x	x	x	x	x	x	x
D20	x	x	x	x	x	x	x	x	x
D21	x	x	x	x	x	x	x	x	x
D22	x	x	x	x	x	x	x	x	x
D23	x			x			x		x
D24	x	x		x	x		x	x	x
D25	x	x	x		x	x	x
D26		x	x		x	x	x
D27	x	x	x	x	x	x	x	x	x
D28	x	x	x	x	x	x	x	x	x

x: It indicates that there is information about the sub-dimension of interest.

is examined, it can be seen that 28 documents were evaluated. Digital citizenship is discussed in nine dimensions across sixteen documents. As a result of the document analysis, it is seen that all nine dimensions of digital citizenship are important. According to this finding, nine dimensions should be included when developing the higher education digital citizenship curriculum.

3.6. Efficiency of the Developed Digital Citizenship Curriculum

The findings regarding the questions, “What are the students’ digital citizenship attitude levels before and after the experimental application?” and “What are the students’ digital citizenship behaviour levels before and after the experimental application?” are given below.

3.6.1. Pre-Test and Post-Test Findings of the Digital Citizenship Attitudes Scale for Teacher Candidates

When the pre-test and post-test paired sample t-test analysis results of the digital citizenship attitude scale for the teacher candidates are examined (t = 11.69, p = 0.000, p < 0.05), a significantly significant difference is observed in favour of the post-test scores. The paired groups’ t-test formula was calculated by taking into account the arithmetic mean and standard deviation values (according to a margin of error of 0.05). Considering the number of samples and df value here, the difference can be interpreted as significant (critical t value = 11.69). The significance of the resulting difference was found to be strong (impact level = 2.50). Also, it is seen that the post-test ($\overline{\mathrm{X}}$ = 3.93) mean score is higher than the pre-test ($\overline{\mathrm{X}}$ = 3.08) mean score. Based on this finding, it can be said that there was a positive meaningful change in the digital citizenship attitude of the teacher candidates. As can be seen from

Table 8. Pre-test and post-test paired sample t-test analysis results for the teachers’ digital citizenship attitude scale.

	N	$\overline{\mathbf{X}}$	Ss	Sh	t	Sd	p	Significance Level	Impact Level—Cohen’s d
Pre-Test	39	3.08	0.21	0.03	11.69	38	0.000	0.05	2.50
Post-Test	39	3.93	0.37	0.06

N: participants; $\overline{\mathrm{X}}$: mean; Ss: std. deviation; Sh: std. error mean; t: t-test significance level; Sd: df.

, it is seen that the digital citizenship curriculum provides a change in students’ attitudes.

3.6.2. Pre-Test and Post-Test Findings on the Digital Citizenship Behaviour of Teacher Candidates

When the digital citizenship behaviour scale’s pre- and post-paired sample t-test analysis results for the teacher candidates are examined (t = 12.70, p = 0.000, p < 0.05), a significantly significant difference in favour of the post-test scores is observed. The paired groups’ t-test formula was calculated using the arithmetic mean and standard deviation values (according to a margin of error of 0.05). Considering the number of samples and df value here, the difference can be interpreted as significant (critical t value = 12.70). The significance of the resulting difference was found to be strong (impact level = 3.11). Also, it is seen that the post-test ($\overline{\mathrm{X}}$ = 4.18) mean score is higher than the pre-test ($\overline{\mathrm{X}}$ = 2.96) mean score. In light of this finding, it can be said that there were positively significant changes in the digital citizenship behaviours of teacher candidates. As can be seen from

Table 9. Pre-test and post-test paired sample t-test analysis results for the teachers’ digital citizenship behaviour scale.

	N	$\overline{\mathbf{X}}$	Ss	Sh	t	Sd	p	Significance Level	Impact level—Cohen’s d
Pre-Test	39	2.96	0.43	0.06	12.70	38	0.000	0.05	3.11
Post-Test	39	4.18	0.32	0.05

N: participants; $\overline{\mathrm{X}}$: mean; Ss: std. deviation; Sh: std. error mean; t: t-test significance level; Sd: df.

, it is found that the digital citizenship curriculum provides changes in the students’ behaviours.

3.7. Self-Evaluation Findings of Teacher Candidates during the Digital Citizenship Course Process

The findings of the question, “What are the students’ self-evaluations during the implementation process of the developed digital citizenship curriculum?” are given below.

In

Table 10. Analysis of pre-service teachers’ weekly self-assessment forms.

Course	Degree	(%)
Course 1, Introduction to Digital Citizenship	Good	7.50
Course 2, Digital Access	Very good	8.31
Course 3, Digital Commerce	Very good	8.01
Course 4, Digital Communication	Very good	8.26
Course 5, Digital Literacy	Very good	8.34
Course 6, Digital Ethics	Very good	8.00
Course 7, Digital Law	Very good	7.91
Course 8, Digital Rights and Responsibilities	Very good	8.26
Course 9, Digital Health	Very good	8.06
Course 10, Digital Security	Very good	8.58

, the answers that teacher candidates gave to the self-evaluation forms have been examined on a weekly basis and scored using the rubric score method. Four criteria have been specified for scoring. In order to be able to calculate out of 10 points, each criterion was determined as 2.5 points and a total of 10 points were given per week, and the general average of the classroom was determined. The scores were separated into four worded categories in multiples of 2.5: 1. Bad (0–2.5), 2. Moderate (2.5–5.0), 3. Good (5.0–7.5), and 4. Very Good (7.5–10.0). Generally, when the table is examined, it can be seen that the self-evaluation scores are high and positive, and they stated that they had learned the topic in a “Very Good” manner at the end of each course. According to the students’ self-evaluation after the implementation of the curriculum, we can say that their digital citizenship skills are positive.

3.8. Views of Teacher Candidates towards Digital Citizenship Course Application

The findings of the question, “What are student opinions about the implemented digital citizenship course?” are given below.

When the outcomes of the digital citizenship course for the teacher candidates’ are considered and their replies to the question, “Do you define yourself as a digital citizen?” are examined, most of them (n = 32) stated that they think they do exhibit digital citizenship behaviours. After the experimental study, a small number of participants (n = 6) thought that they partially possessed digital citizenship behaviours. When the findings related to the question, “When you think about the digital citizenship sub-dimensions of the teacher candidates, which sub-dimension did you find most interesting and useful?” are examined, it can be seen that Digital Security (n = 9) is the sub-dimension in which they were most interested. The least interesting sub-dimensions to them were the Digital Rights and Responsibilities and Digital Access dimensions (n = 2). Additionally, there were people who found all dimensions useful (n = 5). When they were asked, “What are your thoughts on the scenario-based teaching approach/activity applied in the teacher candidates’ digital citizenship course?”, they answered that it enabled efficient learning (n = 9). When asked, “Did the digital citizenship teaching programme ensure that you actively participated in the learning–teaching stages of the course? Why?”, they said yes, and gave the reason that the active participation (n = 13) and scenario-based learning method was effective for this. When their answers to the question, “How do you find that the digital citizenship curriculum allows you to evaluate yourself with self-evaluation forms?” were examined, they stated that this method provided academic success and enabled them to see their own shortcomings. When their answers to the question, “Do you think that all teacher candidates should take the digital citizenship course as a regular course in the future? Why?” were examined, they said yes, stating their reason to be the necessity of correct behaviour in digital environments and the necessity to adapt to the digital age.

As can be seen from

Table 11. Analysis of teacher candidates’ views on the digital citizenship course.

Dimension	Category		(f)	(%)
Findings related to the question, “Considering the achievements of the teacher candidates’ digital citizenship course, do you define yourself as a digital citizen?”	Thinking you have digital citizenship behaviours		32	82.05
	Thinking you partially possess digital citizenship behaviours		6	15.40
	Thinking you do not have digital citizenship behaviours		1	02.05
Findings related to the question, “When you think about the digital citizenship sub-dimensions of the teacher candidates, which sub-dimension did you find most interesting and useful?”	Digital Security		9	23.07
	Digital Commerce		5	12.82
	Digital Law		5	12.82
	Digital Communication		5	12.82
	Digital Health		3	07.69
	Digital Literacy		3	07.69
	Digital Access		2	05.12
	Digital Rights and Responsibilities		2	05.12
	All		5	12.82
Findings related to the question, “What are your thoughts on the scenario-based teaching approach/activity applied in the teacher candidates’ digital citizenship course?”	Effective learning		9	23.07
	Learning by living		5	12.82
	Quick learning		5	12.82
	Active participation		4	10.25
	Permanent learning		3	07.07
	Thinking skills		3	07.07
	Common learning environment		1	02.05
	Effective learning		9	23.07
Findings related to the question, “Did the learning–teaching process of the teacher candidates’ digital citizenship course curriculum enable you to actively participate in the course?”	Answer	Why?	(f)	(%)
	Yes	Active participation	13	33.33
	Yes	Scenario-based learning	8	20.51
	Yes	Self-assessment	8	20.51
	No	-	0	0
Findings related to the question, “Do you think that all prospective teacher candidates should take the digital citizenship course as a regular course in the future?”	Answer	Why?	(f)	(%)
	Yes	Correct behaviour in digital environments	15	80.00
	Yes	Adapting to the digital age	13	33.33
	Yes	Forward thinking	8	20.51
	Yes	Professional business skills	3	07.07
	No	-	00	00.00

, it is seen that the students generally expressed positive views towards the digital citizenship course implemented and its process.

4. Discussion, Conclusions and Recommendations

This study aimed to develop a digital citizenship curriculum and evaluate its efficiency. Firstly, the need for a digital citizenship curriculum was proposed. Then, an experimental study was planned and applied in an undergraduate level class for 10 weeks. In order to determine the efficiency of the programme, the participants, who were teacher candidates, were given interviews, self-evaluation forms and pre-test–post-test attitude and behaviour scales. As a result of this, it was determined that this undergraduate-level digital citizenship teaching programme is effective.

With the first research question of this study, it was revealed that undergraduate students’ digital citizenship skills and attitudes were low. When the students’ compositions were evaluated within the scope of the literature, it was understood that only very few students had knowledge about Digital Security and Rights and Responsibilities, and had no idea about other citizenship skills. In addition, experts’ opinions have shown that students’ digital citizenship skills need to be improved. The study conducted by Sincar [74] examined the digital citizenship levels of students and discovered that their digital citizenship behaviours were poor. According to Kocadağ [75], teachers having skills of digital citizenship will affect the future generations and ensure that students gain digital citizenship perception. This research is supported by the findings of current research. Logan [76] examined how digital citizenship relates to 21st century skills; their research revealed that only 32% of the sample group of 88 participants had sufficient knowledge about digital citizenship. Ata and Yıldırım [77] stated that prospective teachers’ perceptions of digital citizenship, especially digital rights, responsibilities and ethics, could be further investigated. Ji-eun and Kyung-ah [78] determined the digital citizenship education needs of university students in Korea. The results of their needs analysis of digital citizenship education showed that ‘balanced use of digital technology’, ‘digital security’, ‘digital rights and responsibilities’, and ‘digital literacy and critical thinking’ were of the highest priority, and that ‘social engagement through digital technology’ was a secondary priority.

Documents were examined during the needs analysis phase of our research. With a detailed literature review, it was concluded that digital citizenship education should be given in nine dimensions and the training programme should be developed within the framework of nine dimensions. Logan’s [76] digital citizenship program that addresses access, commerce, communications, etiquette, law, health, wellness, rights, and responsibilities is recommended to support children online. Armfield and Blocher [79] explored the details of the redesign of a digital citizenship course unit for pre-service teachers. The blog publishing process of teacher candidates was managed. When the research results were examined, it was revealed that content should be developed on the basis of fair access, global, universal, cultural understanding, and being technological safe, healthy, legal, ethical and responsible. Sarı [80] stated that the digital citizenship teaching programme should be regulated, that the knowledge, perceptions and experience of all educational stakeholders should be benefited from, and that digital citizenship should be taught as a course. Kilci [81] stated that students need to gain all digital citizenship skills. Ji-eun and Kyung-ah [78], in their research, stated that in the age of great digital transformation, there should be an education plan to improve the digital citizenship competencies of university students.

In the third and most important research question of this study, the effectiveness of the developed digital citizenship curriculum was examined. The digital citizenship curriculum developed for university students was assessed with an experimental study. The results were in favour of the post-test scores and showed that the students’ initial skills and attitudes changed positively after the application. This result shows that the developed programme can improve the digital citizenship skills and attitudes of university students. In addition, student self-evaluations and opinions about the digital citizenship programme, discussed in the third and fourth research questions of this study, also revealed that this programme is effective. Hui [82], in his research, investigated the teaching of digital citizenship in order to support digital citizenship. After the application, he obtained anecdotes showing the positive results of the course presentation. Overall, this course was well received by students. Haskey, Gould and Von Gillern [83] determined in their study that when student candidates have digital citizenship skills, they become individuals who can create knowledge and use this knowledge for the benefit of their society. Akcil and Baştaş [84], who examined digital citizenship behaviours throughout the COVID-19 pandemic, stated that individuals with digital citizenship skills have a more positive attitude to education teaching processes in comparison with those who do not have these skills. Neyişci and Sarı [85], who conducted a similar study, emphasised that digital citizenship skills had positive effects on the learning process of students during the COVID-19 pandemic. Ribble et al. [20] stated that digital citizenship has become a priority in schools that see its importance in both education and professional life after education in the 21st century.

It is an important fact that the digital citizenship curriculum developed with this research will make significant contributions to both a student’s educational process and the process of preparing them for a profession. Many studies in the literature also support this statement. Lindsey [86] mentioned that teacher candidates must be educated in the field of digital citizenship in order to become better role models. Choi, Cristol and Gimbert [87] stated that after becoming accustomed to technology, teachers’ technological knowledge and skills significantly influenced their views and attitudes towards digital citizenship. Von Gillern, Gleason and Hutchison [88] stated that it is critical for teachers to have digital citizenship skills and reflect these skills to their students in order to ensure they become digital citizens. With their study conducted during the COVID-19 pandemic, which is a current universal issue, Capuno et al. [89], when talking about the positive effects of digital citizenship skills, emphasised the importance of digital citizenship education, particularly as students are faced with significant risks on digital platforms. Palmberger [90] emphasised the importance of digital citizenship skills and the need for education in this field for the education of refugees and the process of their adaptation, which is also a current global issue. In another study, Karaduman and Öztürk [91] stated that digital citizenship education will positively develop the approaches of students. Öztürk [28], in the study titled “Digital citizenship teaching: a literature review”, stated that in the revised curriculum, there were no courses completely focused on teaching digital citizenship and that digital citizenship was mentioned in some of the courses of computer and teaching technologies departments. It can be said that this study has provided a digital citizenship teaching programme for both the literature and for undergraduate-level programmes.

When the results of this study are examined, based on our finding that the applied digital citizenship programme is effective, it is recommended that it be used for undergraduate-level students and in teacher training programmes as a separate course. Especially when evaluated within the framework of the participants in this study, it is thought that the digital citizenship curriculum is important for teacher candidates.

5. Limitations

This study has several limitations. First, a random selection of participants would increase the validity of the study. This was not possible due to the eight-term curriculum regulations of the higher education institution in which the application was made, and the students who could participate in the application period were limited.

Second, this study had only an intervention group and no control group because only a small number of student groups (n = 39) were able to participate in this study. A significant difference emerged between the pre-test and post-test scores of the experimental group’s digital citizenship attitudes and behaviours. The effect size of the resulting difference was calculated and found to be high. However, when only the average scores obtained from the attitude scale were examined, there was no significant difference in the scores.

As a promising result, this study provides insights into the potential of digital citizenship education to improve participants’ behaviours and attitudes. This developed programme can be applied as is, but due to the constant advancement and effects of technology, the programme must be constantly updated. Additionally, it should not be forgotten that the conditions of each country and the demands of the workforce shape the studying of this programme.