Digital Competence Assessment Methods in Higher Education: A Systematic Literature Review

: The rapid increase in recent years in the number of different digital competency frameworks, models, and strategies has prompted an increasing popularity for making the argument in favor of the need to evaluate and assess digital competence. To support the process of digital competence assessment, it is consequently necessary to understand the different approaches and methods. This paper carries out a systematic literature review and includes an analysis of the existing proposals and conceptions of digital competence assessment processes and methods in higher education, with the aim of better understanding the ﬁeld of research. The review follows three objectives: (i) describe the characteristics of digital competence assessment processes and methods in higher education; (ii) provide an overview of current trends; and, ﬁnally, (iii) identify challenges and issues in digital competence assessment in higher education with a focus on the reliability and validity of the proposed methods. On the basis of the ﬁndings, and as a result of the COVID-19 pandemic, digital competence assessment in higher education requires more attention, with a speciﬁc focus on instrument validity and reliability. Furthermore, it will be of great importance to further investigate the use of assessment tools to support systematic digital competence assessment processes. The analysis includes possible opportunities and ideas for future lines of work in digital competence evaluation in higher education. a design was used in and for


Introduction
Digital technologies have brought changes in, and, thus, challenges to, our everyday life, in which the use of technology is inevitable. Therefore, the concept of digital competence has become a central part of our lives, and this set of skills has quickly risen to become a key area of competence when dealing with different policy-related documents, and is a clear topic of focus in European policies. It can be assumed that the concept of digital competence has become a focus of policy because of its future-oriented nature: it comprises skills that are expected from a workforce that, in the future, will need to operate effectively in a knowledge-intensive society. Thus, when talking about supporting the development of such skills, it is also important to find ways to evaluate digital competence.
The situation today is similar to that described by Bawden [1], who already stated more than 15 years ago, in 2001, that the literature regarding digital skills is not consistent with respect to the terms used or the underlying concepts. When talking about digital competence, we use a variety of similar concepts: digital skills, digital literacy, media literacy, information literacy, transversal skills, new media literacy, e-skills, e-competences, and in some cases digital intelligence. Often, these concepts are discussed as being 21st-century skills. Furthermore, while the concept of digital competence was once considered mainly to comprise skills related to computer use, today the concept refers to a wider sense of knowledge, skills, and attitudes that are largely affected by the labor market. As we move into more comprehensive work situations that include open learning spaces, informal learning and working situations, and a greater amount of interactive technologies, digital 2 of 13 competence is taking on new dimensions, becoming more context specific, as well as more conceptually intricate. Following on from this, different digital competency models and frameworks have been developed for different target groups. Most notably, in Europe, these include the DigComp Frameworks for Citizens [2], which was first published in 2013 and last renewed in 2017, while in the United States, these include the ISTE [3] digital competency strategy for teachers and the NETS model for students' digital skills. Additionally, the UNESCO ICT Competency Framework for Teachers [4] aims to help countries to develop comprehensive policies and standards for teachers, while the DigCompEdu [5] framework was published late 2017 with the aim of creating standards for evaluating the digital competence of educators in Europe. The rapid increase in the number of different digital competency frameworks, models, and strategies has profoundly shifted the focus from the measurement aspects and operational interpretations of digital competence towards aspects related to definitions, indicators, and indexes [6].
In addition, among the various concepts of digital competence and the frameworks that set out the initial scales for their measurement, one of the most complex topics is how to assess such future skills, when digital competence is considered to comprise essential future skills at a policy level in knowledge-intensive societies. The evaluation of digital skills has proven to be a real challenge, and existing systems have failed to initiate effective and systematic processes [7]. The aforementioned arguments frame the lack of and need for systematic digital competency assessment methods and, furthermore, tools that can be implemented in consideration of the particular field of expertise. Several attempts have been made to assess and measure digital competence, but an overview of the different possibilities with their strengths and weaknesses is lacking. At the same time, such an overview is needed to guide the various policy-level initiatives and research groups aiming to assess such competence, but who are struggling with similar issues: what instruments are less time-consuming, give enough evidence, are based on real-life situations and tasks, and, at the same time, are valid and reliable.
In light of the statements mentioned and the small number of previous literature reviews focusing on the assessment of digital competence in the context of higher education, including the students and academic staff within the sample, the following gives a brief overview of various definitions of digital competence and equivalent terms. The large number of different attitudes, values and approaches to digital competence provides us with a plethora of similar concepts [1]. Based on DigComp 2.1, The Digital Competence Framework for Citizens [2], which includes eight proficiency levels and examples of their use, includes the following competence areas: information and data literacy; communication and collaboration; problem solving; content creation; and safety.
Ala-Mutka [8] presented in her research, which might be considered as the first holistic visual representation of digital literacies in the context of 21st century skills in the age of knowledge society. The 21st century skills and digital competence are seen as important future skills in a knowledge-intensive society, although the relationship between those concepts is unclear. Van Laar et al. [9] have proposed using the term 21st century digital skills, which are critical for both people and organizations for keeping up with developments and innovating products and processes. However, these skills go beyond the technical annotation and have a greater impact on a person's ability to function in a technologically rich society than just being knowledgeable about specific software. Although technologies are the foundation of innovation, technology does not create a knowledge-based innovation-people do that by making human capital within the workforce decisive, which means that workplaces in the future demand a workforce with 21st century skills: collaboration, communication, digital literacy, citizenship, problem solving, critical thinking, creativity, and productivity [10]. In the current fast-changing knowledge economy, 21st century digital skills drive organizations' competitiveness and innovation capacity [9].
The ongoing COVID-19 pandemic has highlighted the importance of digital competence in implementing technology enhanced learning practices in all levels of education.
Zhao, Pinto Llorente, and Sanchez Gomez [11] stated in their literature review that digital competence perceptions vary, which is evident in their level of digital competence based on self-assessment.
The paper provides an overview of the field of digital competence assessment methods, instruments and tools in higher education using a systematic literature review of digital competency assessment practices and conceptions focusing on the instruments used for digital competence assessment. Bearing in mind the previous, the review focuses on three main goals. The first goal is to (i) describe the characteristics of digital competence assessment processes and methods in higher education. The second aim of the review is to (ii) provide an overview of the current trends. Finally, the review aims to (iii) identify the challenges and issues in digital competence assessment in higher education focusing on the reliability and validity of the proposed methods. The review concludes with possible opportunities and ideas for future lines of work in digital competence evaluation, mainly in higher education.

Systematic Review Methodology
The current literature review follows the principles of a systematic literature review presented by Siddiq, Hatlevik, Olsen, Throndsen, and Scherer [12] where a proposed predefined procedure by Gough, Oliver, and Thomas [13] and Petticrew and Roberts [14] was used. The review lies on the principles of The Cochrane Community who have defined the systematic literature review as collecting empirical evidence that presents itself coherent with the predetermined eligibility criteria in order to provide a specific answer to a research question [15]. The literature review focuses on minimizing the possibility of bias in research as the conclusions and decisions are drawn from a vast amount of diverse research findings. With the aim of analyzing evidence on the evaluation of digital competences in higher education the review includes the formulation of research questions, research and assessment of literature based on the criteria, description of the analysis and assessment of the literature suitability.

Search Procedures
The review focuses on higher education with the aim of understanding and describing the digital competence assessment processes and implications. When describing the topic of digital competence, a variety of notions is used. For a clear overview of the field a set of synonyms and alternative phrases were used in the literature search, resulting in using the following terms: Digital competence: Digital competency, ICT literacy, digital literacy, ICT skills, digital skills, computer skills, technology literacies, digital competencies, 21st century skills; Measurement: Assessment, evaluation, testing, measuring, questionnaire.
The current research mainly focused on two lines of research. On one hand, it includes the description of the digital competence instrument and tool development process which includes a sample in the validation process. The second line of research focused specifically on the assessment of digital competence. Both lines of research were limited to higher education settings, thus giving an overview of the used methods, instruments, and tools implemented for digital competence assessment in adult learning.
The literature search analysis was carried out from March 2018 to January 2019. The search in the databases was restricted to years 2000-2018. The data were queried from five academic databases: IEEE Xplore, Science Direct, Springer Link, ACM Digital Library, and Web of Science. This selection of databases was used due to the large capacity of computer science and ICT topic research published. In addition, Google Scholar was used to find grey-literature and have an overall view of the topic of digital competency assessment in higher education. In addition to the database search, professional social networks, such as Academia.edu and ResearchGate, were analyzed for relevant articles. In the final step experts were included, who provided some extra studies carried out in the field.

Eligibility Criteria and Screening Process
Initial screening of the literature was done by reading the titles and abstracts. During the process all studies which included the context of higher education and presented digital competence assessment as the core topic were included in the next round of analysis. This was considered as the initial eligibility criteria. Before the next stage of the screening process the final eligibility criteria were set out.
The criteria include the following: • Research focused on digital competence assessment; • ICT literacy or other equally defined concept assessment in higher education; • The articles describe the development process of a digital competence (or equal concept) assessment tool and include a sample in the validation process; • The research method includes digital competence self-assessment or self-reporting; • Research which focuses on variety of interventions in technology enhanced learning and include digital competence assessment as a preliminary stage of the research.
In the elimination process the following criteria were used: • Insufficient or no reporting on the sample, digital competence assessment instrument or educational setting; • Characteristics of the used tool or instrument was not given.
The full screening process of the literature is presented in Figure 1, which shows that the initial search produced 6399 studies by screening the titles and abstracts. Relevant studies were selected based on the initial eligibility criteria. After the database query 316 papers were left for further review. Each of the 316 studies were analyzed based on the eligibility criteria and by reading the full text. All literature that focused on digital competency assessment in secondary or lower education was excluded. All together 274 publications were excluded based on the context of the research, used methodologies and research focus. The focus was on the studies that included a sample for validation in the research.
Academia.edu and ResearchGate, were analyzed for relevant articles. In the final step ex perts were included, who provided some extra studies carried out in the field.

Eligibility Criteria and Screening Process
Initial screening of the literature was done by reading the titles and abstracts. During the process all studies which included the context of higher education and presented dig ital competence assessment as the core topic were included in the next round of analysis This was considered as the initial eligibility criteria. Before the next stage of the screenin process the final eligibility criteria were set out.
The criteria include the following: • Research focused on digital competence assessment; • ICT literacy or other equally defined concept assessment in higher education; • The articles describe the development process of a digital competence (or equal con cept) assessment tool and include a sample in the validation process; • The research method includes digital competence self-assessment or self-reporting; • Research which focuses on variety of interventions in technology enhanced learning and include digital competence assessment as a preliminary stage of the research.
In the elimination process the following criteria were used: • Insufficient or no reporting on the sample, digital competence assessment instrumen or educational setting; • Characteristics of the used tool or instrument was not given.
The full screening process of the literature is presented in Figure 1, which shows tha the initial search produced 6399 studies by screening the titles and abstracts. Relevan studies were selected based on the initial eligibility criteria. After the database query 31 papers were left for further review. Each of the 316 studies were analyzed based on th eligibility criteria and by reading the full text. All literature that focused on digital com petency assessment in secondary or lower education was excluded. All together 274 pub lications were excluded based on the context of the research, used methodologies and re search focus. The focus was on the studies that included a sample for validation in th research.  Furthermore, as the same keywords were used in all five academic databases some of the results included duplicates. Finally, as the databases do not allow a detailed search of keywords only in the titles, in some cases it resulted in the low relevance of papers for the literature review based on the eligibility criteria. Consequently, the literature screening process revealed 40 suitable studies which make up the literature used in the systematic review. Based on the research objectives, a content analysis was carried out which supported the aim of providing an overview of the characteristics of digital competence evaluation in higher education by identifying the trends and challenges.

Results
The systematic review includes 40 of publications gathered from sources presented and visualized in the methodology chapter. The research includes both technological tools and instruments developed with the purpose of digital competence assessment. This includes fully developed platforms [16,17], digital competence scales [18,19], and tools developed within international projects [20].

RQ1-Characteristics of Digital Competence Assessment Processes and Methods in Higher Education
To give a clear overview of the characteristics of digital competence assessment process and methods in higher education we focused on defining the geographical mapping of the analyzed research. We also viewed other characteristics like the year of publication, number of respondents, publications, research methods, task design, and used competency frameworks or models. Table 1 gives an overview of the characteristics of the research included in the literature review. The final set of publications used in the research included studies from 25 different countries, of which four include a sample from two countries [9,[20][21][22]. The most represented county was Spain, with nine instances, which is affected by the national level policies which state the importance of supporting the development of digital competence in higher education. Similar conclusion can be made based on the articles published based on the sample from United States and Portugal. The remainder of the studies focus mainly on field-specific digital competence which again shows that there is a growing trend in describing and evaluating digital competence in a specific context. Note. NR-no recording. In the category Framework-the 'F' refers to an international framework used in the digital competence assessment process. The 'L' indicates that the research was based on a localized framework, model, curriculum or strategy; In the task/item design category-INT indicates tasks with interactive content, MC indicates multiple-choice tasks/items, MC-INT indicates that both multiple-choice and interactive content was used in the digital competence evaluation, and AUTH indicates that the evaluation process was carried out in completely authentic digital platform. In the tool category, the used technological solutions have been fully written out. This includes both already existing testing platforms and new developed tools for digital competence assessment. We also looked into the year of publication and the number of respondents to understand the scope of the research. The studies included were published between 2007 [23] and early 2019 [54] and sample sizes varied between 35 [24] and 35,000 subjects [22], covering both the academic staff of higher education, but also student teachers [54] and students from other fields [51]. In one case a sample size was not introduced [45] but was included in the study due to the use of a specially developed tool, which was also used in another study [17]. The majority of the research was published in 2018 (n = 13).
One of the objectives of the study was to understand what research methods were used in the included studies and what were the underpinnings of digital competence instrument task and item design.
Majority (n = 35) of the studies included in the review used quantitative methodology where either a self-assessment instrument or knowledge-based testing was implemented. The rest of the studies included in the research employed either a qualitative methodology (n = 3) or a mixed method (n = 2) design in the digital competence test validation and evaluation process. Sixteen of the analyzed studies include a multiple-choice test as an instrument for digital competency assessment and are categorized as MC (Table 1). In most cases, a simple computerized questionnaire was used (n = 14), but in two cases a digital competence assessment tool was developed [17,45]. Based on further investigation, it can be said that it was not an open-access tool. Fourteen of the studies included a multiplechoice and interactive task (MC-INT) in the competency evaluation instrument. This means that the participants had to interact with a variety of tasks to provide a holistic view of the level of digital competence. Four of these cases included a technological platform development as part of the research. The literature revealed that in the task and item design it is important to include multiple scales in item development to assess a variety of skills.
Studies introduced factor analysis as a basis for item development to prove the structural validity. The researchers included that although the participant number for factor analysis was sufficient it is not sufficient to use the same dataset for the confirmatory analysis and, thus, a replication of the study would be advised [40]. This was mentioned in multiple studies, which suggests that although in most cases the studies include a representative sample they do not include a test group for assessment instrument reliability testing. One of the papers implemented item response theory (IRT) to measure the ability of respondents to deal with the given information, interpret and act on it. Methodology for analyzing the validity of the items produced was also introduced. The results indicated that, in most cases, the items fall into three levels of difficulty (easy, medium, difficult) with the focus on medium level items. In the construction stage of the items they based on the conceptual model that included two main dimensions-cognitive and critical, and creative. Based on the used framework and conceptual model a matrix was produced to support the instrument design.
Finally, we analyzed the included research based on the used digital competence frameworks or alternative basis documents. Majority of the studies (n = 30) included a localized competency model as a competence instrument basis. The localized models were based on national curriculum, international frameworks, such as DigComp 2.0 and 2.1 by European Commission [2], or UNESCO ICT competency framework for teachers [4]. In four of the studies DigComp was used as the framework for evaluation instrument development [29,41], in all instances a digital competence tool development was not included. Two of the studies were based on the ISTE standard for teachers [16,25]. The use of alternative frameworks and models further shows that there is a lack of common understanding when describing digital competence, which often influences the instrument design and the digital competence assessment results. The analyzed research state that there is an opportunity to build a wider scale research on digital competences when basing the instrument design on a common framework. One of the options presented is DigCompEdu [5] which could be used in understanding educators. digital competence. Simultaneously, we can argue that localized competence models better describe the implementation context.

RQ2; RQ3-Overview of Current Trends and Challenges in Digital Competence Assessment in Higher Education
To provide a clear answer on the second research objective, we looked into the development of digital competency assessment tools and tests. The analysis of the research revealed [16,20] that, until now, very little has been done to critically evaluate ICT literacy skills in higher education. Thus, it was important to introduce and develop an innovative way to measure the level of such skills that would be internet-delivered and computerized. In one of the studies the researchers set out to develop a tool to measure students' abilities to research, organize, and communicate information while using technology as the initial development expectations [23].
Based on the need for scalable digital competence assessment methods we focused on analyzing how the studies have introduced validity argument in the research and what were the main reliability evidence provided. Several of the studies stressed the fact that in addition to the validity of the framework used in the digital competence assessment it is also important to validate the assessment instrument. The majority of research used selfreporting or self-assessment tools as the main instrument for data collection, it was stated that although the results give a good overview of the perceived level of digital competence in higher competence assessment research done over the past fifteen years included a sample of 4048 university students and use the Educational Testing Service (ETS) online tool. The ICT literacy test included seven proficiency categories which reflected problemsolving and critical thinking aspects: define, access, manage, integrate, evaluate, create, and communicate. The tool is designed for students to undertake information-handling tasks in the context of simulated software where tasks are separated into five-and fifteenminute tasks, and proficiency levels are scalable accordingly. The test was divided into four sections of which the first part was based on demographic and academic performances, second was a self-assessment test, third part was based on the simulated tasks and finally the participants filled out a form based on their experiences, the final section results were used on the validating process. The study used a fairly common approach to validating the ICT literacy assessment, where they administered the assessment and other measures to a sample drawn from the population, which was made up of university students. Convergent validity was used if the assessment scores correlate to other measures related to ICT literacy and discriminant validity in case the scores do not correlate with measures [23].
Additionally, the literature revealed that, as digital technology is fast developing and by nature exists on many levels, it is a subject of constant arguments. This is highlighted in the fact that in most cases it is difficult to access the knowledge on the use of specific technological tools or platforms, as they are almost always based on the context and the users' preference [31]. In many cases, the authors recommend using a maximum of five level proficiency scales as the more complexed scales confused the participants and steered the attention from the task in hand which was to assess ones' digital competence [6].
Based on the study, we identified that one of the biggest challenges was the reliability of the tools used and the evidence collected, due to the use of self-evaluation test where the interpretation of the statement meaning is left to the individual and bias may appear. In most cases, this means that the research is not repeatable as the results on the same sample are subjective in nature [25,29,54]. In the context of the review it highlights that the evaluation and development of digital competence in the majority of the cases are set on a formal educational setting and not in the organizational level, but among small groups. This can be related to the fact that most analyzed proposals aimed to get a wider understanding of digital competence in the area of research rather than developing and validating a digital competency assessment instrument.

Discussion
Digital competence assessment processes, instruments, and tools have been the focus of international research since the start of rapid development of technologies. Analyzed research revealed the most commonly self-assessment instruments are used for digital competence assessment. This can be related to the fact that the analyzed proposals aim to get a understanding of digital competence in the area of research rather than developing and validating a digital competency assessment instrument for universal use. This is also evident in the use of multitude of frameworks when designing the assessment instruments. The European Commission Joint Research Centre (JRC) has focused their efforts in mapping and understanding digital competence assessment of citizens and educators [2,5]. The future research focuses include using various digital competency models, frameworks, and strategies in implementing similar competence evaluation instruments to find common characteristics. It is also important to analyze the suitability of the digital competence assessments instruments based on the field-specific concepts. Understanding the concept of digital competence is a challenge and, thus, digital competence assessment should not be addressed with one type evaluation instrument. To ensure the validity of the instrument and value of the results it is important to consider other assessment approaches and implementing multi-level competence assessment.
The authors of the reviewed studies bring into light that, although there is a significant amount of theoretical approaches for digital competency assessment, including the conceptual bases, a more flexible and adaptable theoretical approach is needed in order to compare data and independently assess digital competence either for educational setting or personal development.
Digital competence is central in several policy documents and national trends as key competence and future skill in knowledge-intensive economy supporting digital transformation of the society. Due to the COVID-19 pandemic digital competence has become even more important in understanding the use digital technologies in educational settings [55]. Additionally, we need to understand educators' digital competence to support their professional development and consequently the quality of education migrating to more online teaching and learning.
The research revealed that the connection between emerging new technologies and educators' barriers in integrating technology in evident and requires a deeper analysis. Analyzed research showed that in multiple cases the focus was on the development of digital competence not on the evaluation but consequently includes the competency assessment in the process as a by-product. It is important to include digital competence assessment as a starting point of digital competence development when designing wholistic and systematic approach.
As the focus on digital competency assessment has been on general education it is vital to provide adaptable solutions for digital competency assessment in higher education based on the research results in current studies. Similarly stated by Zhao, Pinto Llorente, and Sanchez Gomez [11], who suggest that there is little knowledge on how digital competence is immersed in teaching and learning in higher education. This incorporates the importance of understanding specifics and scope where digital competence assessment is implemented and the digital competence needs. In order to provide high-quality professional development in higher education we are forced to consider the field-specific digital competence and how we can support the development of such competence cross-curricula when implementing competence assessment and development into teaching strategies and teacher training. Furthermore, to support the assessment process of field-specific digital competence requires the understanding of such competence and related constructs. The failure to develop effective and scalable methods for digital competence assessment demands an adaption of a methodological framework which would support a more universal methodological approach to digital competence assessment. One of the suitable frameworks considered is the evidence-centred assessment design, which has been previously used for evaluating the more traditional learning outcomes but, nevertheless, has the potential to be included in the evaluation process of digital competence. The evidence-centered framework allows to both clearly articulate design goals and decisions but also to analyze and process large amounts of data which is imperative in digital competency assessment. Furthermore, the evidence-centred framework pull focus on how the data are collected and if the analysis process is coherent with the purposes of the assessment intended to address [56].
Furthermore, one of the principal issues of digital competence assessment is the scale of the research completed until now. Digital competency assessment necessity is mostly recognized in formal educational settings and, consequently, gives a partial overview of the situation. The studies also bring out the size of the used sample and the reliability of the sampling methods, which in most cases is either a random selection of participants or based on geographical distribution. In order to make more ground-breaking conclusions, future research must include a more comprehensive approach and research-based decisions already in the research design process.

Conclusions
It is important to state the complexity of the topic, as the field of competency assessment includes multiple layers. The reviewed research highlights one of the biggest issues in digital competence assessment, which is the focus on quantitative studies using self-assessment tools. Furthermore, the analyzed research presents the lack of qualitative research to accompany the results in analyzing the reliability and validity of the instruments and digital competence assessment process. As most of the research focuses on applying self-assessment instruments for digital competence assessment it would be useful to look into alternative assessment approaches by including summative, formative, and diagnostic assessment. Self-assessment is often one-dimensional, meaning there is a low possibility to understand why and how students and educators in higher education approach digital competence self-assessment. Thus, it is important to embed authentic assessment approaches-including portfolios, reflective journals, and observations to understand educators' perceptions of digital competence and giving evidence on their technology enhanced learning practices. Additionally, future research be conducted in the field of digital competency assessment should be based upon developing validated and adaptable guidelines for the competency assessment processes. The future of digital competency assessment research should include a participatory design process where the different focus-groups are already included in the process of development and validation of the digital competency model and design of the assessment instrument.
The current literature review gives an overview of the constant development in the field of digital competence assessment. The analyzed proposals describe the state of digital competence assessment we face today, and highlight the importance of systematic and repetitive processes in the context of digital competence assessment in higher education.