Next Article in Journal
Internationalization at Home: Enhancing Global Competencies in the EFL Classroom through International Online Collaboration
Previous Article in Journal
Natural Disaster Prevention Literacy Education among Vietnamese High School Students
Previous Article in Special Issue
Learnings Networks and Professional Development in Building Energy Management Systems Industry
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 13
Issue 3
10.3390/educsci13030263
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Validation and Evaluation of a Tool for Developing an Integrated View of ICT-Based Educational Innovation

by
Dorien Hopster-den Otter
1,* and
Iwan Wopereis
2
1
Faculty of Behavioural, Management, and Social Sciences, University of Twente, 7500 AE Enschede, The Netherlands
2
Faculty of Educational Sciences, Open Universiteit, P.O. Box 2960, 6401 DL Heerlen, The Netherlands
*
Author to whom correspondence should be addressed.
Educ. Sci. 2023, 13(3), 263; https://doi.org/10.3390/educsci13030263
Submission received: 30 January 2023 / Revised: 22 February 2023 / Accepted: 27 February 2023 / Published: 1 March 2023
(This article belongs to the Special Issue The Values and Risks of Emerging Technologies in Higher Engineering Education)
Browse Figures
Review Reports Versions Notes

Abstract

:
The increasing digitalization of education requires an integrated view of innovation and related expertise development. This study validated and evaluated the content and use of the ”Integrated ICT Motion Sensor”, a tool for stimulating awareness about ICT-based educational innovation among multiple stakeholders within higher education institutions. The Motion Sensor offers administrators, managers, ICT specialists, educational designers, and lecturers a guiding tool for entering into a dialogue about substantive themes such as vision and policy, leadership, professional development, and ICT infrastructure within the institution. The validation of the content by questionnaires and a focus group with 10 experts showed that the experts’ input was almost in line with the Motion Sensor prototype, which was based on the literature. The use of the Motion Sensor was evaluated in a pilot study within four higher educational institutions. Results from the questionnaires showed that participants generally appreciated the nature and course of the conversation. Moreover, the Motion Sensor led to more awareness about ICT-based educational innovation among participants.

1. Introduction

Digitalization has become a common and proliferating phenomenon in higher education [1]. New information and communication technologies (ICTs) to facilitate teaching and learning enter the educational arena at an increasing pace. Some of them offer opportunities for educational innovation, while others just aim to make professional and student life a little easier. Although digitalization has not yet led to a fundamental change in higher education systems, it has contributed to an incremental and ongoing process of renewal and improvement for several decades. One might say that, today, change is a constant presence in higher education institutes (HEIs) and that emerging digital technologies are important drivers for such change. Unfortunately, digitalization in education does not always run smoothly. Too often, highly funded initiatives yield short-lived or local successes [2], which have their origins in organizational ignorance and oblivion [3]. For digitalization to be successful, it is important to overcome silo structures in HEIs and explain and share the knowledge and ideas of various HEI employees. This includes knowledge about the history of experiences with both inner and outer university educational technology (EdTech) implementations, as well as ideas for coping with the digital and cultural transformations ahead [4,5,6,7].
The present study aims to contribute to the success rate of digitalization in HEIs. It presents a study on the use of a tool for entering into an integrated (multiple-stakeholder) dialogue in order to create awareness about ICT-based educational innovation. In addition, it specifies what is needed for instructors’ professional development, an important factor for innovation success [8,9]. The tool is named the “Integrated ICT Motion Sensor”, which emphasizes its focus; namely, eliciting relevant change-related information from various stakeholders concerning ICT-based educational innovation within the institution. This information is needed to move it in the right direction. Before elaborating on the study, we briefly describe the context in which the tool was developed.

1.1. Context

The “Integrated ICT Motion Sensor” (or “Motion Sensor” for short) was developed by project members of the Dutch “Acceleration Plan Educational Innovation with ICT” (“Acceleration Plan”, or AP), a 4-year nationwide initiative to promote ICT use for educational innovation [10]. The AP’s mission was to take substantial steps in digitalization in higher education in the Netherlands. To this end, several project groups (called “zones”) were launched to address topics such as strengthening digital human capital, the secure and reliable use of educational data, making education more flexible, moving toward digital (open) educational resources, and facilitating professional development for lecturers. Within the latter “zone”, a group of faculty from various universities dealt with the development of the Motion Sensor.
The Motion Sensor was created to anticipate new technological, organizational, and pedagogical developments and provide input for programs aimed at expertise development. Research on ICT-based educational innovation has shown that support and encouragement were often too fragmented in the past and not sufficiently focused on sustainability [2,11]. A more integrated approach to innovation is thus needed, in which different stakeholders in the organization should be involved. In the current time frame, also referred to as the Fourth Industrial Revolution [4,12], the introduction of new and updated technology such as the Internet of Things and artificial intelligence (AI) is (again) creating optimism in education. New technology offers openings for far-reaching individualization (e.g., adaptive education) and virtualization (e.g., gaming and simulation). However, it also raises concerns when key stakeholders, such as lecturers, are not properly supported and guided (consider technostress, see [13]). Benavides et al. [12] described far-reaching digitalization (i.e., digital transition) as a process that needs to be comprehensive and holistic. This is reflected in their recommendations, in which the importance of institutional visioning and setting up centers of expertise for digitalization and professional development is emphasized. Bygstad et al. [14] identified a situation of dual digitalization in higher education, where governance related to digitalization is organized both top-down and bottom-up. In the first case, digitalization refers to the implementation of major systems, such as the use of a learning management system or social media. In bottom-up digitalization, the teacher/researcher is in the lead in choosing ICTs in education, often motivated by the domain they are trained for or in which they work. Ideally, a learning space should be offered in which there is alignment of the two digitalization streams and where there is also space for transcending institutional borders and room for new pedagogical insights [14].
Creating such a situation requires coordination between stakeholders in education. This means consultation, in which there is room for sharing information. There are different ways to initiate and encourage knowledge sharing, where the degree of support can vary [15]. Constraints, such as the available time participants have, determine the choice of method/intervention. Views about information sharing also influence the choice. For example, when synchronous communication is undesirable, the group concept-mapping method can be applied. This method is suitable to generate the mental model of a group of experts on a topic via individual brainstorming, clustering, and rating of ideas [11,16]. However, when synchronous communication is important, for instance, to also get to know each other (better), group meetings are a better option (e.g., a focus group or workshop; [17]). The project group in the zone selected a format where synchronous communication is an important constituent. A structured format was also chosen so that enough information could surface within a certain time span. The tool and associated procedure are explained in detail in Section 2.2.
The main feature of the Motion Sensor is dialogue. Dialogue can be described as constructive interaction. Isaacs [18] (see also [19]) considered dialogue as “the creation of tangible, self-organising charged ‘fields’ of meaning in which profound collective insight and reorientation appear, and out of which people can take alignment and effective action.” (p. 20). In the process of ICT-based educational innovation that impacts a large part of an organization (“system”), such an approach is relevant. It also helps to obtain focus on the areas in which lecturers need professional development. Understanding the vision and opinion of stakeholders such as management, educationalists, human resource managers, students, and lecturers from a variety of backgrounds helps sharpen the focus for expertise development. In this context, a striking feature that Kempen [20] mentioned about dialogue is relevant: “If anything, the dialogue stands for an invitation to slow down.” Thus, in situations of dynamic change, dialogues are welcome benchmarks. It therefore makes sense that the moments of awareness sought by the Motion Sensor should be repeated periodically; for example, annually.

1.2. Focus

The present study describes the validation and evaluation of the Motion Sensor. The first phase of the study addressed validation. The evaluation of its use was investigated in the second phase among several HEIs in the Netherlands, which used the tool in an integrated (multi-stakeholder) way. When using the Motion Sensor, a distinction can be made between conceptual utilization and instrumental utilization [21]. Instrumental use involves making decisions and taking actions based on the results of the actual use of the Motion Sensor. This usually takes more time. If no direct action has been taken, but the use of the Motion Sensor has led to awareness, then this can be called conceptual utilization. This can potentially lead to concrete actions and thus instrumental utilization in the future. In this study, we investigate whether the conversation tool we developed is a useful and usable method of creating awareness around ICT-based educational innovations (i.e., conceptual utilization). In the longer term, it can then be investigated whether these new insights have been acted upon (i.e., instrumental utilization).

1.3. Research Questions

The purpose of this study is twofold. First, it aims at validating and adapting the conversation tool (i.e., the Motion Sensor), such that the results of the validation are used to perfect the tool. Second, the study evaluates the usability of the updated tool, focusing on an understanding of its conceptual use. To achieve this purpose, three research questions (RQs) were formulated:
RQ 1:
What themes are relevant for integrated conversations with the aim of creating awareness about ICT-based educational innovation?
RQ 2:
How do participants experience the use of the Motion Sensor?
RQ 3:
To what extent does the use of the Motion Sensor lead to more awareness of ICT-based educational innovation among participants?

2. Materials and Methods

The study consisted of two phases. The first phase (March–April 2020) involved the validation of the content of the Motion Sensor by 10 experts (RQ 1). Following this content validation, the original tool was refined and then used in the second phase (January–July 2021), which was a pilot study on the practical use of the Motion Sensor within HEIs (RQs 2 and 3). Table 1 shows an overview of the design, including the phases, corresponding participants, instruments, and research questions.

2.1. Participants

Ten experts participated in the content validation of the Motion Sensor in Phase 1. They were selected based on their expertise in ICT and technological innovations (n = 3) or governance and policy (n = 3) or being a lecturer in professional development (n = 6) and/or higher education (n = 5). They were from five different HEIs and companies and held positions as professors (n = 3), associate professors (n = 2), an assistant professor (n = 1), doctoral candidates (n = 2), a senior advisor (n = 1), and a lecturer (n = 1). Each expert provided consent prior to the study for the use of their (anonymized) data.
In Phase 2, the Motion Sensor was used at four different Dutch HEIs: the University of Twente, Avans University of Applied Sciences, HAN University of Applied Sciences, and Van Hall Larenstein University of Applied Sciences. Table 2 provides an overview of the number of participants and how the conversation with the Motion Sensor was designed.

2.2. Materials

2.2.1. The Integrated ICT Motion Sensor

The Motion Sensor aims to create awareness about ICT-based educational innovation and what is needed to support lecturers’ professional development in this area. The intention is that the tool will be used periodically with a diverse group of stakeholders, such as administrators, managers, ICT specialists, educational designers, students, and lecturers.
The content of the tool is based on the scientific literature, which shows that many indicators influence lecturers’ professional development concerning ICT-based educational innovation (e.g., [22,23,24,25]). These indicators can be divided into four main themes or pillars. The first pillar is vision and policy, which is about having a clear idea of the added value of ICT-based educational innovation. Research shows that a shared vision and specific associated objectives that will be monitored stimulate the effective use of ICT in the educational process [22,26,27,28]. The second pillar is leadership, which is about the pivotal role formal and informal leaders play during the process of initiating and maintaining ICT-based educational innovation [22,25,29,30,31,32]. They act as role models [22,25] and pacemakers [25,33] and create a safe culture for learning and experimentation [34,35]. The third pillar is professional development, indicating the importance of allowing lecturers and staff members to develop their competencies relating to the integration of ICT. Various studies have shown that professional development has a significant influence on the quality of ICT use and people’s attitudes toward ICT [22,25,28]. The fourth pillar is ICT infrastructure, which is about the availability, accessibility, and quality of hardware, software, and educational support [22,25,26,36]. The ICT infrastructure is an important condition for the implementation of ICT, since its absence can cause frustration and discourage the use of ICT [37]. Appendix A details the four pillars and all associated indicators, including definitions and references.
The tool is intended to be used for an (online) conversation among the stakeholders about the indicators as described in Appendix A. Various materials are available for this conversation. First of all, four reflection cards are available (one for each pillar), which the stakeholders can use to evaluate the current state of affairs within their own institution. Prior to the interview, stakeholders rate each indicator from not present/developed (scored 0) to fully present/developed (scored 3). An example of a reflection card is shown in Figure 1. Second, there is an inspirational cartoon in which the pillars and indicators are visualized (Figure 2). This cartoon is used during the conversation as a way to inspire stakeholders and encourage them to think about the substance of the discussion. Third, there are four conversation cards, one for each pillar, showing the indicators, definitions, and sample questions for each indicator. Sample questions are: “What position does the institution see ICT occupying in education in 2025?” and “How much (extra) time are lecturers given to experiment with new forms of teaching using IT?” The conversation is purposefully and effectively supervised by a moderator and has a duration of 2 h. The HEI selects heterogeneous group compositions, in which multiple stakeholders with different functions discuss all pillars and indicators.

2.2.2. Data Collection and Measurement Instruments

Prior to the focus group in Phase 1, the experts completed a Dutch digital questionnaire. This questionnaire included four parts: a brainstorming question, a matching task, a scaling question, and an open-ended question. In the brainstorming question, the experts were asked to mention relevant indicators that might influence ICT-based educational innovation based on a fictitious case. In the matching task, experts had to match the correct indicator to each definition from the prototype (the description of the indicator). In the scale question, experts had to specify, for each indicator, the extent to which they considered each indicator relevant to a conversation about ICT innovation at the institutional level, on a scale of 1 to 10. Finally, the experts were asked the open-ended question of whether they felt any indicators were missing from this list, and if so, which ones. The experts completed the questionnaire individually and did not receive any information about the content of the Motion Sensor beforehand. During the focus group, the classification of pillars and indicators was presented and the experts could comment on this classification in a Padlet (a kind of digital bulletin board). Then, the results of the digital questionnaire were presented, where further explanations were requested and the experts discussed the different pillars, indicators, and their definitions.
During Phase 2, the Motion Sensor was used in four pilots; the adjustments based on Phase 1 had already been implemented. Afterward, both the stakeholders and the moderators completed an online questionnaire. This questionnaire was available in both English and Dutch, depending on the language spoken. The questionnaire for stakeholders included two open-ended questions about positive points and areas for improvement. In addition, stakeholders rated the tool on a scale of 1–10 in terms of their overall impression. They were then presented with 10 statements and asked to indicate the extent to which they agreed with the statement on a 5-point Likert scale (1 = completely disagree, 5 = completely agree). These statements focused on the course of the conversation, such as the amount of time, the perceived atmosphere, and the guidance for the conversation. There were also some statements specifically focused on the materials used in the Motion Sensor. Finally, the stakeholders were presented with eight more statements about the outcomes of the conversation, focusing on the extent to which the Motion Sensor led to awareness (conceptual use) regarding ICT-based educational innovation. The moderators were asked five open-ended questions about the context of the conversation, such as the name of the HEI, the number of stakeholders, and the group composition. Next, the moderators were presented with the same statements about the course and outcomes of the conversation but had to answer them (where applicable) from the stakeholders’ point of view in order to triangulate the results.

2.3. Data Analysis

The expert questionnaire responses were analyzed quantitatively and qualitatively. The brainstorming question required qualitative analysis, with responses coded in Excel. Each expert’s listed indicators were linked with the indicators from the literature. If an expert named an indicator that was not one from the literature, a new category was created. The matching task and scaling question scores were analyzed quantitatively using frequency analysis. The conversation during the online focus group was transcribed and the experts’ input was categorized and summarized by pillar. In the results section, these inputs were translated and anonymized.
Among the participants in Phase 2, 24 stakeholders and 10 moderators completed the questionnaire. The quantitative data were analyzed using descriptive statistics in SPSS. Possible significant differences between interviewers and stakeholders were tested with a Mann–Whitney U test, since the distribution was non-normal. The responses to the open-ended questions were content-analyzed by researcher A, coded in Atlas.ti, and clustered based on emergent coding [38]. Two main categories emerged from this: positives and areas for improvement. The codes were checked by researcher B; there was no disagreement on the (clusters) of codes.

3. Results

3.1. Relevant Themes for Integrated Conversations about Educational Innovation with ICT (Research Question 1)

The experts agreed with the categorization using the four pillars: vision and policy, leadership, professional development, and ICT infrastructure. Table 3 shows the results for each indicator in the Motion Sensor prototype. According to the experts, the indicators related to vision and policy are relevant to discuss within HEIs (M ≥ 7.7). During the focus group, three additions were made to the indicator regarding vision. First, the vision must gain support in an organization. For example, expert E said, “A vision is one thing, but it is also about a vision having a certain effect; support or connection in the organization.” In addition, questions must be asked about the “why” of the vision. Finally, the vision must be based on knowledge. Regarding the indicators related to the policy, two suggestions were made. First, expert G indicated that policy is not only about objectives but also about the means and time frame. In addition, expert F said that monitoring development only becomes relevant once educational innovation with ICT is initiated.
“A culture of learning and experimentation” turned out to be the most relevant indicator for leadership (M = 7.9; SD = 2.3). The indicator “leader as a pacemaker” scored the lowest (M = 5.9; SD = 1.7); this was due to a lack of clarity about what kind of leader was meant. Expert F indicated that there are many formal leaders in university education, such as department heads, education coordinators, deans, and program directors. These leaders are not necessarily role models and pacemakers; these positions can also be filled by close colleagues. In addition, the experts missed seeing the relational aspect of leadership in the definition. According to expert F, it is not only about expertise and attitude but also about personal contact with lecturers.
The experts considered the four indicators concerning professional development to be relevant. Expert G indicated that in addition to opportunities for professional development, the professional development needs of lecturers also matter. Expert A mentioned that the focus of professional development should be on the lecturers’ attitude in addition to their skills: “Attitude is a determining indicator for whether or not lecturers apply ICT in their own teaching, and for the extent to which lecturers are dependent with respect to, for example, helpdesk or ICT services.” According to experts D and I, the intertwining with research was missing in the indicator “Benefits for the teaching practice”.
Finally, Table 3 shows that the experts found “ICT and educational support” particularly relevant (M = 8.3; SD = 1.9). Expert H indicated that education with ICT requires adequate preparation. Expert A added that privacy and ethical aspects must also be taken into account with regard to using software. In addition, expert F believed it is important to make an inventory of both current ICT and innovative ICT.
The final content of the Motion Sensor was determined based on the input of the experts. Some of the names of the indicators were abbreviated: for example, “Leader as a role model” and “Leader as a pacemaker” were shortened to “Role models” and “pacemakers”. The description of these indicators was adjusted to “formal and informal leaders”. The indicator “Insight into ICT expertise, attitude and use” was moved to the professional development pillar and renamed “professional development needs”. The pillars and indicators from the final tool can be found in Appendix A. This final version was used during the pilots at four different HEIs in Phase 2.

3.2. Experiences with the Integrated ICT Motion Sensor (Research Question 2)

Stakeholders (n = 24) rated the Motion Sensor on average with a 7.04 (SD = 1.63) on a scale of 1 to 10 in terms of overall impression after their first use. Moderators (n = 11) gave it an average rating of 7.64 (SD = 0.67). Table 4 shows the positive points and areas for improvement mentioned by all participants (N = 35). A total of 16 participants mentioned the structure of the Motion Sensor (the framework of pillars and indicators) as positive. Furthermore, the interactive nature of the conversation (n = 5) was appreciated. In addition, five participants indicated that the conversation led to awareness in the area of ICT-based educational innovation. The most common area for improvement focused on the preparation for and guidance of the interview, often combined with a lack of time. As a result, the interview did not cover all the topics and not everyone was able to speak.
Table 5 shows how stakeholders and moderators experienced the course of the conversation. Most of the statements were answered positively by both the stakeholders and the moderators, with scores above 3.0 (varying between “neutral” and “completely agree”). Participants were most positive about the atmosphere during the conversation (stakeholders: M = 4.38; SD = 0.67; moderators: M = 4.27; SD = 0.47), while the available time to conduct the conversation was rated lowest (stakeholders: M = 3.00; SD = 1.07; moderators: M = 2.55; SD = 1.04). Significant differences between moderators and stakeholders were found for just one statement; stakeholders themselves felt they were able to contribute enough to the conversation more strongly than the moderators’ estimates for them.

3.3. The Awareness of Stakeholders and Moderators (Research Question 3)

As Table 6 shows, stakeholders and moderators mostly became more aware of ICT-based educational innovation by using the Motion Sensor (conceptual use). They were predominantly positive about the conversation outcomes, with scores above 3.0. In particular, they found it would be relevant to have another conversation about educational innovation with ICT using the Motion Sensor in the future (stakeholders: M = 4.50; SD = 0.61; moderators: M = 4.60; SD = 0.52). There were no significant differences between stakeholders and moderators.

4. Discussion

There are a few points to consider when interpreting these results. First, none of the HEIs in the pilot chose to include students in the conversation. As a result, certain perspectives may have been underexposed, because students were an important stakeholder group in developing the Motion Sensor. Second, the four HEIs used the Motion Sensor for different reasons and with different types of moderators. This makes the results highly context-dependent. Third, the response rate among the stakeholders was quite low. As a consequence, the results for the second and third research questions should be interpreted with caution. Fourth, the tool was used digitally due to COVID-19 measures. The Motion Sensor was deliberately designed as a paper tool to facilitate physical conversations. Using it online may have influenced the results.
A number of theoretical and practical implications follow from this study. First of all, the study shows the importance of and appreciation for using an integrated approach, in which several stakeholders from different layers of the organization enter into a dialogue. This is in line with previously mentioned literature [18,19,20] that emphasizes the potential of dialogue for social change. At the same time, the study showed that there was often not enough time to discuss all four pillars in a single conversation. The guide should therefore specify that more time or several sessions must be planned in order to obtain a complete picture. The added value of students as a target group should also be emphasized as just described. Furthermore, the results showed that using the Motion Sensor led to more awareness of ICT-based educational innovation among participants. For example, it prompted them to think about ICT-based educational innovation within their institution. In this sense, we can conclude that this one-time use of the Motion Sensor led to conceptual use [21,39]. Further research is needed to determine whether periodic use leads to concrete actions in the field of ICT-based educational innovation; that is, whether the results are ultimately also used instrumentally [21,39] and thus lead to educational improvement. Moreover, the desire for a digital tool arose during the COVID-19 closures. That is why, at the time of this writing, a digital version of the Motion Sensor has been released in Dutch, so that integrated conversations about educational innovation with ICT can now be held remotely (for more information, see https://sensor.versnellingsplan.nl/). Because several participants found face-to-face conversations very valuable, especially with complex themes such as professional development in the field of ICT-based educational innovation, future research is needed to find out to what extent virtual stakeholder conversations lead to the same results as physical ones. Research by Ziegele, Kurtze, and Zerfaß [40] showed that virtual conversations offer various advantages and disadvantages. They identified eight success factors that exploit the advantages, including the rigorous preparation of the moderator, the activation of all stakeholders, and paying attention to privacy and confidentiality. These factors are important to take into account when implementing and evaluating a digital version of the Motion Sensor. Finally, emerging technologies, such as big data and AI, are substantially present in our society and are also permeating higher education [4,41,42]. This includes, for example, AI chatbot systems such as ChatGPT, with which students can respond to essay questions or write academic papers [43]. The Motion Sensor can be a useful tool for entering into a dialogue about these technologies. However, future research is needed on the appropriateness of the Motion Sensor for discussing these technologies, as the technologies are accompanied by potential benefits and risks to individuals. This could result, for example, in the need for a separate indicator on the ethical dimension, including transparency, privacy, safety, and accountability [5]. It may also be beneficial to involve new stakeholder groups in the dialogue, such as alumni and other representatives of the future work field where emerging technologies are already being used. These groups can be supplemented by engineers (e.g., colleagues from relevant engineering schools) who have an understanding of the technological developments ahead.

5. Conclusions

This study evaluated the content and the use of the Integrated ICT Motion Sensor, a tool that aims to create awareness about ICT-based educational innovation and what is needed to stimulate lecturers’ professional development in this area. We concluded that several indicators are relevant to discuss, where the input of the experts was almost in line with the prototype of the Motion Sensor, which was based on literature. Furthermore, the experiences of participants show that the nature and course of the conversation are generally appreciated, although some points for improvement were mentioned. Finally, the Motion Sensor led to more awareness about ICT-based educational innovation among participants.

Author Contributions

Conceptualization, D.H.-d.O. and I.W.; methodology, D.H.-d.O.; formal analysis, D.H.-d.O.; investigation, D.H.-d.O.; resources, Acceleration Plan; data curation, D.H.-d.O.; writing—original draft preparation, D.H.-d.O. and I.W.; writing—review and editing, D.H.-d.O. and I.W.; visualization, D.H.-d.O.; project administration, D.H.-d.O. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Acceleration Plan Educational Innovation with ICT.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of the University of Twente (protocol code 200328 on 26/03/2020.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy issues.

Acknowledgments

This article is based on the Dutch article: Ter Beek, M., Hopster-den Otter, D., & Schildkamp, K. (2021). Bewustwording creëren rond onderwijsinnovatie met ICT op integraal niveau. Tijdschrift voor Hoger Onderwijs, 40(2), 79-102.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A

Table
Table A1. Final content of the Integrated ICT Motion Sensor, version 1.0 [44].
Table A1. Final content of the Integrated ICT Motion Sensor, version 1.0 [44].
PillarsIndicatorsDefinitionsReferences
Vision and policiesVision on education and ICT-based educational innovationViews on the relevance and added value of ICT-based educational innovation as it relates to teaching and learning and the extent to which there is support for this within the institution.[22,26,37,45]
Policy: Objectives for ICT-based educational innovationThe objectives that the institution is aiming for over a period of time with regard to ICT-based educational innovation and the extent to which all the institution’s staff are aware of them.[22,23,27,45,46]
Policy: Management of ICT-based educational innovationThe way in which tasks and responsibilities related to ICT-based educational innovation are delegated across the various departments and employees of the institution and, in particular, the role of the executive board.[37]
Policy: Monitoring of developments in ICT-based educational innovationThe way in which progress (measured against the objectives) in ICT-based educational innovation is evaluated, reflected on, and reported.[28,45,47]
Policy: Professional development and rewardsHow lecturer professional development in the field of educational innovation using ICT is integrated into the institution’s human resources policies. This could include a competence profile for employees, the professional development initiatives available, the (financial) valuation and assessment in job appraisal interviews, and the role of ICT skills during the recruitment process.[30,35,46]
LeadershipRole modelsThe extent to which formal or informal leaders are inspirational role models for other lecturers in relation to the use of IT in teaching, for example, through their positive attitude or expertise.[22,24,25,48]
PacemakersThe extent to which formal or informal leaders encourage ICT-based educational innovation.[25,33]
Culture of learning and experimentationThe way in which the professional culture within a department is geared to joint lifelong learning and experimentation in the context of ICT-based educational innovation, for example, by means of extra facilities for lecturers, the scope for innovation, (critical) reflection, feedback, and the sharing of experiences.[22,34,35,37]
Professional developmentProfessional development needsThe professional development needs that lecturers within the institution have in relation to ICT, based on their current (1) knowledge and skills, (2) attitude and convictions, and (3) IT use.[22,24,29,35,45,49]
Professional development supplyThe professional development opportunities available to lecturers within the institution in the field of ICT, such as courses, training, and professional learning communities. The format, duration, and content are all relevant here.Added by experts
Facilitation of professional development in terms of time, money, and spaceThe time available, the financial resources, and the freedom of choice in the professional development activities for lecturers in the longer term.[22,26,36,45,46,50,51,52]
Sharing knowledgeThe extent to which knowledge of ICT in education is shared internally (among staff) or externally (with other institutions).[29,30,36,37,53,54]
Benefits for teaching practiceThe extent to which professional development leads to desired effects in teaching practice (for students). This depends on the focus and responsibility of those involved in achieving these effects.[29,55]
ICT-infrastructureSoftwareThe availability, accessibility, and quality of software that can be used within education. This includes digital learning resources, electronic learning environments, student management systems, apps, MOOCS, artificial intelligence, software programs, or tools.[25]
HardwareThe availability, accessibility, and quality of hardware that enables the practical use of software in the education process. This includes computers, laptops, tablets, smartphones, augmented reality/virtual reality, robots, interactive whiteboards, an accessible and fast (wireless) network, and licenses for the use of software or cloud services.[22,23,24,25]
ICT and education supportThe availability, accessibility, and expertise of staff within the institution who support the process of ICT innovation in education. This includes ICT specialists who provide support for technical problems and ICT coaches who provide educational support for the organization of education using ICT and the extent to which these parties work together.[22,23,24,25,26,50]

References

  1. Bates, A.W. Teaching in a Digital Age: Guidelines for Designing Teaching and Learning, 3rd ed.; Tony Bates Associates: Vancouver, BC, Canada, 2019. [Google Scholar]
  2. Kirschner, P.A.; Hendriks, M.; Paas, F.; Wopereis, I.; Cordewener, B. Determinants for Failure and Success of Innovation Projects: The Road to Sustainable Educational Innovation. In Proceedings of the AECT Conference, Chicago, IL, USA, 19–23 October 2004. [Google Scholar]
  3. Weller, M. Issues in Distance Education: 25 Years of Ed Tech; AU Press: Edmonton, AB, Canada, 2020. [Google Scholar]
  4. Bühler, M.M.; Jelinek, T.; Nübel, K. Training and Preparing Tomorrow’s Workforce for the Fourth Industrial Revolution. Educ. Sci. 2022, 12, 782. [Google Scholar] [CrossRef]
  5. De Vries, P. The Ethical Dimension of Emerging Technologies in Engineering Education. Educ. Sci. 2022, 12, 754. [Google Scholar] [CrossRef]
  6. O’Brien, J. Higher Education in Motion: The Digital and Cultural Transformations Ahead. EDUCAUSE Rev. 18 October 2022. Available online: https://er.educause.edu/articles/2022/10/higher-education-in-motion-the-digital-and-cultural-transformations-ahead (accessed on 25 January 2023).
  7. Ter Beek, M.; Wopereis, I.; Schildkamp, K. Don’t Wait, Innovate! Preparing Students and Lecturers in Higher Education for the Future Labor Market. Educ. Sci. 2022, 12, 620. [Google Scholar] [CrossRef]
  8. Bond, M.A.; Blevins, S.J. Using Faculty Professional Development to Foster Organizational Change: A Social Learning FrameWork. TechTrends 2020, 64, 229–237. [Google Scholar] [CrossRef]
  9. Schildkamp, K.; Wopereis, I.; Kat-de Jong, M.; Peet, A.; Hoetjes, I. Building Blocks of Instructor Professional Development for Innovative ICT Use during a Pandemic. J. Prof. Cap. Commun. 2020, 5, 281–293. [Google Scholar] [CrossRef]
  10. Association of Universities, Association of Universities of Applied Sciences, SURF. Acceleration Plan Educational Innovation with ICT. 2018. Available online: http://www.vsnu.nl/files/documents/Acceleration%20plan%20educational%20innovation%20with%20ict.pdf (accessed on 25 January 2023).
  11. Wopereis, I.G.J.H.; Kirschner, P.A.; Paas, F.; Stoyanov, S.; Hendriks, M. Failure and Success Factors of Educational ICT Projects: A Group Concept Mapping Approach. Br. J. Educ. Technol. 2005, 36, 681–684. [Google Scholar] [CrossRef]
  12. Benavides, L.M.C.; Tamayo Arias, J.A.; Arango Serna, M.D.; Branch Bedoya, J.W.; Burgos, D. Digital Transformation in Higher Education Institutions: A Systematic Literature Review. Sensors 2020, 20, 3291. [Google Scholar] [CrossRef]
  13. Li, L.; Wang, X. Technostress Inhibitors and Creators and Their Impacts on University Teachers’ Work Performance in Higher Education. Cogn. Technol. Work 2021, 23, 315–330. [Google Scholar] [CrossRef]
  14. Bygstad, B.; Øvrelid, E.; Ludvigsen, S.; Dæhlen, M. From Dual Digitalization to Digital Learning Space: Exploring the Digital Transformation of Higher Education. Comput. Educ. 2022, 182, 104463. [Google Scholar] [CrossRef]
  15. Helbig, C.; Hofhues, S.; Lukács, B. Multi-stakeholder Dialogues as Tool for Design and Qualitative Research in Educational Organisations. In Digital Transformation of Learning Organizations; Ifenthaler, D., Hofhues, S., Egloffstein, M., Helbig, C., Eds.; Springer: Cham, Switzerland, 2021; pp. 23–40. [Google Scholar]
  16. Biedermann, D.; Kalbfell, L.; Schneider, J.; Drachsler, H. Stakeholder Attitudes Towards Digitalization in Higher Education Institutions. In Die 17. Fachtagung Bildungstechnologien, Lecture Notes in Informatics (LNI); Pinkwart, N., Konert, J., Eds.; Gesellschaft für Informatik e.V.: Bonn, Germany, 2019; pp. 57–66. [Google Scholar]
  17. Moens, N.; Broerse, J. Innovating in Sectoral Governance and Development with ICT: Conceptualising the ICT Roundtable Process. JSCI 2006, 4, 33–40. [Google Scholar]
  18. Isaacs, W.N. The Process and Potential of Dialogue in Social Change. Ed. Tech. 1996, 36, 20–30. [Google Scholar]
  19. Bohm, D. On Dialogue, 2nd ed.; Routledge: London, UK, 2004. [Google Scholar]
  20. Kempen, A. ‘An Invitation to Keep with the Other’ or Refining Definitions of the Dialogue: Gadamer, Habermas, Derrida. Locus Tijdschr. Voor Cult. 2022, 25. Available online: https://locus.ou.nl/locus-dossier-de-dialoog/an-invitation-to-keep-with-the-other-aldo-kempen (accessed on 25 January 2023).
  21. Weiss, C.H. Have We Learned Anything New About the Use of Evaluation? AJE 1998, 19, 21–33. [Google Scholar]
  22. Afshari, M.; Bakar, K.A.; Luan, W.S.; Samah, B.A.; Fooi, F.S. Factors Affecting Teachers’ Use of Information and Communication Technology. Int. J. Instr. 2009, 2, 77–104. [Google Scholar]
  23. Tondeur, J.; Van Keer, H.; Van Braak, J.; Valcke, M. ICT Integration in the Classroom: Challenging the Potential of a School Policy. Comput. Educ. 2008, 51, 212–223. [Google Scholar] [CrossRef] [Green Version]
  24. Mumtaz, S. Factors Affecting Teachers’ Use of Information and Communications Technology: A Review of the Literature. JTATE 2000, 9, 319–342. [Google Scholar]
  25. Buabeng-Andoh, C. Factors Influencing Teachers ’ Adoption and Integration of Information and Communication Technology into Teaching: A Review of the Literature. Int. J. Educ. Dev. Using Inf. Commun. Technol. 2012, 8, 136–155. [Google Scholar]
  26. Law, N.; Chow, A. Teacher Characteristics, Contextual Factors, and How These Affect the Pedagogical Use of ICT. In Pedagogy and ICT Use. CERC Studies in Comparative Education; Law, N., Pelgrum, W.J., Plomp, T., Eds.; Springer: Dordrecht, The Netherlands, 2008; Volume 23, pp. 181–219. [Google Scholar]
  27. Strudler, N.B.; Wetzel, K. Lessons from Exemplary Colleges of Education: Factors Affecting Technology Integration in Preservice Programs. Educ. Technol. Res. Dev. 1999, 47, 63–81. [Google Scholar] [CrossRef]
  28. Desimone, L. How Can Comprehensive School Reform Models Be Successfully Implemented? Rev. Educ. Res. 2002, 72, 433–479. [Google Scholar] [CrossRef]
  29. Timperley, H. Teacher Professional Learning and Development. In The Educational Practices Series; Brophy, J., Ed.; International Academy of Education: Brussels, Belgium, 2008; Volume 18, pp. 1–31. [Google Scholar]
  30. Van Veen, K.; Zwart, R.C.; Meirink, J.A.; Verloop, N. Professionele Ontwikkeling van Leraren: Een Reviewstudie naar Effectieve Kenmerken van Professionaliseringsinterventies van Leraren; ICLON/Expertisecentrum Leren van Docenten: Leiden, The Netherlands, 2010. [Google Scholar]
  31. Harris, A.; Spillane, J. Distributed Leadership through the Looking Glass. Manag. Educ. 2008, 22, 31–34. [Google Scholar] [CrossRef] [Green Version]
  32. Leithwooth, K.; Harris, A.; Hopkins, D. Seven Strong Claims About Successful School Leadership Revisited. Sch. Leadersh. Manag. 2020, 40, 5–12. [Google Scholar] [CrossRef]
  33. Moolenaar, N.M.; Daly, A.J.; Sleegers, P.J. Occupying the Principal Position: Examining Relationships Between Transformational Leadership, Social Network Position, and Schools’ Innovative Climate. Educ. Adm. Q. 2010, 46, 623–670. [Google Scholar] [CrossRef]
  34. Fullan, M. The Moral Imperative of School Leadership; Corwin: Thousand Oaks, CA, USA, 2003. [Google Scholar]
  35. Losada, D.; Karrera, I.; de Aberasturi, E.J. Factors Facilitating Successful Educational Innovation with ICT in Schools. Rev. Psicodidact. 2012, 17, 113–134. [Google Scholar]
  36. Albion, P.R.; Tondeur, J.; Forkosh-Baruch, A.; Peeraer, J. Teachers’ Professional Development for ICT Integration: Towards a Reciprocal Relationship Between Research and Practice. Educ. Inf. Technol. 2015, 20, 655–673. [Google Scholar] [CrossRef] [Green Version]
  37. Eickelmann, B. Supportive and Hindering Factors to a Sustainable Implementation of ICT in Schools. J. Educ. Res. Online 2011, 3, 75–103. [Google Scholar]
  38. Creswell, J.W. Qualitative Inquiry & Research Design: Choosing Among Five Approaches, 3rd ed.; SAGE: Thousand Oaks, CA, USA, 2013. [Google Scholar]
  39. Johnson, K.; Greenseid, L.O.; Toal, S.A.; King, J.A.; Lawrenz, F.; Volkov, B. Research on Evaluation Use: A Review of Empirical Literature from 1986 to 2005. AJE 2009, 30, 377–410. [Google Scholar] [CrossRef]
  40. Ziegele, D.; Kurtze, H.; Zerfaβ, A. From Reality to Virtuality. Moving Stakeholder Dialogues from Face-to-face to Online Formats. Commun. Insights 2021, 12. Available online: http://hdl.handle.net/10419/246739 (accessed on 25 January 2023).
  41. Holmes, W.; Tuomi, I. State of the Art and Practice in AI in Education. Eur. J. Educ. Res. 2022, 57, 542–570. [Google Scholar] [CrossRef]
  42. Molenaar, I. Towards Hybrid Human-AI Learning Technologies. Eur. J. Educ. Res. 2022, 57, 632–645. [Google Scholar] [CrossRef]
  43. Sharples, M. Automated Essay Writing: An AIED Opinion. Int. J. Artif. Intell. Educ. 2022, 32, 1119–1126. [Google Scholar] [CrossRef]
  44. Hopster-den Otter, D.; ter Beek, M.; Nouta, J.; Alvarez, C.; Kuypers, M.J. The Integrated ICT Motion Sensor: A Guide to Accelerate Educational Innovation with IT; Acceleration Plan Educational Innovation with ICT: Utrecht, the Netherlands, 2020. [Google Scholar]
  45. Van Rhijn, S. De Implementatie van Blended Learning in Het Medisch Onderwijs: Een Multiple Case-Studie naar de Factoren die de Implementatie van Blended Learning op Curriculumniveau Beïnvloeden. Master’s Thesis, University of Amsterdam, Amsterdam, The Netherlands, August 2008. Available online: https://scripties.uba.uva.nl/search?id=686652 (accessed on 25 January 2023).
  46. Gast, I.; Schildkamp, K.; Van der Veen, J.T. Team-based Professional Development Interventions in Higher Education: A Systematic Review. Rev. Educ. Res. 2017, 87, 736–767. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Uerz, D.; Volman, M.; Kral, M. Teacher Educators’ Competences in Fostering Student Teachers’ Proficiency in Teaching and Learning with Technology: An Overview of Relevant Research literature. Teach. Teach. Educ. 2018, 70, 12–23. [Google Scholar] [CrossRef]
  48. Stuart, L.H.; Mills, A.M.; Remus, U. School Leaders, ICT Competence and Championing Innovations. Comput. Educ. 2009, 53, 733–741. [Google Scholar] [CrossRef]
  49. Ward, L.; Parr, J.M. Revisiting and Reframing Use: Implications for the Integration of ICT. Comput. Educ. 2010, 54, 113–122. [Google Scholar] [CrossRef]
  50. Lam, I.; De Jong, R. De ICT-Bekwaamheid van Docenten Verbeteren: Discussion Paper Met 7 Aanbevelingen; SURFNET: Utrecht, The Netherlands, 2015. [Google Scholar]
  51. Lam, I.; De Jong, R. Van Docentprofessionalisering naar Onderwijsontwikkeling: Inventarisatie van de Status Quo van ICT-Docentprofessionalisering; SURFNET: Utrecht, The Netherlands, 2015. [Google Scholar]
  52. Prenger, R.; Poortman, C.L.; Handelzalts, A. Factors Influencing Teachers’ Professional Development in Networked Professional Learning Communities. Teach. Teach. Educ. 2017, 68, 77–90. [Google Scholar] [CrossRef]
  53. McIntyre-Bhatty, T. Continuous Development of Teaching Competencies; European University Association: Brussels, Belgium, 2019. [Google Scholar]
  54. Stoll, L.; Bolam, R.; McMahon, A.; Wallace, M.; Thomas, S. Professional Learning Communities: A Review of the Literature. J. Educ. Chang. 2006, 7, 221–258. [Google Scholar] [CrossRef]
  55. Nelissen, T.; Jansen, M.; Olvers, D. Succesfactoren Docentprofessionalisering; Avans University of Applied Sciences: Breda, The Netherlands, 2017. [Google Scholar]
Education 13 00263 g001 550
Figure 1. Example of a reflection card.
Figure 1. Example of a reflection card.
Education 13 00263 g001
Education 13 00263 g002 550
Figure 2. Inspirational cartoon.
Figure 2. Inspirational cartoon.
Education 13 00263 g002
Table
Table 1. Design of the study.
Table 1. Design of the study.
PhaseParticipantsInstrumentsResearch Question(s)
1ExpertsQuestionnaire with a brainstorming question, a matching task, a scaling question and an open-ended question.
Focus group.
  • What themes are relevant for integrated conversations with the aim of creating awareness about ICT-based educational innovation?
2Stakeholders and moderatorsQuestionnaire after the conversation using the tool, with items about positive points and areas for improvement, the course of the conversation, and conversation outcomes.
2.
How do participants experience the use of the Motion Sensor?
3.
To what extent does the use of the Motion Sensor lead to more awareness of ICT-based educational innovation among participants?
Table
Table 2. Features of the different conversations using the Integrated ICT Motion Sensor.
Table 2. Features of the different conversations using the Integrated ICT Motion Sensor.
Educational InstitutionNumber of ParticipantsSettingProcedure
University of Twente14
(executive board members, HR, managers) 		Online, using Microsoft TeamsThe Motion Sensor was used to reflect on teaching during the period of the COVID-19 pandemic. The reflection tool was digitized and completed prior to the conversation. The results provided input for the content of the conversation. The conversation was moderated by one internal and one external staff member.
Avans University of Applied Sciences≈60
(educational ICT specialists, educational professionals, product owners, management, executive board members)		Online, using Microsoft Teams, an online Mural environment.The working format was a World Café, which means that stakeholders rotated to different conversations, divided among the four pillars 1. There were a total of four 30 min conversations, with a joint introduction and conclusion of the main points. The conversation was moderated by eight internal employees (in pairs per pillar).
HAN University of Applied Sciences≈50
(program managers, lecturers, support staff (educational ICT specialist, scheduling, secretariat) of the Business Administration degree program) 		Online, using Qualtrics (blended learning scan), Scorm Articulate Emodule, Microsoft Teams/ Powerpoint/ whiteboard, and Mentimeter.The format was a Flipped World Café. Stakeholders prepared for the session with a module on blended learning. Then, there was an online World Café in which stakeholders rotated over four 15 min conversations divided among the four pillars. There was a common introduction and conclusion of the main points. The conversation was moderated by eight internal employees (in pairs per pillar).
Van Hall Larenstein University of Applied Sciences11 (HR, managers, support staff)Online, using a beta version of the digital Motion Sensor.The working format was a World Café, which means that stakeholders rotated to different conversations, divided among the four pillars. There was a joint introduction and conclusion of the main points. Stakeholders did not prepare anything in advance. The conversation was moderated by two internal staff members.
1 The four pillars of the Motion Sensor: vision and policies, leadership, professional development, and ICT infrastructure; see Appendix A and Section 2.2.
Table
Table 3. Phase 1 results for each indicator in the Motion Sensor prototype.
Table 3. Phase 1 results for each indicator in the Motion Sensor prototype.
IndicatorBrainstorming
n 1		Definition
n 2		Relevance
M (SD) 3
Vision on education and ICT-based educational innovation 789.2 (1.4)
Policy: Objectives for ICT-based educational innovation488.8 (2.0)
Policy: Management of ICT-based educational innovation167.9 (1.6)
Policy: Monitoring of developments in ICT-based educational innovation 187.7 (1.7)
Policy: Professional development and rewards388.4 (1.2)
Leader as a role model277.2 (1.7)
Leader as a pacemaker 255.9 (1.7)
Insight into ICT expertise, attitude, and use376.4 (2.3)
Culture of learning and experimentation687.9 (2.3)
Professional development supply787.1 (2.0)
Facilitation in terms of time, money, and space798.3 (1.5)
Sharing knowledge497.8 (2.4)
Benefits for the teaching practice278.7 (2.0)
Software386.6 (2.4)
Hardware396.4 (2.2)
ICT and education support778.3 (1.9)
1 Number of experts who mentioned this indicator in the brainstorming question. 2 Number of experts who matched the indicator and definition correctly. 3 Score range of 1 (not relevant) to 10 (highly relevant) based on the total of 10 experts.
Table
Table 4. Positive points and areas for improvement of the Motion Sensor according to stakeholders and moderators, clustered by topic.
Table 4. Positive points and areas for improvement of the Motion Sensor according to stakeholders and moderators, clustered by topic.
Positive Pointsn *Areas for Improvementn *
Structure using framework of pillars and indicators16More preparation and guidance needed before and during the conversation5
Interactive, exchange, conversation, colleagues speaking5More time needed for the conversation 4
Awareness, insight into the current state of affairs 5Differentiating the content of pillars and indicators4
Breadth of topics covered in conversation3Working more toward concrete next steps 3
Conversation cards and questions per pillar as input for conversation3Uncomfortable switching between online environment Motion Sensor and online conversation 3
Integrated approach, contributions from different perspectives 3
Clear action and points for improvement points 3
* Number of times a topic appeared in answers to open-ended questions in the evaluation questionnaire. The data from stakeholders (n = 22) and moderators (n = 11) were combined. Only topics that occurred three times or more are included in this table.
Table
Table 5. Results related to the course of the conversation.
Table 5. Results related to the course of the conversation.
StatementStakeholders
M (SD), n = 22		Moderators
M (SD), n = 11
I was sufficiently aware of the nature and content of the conversation beforehand.3.64 (1.09)4.09 (0.70)
There was enough time to conduct a substantive conversation.3.00 (1.07)2.55 (1.04)
I feel was able to contribute enough to the conversation. *4.05 (0.81)3.09 (1.22)
There was a safe atmosphere during the conversation.4.38 (0.67)4.27 (0.47)
The moderator effectively guided the conversation. 4.10 (0.83)3.80 (0.42)
The inspirational cartoon inspired the conversation. 3.26 (1.10)3.44 (0.53)
The inspirational cartoon contributed to the image of ICT-based educational innovation.3.26 (1.15)3.56 (0.53)
The conversation cards gave direction to the conversation. 3.79 (0.92)3.91 (1.04)
The sample questions on the conversation cards were specific and clear enough. 3.57 (1.08)3.64 (1.12)
By filling in the reflection card(s), I have a better understanding of my institution’s position with regard to the field of ICT-based educational innovation. 3.60 (1.00)4.27 (0.65)
Note: 1 = completely disagree, 5 = completely agree. The statements in the questionnaire for the moderators were filled in from their own perspective (“I feel I was able to effectively guide the conversation.”) or the stakeholder’s perspective (“Each stakeholder was able to contribute enough to the conversation.”). * p < 0.05 for Mann–Whitney U test comparing stakeholder and moderator mean scores.
Table
Table 6. Results related to the conversation outcomes.
Table 6. Results related to the conversation outcomes.
StatementStakeholders M (SD), n = 22Moderators
M (SD), n = 10
The conversation using the ICT Motion Sensor prompted me to think about ICT-based educational innovation within my institution. 3.77 (0.75)3.90 (0.74)
This conversation has made me more aware of the role of ICT-based educational innovation within my institution. 3.45 (1.06)4.20 (0.42)
Due to this conversation, I want to make further plans in the field of ICT-based educational innovation within my institution. 3.45 (0.96)3.70 (0.68)
After this conversation, I expect my institution to develop further in the field of ICT-based educational innovation.3.77 (0.92)4.10 (0.74)
I am confident that the conversation using the ICT Motion Sensor will lead to more ICT-based educational innovation within my institution. 3.77 (1.02)3.40 (0.97)
I think it is relevant to have another conversation about ICT-based educational innovation using the ICT Motion Sensor in the future. 4.50 (0.61)4.60 (0.52)
Note: 1 = completely disagree, 5 = completely agree. The statements in the questionnaire for the moderators were filled in from the stakeholder’s perspective.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hopster-den Otter, D.; Wopereis, I. Validation and Evaluation of a Tool for Developing an Integrated View of ICT-Based Educational Innovation. Educ. Sci. 2023, 13, 263. https://doi.org/10.3390/educsci13030263

AMA Style

Hopster-den Otter D, Wopereis I. Validation and Evaluation of a Tool for Developing an Integrated View of ICT-Based Educational Innovation. Education Sciences. 2023; 13(3):263. https://doi.org/10.3390/educsci13030263

Chicago/Turabian Style

Hopster-den Otter, Dorien, and Iwan Wopereis. 2023. "Validation and Evaluation of a Tool for Developing an Integrated View of ICT-Based Educational Innovation" Education Sciences 13, no. 3: 263. https://doi.org/10.3390/educsci13030263

APA Style

Hopster-den Otter, D., & Wopereis, I. (2023). Validation and Evaluation of a Tool for Developing an Integrated View of ICT-Based Educational Innovation. Education Sciences, 13(3), 263. https://doi.org/10.3390/educsci13030263

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop