Development and Application of a Novel Engineering-Based Maker Education Course for Pre-Service Teachers

Abstract

This study aimed to raise awareness of maker education for pre-service teachers and discuss maker education in their major subjects by developing and applying a maker education course for pre-service teachers with various majors based on novel engineering (NE), a teaching and learning method that combines humanities and engineering. Accordingly, the course was developed following the procedure of the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model, and the educational effectiveness was investigated using test tools. The educational effect and difficulties were also examined through the analysis of reflective journals written by 20 pre-service teachers with various majors who participated in the course. To investigate the educational effectiveness of the developed course, the science, technology, engineering, arts, and mathematics (STEAM) literacy of the participants—before and after the course—was measured, with the results demonstrating a statistically significant improvement. Analysis of the reflective journals identified a recognition of sharing effectiveness, the joy of making, and an in-depth understanding of maker education as education effects, and a lack of understanding of techniques, the burden of prototype fabrication, and the limitation of majoring subjects as difficulties experienced during the activities. This study verified that NE could be used as a significant maker education measure for pre-service teachers with various majors. Based on this verification, this study also proposes a strategy to develop more effective NE-based maker education.

1. Introduction

Society has rapidly changed in the era of the Fourth Industrial Revolution [1,2,3]. Machines and artificial intelligence (AI) are increasingly penetrating into what has been considered as human jobs. In particular, 70% of jobs available as of 2016 will not exist or become completely new jobs in the future when current primary students are employed. Thus, it is difficult to predict the future of employment [2].

One of the primary challenges is that human jobs are being replaced by machines. Some of the work of medical doctors, pharmacists, and lawyers, which are classified as professional fields, has already been replaced by AI. AI has displaced human jobs because it can easily exceed human capability based on the knowledge accumulated by humans over several centuries. AI can make an accurate decision based on big data, which contains a large amount of information that cannot be identified and managed by humans. Furthermore, AI does not require rest (as humans do) and is not affected by body conditions. Under these circumstances, simple knowledge or repetitive jobs are replaced by machines and will be completely displaced by machines in the near future. Consequently, a considerable change will occur among occupations. The World Economic Forum (WEF) has stressed that governments must urgently establish education policies to support personal growth in this period. In particular, it has emphasized the need to improve personal competence for all ages and establish policy measures for improved competence in science, technology, engineering, and mathematics (STEM) and non-cognitive soft skills to develop unique human capabilities [3].

In this circumstance, convergence education and competency education are two keywords relevant to the global education field. First, various types of convergence education have been attempted across the world, such as STEM, STEM + Art (STEAM), and STEAM + Reading (STREAM). Problems we face in in our daily life will become more complex as society advances. To solve those problems, several subjects should be used in a combined manner rather than simply relying on the knowledge of a single subject. Thus, the experience of problem-solving based on convergence knowledge is essential to future generations. Accordingly, convergence education with a broader curriculum is critical. Competency education is a method to raise the competency of future generations to adapt to society at the rapid pace of change when the current common sense or knowledge is no longer absolute. The non-cognitive soft skills emphasized by the WEF are also related to various competencies, such as collaboration, work ethic, self-evaluation, critical thinking, communication skills, and integrity [4].

Maker education is an educational approach that is highly associated with convergence education and competency education. In maker education, problem situations are found in our daily life, and deliverables are produced to solve these problems, in which the convergence knowledge of various subjects is used. Furthermore, competencies, including sharing, communicating, and collaborating with others, can be developed through maker education. With this dual emphasis on convergence and competencies, maker education has emerged as a key for future education [5]. Nonetheless, few studies have been conducted on maker education, and, in particular, very few studies have been conducted on teachers or pre-service teachers [5,6,7]. The role of teachers is critical because teachers are the main actors who set the curriculum and determine the educational content. Only when teachers understand maker education, share its advantages, and are willing to accept it, can maker education be realized in the field of education. Thus, studies on teachers and pre-service teachers are essential to maker education research.

This study aims to investigate the effectiveness of maker education by developing and applying a maker education course for pre-service teachers through teacher training and apply novel engineering (NE), which is applied to various subjects in which maker education is used. Accordingly, this study concerns a measure for pre-service teachers with various majors to apply maker education to their major subjects and aims to provide a basis for the application of maker education in the field of education.

2. Related Works

2.1. Maker Education

Maker education is a compound word of maker and education, which is tied to constructionism, derived from the maker movement. The maker movement, one of the two roots of maker education, was a flow of sharing and developing a making method by makers who make things that they need. The maker movement is not a completely new concept but is extended from the do it yourself (DIY) movement, which already existed [8]. The difference between the DIY and maker movements is that the DIY movement is focused on individual hobby activities, whereas the maker movement extends the making range from individual hobbies through startup and manufacturing [8]. As one of the small-quantity batch-production systems considered appropriate for manufacturing in the Fourth Industrial Revolution, much attention has been paid to the maker movement by nations across the world. In particular, former U.S. president Obama held a maker fair in the White House for the first time in his presidency and mentioned the importance of the maker movement through the phrase “Today’s D.I.Y. is tomorrow’s ‘Made in America’” [9]. Since then, policies for the maker movement have been supported at the national level not only in the U.S. but also in Europe and China.

The educational root of maker education is constructionism, which was an education trend by Seymour Papert—a mathematician, computer scientist, and educator. His idea about learning is well expressed by the phrase, “learning-by-making” [10]. He thought students learned deeply during a process when making their ideas into tangible and shareable objects and sharing their ideas with others [11]. That idea became the root of maker education, which has been the educational foundation of learning through making ever since.

Maker education is an activity where knowledge and skills from various fields are fused and used in the process of activities to make something to solve a problem. It shares similarities with problem-based learning because it also aims to solve a problem, project-based learning because it focuses on project activities, and design thinking because it makes something based on demand. Maker education is characterized by a convergence where various subjects such as art, craft, making, math, science, and engineering are closely related, as depicted in Figure 1 [12], and can be applied even to the humanities and social science fields, in addition to the previously proposed areas, depending on the subject.

The association between maker education and competence education can be found in the maker mindset. The maker mindset refers to several competencies trained through making. Various types of maker mindsets have been proposed. Martin (2015) proposed several maker mindsets: playful, asset- and growth-oriented, failure-positive, and collaborative. Playful refers to the joy and interest felt by engaging in the making activity. Such joy can induce inner motivation for making and for developing perseverance without giving up when facing difficulties. Asset- and growth-oriented refers to an attitude of willingness to learn the skills required for one’s own work in the making process to enable personal growth. Failure-positive refers to an attitude of positive thinking to leverage failure in the making process as a foundation of growth. Although much trial-and-error occurs in the making process, makers evaluate these failure experiences positively as a basis of development. Finally, collaborative refers to an attitude naturally bred in the communication process when sharing ideas and helping each other in a making project that is collaborative. Furthermore, the unlimited competition environment of existing education is discouraged, but a cooperative atmosphere can be raised through sharing and feedback via online maker communities [13]. Hatch (2013) expressed the maker mindset in verb form through the “maker movement manifesto,” including nine types: make, share, give, learn, tool up, play, participate, support, and change [8].

2.2. Novel Engineering

NE is a compound word of novel and engineering, which is an engineering education technique proposed by the Center for Engineering Education and Outreach (CEEO), the affiliated institution at Tufts University in the U.S. NE is a problem-solving-oriented education method that identifies the problem context or challenging situations of characters through reading a book and solves them through an engineering approach. The characters take the customer’s stance, while the students take the position of engineering designers. NE is an excellent tool to combine various education methods such as humanities, computer science education, STEM education, and maker education [14]. Consequently, several NE-related studies have been conducted on the basis of convergence education [15,16,17,18]. The educational advantages of NE proposed by the CEEO are as follows [18].

• Works within ELA(English Language Arts) curriculum
• Enhances reading comprehension
• Engages all learners
• Provides integration of different disciplines
• Introduces realistic engineering problems

The procedure of NE differs slightly depending on the study, as summarized in

Table 1. Steps of novel engineering.

Study	Novel Engineering [18]	Choi (2019) [17]	Hong and Cho (2019) [14], Novel Engineering Education Research Group [19]
Step	Read a book and identify problems	Read a book and identify problems	Picking a book
			Identifying problems
	Scope problems and brainstorm solutions	Define problems and brainstorm solutions	Designing solutions
	Design a solution	Design solutions	Building
		Get feedback	Feedback
		Improve solutions	Improving solutions
		Share	Reconstructing stories

.

In this study, the step proposed by Choi (2019) was applied. The activities for each step are as follows [17]:

• Read a book and identify problems. Students read a book proposed by a teacher, and problems or challenges faced by the characters in the book are identified. Students conduct debates about the problems or challenges identified while reading and make and publish a list of the challenges according to the debate results, thereby sharing them with all students.
• Define problems and brainstorm solutions. The problems to be solved are selected and solutions discussed. Students devise as many solutions as possible through idea exchange between team members and brainstorming. The problems are understood more clearly in this process, and in-depth sharing about characters and situations in the problem context can be achieved.
• Design solutions. One of the solutions is selected, considering the limitations and usable materials in the book, out of the solutions devised in the previous step. The selected solution is visualized using figures or descriptions. The solution is announced to be shared with other students.
• Get feedback. The designed solution is announced to be shared with others, and ideas are exchanged freely. In this step, students can obtain feedback to improve the solution, such as with errors in the student’s solutions, problems, or additional ideas.
• Improve solutions. The student’s solutions are improved and developed based on the feedback. The actual prototype is fabricated based on the improved solution, and deliverables are completed through simulations and tests.
• Share. The completed deliverables are shared with other students through an announcement. The announcement can be made using diverse forms suitable to the deliverables, such as a presentation, re-constructing book stories, and making solution advertisements.

NE can be freely used in various subjects according to the content of the book used. Thus, it is a teaching and learning method suitable for combining with the major subjects of pre-service teachers with various majors at their educational institutions [14]. Accordingly, this study aimed to combine NE to develop a maker education course for pre-service teachers and operate the course for pre-service teachers considering how to apply maker education to their own major subjects.

3. Materials and Methods

3.1. Participants

This study targeted 20 participants in the liberal arts subject “Information and Communication Technology Education” at the K University of Education. All study subjects were pre-school teachers with various majors. Most had no experience in maker education. The characteristics of the study participants are presented in

Table 2. Characteristics of participants [20].

Characteristics	Sex		Major				Experience in Maker Education
Category	M	F	Elementary education	Humanities and Social education	Natural science education	Arts education	Experienced	Not experienced
n	10	10	2	5	10	3	2	18
Percentage (%)	50	50	10	25	50	15	10	90

.

3.2. Study Design

This study was conducted with a one-group pretest–posttest design to verify the educational effect of the NE-based maker education. The STEAM literacy test [21], which is widely used in maker education studies, was conducted before and after the experimental treatment to verify the educational effect. For experimental treatment, an NE-based maker education course developed in this study was applied, and additional satisfaction tests were conducted as a posttest to verify the satisfaction level of the learners of the developed course. Furthermore, whenever a course was complete, students wrote their thoughts in a reflective journal to investigate the responses of the participants.

3.3. Course Design

The purpose of this course was to help pre-service teachers with various majors understand maker education and its educational effects, thereby applying maker education to their major subjects. Accordingly, this course was developed according to the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model, which was the instructional system design framework. The ADDIE model consists of the five steps indicated by the model acronym, with a sequential or cyclic process in each step [22].

First, in the analysis step, the characteristics of study participants were analyzed and associated tasks derived. The results were as follows.

• Because most of the participants had no experience in maker education, the participants required a basic theory of maker education, related skill learning, and experience in making projects.
• Because this course targets pre-service teachers, it should be designed to help them understand the teaching and learning methods of maker education. In particular, it is necessary to enable pre-service teachers to take maker education without significant difficulty by presenting a realistic measure to link maker education to their own major subjects.
• Because most learners (90%) majored in fields with minimal or no relation to skills used in maker education, it is necessary for them to select teaching tools and content that are not too difficult.

In the design step, a learning goal was set and an appropriate evaluation method, education method, and teaching tools selected. Furthermore, the overall structured course plans were established. The learning goals established based on the task derivation results in the analysis step are as follows:

• The concepts and teaching and learning methods of maker education can be understood.
• The skills for making activities are acquired, and making projects can be performed.
• Students can relate their majors to maker education.

This course is not a simple knowledge acquisition of learners but a practice based on hands-on learning and projects. Thus, it is necessary to apply various evaluation methods rather than a simple written test evaluation to identify whether the learning goals are achieved. Accordingly, the following evaluation methods were established: project evaluation through project process and deliverables, mutual evaluation based on sharing (which was the core activity of maker education), and self-evaluation through writing in a reflective journal.

To achieve the learning goals, this course consisted of the following steps:

• Maker education theory learning.
• Skills acquisition.
• NE-Based Making Project 1: experience in making project.
• NE-Based Making Project 2: link between major subject and maker education.

As an education method for the making project, NE was selected to enable a straightforward link with various subjects and textbooks, and the project progress direction was devised. For teaching tools, Micro:bit tinker kit, a physical computing tool, a product from ElecFreaks (Shenzhen China), and MakeCode, a block-based programming language, a product from Microsoft (USA), were selected to enable non-major students to easily acquire the skills. The structured course plan was created by summarizing the above content, which is presented in

Table 3. Plan of NE-based maker education course [20].

Week	Learning Theme	Step	Assessment
1	Orientation	Maker education theory learning
2	The Fourth Industrial Revolution and maker education
3	Instructional model and method of maker education
4	Micro:bit and MakeCode experience	Skills acquisition	Self-evaluation
5	Micro:bit single-item-based project
6	Micro:bit tinker-kit-based project 1
7	midterm exam
8	Micro:bit tinker-kit-based project 2
9	Micro:bit tinker- kit-based project 3
10	NE problem-solving 1	NE-based Making Project 1	Self-evaluation, mutual evaluation, project evaluation
11	NE problem-solving 2
12	NE result-sharing 1
13	NE result-sharing 2
14	Development of major-by-major NE instruction material	NE-based Making Project 2	Project evaluation
15	Final exam

.

In the development step, teaching materials to be used in the course were developed, and the learning content for each learning step was created to achieve the learning goals. Accordingly, the basic concept of maker education was acquired, and teaching and learning models and methods were trained during the maker education theory learning step, thereby equipping learners with the theoretical foundation to combine their own majors with maker education. In the skills acquisition step, learners performed a topic of interest project using the Micro:bit tinker kit and acquired required skills in context. In the NE-based Making Project 1 step, an activity was devised for learners to read The Wizard of Oz and solve the problem through making by identifying the appropriate problem scenario. Finally, in the NE-Based Making Project 2 step, a measure to combine maker education with the teachers’ major subjects by summarizing the learned and practiced content was deeply pondered, and NE-based maker education classes suitable to majoring characteristics and proper prototypes were designed.

In the implementation step, the designs and developed course were applied practically. The course was performed 3 h per week over 15 weeks from September 2 to December 13, 2019.

In the evaluation step, a STEAM literacy test [21] and course satisfaction test [23] were conducted to investigate the educational effectiveness of the course. Furthermore, participants prepared their own reflective journals in the learning management system (LMS) when the course was complete, and these were analyzed to investigate the student response to the course.

3.4. Measures

This study used the STEAM literacy test tool proposed by Choi et al. (2013) to measure the educational effects of the developed course [21], which consists of four sub-elements: convergence, creativity, caring, and communication. The convergence in the sub-elements refers to an ability to understand, create, and use convergent knowledge. Furthermore, creativity refers to pursuing creativity and innovation, which also includes an ability to understand a problem and solve the problem creatively by using information. Caring refers to practicing consideration and respect, which is related to personality, including self-love, self-confidence, respect for others, and understanding multiple cultures. Finally, communication refers to an understanding of oneself and others, and, furthermore, the ability to communicate for cultural understanding in a global society [21,24]. The STEAM literacy test tool consisted of 21 questions, and a Likert five-point scale was used for the questions to collect answers from the learners. A Cronbach-α of 0.897 in the original test sheet was measured, which indicated that the test results were reliable. The composition of the questions in the sub-elements is presented in

Table 4. Composition of questions for test tool of STEAM literacy [21].

Area	Convergence	Creativity	Caring	Communication	Sum
No. of questions	5	7	4	5	21
Full mark	25	35	20	25	105

[21].

Furthermore, four questions from the user satisfaction survey proposed by Holsapple and Lee-Post (2006) were used to investigate the satisfaction level of the course [23], and reflective journals were written for the users’ activities in the LMS bulletin board when the course was completed.

4. Results

4.1. Case of Applying NE-based Maker Education Course

4.1.1. Theory Learning Step of Maker Education

The maker education theory learning step was conducted for three weeks. In this step, learners learned the basic theories of maker education. The background and needs of maker education, basic concepts, teaching method, and the model of maker education were explained along with the cases based on the learner’s characteristics. In this step, NE, which was one of the teaching methods in maker education, was introduced to learners, and problem-solving practices were conducted according to the NE procedure. Because the intent of the hands-on practice was a conceptual understanding of NE, a prototype was not fabricated; instead, only problem context discovery and simple solution-setting were conducted. Accordingly, learners discovered challenges or problem situations after reading the fairy tale The Station Mouse, produced their own solutions, and wrote them on the LMS bulletin board. After that, ideas were shared between learners, and feedback was exchanged through comments. Then, solutions were revised according to feedback, and a final solution and measure were derived and shared. Figure 2 illustrates the book used in NE and several solutions produced by students.

4.1.2. Skills Acquisition Step

In this step, learners learned how to use the Micro:bit tinker kit prior to making project. An exemplary project was completed [26,27], which was led by the teacher focusing on the topic, and then several additional missions were suggested to enable students to review and apply what they learned.

The teaching tool, Micro:bit, used in this step, is a physical computing tool developed by the British Broadcasting Corporation (BBC), which enables anyone to easily learn to program [26]. The specific kit used in the course had the benefit that even novice learners who had minimal electric and electronic knowledge could connect Micro:bit and various external input/output modules easily to use the tool [28]. Thus, Micro:bit was determined to be an appropriate tool to understand and experience maker education at a level of liberal arts without spending excessive time. The Micro:bit tinker kit includes a PIR(passive infrared) sensor, moisture sensor, crash sensor, and AD keypad as input devices, and an LED(R/G/B), OLED display, passive buzzer, and mini servo as output devices. The configuration of the Micro:bit tinker kit is depicted in Figure 3.

The Micro:bit programming can be performed using Java and Python, which are text-typed languages, and MakeCode, which is a block-typed language. In this course, MakeCode was used, whose difficulty level is low, considering that most study participants were non-major students.

4.1.3. NE-Based Making Project Step

This step consisted of two projects: NE-Based Making Projects 1 and 2. First, in NE-Based Making Project 1, learners read the fairy tale The Wizard of Oz and performed the making project according to the NE procedure proposed in Table 1. Learners searched various problem situations and challenges based on the characters in the book and overall story flow and shared them in the LMS. Then, the discovered problem situations were identified, and solutions to solve the problems were searched in several ways. A solution that was achievable and did not violate the proposed limitations excessively was selected and developed. In the next step, the solution was uploaded in the LMS, and students exchanged their feedback about the solution. Students revised and advanced the solution based on the feedback received, and a prototype was fabricated accordingly. Finally, students shared their work with other students. Through these steps, student projects were completed and shared, as depicted in Figure 4.

In the NE-Based Making Project 2 step, learners thought deeply about how to apply NE-based maker education to their major subjects and created their own exemplary projects. In this step, learners were able to select whether Micro:bit could be used, according to their judgment, to establish an education plan considering their majors. Based on the results, all of the pre-service teachers who participated in this study achieved the goals established in the course design by appropriately presenting measures to apply maker education to their own major subjects. Figure 5 illustrates the project examples made by students.

Figure 5a, made by a student who was a pre-service teacher majoring in mathematics education, illustrates an alarm watch for the rabbit based on the story of “The Tortoise and the Hare” from Aesop’s Fables. The alarm watch was made considering the distances of the tortoise and hare and speed of the tortoise, in which the time the hare slept was calculated and an alarm triggered before that time to wake up the hare. The pre-service teacher who made this watch expected students to understand and apply the mathematical concepts of distance, speed, and time when they made the alarm watch.

Figure 5b illustrates a Cao Cao’s wind vane made by a pre-service teacher majoring in history education after watching the Battle of Red Cliffs in Romance of the Three Kingdoms. The pre-service teacher identified a problem scenario where Cao Cao did not know the wind direction in the Battle of Red Cliffs and, if Cao Cao had known, he would have won the battle. The teacher manufactured a making work for Cao Cao in which the wind direction information was received by the wind vane and displayed in Cao Cao’s receiver.

Finally, Figure 5c illustrates the separate collection playset made by a pre-service teacher majoring in environmental education. The pre-service teacher selected the picture book Where’s the Starfish? [29] written by Barroux for NE-based maker education. This book discusses pollution in the marine ecosystem, and the pre-service teacher intended that her students realized the seriousness of the contamination in the marine ecosystem and thought about how to solve this problem in different ways. The pre-service teacher fabricated the separate collection playset as an example, expecting that students would practice separation and dumping several types of plastic rubbish, which led to the correct practice of separate collection in daily living.

4.2. Statistical Analysis

The data collected through the survey in this study were statistically analyzed at a 0.5 significance level using SPSS Statistics 21 software, a product from IBM (USA). The pretest and posttest of STEAM literacy were conducted to verify the educational effectiveness of the developed course, and for the satisfaction test, only the posttest was conducted. To determine how the STEAM literacy of the participants changed before and after the experiment, a paired sample t-test was conducted, and a descriptive statistic was derived to analyze satisfaction. In the pretest, 20 students participated, and in the posttest, 19 students (excluding one student who did not respond) participated.

4.2.1. STEAM Literacy

After the pretest, participants enrolled in the NE-based maker education course for 15 weeks, and the posttest was conducted to verify the educational effects of the course. The posttest of STEAM literacy was conducted with the same test sheet used in the pretest. A paired sample t-test was conducted to determine the change before and after the course. The results of the analysis are presented in

Table 5. Pretest–posttest results in sub-categories of STEAM literacy.

Area	Category	n	Mean	SD	t	p (2-tailed)
STEAM Literacy	Pretest	19	77.31	8.39	−3.247	0.004 *
	Posttest	19	86.05	11.50
Convergence	Pretest	19	20.42	2.27	−2.527	0.021 *
	Posttest	19	21.94	2.57
Creativity	Pretest	19	22.26	4.21	−4.045	0.001 *
	Posttest	19	26.89	5.21
Caring	Pretest	19	15.58	2.14	−2.248	0.037 *
	Posttest	19	16.79	2.23
Communication	Pretest	19	19.05	2.80	−1.916	0.071
	Posttest	19	20.42	3.53

* p < 0.05.

.

The comparison of the pretest and posttest of STEAM literacy demonstrated that the overall posttest mean (86.05) of the STEAM literacy was higher than the pretest mean (77.31). The paired sample t-test that compared two values exhibited that the t-value and significance probability were −3.247 and 0.005, which statistically also significantly improved (p < 0.05). Furthermore, the convergence, creativity, and caring sub-elements of the STEAM literacy had significant results at the 0.05 significance level (p < 0.05). In contrast, the posttest mean of the communication was 20.42, which was higher than the pretest mean, 19.05, but it was not statistically significant (p > 0.05). This was because the developed course in this study was focused on an individual project rather than a team project, which required strengthening team project opportunities to expand the communication probability when performing the NE-based maker education course in the future. These responses were also indicated in the reflective journals written by students after the course completion as follows:

• “Although the classes had more individual work, it would be better if team projects were used,” as excerpted from the reflective journal of Student 20.

Although team projects were encouraged and teams were formed freely to avoid limiting creative thinking, students had difficultly forming teams proactively. Thus, it was necessary to compose the course and strengthen the strategy of the teacher to further induce team projects. This was the same argument implied by Kim et al. (2020); team projects contribute to not only the expansion of communication ability but also reducing the burden of prototype production and skills felt by pre-service teachers with various majors [20].

4.2.2. Satisfaction

The course satisfaction survey was conducted during the last course session. The survey revealed that the overall satisfaction mean was 4.61, which was relatively high.

Table 6. Test results of course satisfaction.

Sequence	Question	n	Mean	SD
1	You are satisfied with the course.	19	4.58	0.59
2	You enjoyed the learning experience.	19	4.63	0.48
3	You believe the system is successful.	19	4.58	0.49
4	You will recommend the course to others.	19	4.63	0.58
Overall satisfaction		19	4.61	0.54

presents the results of the satisfaction survey.

4.3. Reflective Journals

4.3.1. Educational Effects

The reflective journals written by learners each time the course was complete were analyzed, and the results exhibited the educational effects of the NE-based maker education felt by students as follows: recognition of sharing effectiveness, recognition of making joy, and in-depth understanding of maker education. These journals were matched against the understanding of maker education held by pre-service teachers with various majors and their recognition of educational effectiveness, which were the goals established in the course design. Consequently, the previously designed learning goals were achieved.

The educational effects are summarized as follows. First, learners recognized the sharing effectiveness of NE-based maker education. More specifically, students could acquire various insights just by sharing their ideas or deliverables and expand their ideas because they felt that various solutions could apply to a single problem. Furthermore, although students exchanged feedback about their ideas, collective intelligence could be exhibited to develop their overall ideas, which verified sharing effectiveness.

Of the reflective journals, 39 reported sharing effectiveness, accounting for 24% of the 164 journals. This study analyzed sharing effectiveness by step. Learners experienced sharing effectiveness most significantly during the maker education theory learning step. In this step, learners learned the concept and procedure of NE-based maker education for the first time and shared their opinions through case-oriented experiences—thereby developing their ideas. Thus, many pre-service teachers in this step felt the advantages of sharing by themselves and reported those in their reflective journals. The participants also felt that their ideas were improved during the idea sharing and feedback exchange process with others in the NE-based making project and expanded in the sharing step of their final deliverables. In contrast, the skills acquisition step, which was a technical acquisition-oriented content learning step, did not include opinion-sharing activities. Consequently, no reflective journal reported results concerning this step.

Second, learners recognized the joy of making through project execution. Most students had minimal or no experience in maker education before the course and felt challenged when they learned the technical aspects during the early days of the course. However, they felt joy in the making process when they performed the NE-based making project.

Of the reflective journals, 39 reported the joy of making, accounting for 24% of the reflective journals. Learners felt the joy of making most significantly during the skills acquisition step. In this step, learners created maker works using teaching tools based on teaching themes and missions. Thus, the reflective journals verified that learners felt joy when they made something. Moreover, this study verified that learners felt the accomplishment of making their own works and advancing to become experienced makers during the NE-based making project step. The maker education theory learning step did not involve performing making. Thus, the number of reflective journals in that step was zero.

Third, the understanding of maker education deepened during the NE-based maker education. The study participants lacked the understanding of maker education during the early days of the study and thought about maker education somewhat ambiguously. The pre-service teachers experienced maker education through NE-based maker education, in which reading and engineering education were combined, and started to understand the concept of maker education as a convergence education.

Of the reflective journals, 21 reported the deepening of understanding of maker education, accounting for 13% of the reflective journals. The largest number of cases occurred during the NE-based making project because learners deepened their understanding of maker education and recognized the advantages of maker education as they performed making projects by themselves.

Table 7. Cases that recognize the educational effects.

Effect	Content from Reflective Journal	No. of Cases (%)
		M1	S2	N3	T4
Sharing effectiveness	I felt my ideas were also developed when I looked around the ideas from others. I tried to find a solution that other students could not come up with easily, but I was surprised how other students had such creative answers. My solution seemed greatly advanced as I thought about the solution myself and re-consider the problem-solving method after getting feedback from others.	12 (63)	0 (0)	27 (36)	39 (24)
Joy of making	It was a joyful time of creation. It was fun to make and run the code as I wished like I was a designer and I felt good when things were working as I expected. Making something what I wanted using Micro:bit was not easier than I originally thought. However, after some trial-and-errors, I was successful, and I felt great.	0 (0)	18 (26)	11 (14)	39 (24)
In-depth understanding of maker education	In maker education, I realized that thinking freely without having a stereotyped idea was effective in discovering and solving a problem. Maker education seemed advantageous because it could exhibit collective intelligence by coming up with more advanced results when exchanging ideas with others rather than shutting myself up in the corner and concentrating on one idea. Now, maker education is no more unfamiliar, and I become confident it is a wonderful education method that can draw an interest and development from children at the same time. I did not completely understand what maker education at the previous classes because it was new to me. However, I had an idea now that maker education was a method to solve a problem with creative ideas after today’s class.	4 (21)	5 (7)	12 (16)	21 (13)

¹ Maker education theory learning step (19 journals), ² Skills acquisition step (70 journals), ³ NE-based making project step (76 journals), ⁴ Total (164 journals).

presents the educational effects of NE-based maker education and some excerpts from the reflective journals.

4.3.2. Difficulties

The difficulties faced by students in the NE-based maker education course were written in the reflective journals and analyzed. The difficulties were as follows: a lack of understanding of the techniques, and the burden of prototype production.

First, students had difficulties due to their lack of understanding of technical procedures; 17 cases, constituting 10% of the reflective journals, reported technical difficulties. The largest number of cases were in the skills acquisition step. Several students felt interested and challenged in the same course even with the same content. There were individual differences in the skill competence of learners as they dealt with teaching tools of program language and physical computing tools for the first time and performed more advanced missions. Students tended to easily follow the tutorial project but felt difficulties in performing applied tasks such as additional missions.

Next, students felt challenged with prototype production; 11 cases, constituting 7% of the reflective journals, reported the burden of prototype production. This was evenly reported for skills acquisition and NE-based making project steps, which contrasts with what many students reported about the joy of making in the same steps as in Section 4.3.1. Most students felt joy while performing the exemplary making project during the skills acquisition step but felt the burden when they realized they had to perform the self-directed making project in the future. During the NE-based making project step, students reported that they felt the challenge of the design and production process but also felt achievement and joy in the process when they overcame difficulties and completed the prototype.

Table 8. Cases of recognizing difficulties.

Difficulties	Content from Reflective Journal	No. of Cases (%)
		M1	S2	N3	T4
Lack of understanding of techniques	It was not difficult when I followed the tutorial, but it was really hard to review the previous part today without tutorial. It was easy to follow by watching the procedure, but it was difficult to fully understand the procedure myself. I failed again. When will I succeed? Actually, I did not fully understand the principle of the game. It would be really strange if I succeed.	0 (0)	12 (17)	5 (7)	17 (10)
Burden on prototype production	I saw many examples but when I designed it myself, it was really complex. It was difficult to come up with some concrete idea how to implement the solution by applying functions that I had learned so far. I felt worry whether I can make it next time. It would be very difficult to combine and apply functions that I had learned so far in several ways if I did practice myself with this kit.	1 (5)	5 (7)	5 (7)	11 (7)

¹ Maker education theory learning step (19 journals), ² Skills acquisition step (70 journals), ³ NE-based making project step (76 journals), ⁴ Total (164 journals).

presents the difficulties experienced by students in the NE-based maker education course and some excerpts from the reflective journals in relation to difficulties.

5. Discussion

This study developed and applied a maker education course based on NE for pre-service teachers with various majors. The course was designed with a 15-week liberal arts course according to the ADDIE model’s procedure, which consisted of four steps: maker education theory learning, skills acquisition, NE-based Making Project 1, and NE-Based Making Project 2. The STEAM literacy measurement results, which aimed to verify the educational effects of the developed course, demonstrated a statistically significant result. Satisfaction with the course was also high.

The purpose of the development and application of this course was to help pre-service teachers understand maker education, experience the associated educational effects, and apply maker education to their major subjects. The analysis results of the reflective journals written by the participants demonstrated effectiveness of the following: recognition of sharing, joy of making, and in-depth understanding of maker education. These results are consistent with another study [30] in which a program combining reading and maker education was applied to pre-service elementary teachers, improving the recognition of maker education. This study also verified that all of the participants could produce exemplary works appropriately by applying maker education to their major subject, achieving the goal of combining a subject major with maker education. These effects imply that the NE-based maker education was effective in deepening the understanding of maker education, and it also had a positive effect on playful, failure-positive, and collaborative mindsets, which constitute the maker mindset [15].

The NE-based maker education course is matched with the Eight Big Ideas of maker education based on Papert’s constructionism [31]. Students pursued in-depth learning by sharing their ideas with other students in the process of translating their ideas into actual prototypes (learning-by-making (Idea 1)) [9,10]; digital technology was utilized for more interesting prototypes (technology as building material (Idea 2) and digital world (Idea 8)); and learners felt the joy of creating by overcoming failures and hardship (difficult fun (Idea 3)), because you cannot get something right without getting it wrong (Idea 6). In addition, learners discovered what they needed to do to make their ideas a reality (learning to learn (Idea 4)) and completed their work by managing their time (the proper time for the job (Idea 5) and do unto ourselves what we do unto our students (Idea 7)) [31]. This result is consistent with previous study results [14,19] that found that NE could be a suitable framework for maker education. Accordingly, NE can be a strategy for the operation of a maker education course for pre-service teachers with various majors.

However, the lack of understanding of techniques, burden on prototype production, and no statistically significant changes in communication areas among the sub-elements of STEAM literacy implied that it was not easy to develop and apply NE-based maker education successfully for pre-service teachers. The following discussion summarizes strategies of how to apply NE-based maker education to a course as a maker education measure for pre-service teachers with various majors.

First, the opportunity for team projects should be ensured. Students were empowered to exhibit as much creativity as possible in their groups and reduce the limitations in the making project in this study. However, students were reluctant to form groups by themselves, such that all students performed their own projects individually. Students had difficulties when asked to solve all problems alone: a lack of understanding of techniques and the burden on prototype production. Although making was not necessarily based on a team project, a team project was an opportunity to develop the maker mindset during the sharing and collaboration process [13]. Thus, it was necessary for students to experience the advantages of team projects by ensuring an opportunity to perform team activities several times while making projects. A previous study [32], which found that team activities in NE projects had a positive impact on the creation of ideas and cooperation, also supported this study’s results.

Although no study has yet clarified the relationship between maker education and cooperative learning, cooperative learning has been shown to be effective in problem-based learning (PBL), which is a teaching-learning method and the basis of maker education. This is also called cooperative problem-based learning (CPBL) [33,34,35,36]. More specifically, cooperative learning in PBL is effective in teaching learning skills, interpersonal skills, problem-solving skills, writing skills, awareness of the benefits of teamwork, and how to enjoy team activities [33,34,35,36]. These advantages make it clear that cooperative learning in PBL can be a good exercise for enhancing cooperative ability, which is essential for the learners’ future social activities. Thus, the adoption of team projects in maker education can be useful. However, we need to consider certain factors, such as team size, team formation, accountability, and positive interdependence in cooperative learning. Therefore, it is important to find the most effective method for changing the factors [37].

Maker education in the school field will necessarily consist of convergence education [12,13,38], and a theme-based convergence education course must be developed through collaboration between teachers with various majors. Consequently, collaborative experience in making education among pre-service teachers with various majors will be helpful in understanding the place where their major subjects should be positioned in maker education when composing real-world maker curricula in schools in the future. The collaboration experience will also be helpful for understanding the relationship between major and maker education for pre-service teachers who were not able to manage the technical aspects of maker education directly. Furthermore, if cases of maker programs produced by converging various subjects are presented, and opportunities for composing maker curricula that fuse major subjects through team projects are provided, it would be helpful for pre-service teachers to understand maker education as convergence education. Additionally, the burden felt by students concerning technical aspects and prototype production, which were identified in this study as challenges, could be reduced slightly if team members could exchange help with each other through the team’s project performance.

Second, the results of not having a statistically significant outcome in the communication sub-element during the STEAM literacy test implies that an opportunity for communication and idea exchange should be sufficiently encouraged in the process of an NE-based maker education course. The communication sub-element is related to not only verbal and audiovisual communication but also to communication attitude and cooperative behavior [21,23]. Thus, an opportunity for communication and collaboration should be sufficiently encouraged to enhance communication ability. Accordingly, various teaching strategies should be used, including the team project activities proposed previously.

An effective method to increase opportunities for communication in the NE-based maker education course is to expand opportunities for sharing and openness. The spirit of sharing and openness is a core element of making activities and involves activities based on communication [8,39]. Existing studies also verified that students felt significant educational effects when they shared ideas and exchanged opinions during making activities [20,40]. The results of this study also demonstrated that students obtained insights from not only receiving feedback on their own ideas but also watching other students’ ideas. This can be understood using Dewey’s experientialism and is well expressed in the following sentence “we do not learn from experience... we learn from reflecting on experience” [41]. The study’s results meant that learners grew through meaningful experience (making) and the process of reflection (sharing and openness).

Thus, to expand sharing opportunities, it would be a significant activity for students to not only see their projects through presenting but also freely display the presentation materials in a maker fair context before and after the presentation and observe each other’s projects.

Finally, the limitations and future research of the present study are as follows. First, the study results may not be generalized to all groups because it was a case study only for students from a single university of education. In the future, it is necessary to identify whether the effects of the activities can be generalized as a maker education method for pre-service teachers after analyzing the differences due to student characteristics and how the study is executed by performing a similar study with pre-service teachers with various majors and school classes. Control and treatment groups could be established and NE-based maker education applied as the experimental treatment. Second, this study combines maker education with an existing NE procedure. In the future, it is necessary to conduct a study on the development and procedure of the NE-based maker education model that has the advantages of two teaching and learning methods—maker education and NE—after analyzing the characteristics of maker education and comparing them with those of NE. Third, it is necessary to empirically analyze actual effects and problems revealed by applying the strategies proposed in this study (addition of team project activities and opportunities for communication and sharing) and executing the course in a real-world context.