Developing a Professional Performance Evaluation System for Pre-Service Automobile Repair Vocational High School Teachers in Taiwan

Abstract

Educational quality is one of the most important factors for having well-trained human resources that support the productivity, competitiveness, and economic growth of a nation. Teachers have always played dominant roles in ensuring education quality. Thus, teacher education is becoming ever more important. However, scholars have long warned about the ineffectiveness of teacher preparation. Indeed, a significant gap still exists between current pre-service teacher education and ideal training. Thus, a new professional performance evaluation system for pre-service teachers will be very helpful in resolving long-term public dissatisfaction with schools. To resolve this problem, developing a performance evaluation system that contains professional standards, performance evaluation criteria, and a precise performance evaluation mechanism is essential to guarantee the highest teacher quality in Taiwan. The authors designed a two-stage research effort for a professional performance evaluation system for Taiwanese pre-service teachers of automobile repair. In the first stage, the Delphi method is introduced to develop pre-service teachers’ professional standards and key evaluation criteria using two iterations of Delphi processes and 14 iterations of focus group meetings based on experts’ opinions. In the second stage, a professional performance evaluation and review mechanism for pre-service teachers is developed using focus group meetings to collect opinions from six vocational education experts. The final performance evaluation system will enhance the quality of pre-service teachers in vocational high schools and serve as a useful model for other countries and economies.

1. Introduction

The excellence of an educational infrastructure is one of its most important characteristics, and it can contribute to the productivity and competitiveness of all nations [1]. Educational quality, the key factor of any successful educational infrastructure, will have dominant effects on personal income, wealth disparity, and economic sustainability [2]. Teachers thus play a pivotal role in providing that quality education [3]. Some people even argue that teacher quality is the most effective school-based determinant that will influence student accomplishment [4,5]. Therefore, during the past decades, national governments have urged the enhancement of teacher quality by producing and expanding teacher policy definitions [6].

Teacher education plays a daily critical role in our lives. Nonetheless, there has been increased criticism of educational schools. The major reason is the ineffectiveness seen in preparing teachers to perform their duties. Reform efforts in teacher education that include structural and conceptual changes are thus currently under way. Therefore, researchers in some nations have initiated teachers’ evaluations, and these have become an important part of professional development with possible benefits accruing to both schools and teachers [7]. Teacher evaluation systems should play a crucial role in all teachers’ professional development [8]. Nonetheless, there still is no comparable assessment for pre-service teachers who are ready to teach and be effective [9]. In recent decades, educational researchers and reformers have focused on technical, economic, and legal strategies to improve education and the professional status of teachers. Little attention was paid to the social norms or ethical aspects of the professional practice. However, Harrison and Dymoke [10] emphasized the need to foster a more personal and professional led strategy for teacher performance management since both a system-led approach and recommended mentoring are very suitable as tools that can assist pre-service teachers in developing their professional and personal autonomy in the classroom.

Fenstermacher [11] stated that the goal of teacher education is to train teachers who can reason about their teaching as well as teaching philosophy, and perform skillfully. Such rational reasoning capabilities require a specific process for reflecting on what the teacher is doing, an adequate basis of facts, principles, and experience based on the goal of precise reasoning [12]. Crowe [13] argues that there is a clear need for new assessments that can be used to learn whether pre-service teachers have developed the classroom teaching skills they need to appropriately teach students. Meanwhile, some research has found that although teachers’ evaluations can serve as a primary lever for increasing the focus on teaching quality, a lot of the earlier reforms have failed. Therefore, a better understanding of the numerous aspects of successful performance evaluation methods is still necessary [14]. A new professional performance evaluation system for pre-service teachers will be extremely helpful in resolving the current long-term public dissatisfaction with schools and teacher education. Such dissatisfaction can be found in almost all the developed and developing countries and economies today, including Taiwan.

Taiwan placed a high priority on vocational education at an early stage of their industrialization process [15]. The Taiwanese vocational high schools historically have served as the final phase of education and focused primarily on training workers in basic skills [16]. The Taiwan economic “miracle” is clearly indebted to its system of vocational education and training (VET). However, in the past few years, teacher development in the Taiwanese vocational school system has failed to keep pace with technology evolution and the related industry development. A significant competency gap also exists between the education of pre-service teachers and the ongoing requirements for in-service teachers. This phenomenon has been especially noted in the training of pre-service teachers in automobile repair.

Further, the education of pre-service teachers in automobile repair is becoming even more challenging because of the variety of car models, recent advances in automobile technology, and the long-term neglect of the environmental pollution issue and its key sustainability-related issues. This variety in the models of cars increases the complexity of pre-service teachers’ vocational education. Further, recent advances in automobile technology, such as fuel cell and hybrid car technologies, have only worsened the situation. Automobiles and the automotive repair industry both have a huge impact on the environment, and indeed, environmental pollution remains a threat to global sustainable development. However, environmental pollution and sustainability-related issues are too often neglected by teacher-training institutes. The automotive repair industry remains one of the most polluting of all the small-scale industries in Taiwan.

The Environmental Pollution Centers [17] have stated that in many localities automobile shops are actually the main source of pollutants. These shops are likely to cause pollution in the environment because they will not manage the production and recycling of chemicals and wastes [17]. According to Onianwa et al. [18], an automobile workshop is usually operated in a semi-stationary or stationary mode. This mode tends to emit pollutants directly into the environment [18]. In addition, these facilities are responsible for handling other contaminants, including lubricating oil spills, gasoline-burning fumes, and chemical inputs for car operations [19]. The pollution resulting from these automobile shops has greatly affected the health of the public. Possible effects include chronic respiratory, premature death, and even cancer. Since the automobile as well as the automobile repair industry are major venues that graduates of vocational high schools join, environmental pollution and sustainability-related issues should also be a dominant topic in pre-service teachers’ vocational education.

The development of a Taiwanese professional performance evaluation system for pre-service teachers’ professional standards, performance evaluation criteria, and a performance evaluation mechanism is indeed essential to close the current gap in delivering well-qualified, pre-service teachers of automobile repair. To fulfill this objective, the authors undertook a two-stage research effort to develop a Taiwanese professional performance evaluation system for training pre-service vocational high school teachers of automobile repair. In the first stage, pre-service teachers’ professional standards and evaluation criteria were defined using opinions provided by Taiwanese experts in the field. The descriptive statistics of reliability and construct validity of evaluation criteria were reviewed in the pilot test based on questionnaires being collected from practical training supervisors. These practical training supervisors are in-service vocational high school teachers in automobile repair who are supervising or have supervised pre-service teachers in practical training. In the second stage, focus groups were formed to develop and then evaluate and confirm the proposed evaluation mechanism for these Taiwanese vocational education experts.

The remainder of this paper is thus organized as follows. Section 2 reviews the literature on the goals of teacher education and the dissatisfaction with teacher education, competencies for teachers’ professional development, professional standards for teachers, and pre-service teacher assessment. In Section 3, the research design and methods are introduced. Section 4 presents the analytical results. Discussion is presented in Section 5. Finally, Section 6 offers conclusions and recommendations for future research.

2. Literature Review

The literature on the goals of teacher education and dissatisfaction with teacher education is discussed here first. Then, past work related to competencies and professional standards for teacher development are summarized and discussed in the second sub-section. Finally, past work on pre-service teacher assessments is addressed in the third sub-section. Findings from the literature review served as the basis for the development of a new/better performance evaluation system.

2.1. Goals of Teacher Education and Dissatisfaction with Teacher Education

Okoro [20] pointed out that teacher education has five goals: Provide highly motivated, efficient, and conscientious teachers; promote the creativity and equality of teachers; assist teachers in fitting into their social life and enhance their commitment to national goals; provide a professional and intellectual background to teachers to ensure they adapt well to changing situations; and strengthen teachers’ commitment to the teaching profession. Kennedy [21], further categorized these goals and positioned the teachers as such. According to Kennedy [21], the goals of teacher education can be categorized into three aspects:, namely, socialization, human-resource development, and subjectification. Teachers can be properly positioned based on three aspects, namely, novel professional members who need to instill the existing professional culture and practice, officials who will enable students to enhance the national situation, and autonomous educators who can strengthen their own specific interests and talents through creative means and thus further contribute to the common good [21]. Generally, the primary goal of teacher education is to give teachers the theoretical skills and knowledge they need using a series of policies and procedures. This process will enable teachers to continuously acquire efficient teaching capabilities in their area of specialization and thereby contribute to their schools and communities [22].

Over the past decade, teacher education has become one of the public dissatisfactions noted in schools. Schools have been criticized for their ineffectiveness in preparing teachers for their work, being unresponsive to new demand, being situated too far from actual teaching practice, and presenting barriers that prevent appointment of the smartest college students into the teaching profession [23]. For instance, the Chester Finn and Thomas Fordham Foundation argued that the actual requirements for teacher education are the major obstacles to entering teaching [23]. Critical stakeholders like the National Network of Educational Renewal have proposed fundamental changes in pre-service teacher education curricula. These proposals include creating partnerships with public schools, making available extensive experience for pre-service teachers in school settings, and closing the gap between teaching theories and practice. A given percentage of the universities have accepted these recommendations and updated their teacher education programs [24].

Until recently, scholars still criticized the problems present in teacher education and urged reforms. For example, Morris and Hiebert [25] further argued that teacher education programs in the United States can also do much more to improve teaching quality. Aubusson and Schuck [26] indicated that in many countries the continual concerns about students’ falling behind those in other nations in performance measurement has resulted in the widespread belief that teachers are not adequately prepared. This view adds impetus to the move to reform teacher education [26]. Gelfuso and Dennis [27] summarized recent studies and global appeals for better teacher education, including an increase in the number and quality of field experiences for all pre-service teachers.

2.2. Competencies and Professional Standards for Teacher Development

The development of an individual’s professional role entails having both formal and informal experiences [28]. Teacher professional development is the professional growth that a teacher attains through increasing professional experiences and systematic evaluation of their teaching [29]. Certain features of the pre-service teacher education program can have a precise influence on graduates’ teaching capabilities [30]. Empirical studies had verified the relationship between teacher professional development and greater enhancement of both pedagogical skills and student achievements [31]. Thus, Kunter et al. [32] argued that the concept of professional competence may offer a viable and workable route to understanding the full nature of teacher success.

Competence is the integration of skills, knowledge, and attitudes [33]. Also, competence is a combination of personal characteristics, knowledge, skills, motivation, and attitudes, all of which enable one to act effectively in a specific scenario [34]. Stoof and Martens [35] classified competences into those that relate to tasks and personal characteristics. Per the Australian Competency Standards, competency can be subdivided into seven groups. These include having capabilities in (a) information collection, analysis, as well as organization; (b) idea and information communication; (c) activity planning and organization; (d) collaboration and teamwork with others; (e) manipulation of mathematical thinking and techniques; (f) problem solving; and (g) technology manipulation [36].

As per the literature, certain general competencies for teachers’ professional development are presented. These include pedagogical skills, understanding students, monitoring and analyzing learning and development, school–family and society relationships, knowledge of the syllabus and its content, communication, teamwork, cultural understanding, and problem-solving [37]. Moreover, these competences relate to the fields of research, curriculum, lifelong learning, social–cultural, emotions, communication, information and telecommunication technology, and environment, all regarded as generic competencies necessary for all teachers [37,38]. Kunter et al. [32] categorized teaching knowledge, work motivation, professional beliefs, and self-regulation as the primary elements for teachers to have full professional competence. He also evaluated how these elements can influence instruction and thus, influence the students’ learning results.

Based on these evidences, teacher education institutions in various countries have already restructured their programs and defined their competences as the major goals of their new curricula [39]. Although important, the concept of teacher professional competence remains still limited. It is still unclear whether the factors that can enable effective teaching and the interactions between competence and the teaching environment [40]. Recently, Tang and Cheng [41] identified teaching competence as a crucial factor for having professional competence. However, teaching competence is often a neglected factor in the development of teachers. Apparently, significant disparities still exist between the competences required for pre-service teachers and those being provided by teacher educators. Competency-based performance evaluation is a standard reference summative assessment process that assesses the actual ability of participants to meet predetermined professional standards under controlled conditions and agreements [42]. Because competencies must lead to effective performance [43], competency-based performance evaluations will be very helpful in achieving results ratings to provide a complete picture of individual performance.

Professional teaching standards defines both mature and high quality teaching. These standards are one of the primary instruments by which policymakers and education authorities seek to make teaching practice less variable, more reliable, and increasingly effective. Normally, teaching standards seek to articulate what we value about teaching and state the crucial characteristics of what teachers believe, know, and are capable of doing. Indeed, Donaldson [44] identified two issues of concern if professional standards are to be used more effectively to support professional learning, namely, the establishment of precise professional standards for teaching and leadership and the utilization of those standards by teachers [44].

The National Board for Teaching Standards of the United States has made several contributions to these teaching standards. One of its major contributions was the publication of five core propositions for professional teaching standards in 1989. “What Teachers Should Know and Be Able to Do” is a document that details the five core suggestions. The document is similar to other documents, i.e., the Hippocratic Oath of Medicine. These documents underscore a teacher’s commitment to improving student performance. In this regard, the propositions form the foundation of all the standards being proposed by the National Board for Teaching Standards. These propositions also serve as the groundwork for National Board Certification. The five propositions suggest that a teacher must (1) be dedicated to students and their learning, (2) know the subject they teach and how to teach that subject, (3) be responsible for managing and monitoring students’ learning, (4) think systematically about their professional practice and then learn from experience, and (5) be contributing members of learning communities. Forde and McMahon [45] summarized the international examples and categorized them into three groups of professional teaching standards: Initial preparation and entry into the teaching profession; management and leadership in education, and advanced teaching practice. The pre-service professional performance evaluation system for pre-service high school teachers is classified as the first group, which is the professional teaching standard for the initial preparation and entry into the teaching profession.

While governments are committed to setting professional standards for teachers, the government-imposed standard is generally expressed in general terms and still conforms to the one-size-fits-all principle [46]. Although middle school teachers tend to develop their professional identity and pride from their own disciplines, they still lack specific areas of knowledge and skills [46].

2.3. Pre-Service Teacher Assessment

The success of national initiatives, including the establishment of professional standards and a framework for professional learning, depends greatly on the integrated evaluation strategy that needs to be designed to unpack the processes of learning and implementation as well as their influences on professional practice and student performance. Thus, over the past two decades, major policy proposals for reforming teacher education have been proposed, piloted, and implemented at the local, state, and national levels. The proposed policies have enhanced accountability for teachers and improved the standards of teachers’ performance [47]. In particular, a performance-based assessment of pre-service teachers is considered to be one of the policies clearly aimed at improving the quality of teaching and the advancement of other professional staffs in education [47].

Performance evaluation defines what teachers need to know and the capabilities teachers should have in actual teaching contexts [48]. By utilizing teaching portfolios to evaluate candidates’ competencies, teacher preparation programs can analyze their effectiveness in fulfilling the needs of standards, the measures of accountability, and graduation competencies. When pre-service teachers develop the capabilities of a learner-centered, inquiring professional through this portfolio process, they can establish ongoing habits of reflective inquiry and practice that they can then show to in-service teachers [49]. Campbell and Cignetti [50] recommended that teacher preparation programs make use of the portfolio to evaluate teacher candidates’ development over the full span of the teacher preparation program. Many people do accept the viewpoint that evaluation has a major impact on how students learn. Therefore, researchers have placed key importance on student perceptions of two specific features of any evaluation of students’ learning, namely, authenticity, and feedback [51].

The development of strategies to use to address pre-service teachers, curricula, and assessment techniques actually means expanding the professional role of teachers and educators. Pre-service teachers need to have the opportunity to develop greater understanding of the formative and summative objectives of assessment [52] so as to expand their knowledge of current assessment practices and the nature of relevant curricula, evaluation, and pedagogy [53]. The use of portfolios in pre-service teacher evaluations enables student teachers to demonstrate their knowledge and abilities sooner and more precisely. Portfolios, therefore, provide insight into the ability of the pre-service teacher from two perspectives: (1) the retention ability of the student teacher and (2) the ability to apply or demonstrate the teaching-related skills acquired by that teacher in the course of learning and in other venues [53]. The use of portfolios is also effective because it allows student teachers to link the professional knowledge learned during their coursework with the skills they acquire in the actual teaching process as they learn how to practice instruction [54]. Gelfer and O’Hara [55] argued that both the characteristics of the pedagogical environment and actual pedagogical performance can be recorded by using teaching portfolios. In addition, teachers’ portfolios can be further organized around the professional standards being defined by the government. These standards can help both teachers and administrators evaluate pedagogical performance by clearly demonstrating the facts related to the fulfillment of these competencies [56,57].

According to Struyk and McCoy [58], both formative and summative evaluations are required when evaluating the performance of pre-service teachers. Self-assessment can be used as a main tool and a valuable tool to overcome two limitations, namely, the lack of feedback [58] and the opportunity to reflect on one’s own teaching [59]. Reflection is the key competence that pre-service teachers should develop during pre-service training [60]. By using self-assessment, teachers can improve their performance both as pre-service teachers and fully professional teachers [60]. It is indeed possible to develop valuable reflective skills spontaneously through practice.

Based on the literature review results, although scholars have warned about the ineffectiveness of teacher preparation [23] for a long while, a significant gap has continued to exist between the current state of pre-service teacher education and the aspired and desired level of that education. Based on that evidence, teacher education institutions in several countries have restructured their programs and set specific competencies as the goals of their new curricula [39]. However, more recently, other scholars (e.g., Santagata and Yeh [40] and Tang et al. [41]) still argue there is a research gap remaining in the basic conceptualizations for teacher professional competence. Witte and Jansen [46] argue that teaching standards have become one of the most suitable policy tools to use for monitoring and enhancing teacher quality. As a result, a large number of teacher education reform policies have been proposed and applied at the state and local levels in both developed and developing countries, i.e., the United States, Canada, Taiwan, etc. These policies promote the strengthening of accountability for teacher education and stricter standards for evaluating teacher performance [47]. A new professional performance evaluation system for pre-service teachers will indeed be most helpful in closing the worldwide research gap and resolving the current long-term public dissatisfaction with schools and teacher education. In the following section of this paper, precise research methods are proposed for developing a Professional Performance Evaluation System for Pre-service Teachers in general and Taiwanese Automobile Repair Vocational High School Teachers in particular.

3. Research Design

The successful development of a professional evaluation system includes two stages of research. In the first stage, the professional standard and the performance evaluation criteria are clearly defined. In the second stage, the precise performance evaluation mechanism is defined. In the first stage, the literature on professional performance, checking processes for pre-service teachers in general, and the literature related to pre-service teachers of vocational high schools in particular, will be reviewed. An initial draft of teachers’ professional standards and the performance evaluation system including evaluation criteria can be proposed based on the literature review. Two iterations of the Delphi process were held to evaluate and modify the drafts for both the professional standard and the evaluation criteria. The department heads of all the leading Taiwanese vocational schools, as well as their senior teachers, were invited as input experts. A total of 10 vehicle engineering experts joined that survey. Finally, the final professional standards as well as the performance evaluation criteria were confirmed by 10 experts.

In the second stage, the literature from related fields for pre-service teacher professional performance evaluation and its review mechanism was first reviewed first. Then, the initial draft of the pre-service teacher professional performance evaluation and the review mechanism was proposed based on the results of the literature review. Then, 14 iterations for focus group meetings consisting of six vocational education experts were held to evaluate, modify, and confirm the drafts of the pre-service teacher professional performance evaluation and review mechanism. After these 14 iterations for the focus group meetings, a pilot test was carried out on the evaluation criteria for professional performance evaluation system for pre-service teachers of automobile repair. Fifty-three practical training supervisors were surveyed. Pilot tests are required to modify and supplement the evaluation indicators before the main survey, and to find unsuitable, obscure, and less readable items. The pre-service teacher professional performance evaluation and review mechanism were precisely defined accordingly.

The pilot test will be based on the agreement of the practical training supervisors: the in-service teachers in automobile of vocational high schools who are supervising or have supervised pre-service teachers. In Taiwan, students from teacher training institutes become pre-service teachers in the final year of their degree programs. The pre-service teachers in automobile repair begin a half-year practical training at vocational high schools after the completion of their coursework. The professional performance of a pre-service teacher is rated by practical training supervisors. The 53 practical training supervisors participating in the pilot-test are supervising or have supervised pre-service teachers. The practical training scores, i.e., the professional performance, of the pre-service teachers, are rated by practical training supervisors. According to Arvey and Murphy [61], the use of supervisory ratings on job performance measures is becoming increasingly optimistic; such subjective assessment instruments do not automatically translate into rater error or bias. Recently, Cook [62] argued that the supervisor rating criterion is widely adopted due to the wide-availability, uniqueness, and the results which are difficult to argue with. Therefore, the pilot-test will be based on the agreement of the supervisors on the evaluation criteria.

Firstly, the quality, validity, and readability of the items are analyzed. The Cronbach’s α coefficient is used to analyze reliability. Reliability can be ensured by carefully designed processes to remove or modify low reliability items [63]. The construct validity was verified by using the r_altering-CV and r_contrast-CV indices provided by Western and Rosenthal [64] in the “quantifying construct validity” (QVC) procedure, since the proposed standard was adapted from the “Professional Standard for Teachers” [65,66]. The r_altering-CV is the correlation between the collected patterns and the predicted patterns of correlations no matter what their magnitudes are [67]. The r_contrast-CV takes into account the correlations; therefore, in addition to the order of the correlations, the parameter also compares the absolute values of the predicted and the observed correlations [67]. A selected group of experts independently give their opinions on the pattern of expected correlations so that the median of each value given by the experts is used during the calculations of the indices [63].

The QCV is a relatively recent idea, thus, there are no specific guidelines about what the effect sizes are sufficient to ensure adequate validity [68]. Therefore, a higher effect size provides greater evidence of validity [68]. The range for r_altering-CV and r_contrast-CV is between –1.000 and +1.000. A large positive effect size indicates that the actual pattern of convergent and discriminant correlations are closely matched with the correlation being predicted based on the conceptual meaning of the constructs being assessed. Significance tests is another function that can be achieved by QCV analysis. This significance test determines whether the fitness between the actual and the predicted correlations is likely to have happened by chance. Researchers using the QCV procedure always want to derive large values of the two effect sizes and have statistically significant results [68].

In the following section, the development of (1) the Taiwanese professional performance system for vocational high school teachers in automobile repair and (2) the pre-service teacher professional performance evaluation and review mechanism is presented based on the research methods proposed herein.

4. Developing the Taiwanese Professional Performance System for Pre-Service Automobile Repair Vocational High School Teachers

In the following section, the current status of the Taiwanese VET system for vocational high schools in automobile repair is introduced in Section 4.1. Then, the development of the professional standard and the performance evaluation criteria during the first stage of the research is presented in Section 4.2. Finally, the development of the professional evaluation mechanism is presented in Section 4.3.

4.1. The Taiwanese VET System for Vocational High Schools in Automobile Repair

Taiwan developed the VET system at an early stage of the industrialization process. Vocational high schools have historically served as the final phase of education and focus primarily on training workers in basic skills [16]. During the past decades, the well-developed Taiwanese VET system has played a dominant role in industrial and economic development. However, during the past years, teacher development for the Taiwanese vocational school system has failed to keep pace with social changes, technology evolution, and industry development. Further, teacher development and the training curriculum have mainly been limited to traditional disciplines and has had a lack of coverage for diversified, emerging professions. Meanwhile, since the introduction of a new teacher education system in 1994, all qualified teacher programs are now authorized to educate pre-service teachers [69]. However, Taiwanese teacher education institutes are facing several problems: (1) insufficient faculties to give teacher education related lectures in general, and domain-specific courses and internship programs in particular; (2) a high number of professional domains, e.g., automobile repair, electronics engineering, electrical engineering, accounting, management, etc., require a huge number of teachers, which imposes a very heavy financial burden; (3) in addition to teaching and internship, overloads have been imposed by support for local education and on-the-job training activities; and (4) consistency between the teacher education programs is being provided by normal universities and those programs is being provided by other universities that need to be verified. That is, the quality of the pre-service teachers may not be guaranteed.

Aiming to solve the current problem in vocational education, the Ministry of Education (MOE) of Taiwan defined the competencies being required by pre-service teachers at the 8th National Education Conference held in 2010. These competences were classified into five major aspects and defined as the professional standard for subject-specific pre-service teachers of senior high schools, vocational high schools, junior high schools, elementary schools, kindergartens, and special education. The standards include (1) teacher quality, (2) dedication and attitude, (3) curriculum design and teaching, (4) class management and guidance, as well as (5) research, development, and on-the-job training. Following that development, the domain-specific “Professional Standards” and “Performance Standard for Teachers” were defined. These standards can serve as the basis for evaluations and the professional development of pre-service teachers.

To continue the efforts to enhance teacher quality, in 2011, the MOE further proposed the “Teacher Education Refinement Program” in the Report on Education in Taiwan. Practical strategies proposed by Hsu and Chen [70] include (1) the formation of a work group for the professional standard and evaluation criteria definition and (2) the definition of an evaluation system and support mechanism based on the “Teachers’ professional standard” and “professional performance evaluation criteria”. All these endeavors seek to serve as guidance for pre-service teacher development and training through the planning of quality, legitimacy, systems, and delegation.

4.2. Development of the Professional Standard for Pre-Service Vocational High School Teachers in Automobile Repair

By partially adapting the “Professional Standard for Teachers” [65,66], this research defined the professional standard for pre-service teachers in automobile repair in the first stage of the research. Related literature was reviewed first. The professional standard that includes performance evaluation criteria was defined based on the results of literature review, two iterations of the Delphi process were introduced to confirm the standard and the performance evaluation criteria by summarizing opinions being provided by a total of 10 vehicle engineering experts. The selected experts included 10 teachers from the vocational high school. All experts have more than five years of working experience. Please refer to

Table 1. Background of Experts for the Delphi Process.

No.	Education	Title	Experiences	No.	Education	Title	Experiences
1	M.S.	Director	36 Years	6	M.S.	Director	11 Years
2	M.S.	Teacher	12 Years	7	M.S.	Director	28 Years
3	M.S.	Director	20 Years	8	M.S.	Teacher	29 Years
4	Ph.D.	Teacher	36 Years	9	M.S.	Director	36 Years
5	Ph.D.	Director	30 Years	10	M.S.	Teacher	11 Years

for the background of the experts. Following, the Delphi process follows the works by Jones and Hunter [71] as well as Vidal et al. [72] are demonstrated.

4.2.1. The Delphi Process and Data Analysis

The Delphi questionnaires were first defined by referring to the “Draft of Professional Standard and Performance Evaluation Criteria for Senior High School Teachers” proposed by the Minister of Education [65] and related work by Chen et al. [66]. The questions were formulated based on a scale consisting of agree, disagree, and modification to understand the consistency of each important aspect and its criterion. The participants could also leave commentaries in the blank area of the modification part and ask questions at any moment regarding any point of the Delphi questionnaire to generate discussion to suggest other potential factors [72]. For each round, two weeks were given for the participants to answer the survey.

According to the survey results collected from the participants in the first round of the Delphi process, responses from all participants were received. The proposed standards and performance evaluation criteria were revised based on the experts’ opinions. Following the first round, 32 criteria (see

Table A1. The first iteration result of Delphi method for criteria evaluation.

No.

Criteria

1-1

1-2

1-3

2-1

2-2

2-3

2-4

3-1

3-2

3-3

3-4

4-1

4-2

4-3

5-1

5-2

5-3

1

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

2

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

3

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

4

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

5

agree

agree

disagree

agree

agree

disagree

agree

agree

agree

disagree

agree

agree

agree

agree

agree

agree

agree

6

agree

agree

disagree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

disagree

agree

agree

agree

7

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

8

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

9

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

10

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

100%

100%

80%

100%

100%

90%

100%

100%

100%

90%

100%

100%

100%

90%

100%

100%

100%

disagree

0%

0%

20%

0%

0%

10%

0%

0%

0%

10%

0%

0%

0%

10%

0%

0%

0%

No.

Criteria

6-1

6-2

6-3

7-1

7-2

7-3

8-1

8-2

8-3

9-1

9-2

9-3

9-4

10-1

10-2

1

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

2

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

3

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

4

disagree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

5

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

6

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

7

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

8

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

9

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

10

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

agree

90%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

disagree

10%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

of Appendix A and

Table 2. Pre-service professional standard for industrial subject teachers in vocational high schools.

Professional Standards	Professional Performance Indexes
1. Possess professional knowledge and master key issues in technological and vocational education (D1)	1-1 Possess basic knowledge and literacy for technological and vocational education
	1-2 Understand the characteristics and development of students
	1-3 Understand the trend of development in technological and vocational education
2. The cognitive, psychomotor, affective capability, and literacy and quality in dynamic mechanics-related domain-specific core professional curriculum and associated teaching knowledge and ability (D2)	2-1 Developing capabilities and literacy in the domain-specific core course
	2-2 The domain-specific capabilities and literacy in practical training courses of specific domains
	2-3 Developing capabilities and literacy of the affective aspect
	2-4 Developing domain-specific teaching knowledge and capabilities
3. The cognitive, psychomotor, affective capacity, and quality of automobile-related domain-specific core professional curriculum and associated teaching knowledge and ability (D3)	3-1 Developing capabilities and literacy in the automobile related domain-specific core course
	3-2 The domain-specific capabilities and literacy in the practical training courses for automobile repairing related domains
	3-3 Developing capabilities and literacy of the affective aspect
	3-4 Developing the domain-specific teaching knowledge and capabilities
4. Possess competence in instructional design and materials adjustment (D4)	4-1 Possess competence in instructional design and materials adjustment
	4-2 Adjust instructional design and materials according to students’ needs and progress in learning
	4-3 Integrate knowledge concepts, practical skills, and life experience to activate teaching content
5. Apply instructional strategies for effective teaching (D5)	5-1 Apply appropriate teaching strategies and communication skills to help students’ learning
	5-2 Apply diversified instructional media, information communication technology, and related resources to assist teaching
	5-3 Provide remedial or augmented teaching according to students’ learning achievements
6. Apply appropriate methods for instructional evaluation and diagnosis (D6)	6-1 Apply appropriate evaluation instruments and multivariate data to diagnosis students’ capabilities and achievements in learning/practical training
	6-2 Apply evaluation results as feedback to guide students to develop most effective learning strategies
	6-3 Adjust evaluation methods in a timely fashion according to students’ characteristics and learning needs
7. Exert skills in classroom management and build a supportive learning environment (D7)	7-1 Set up classroom regulations to contribute to learning and build a favorable classroom atmosphere
	7-2 Arrange learning situations contributive to the interaction between teacher and students to promote students’ learning achievements
	7-3 Master learning status of classroom and deal with classroom issues appropriately
8. Master individual differences of students and provide related guidance (D8)	8-1 Understand students’ differences and interests and guild students to adaptive learning and career development
	8-2 Understand students’ cultures and guide students to affirmative social learning
	8-3 Respond to the individual needs of students and provide necessary support and guidance
9. Fulfill responsibilities for the educational profession (D9)	9-1 Demonstrate educational enthusiasm when taking care of students and establish cooperative relationships with educational partners
	9-2 Comply with professional ethics and related laws and regulations for teachers
	9-3 Join in school’s affairs and meetings in response to the development needs of school
	9-4 Participate in the development activities of curriculum and instruction in school
10. Endeavor to grow teacher professional development (D10)	10-1 Join in instructional research and advanced studies to promote abilities in both teaching and professional practice
	10-2 Join in professional societies in specialized field to enhance professional development

Reference: Partially adapted from MOE [65] and Chen et al. [66].

) were identified. To clearly understand whether these criteria meet the minimum level, the collection of the data resulted in the establishment of experts’ reviews and was organized and fed into the second round. Based on the convergence results of the first round, the results of second round also met the suggestion of the Pareto principle wherein all items should be at least about an 80% consensus by the panels. Therefore, the criteria could then be confirmed by the Delphi method as suitable for serving as the professional performance evaluation system.

Overall, the professional standard included (1) establishing vocational education-related professional knowledge; (2) establishing cognitive, psychomotor, and affective capacity and quality in group- or subject-specific core professional curricula and associated teaching knowledge and ability; (3) establishing the ability to prepare curricula, teaching design, and teaching material adjustment; (4) able to teach effectively with good use of teaching strategies; (5) able to adopt adequate methods for evaluation and diagnosis; (6) able to maximize class management skills and establish a supportive learning environment; (7) able to understand students’ differences and provide guidance accordingly; (8) being responsible as an educational professional; and (9) being dedicated to teachers’ professional growth.

Next, Delphi questionnaires were designed based on the modified standards and indicators and sent to the same group of participants to ask for their opinions once again. Utilizing the same respondents as in Round 1, all of the experts’ viewpoints were further consolidated after Round 2. Since all these experts’ opinions reached a consensus, the Delphi process was then terminated. Results of the first iteration of the Delphi process are noted in Appendix A. The results of two iterations of the Delphi process were almost similar except for minor modifications of wording of the proposed process standard and performance evaluation criteria. Thus, the professional standards for pre-service vocational high school teachers in automobile repair were established, as shown in Table 2. The symbols representing each aspect are also defined in Table 2. The evaluation criteria for pre-service vocational high school teachers in automobile repair are fully noted in

Table A2. Professional standards, performance indexes, requirements, and evaluation criteria for pre-service vocational high school teachers in automobile repair.

Professional Standards	Professional Performance Indexes	Requirement	Performance Evaluation Criteria
1. Possess professional knowledge and mastery of key issues in technological and vocational education	1-1 Possess basic knowledge and literacy in technological and vocational education	Core	1-1-1 Possess basic knowledge and ability regarding technological and vocational education
			1-1-2 Understand the system of technological and vocational education
			1-1-3 Understand the administration of technological and vocational education
		Advanced	1-1-4 Understand the laws and regulations of technological and vocational education
	1-2 Understand the characteristics and development of students	Core	1-2-1 Possess knowledge regarding the development and guidance of adolescents
		Advanced	1-2-2 Possess knowledge of the distinguishing characteristics of adolescents’ learning
	1-3 Understand the trend of development in technological and vocational education	Core	1-3-1 Understand the trend of thought in technological and vocational education
			1-3-2 Understand the trend of reform in technological and vocational education
			1-3-3 Understand the current key issues of technological and vocational education
		Advanced	1-3-4 Understand the trend of development in industries related to the specialized cluster
			1-3-5 Understand the trend of development in industries related to the specialized field or discipline
2. Have cognitive, psychomotor, affective capability and literacy and quality in dynamic mechanics-related domain-specific core professional curriculum and all associated teaching knowledge and abilities	2-1 Developing both capability and literacy in the domain-specific core course	Not Applicable	2-1-1 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy for applied mechanics
			2-1-2 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of electricity
			2-1-3 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of electronics
			2-1-4 Understanding the fundamental concept and developing the calculation, analysis and application capabilities and literacy for engines
			2-1-5 Understanding the fundamental concept and developing the calculation, analysis and application capabilities and literacy for hydraulic and gas pressure
	2-2 Developing both domain-specific capabilities and literacy in the practical training courses of specific domains		2-2-1 Developing the capabilities and literacy for basic power mechanical tools
			2-2-2 Developing the capabilities and literacy for basic electricity tools
			2-2-3 Developing the capabilities and literacy for maintenance and repair of power mechanical equipment
			2-2-4 Developing the capabilities and literacy for electric wiring and problem analysis
			2-2-5 Developing the capabilities and literacy of electronics circuit repair and problem analysis
			2-2-6 Developing the capabilities and literacy for engine wiring and problem analysis
			2-2-7 Developing the capabilities and literacy for hydraulic and gas pressure and problem analysis
	2-3 Developing capability and literacy for affective aspects		2-3-1 Developing the capabilities and literacy for dynamical machine -related environment protection capabilities and literacy
			2-3-2 Developing the capabilities and literacy for dynamical machine-related environment work safety protection capabilities, and literacy
			2-3-3 Developing the capabilities and literacy for dynamical machine related business ethics
	2-4 Developing domain-specific teaching knowledge and capabilities		2-4-1 Understanding core professional courses in the related field of dynamic mechanics
			2-4-2 Developing the capabilities and literacy for fusing knowledge and the capability of automobile repair into course content
			2-4-3 Developing the capabilities and literacy for fusing knowledge and capability of automobile repair into teaching activities
			2-4-4 Developing the capabilities and literacy for fusing teaching materials, principles of instruction and evaluation methods into teaching activities
3. Have the cognitive, psychomotor, affective capacity, and quality of automobile-related domain-specific core professional curriculum and associated teaching knowledge and ability	3-1 Developing capability and literacy in the automobile-related, domain-specific core course	Not Applicable	3-1-1 Understanding the fundamental concept and developing the calculation, analysis and application capabilities and literacy of automobile-related technology
			3-1-2 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of dynamic mechanics
			3-1-3 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of automobile engine
			3-1-4 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of automobile chassis
			3-1-5 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of factory safety
			3-1-6 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of automobile electricity
			3-1-7 Understanding the fundamental concept and developing the calculation, analysis, and application capabilities and literacy of mechanical and electrical engineering drawing and reading
	3-2 Developing domain-specific capabilities and literacy in practical training courses for automobile repair-related domains		3-2-1 Developing the capabilities and literacy of basic tools
			3-2-2 Developing the capabilities and literacy of related equipment
			3-2-3 Developing the capabilities and literacy of maintenance and repair of related equipment
			3-2-4 Developing the capabilities and literacy of training in reading engineering drawing, wiring, piping, practical training, measurement, and error analysis of automobile engines
			3-2-5 Developing the capabilities and literacy for training in reading engineering drawing, wiring, piping, practical training, measurement, and error analysis of automobile chassis
			3-2-6 Developing the capabilities and literacy of training in reading engineering drawing, wiring, piping, practical training, measurement, and error analysis of electric automobile
			3-2-7 Developing the capabilities and literacy for mechanical and electrical engineering drawing and reading
			3-2-8 Developing the capabilities and literacy for machinery work and practice
			3-2-9 Developing the capabilities and literacy for automobile air conditioner repair and maintenance
	3-3 Developing capability and literacy of the affective aspect		3-3-1Developing the capabilities and literacy of automobile repair-related environment protection capabilities and literacy
			3-3-2 Developing the capabilities and literacy of automobile repair-related environment work safety protection capabilities and literacy
			3-3-3 Developing the capabilities and literacy of automobile repair-related business ethics
	3-4 Developing domain-specific teaching knowledge and capabilities		3-4-1 Understanding the curriculum and contents of automobile repair-related courses
			3-4-2 Developing the capabilities and literacy for fusing knowledge and capability of automobile repair into content of courses
			3-4-3 Developing the capabilities and literacy for fusing knowledge and capability of automobile repair into teaching activities
			3-4-4 Developing the capabilities and literacy for fusing teaching materials, principles of instruction and evaluation methods into teaching activities
4. Possess competence in instructional design and materials adjustment	4-1 Possess competence in instructional design and materials adjustment	Core	4-1-1 Formulate unit teaching objectives in accordance with content of course outline and students’ prior knowledge
			4-1-2 Develop lesson plans according to unit teaching objectives
			4-1-3 Conduct teaching activities according to course outline and students’ abilities and interests
		Advanced	4-1-4 Possess ability in designing special topic courses and related instructional activities
			4-1-5 Possess ability in planning training plans and designing instructional materials for students to participate in skill certifications and skill contests
			4-1-6 Plan featured courses according to school vision and development goals of the department
			4-1-7 Integrate needs and resources of industry in school district to develop school-based curriculum
	4-2 Adjust instructional design and materials according to students’ needs and progress in learning	Core	4-2-1 Provide remedial instruction according to students’ abilities and learning achievements
			4-2-2 Adjust instructional design and materials in timely fashion according the learning needs of students and teaching environment and facilities
			4-2-3 Conduct diversified teaching activities in accordance with interests, characteristics, and individual differences
		Advanced	4-2-4 Carry out school-industry cooperation and integrate available resources to design augmented teaching activities
	4-3 Integrate knowledge concepts, practical skills, and life experience to activate teaching contents	Core	4-3-1 Combine students’ life experience into the design of teaching activities to inspire learning motivation and interests and promote the level of implementing skills of students
			4-3-2 Link courses before and after longitudinally and integrate professional knowledge and skills to facilitate students’ learning
		Advanced	4-3-3 Integrate multivariate information and instructional units to carry out extending teaching
			4-3-4 Apply the principle and spirit of interdisciplinary and curriculum integration to the design of curriculum and instruction to promote students’ practical skills
5. Apply instructional strategies for effective teaching	5-1 Apply appropriate teaching strategies and communication skills to help students’ learning	Core	5-1-1 Apply appropriate methods to conduct teaching according to preset instructional objectives and lesson plan
			5-1-2 Apply teaching skills, such as lectures, questions and answers, blackboard writing, demonstration and eye contact to guide students’ learning
			5-1-3 Guide students to the use and safety of tools, instruments, and equipment and carry out their proper management
			5-1-4 Adjust teaching schedule and strategies in accordance with students’ learning achievement and reactions
			5-1-5 Apply diversified teaching strategies and methods in accordance with different teaching contents to inspire students’ learning and motivation
			5-1-6 Design and conduct individual or small group teaching for students to facilitate the learning of professional and practice courses
			5-1-7 Improve teaching contents, teaching methods, and evaluation strategies in connection with personal teaching history
			5-1-8 Extend instructional materials and innovate teaching strategies to guide students’ self-learning
		Advanced	5-1-9 Make good use of teaching resources and create learning opportunities to encourage students to the learning of specialty and skills
			5-1-10 Develop and upgrade professional knowledge and skills continuously through self-evaluation and reflection to enhance teaching
	5-2 Apply diversified instructional media, information communication technology, and related resources to assist in teaching	Core	5-2-1 Apply various instructional media, information communication technology, and digital in teaching to enhance students’ learning
		Advanced	5-2-2 Apply teaching technology and information retrieval to solve teaching needs and problems
	5-3 Provide remedial or augmented teaching according to students’ learning achievements	Core	5-3-1 Understand students’ learning status and achievement and provide appropriate assistance when needed
		Advanced	5-3-2 Provide students with effective learning strategies and teaching guidance in accordance with students’ learning achievements and available resources inside or outside of school
			5-3-3 Analyze the learning status of students and design teaching strategies and methods for remedial or augmented instruction to help students’ learning
6. Apply appropriate methods for instructional evaluation and diagnoses	6-1 Apply appropriate evaluation instruments and multivariate data to diagnosis students’ capabilities and achievement in learning/practical training	Core	6-1-1 Evaluate and examine students’ learning progress and achievements in a timely manner during the teaching process
			6-1-2 Apply diversified evaluation methods to examine the professional and skill learning achievements of students
			6-1-3 Develop diversified evaluation instruments to examine the professional and skill learning achievements of students
			6-1-4 Develop appropriate assessment gauges and tools for teaching evaluations according to preset professional standards in course outlines
		Advanced	6-1-5 Develop and apply digital teaching platform and evaluation tools to assess students’ learning achievements
	6-2 Apply evaluation results as feedback to guide students to develop effective learning strategies	Core	6-2-1 Analyze students’ learning achievements and master students’ learning problems based on evaluation results and provide necessary feedbacks
			6-2-2 Guide students to estimate individual learning outcomes and formulate individual learning strategies based on their learning achievements
		Advanced	6-2-3 Apply evaluation results to help students develop their ability for self-growth and initiative learning
	6-3 Adjust evaluation methods in a timely manner according to students’ characteristics and learning needs	Core	6-3-1 Adjust evaluation methods in a diversified way to meet students’ learning needs
		Advanced	6-3-2 Observe and discover students’ learning difficulties or needs in timely manner and transfer students with special needs when necessary
			6-3-3 Design and carry out individual teaching and evaluations in accordance with students’ learning needs
7. Exert skills in classroom management and build a supportive learning environment	7-1 Set up classroom regulations to contribute to learning and build a favorable classroom atmosphere	Core	7-1-1 Establish clear and reasonable classroom regulation for students to follow
			7-1-2 Communicate the content and implementing procedures of class regulations with students and remind students to follow them in timely way
			7-1-3 Apply diversified strategies of classroom management and encourage students to join in class affairs actively
			7-1-4 Value students’ diversified cultural backgrounds and individual differences to establish a good human relationship between teacher and students
			7-1-5 Encourage students to explore actively and cooperate with peers to create good learning atmosphere
		Advanced	7-1-6 Plan activities for the whole class to participate in and create a caring, friendly, fair, and active learning environment to cohere centripetal force and identity of the class
	7-2 Arrange learning situations contributive to the interaction between teacher and students to promote students’ learning achievements	Core	7-2-1 Arrange learning environment and learning activities appropriately to facilitate the interaction between teacher and students
			7-2-2 Apply the available resources of school well so as to create a safe, clean, and happy environment
			7-2-3 Provide all students with equal opportunities to participate in learning activities
			7-2-4 Arrange the teaching environment of classrooms or internship workshops in accordance with instructional units to inspire students’ learning interest and promote learning achievement
			7-2-5 Encourage students to express and give positive feedback and guidance to activate the interaction and communication between teacher and students
			7-2-6 Encourage students to join various learning activities according to students’ abilities
			7-2-7 Design and arrange learning situations and guide students to conduct learning and problem investigation through group collaboration
		Advanced	7-2-8 Create a learning atmosphere of cheerful expression and sharing to promote students’ ability for independent thinking and problem-solving
	7-3 Master learning status of classroom and deal with classroom issues appropriately	Core	7-3-1 Apply positive measures of guidance and discipline and related resources to correct unfair behaviors of students
			7-3-2 Understand the process flow of accidental events and promulgation mechanism of school safety incidents and apply when necessary
			7-3-3 Be familiar with the principles and skills for dealing with various class events during the teaching process and handle them appropriately
		Advanced	7-3-4 Observe the symptoms of unfair behaviors of students and exert the ability of crisis management in a timely manner to resolve learning interference problems beforehand
8. Master the individual differences of students and provide related guidance	8-1 Understand students’ differences and interests and guide students toward adaptive learning and career development	Core	8-1-1 Collect, analyze and use related information to understand students’ backgrounds and individual differences and provide recommendations for adaptive learning
			8-1-2 Recognize the learning ability of students and help students in using available resources to enhance the effectiveness of learning
		Advanced	8-1-3 Apply diversified measures of guidance and resources to assist students in exploring self -interests and aptitudes and develop personal expertise
	8-2 Understand students’ culture and guide students to affirmative social learning	Core	8-2-1 Collect related information to understand students’ human relationships, life experiences, and values to guide affirmative learning of students
			8-2-2 Make good use of peer learning to guide students to respect diversified cultural difference and establish good human relationships
		Advanced	8-2-3 Guide students to affirmative social values and behaviors and the ability of having social interaction in accordance with students’ differences in culture and values
	8-3 Respond to the individual needs of students and provide necessary support and guidance	Core	8-3-1 Confront students’ problems and provide responsible guidance and carry out necessary arrangements
			8-3-2 Discover the problems of students actively and seek available resources to provide assistance or transfer
		Advanced	8-3-3 Make good use of various guidance mechanisms and website resources to help students in solving problems
9. Fulfill responsibility for the educational profession	9-1 Demonstrate educational enthusiasm in taking care of students and establishing cooperative relationships with educational partner	Core	9-1-1 Demonstrate educational love and passion, education for everyone, irrespective of background spirit, and respect for differences of students to provide assistance on the learning
			9-1-2 Take the initiative to care for students and actively seek resources to take care of disadvantaged groups, safeguard the interests of students.
		Advanced	9-1-3 Meet the needs of the curriculum and teaching activities and build relationships with the community and industry partners to integrate related resources and teaching content to expand students’ learning experiences and promote related activities.
	9-2 Comply with professional ethics and related laws and regulations for teachers	Core	9-2-1 Comply with professional ethics and maintain the professional image of a teacher.
		Advanced	9-2-2 Understand and comply with laws and regulations related to education and perform teacher duties.
	9-3 Join in school’s affairs and meetings in response to the development needs of schools	Core	9-3-1 Concerned with school development, actively participate in the affairs of the school and relevant committee meetings.
		Advanced	9-3-2 Act as mentors or serve in administrative positions to participate in school affairs.
			9-3-3 Make good use of teaching experience and practical knowledge to assist schools in planning and implementing school development plan.
	9-4 Participate in the development activities of curriculum and instruction in school	Core	9-4-1 Participation in school curriculum and counseling and assist peer teachers to develop their professional abilities.
		Advanced	9-4-2 Actively involved in the school’s development plan and school’s curriculum elaboration and promotion.
10. Endeavor to obtain teacher’s professional development	10-1 Participate in instructional research and advanced studies to promote abilities in teaching and professional practice	Core	10-1-1 Meet the needs of teaching through discussion with peers to elaborate on teaching plan.
		Advanced	10-1-2 Reflect on the effectiveness of teaching process and review professional practice and teaching content to elaboration solutions.
			10-1-3 Participate in the needs of own knowledge and ability in professional training, research, or related growth activities, and apply learning outcomes to teaching activities and practice course.
			10-1-4 Perform action research to develop problem-solving strategies and improve teaching practice.
			10-1-5 Update and embed the most advanced knowledge into courses to improve teaching quality.
			10-1-6 Be actively involved in curriculum research and practical training on the development of teaching materials and methods of professional courses that put teaching into action and enhance the effectiveness of teaching.
			10-1-7 Fit in with school development plan and enrich own learning to widen research capacity and the development of characteristic education.
	10-2 Participate in professional societies in specialized field to enhance professional development	Core	10-2-1 Participate in school teaching and research meetings, teaching growth groups to have professional dialogue and improve teaching programs that promote teaching effectiveness.
			10-2-2 Participate in internal and external professional learning communities to observe, learn, and promote professional development, and share those experiences with other teachers.
		Advanced	10-2-3 Seek industry resources and establish a supportive and effectiveness of partnerships with industry, through collaborative research to implement teachers’ professional development.
			10-2-4 Participate in or establish inter-institution/cross-community, strategic alliances and other teams for integration of theoretical and practical experience into teaching activities and continued diligent teaching to promote further professional growth.

Reference: Partially adapted from MOE [65] and Chen et al. [66].

of Appendix B.

4.2.2. Pilot Test Results

Descriptive statistics, reliability, and construct validity of evaluation indicators were reviewed in the pilot test based upon the questionnaire being collected from 53 practical training supervisors. According to the results being derived by the descriptive statistical analysis (n = 53), no element hindering item quality was identified. Overall Cronbach’s α was 0.992 (for each variable, 0.991 ≤ α ≤ 0.992) in the pilot test. Therefore, the measures are relatively reliable.

In this study, predicted correlations and λ values used to compute r_altering-CV and r_contrast-CV can be derived based on the analytic procedure proposed by Westen and Rosenthal [64]. The results of these analyses for the newly added aspects are shown in

Table 3. Predicted correlations, raw λs, and integer values of raw λs versus new aspects.

Construct	Predicted Correlations	Raw λs	Integer Values of Raw λs
D1	0.713	−0.414	−1
D4	0.780	0.252	0
D5	0.833	0.779	0
D6	0.702	−0.529	−1
D7	0.751	−0.038	−1
D8	0.762	0.075	0
D9	0.755	0.000	−1
D10	0.742	−0.124	−1

and

Table 4. Construct validity indices.

Statistics	raltering-CV(1)	rcontrast-CV	95% CI	zcontrast	tcontrast	pcontrast
Values	0.996	0.417	0.166–0.721	3.268	3.273	1.912 × 10−3 (2)

Notes: ⁽¹⁾ CI: Confidence Interval of r_contrast-CV. ⁽²⁾ The r_contrast-CV coefficient is statistically significant (p_contrast < 0.01).

. Large and moderate values for r_altering-CV and r_contrast-CV were found (r_altering-CV = 0.996; r_contrast-CV = 0.417). This suggests substantial correspondence between theoretical predictions and observed correlations.

Thomas et al. [73] mentioned that no generic benchmarks for evaluating the magnitude of r_contrast-CV coefficients are available since comparatively fewer researches have been conducted using the coefficients proposed by Westen and Rosenthal [64] to test the construct validity. Therefore, judging whether the r_contrast-CV values are comparatively higher or lower is not easy [73]. In order to confirm the construct validity of the performance evaluation system, we compare our results with the analytic results derived in past works. In general, the values derived in the pilot test are similar to the levels of r_contrast-CV coefficients in a previous work by Bombel et al. [74] (most r_contrast-CV values belong within the range 0.39 to 0.59) and Thomas et al. [73] (the minimum acceptable value is 0.41, which is a little bit lower than our result). Therefore, we confirm the construct validity accordingly.

4.3. Pre-Service Teacher Professional Performance Evaluation and Review Mechanism

To pursue expertise and quality in teacher development and training, advanced countries have established professional standards to determine practical objectives for teacher development and training as well as evaluation and professional development. Scheeler and Ruhl [75] proposed that teacher development and training evaluation must adopt multi-element feedback data, particularly for collecting “process-oriented”, “future oriented”, and “purpose of capability development”. The MOE promulgated the “Measures for Examining Teachers’ Qualification for Schools Below High School Level and Kindergartens” on 31 July 2003, and changed the original examination on the formality of teachers’ qualifications to a qualitative examination with a professional certificate, thus truly fulfilling the objective of “keeping the good, and retiring the bad” [65].

Based on the practical features of vocational education, teacher development and training institutes should provide various types of learning opportunities. The best implementation of teaching expertise is through observing teaching on-site and practicing it. Normal universities should adopt teacher evaluation knowledge, the ability for teacher development, and a training system or curriculum. This evaluation system would provide pre-service teachers with a basic understanding of teacher evaluation. Most of the credit hours in education offered at normal universities somewhat lack courses that are designed with evaluation in mind. Teacher evaluation knowledge and ability is almost impossible to cover in a single course. Furthermore, conflicts with other courses should also be considered. It is very important for teachers to understand the meaning of evaluation, the standards, how to observe teaching, and how to conduct an evaluation. The 26 credit hours in most education curricula and the Ministry-stipulated pre-service teacher development and training specialty courses for industrial subjects are limited by time and space. The internship required in industrial subjects offers little chance of practicing teaching. In addition, professional growth has become a subject that requires ongoing learning. Teacher evaluation, in this regard, will be beneficial. Also, there is little room for many teacher evaluation courses, resulting in a lack of understanding of teacher evaluations and the meaning of professional ability, reasons, and methods for evaluating teaching performance.

Therefore, the second-stage research aimed to establish an evaluation standard and construct an examination and review mechanism for pre-service teachers in automobile repair. After development of the professional standard in the first stage, this study formed a focus group consisting of five experts in vocational education and one expert in vehicle engineering. The experts included five professors and one senior lecturer. The professors each have more than five years’ experience in teaching graduate-level courses in vocational education and/or courses in specific professions. The senior lecturer has more than forty years of experience in pre-service teacher education. The background of the experts is demonstrated in

Table 5. Background of experts for the focus group meetings.

No.	Education	Expertise	Title	Experiences
1	Ph.D.	Vocational education	Professor	24 Years
2	Ph.D.	Vocational education	Professor	10 Years
3	Ph.D.	Vocational education and management	Professor	22 Years
4	Ph.D.	Vocational education	Associate Professor	32 Years
5	Ph.D.	Vehicle engineering	Associate Professor	27 Years
6	M.S.	Vocational education	Lecturer	42 Years

.

Based on the opinions being summarized in the 14-iteration focus group meetings, this study has established a three-way evaluation standard and constructed an examination and review mechanism for pre-service teachers in automobile repair. This is practical for vocational education because it adopts professional performance and performance in vocational practical courses and also industry–academics collaboration. It also includes the concept of self-improvement in both teacher evaluation and analysis. The three-way evaluation standard includes the three aspects of cognitive, psychomotor, and affective capacity: (1) Cognitive evaluation aims to allow the teacher’s students to memorize, understand, apply, analyze, evaluate, and create professional course content. The evaluation method can be further classified as a multiple choice questionnaire-based written test and the evaluation of the pre-service teacher’s professional growth file. (2) The psychomotor evaluation aims to allow pre-service teachers to apply cognitive, readiness, guided responses and mechanical, complex, external responses in professional internships in industrial subjects. The evaluation method includes a professional skills test and an oral evaluation of professional knowledge, operations procedure, and simulated teaching in an internship. (3) The affective evaluation aims to allow pre-service students to adopt, respond, evaluate, reconstitute, and formulate integrity in teaching professional course content. The evaluation method includes observation and a review based on a checklist. The three-way evaluation standard and the corresponding professional standards as well as performance indexes are summarized in

Table 6. Three-way evaluation standard.

Aspect	Evaluation Method	Details of the Evaluation Method	Professional Standards	Professional Performance Indexes
Cognitive Evaluation	Written test (Multiple choice)	A standardized written test for industrial subject professional course is prepared to evaluate pre-service teacher students’ professional awareness of professional mechanical drafting.	1, 4, 5, 6, 7, 8, 9, 10	All Professional Performance Indexes belonging to each Professional Standard are applicable.
	File evaluation (Pre-service teachers professional growth file)	Record teacher students’ learning results to enhance their self-improvement, self-reflection, diagnosis, feedback, communication, and evaluation ability for professional courses in industrial subjects.	1, 4, 5, 6, 7, 8, 9, 10	All Professional Performance Indexes belonging to each Professional Standard are applicable.
Psychomotor Evaluation	Professional skills test	Have skills equivalent to Class B Skill Examination stipulated by the Ministry of Labor.	2, 3	2-1, 2-2, 2-4, 3-1, 3-2, 3-4
	Operation tests to show professional skills	Operation tests for operation procedures and simulated teaching in internship subjects.	2, 3	2-1, 2-2, 2-4, 3-1, 3-2, 3-4
Affective Evaluation	Evaluated via observation and examination, as well as check list	Evaluate whether a pre-service teacher can fulfill the objectives of affective education in all phases of professional courses.	2, 3	2-3, 3-3

.

5. Discussion

In this Section, how the professional standards defined in this work can fulfill the specific requirements for domain knowledge and those for generic teacher development and training of a pre-service high school teacher will be discussed. Further, how the validity and reliability of the system and assessment criteria can be accessed appropriately will be discussed. The reasons why two different research methods were introduced to summarize experts’ opinions are worth discussing.

5.1. The Fulfillment of Specific Requirements for Domain Knowledge and Teacher Development

Vocational high schools seek to provide human resources for industries; the curricula primarily cover general and professional subjects. Domain-specific teachers possess specific knowledge and competency, which are different from those of teachers in other fields. Therefore, domain-specific professional standards are especially critical for pre-service teacher education for vocational high school teachers. Teacher evaluation is the process of testing teachers against maintaining teaching standards [76]. Many researchers and policymakers believe that the quality of teachers can only be enhanced by collecting personal productivity information based on the evaluation of teachers and then, dismiss unqualified ones [77]. Since the evaluation of teachers is very critical for assuring teacher quality, results of evaluation can serve as the basis for improvement of teachers and schools [78]. Over the last decade, the role of teacher evaluations is becoming increasingly important. Policymakers are increasingly concerned about the overall quality of teachers and the distribution of effective teachers in schools [79]. Due to the complexity of the external environment as well as the uncertainty and limitation of human understanding, it is not easy to establish a more reasonable and effective method of teacher evaluation [80]. However, in the current research, such concerns have been reflected in the definition of professional standard and performance evaluation criteria.

The development and training of pre-service vocational high school teachers in automobile repair differ from those of generic teachers in terms of its professional performance. Aside from the core courses being defined by the MOE and schools, this study focuses on the current objectives adopted by vocational high schools. Also, except for the overly theoretical courses, this study proposes to adopt project-based, profession-oriented, and internship courses to enrich the pre-service teacher profession and their teaching experiences. To fulfill the above requirements, the possessing of professional knowledge and the mastering of key issues in technological and vocational education have been included as the first Professional Standard: possess professional knowledge and master key issues in technological and vocational education (refer to Appendix A). Meanwhile, the second and third Professional Standards include domain knowledge related to dynamic mechanics and automobile. In addition, industry–academic collaboration plans are required for diversified career paths of current Taiwanese vocational high school students. This requires the participation and assistance of vocational high school teachers so as to close the gap between learning and practice. These specific needs were also considered in Professional Standard 4 for the pre-service vocational high school teachers (please refer to Appendix B for criterion 4-2-4: carry out school-industry cooperation and integrate available resources to design augmented teaching activities). Finally, the specific requirements for preventing pollutions in automotive repair and, thus, enhance sustainability development, have also been emphasized in the professional standard. In Professional Standard 2-3-1, developing the capabilities and literacy for dynamical machine-related environment protection capabilities and literacy has been defined as an index. Also, in Professional Standard 3-3-1, developing the capabilities and literacy of automobile repair-related environment protection capabilities and literacy has been defined as an index. These professional standards fulfill the needs for vocational school teachers’ possessing domain-specific knowledge and competency related to automobile and dynamic mechanics. Domain-specific knowledge is different from those of teachers in other fields.

5.2. The Validity and Reliability of the System and Its Assessment Criteria

This study seeks to determine the reliability and validity of the system as well as the value of the assessment criteria involved. According to Wei and Pecheone [81], the teacher performance evaluation varies in both content and format aspects. In this regard, it is important to always evaluate their technical quality in terms of validity and reliability. Further, three performance-based assessment criteria are always adopted: (1) The usefulness for formative purposes, (2) the credibility and defensibility for summative purposes, as well as (3) the practicability and feasibility [81]. Further, no matter the value-added model or the standard-based observations, the teacher performance evaluation should fulfill the criteria: unbiased, reliable, and valid [82].

As far as testing is concerned, validity is the hallmark of quality and the single most important criterion for evaluating a test [83,84]. Validity is also the most fundamental consideration when developing and evaluating assessments. If an evaluation system does not prove its validity, then the value of its accountability, professional development, or any other legitimate purpose will be seriously compromised [85]. From the viewpoint of teacher assessment, validity is very important; however, in the context of reviewing the validity of contemporary policy-oriented teacher assessment systems, the literature remains relatively scarce [86]. Formal researches, which specifically assess teacher evaluation systems using a validation construct by emphasizing theory-inference evidence, are still very rare [87]. Most studies that have investigated validity claims focus on traditional criterion-based measures for assessing the relationship between academic performance and any assessment of instructional behavior.

The majority of past works on validity focus on traditional standard-based measurement methods used to assess the relationship between academic performance and any evaluation of teaching behavior [88,89,90,91]. Papay [82] further points out that standards-based evaluations are based on traditional teacher observation models, which are beyond simple classroom observations. These evaluations depend on the observation of all teachers in the region, not just on the test scores and subjects of the teacher. The annual total cost of implementing this performance-based evaluation system for all the pre-service teachers in some specific state or country is still unclear [81]. According to earlier estimations, the cost for every pre-service teacher may range from two hundred to four hundred USDs [81].

In general, the validity and reliability of the system should be evaluated based on both planned and long-term observations instead of simple observation of pre-service teachers in classrooms. However, such evaluations are both time-consuming and very expensive. Therefore, in this research, the professional standard and the pre-service teacher professional performance evaluation and review mechanism are only confirmed as having validity, reliability, and feasibility by the experts based on their own experiences. Any evaluations of the validity and reliability of this professional performance evaluation system are worthwhile venues for future studies.

6. Concluding Remarks

Pre-service teacher performance should be carefully evaluated and analyzed to assure education quality. Evaluation results can be used to improve teacher quality and define professional development plans. Such evaluations can be defined as a process used to evaluate and determine teaching performance according to a carefully defined standard. Albeit important, teacher evaluations in general, and the evaluations for pre-service vocational high school teachers in automobile repair in special, can be a daunting task due to the involvement of the evaluations of teaching skills, classroom management skills, contribution to counseling and guidance of students, relationship with parents, compliance with the school’s administrative measures, personality traits, and professionalism. These factors should be included as criteria being needed for teacher evaluation. This study designed a professional performance evaluation system for pre-service vocational high school teachers in automobile repair. Following are the major conclusions of this study.

At first, the performance evaluation criteria of the pre-service vocational school teachers should include the results of technician certification qualification examinations, national skill competitions, research projects, and industry academic cooperation. Further, teacher development and training institutes should provide information for domain-specific certificate qualifications. Also, institutes should urge pre-service teachers in preparing and passing the technician certification qualification examinations through appropriate designs of the evaluation mechanism. Such a mechanism can enhance the expertise and confidence of pre-service teachers before the initiation of their career as teachers. The evaluation methods of vocational high school pre-service teachers should be diversified further for professional development. The 26 credit secondary education courses defined by the MOE and the domain-specific courses for expertise (e.g., automobile repair) may not be sufficient for pre-service teachers to fully develop the competencies required for serving as a teacher. Since the informal curriculum can compensate for such insufficiencies, diversified evaluation methods, including different aspects and criteria, can be included in the evaluation system to assist pre-service teachers in developing teaching experience. Informal curriculum can be divided into different types according to the majors. In the case of the pre-service teachers in automobile repair, the informal curriculum can include off-campus visits, off-campus internships, skill competitions (on-campus, interschool, regional, and national), exhibitions (outcomes, works), extracurricular activities, skills verification, etc. The career and profession can further be developed through assistance in the development and use of pre-service teachers’ portfolios.

Finally, these orientation courses should be included in the evaluation system for pre-service vocational high school teachers. The evaluation of teachers means systematic collections and a subjective analysis of information being related to teachers’ backgrounds and performances. Teachers’ backgrounds include their professional background and work load, while teacher performance can be evaluated via the aspects of teaching, researching, counseling, and guidance as well as service. The complexity of the performance evaluation framework implies the difficulties in cultivating excellent teachers by teacher education systems alone and the importance of the professional performance evaluation system for the pre-service vocational high school teachers. However, the development and enhancement of the current pre-service education curricula only focus on the availability of core courses. Pre-service teachers are evaluated based on such curricula. However, the evaluation criteria are insufficient for the needs of pre-service teacher professional development. An inclusion of these orientation courses into the performance evaluation system will be very helpful for enhancing the professional development of all pre-service vocational high school teachers.