Teaching Ecosystem Design: Teachers’ Satisfaction with the Integrated Course Service System

Abstract

In order to support teachers preparing their teaching resources, a teaching ecosystem for developing an online service system to acquire teaching materials, the Ministry of Education (MOE) in Taiwan, implemented an Integrated Course Service System (ICSS) to assist teachers in searching for instructional resources from distributed Educational Six Learning Networks (ESLN) of MOE. The study designed the ICSS and developed an instrument to investigate the satisfaction for the teachers of primary schools, junior high schools and senior high schools. By analyzing 253 teachers, the results show that demographics, such as age, school level, teaching experience and position, have significant effects on the ICSS satisfaction. Further, the teachers were found to be satisfied with the system content, system accuracy, report format, ease of use, system timeliness and community building offered by the ICSS that helps us to understand the teachers’ online behavior in more detail for the popularization of the course service system.

1. Introduction

An integrated course service system is useful for knowledge generation, teaching and social innovation for sustainability [1]. Social networks are inherent in any e-learning system, and their main actors are teachers, learners and learning resources [2,3]. Therefore, in a digital teaching and learning environment, the teaching resources used for supporting teachers and students are important [4,5]. Creating a virtual community of instructors is one strategy for increasing their satisfaction with regards to the digital teaching and learning environment. When the instructors have a sense of belonging to the site, it will foster their willingness to share materials, enhancing their interests in the system service. The knowledge and instructional resources system platform shapes a professional virtual community, and the teachers can share norms, values and professional dialogues with each other [6,7]. Anderson and Dron [2] further pointed out that the virtual community may finally form a social network to support the digital teaching and learning activities. Thus, the ‘School-Teacher’s Learning Community’ can provide a new path within in-service teachers’ education [8,9]. Additionally, the type of presentation of teaching materials is important for course adaptation and can be supported by a course management system [10,11].

In Taiwan, the Ministry of Education (MOE) created a project called the Integrated Course Service System (ICSS), which integrates the Educational Six Learning Networks (ESLN) to support teachers (including primary school, junior high school and senior high school teachers) to search their instructional materials, displaying the resources with personalized requirements. At present, there are 321 schools participating in the ICSS and more than 2600 teachers using the ICSS to help them prepare teaching materials.

Online communities are virtual spaces over the WWW (World Wide Web), in which groups of people with similar interests or purposes interact with others and share information [12,13,14]. Yilmaz [15] pointed out that users’ willingness to share knowledge and high-quality content will influence the successful use of virtual communities. Besides the content quality, Tam and Oliveira [16] also find that system quality and system service are important determinants of members’ loyalty to the virtual community. As the virtual communities can offer educators the opportunity to interact with other members, they can create professional virtual communities that allow members to share education-related knowledge [7]. Besides the benefits of getting requested teaching materials, the ICSS can also support teachers to socialize with other members through the virtual community [17].

While exploring the contribution of an IS (Information System), the most commonly used measure of effectiveness within the IS field is user satisfaction [16,18,19]. Therefore, satisfaction is often seen as a key variable in predicting the success of an IS or users’ reuse behavior [20]. Commonly, the greater the satisfaction users have with their online services, the stronger will be their feelings about its perceived usefulness [21,22,23]. The measure of satisfaction degree will support the assessment of a successful course service in a teachers’ learning community [8,9].

Previous literature has suggested that user satisfaction should cover the entire customer experience cycle from information retrieval through to purchasing, payments, receipts and post-purchase services [16,18]. The user satisfaction research developed thus far can generally be conceptualized around two different types of satisfaction: transaction-specific satisfaction and cumulative satisfaction [24,25,26]. Transaction-specific satisfaction assessment focuses on the user experience in a product episode or service encounter. Cumulative satisfaction focuses on a user’s overall experience to date with a product or service provider, rather than on a transient perception of a product or service at any given point in time [27]. Min, Khoon and Tan [28] modified the SERVQUAL framework to evaluate learners’ satisfaction, finding that service quality is not just about ‘service operation’ and ‘service process’, but also the real benefits of services as products.

Satisfaction refers to a psychological state that is an emotional response. The emotional basis for satisfaction surrounding disconfirmation expectations is confirmed by an individual’s prior feelings and responses [29]. IS satisfaction covers the extent to which users believe the IS available to them meets their information requirements [30].

There are few studies that have examined teachers’ satisfaction, although instructors’ satisfaction is considered as significant for course participation and increasing student connections with the course activity [31].

Bailey and Pearson [32] considered satisfaction under a given situation as the sum of one’s feelings or attitudes toward a variety of factors affecting that situation and identified 39 factors that affect satisfaction. This definition, which includes attitudinal and affective responses, is also supported by Doll and his colleagues, who used it to develop their own measurement of user satisfaction [33,34,35]. Prior studies highlighted course content, the information system and the facility for accessing and visualizing information on the teaching platform—as well as the possibility of interaction—as keys to enhancing their teaching satisfaction [30,31,36].

This study followed the perspective of Bailey and Pearson [32] and Johnson et al. [27] to measure teachers’ satisfaction with the ICSS. Although Doll and Torkzadeh’s [34] satisfaction scale is popular and has been verified in the online context, it was developed in an end computer context and remains limited in its ability to measure the communication characters of a virtual community [37,38]. Therefore, the study investigated the precise composition of user satisfaction and community building [39,40] to evaluate teachers’ satisfaction with the ICSS.

2. The Implementation of the ICSS

2.1. Strategy of ICSS Development

According to the original concepts and the procedures required for an ICSS user requirements analysis, we interviewed 4 principals, 6 directors of academic affairs, and 10 teachers to determine their requirements for the ICSS. We also sought to identify the resources provided to teachers. Following the interviews, five development strategies of the ICSS were suggested. They were: (1) providing batch functions for the searching of instructional materials; (2) providing a storage and presentation area for instructional materials; (3) reducing the teachers’ workloads when they maintain their own school Course Service System (CSS); (4) promoting the use of the ICSS among teachers and providing a convenient search method for instructional materials; (5) facilitating empathic learning on teaching courses and forming a professional virtual community.

2.2. The Architecture of the ICSS

The ICSS was developed using Moodle Open Source software to serve teachers of primary schools, junior high schools and senior high schools in Taiwan. With this system, teachers can store and display their instructional materials using a personalized format in the ICSS. Although every school has its own CSS, teachers can participate in empathic learning from other schools’ CSSs via the “single sign-on” function authenticated by the external database of Moodle. The architecture of the ICSS is shown in Figure 1 below.

The searching mechanism for digital instructional materials is presented in Figure 2. The database of the ICSS stores all the metadata of the digital instructional materials within the ESLN. When teachers upload digital instructional material to the ESLN, the metadata of the material is written into the ICSS database automatically. Similarly, when teachers want to search for their instructional materials, they merely input the keywords into their own school CSS, and the digital instructional materials of the ESLN will be searched for. When the searching activities are complete, the digital instructional materials will be presented in the user’s personalized format.

2.3. Processes of Searching

This study highlights the need to reform interface design based on the syllabus, in order to develop a convenient method of searching for digital instructional materials. Teachers could then enter the keywords to suit their needs. With a personalized interface design, the ICSS could provide a personalized format to display the searched instructional materials. It is proposed here that when teachers want to search for instructional materials, they should follow four stages: first, users would select the import methods for their syllabuses; second, in editing mode, users would modify their syllabus and keywords; third, users would search for and select the necessary instructional materials; and finally, users would export those instructional materials.

2.4. Operation of the ICSS

According to the concept of function blocks in the ICSS (see Figure 3), ICSS administrators manage course syllabuses and create/delete the course presentation for each school. The process suggested is represented in Figure 3. Teachers select the edited course to form their course packages. When teachers download the course packages, the ICSS can automatically export those course packages with the Moodle format. Teachers can then use their own school CSS to create a course and then import the course packages into their school CSS. More detailed operations are described below according to a number of steps.

Figure 4 shows the first step in creating a course/import syllabus. At the start of the process, teachers can select a personalized online environment for creating or importing a new course. Once teachers have finalized their selection of a teaching subject and edition of a book, onscreen messages related to the topic, keywords and recommended instructional materials are displayed in the second step (see Figure 5). This approach can avoid verbose key-in processes and reduces time. By tracking teachers’ online actions, the ICSS can support knowledge-sharing and achieve the goal of coordinated teaching.

Two sub-functions are provided in the third step. The first is the search for digital materials—after teachers select a keyword (as an example, ‘Pi’ is showed in Figure 6), the ICSS begins the search. The results of a search are shown in Figure 7. Teachers can view the metadata of instructional materials and select the hyperlink to store their chosen instructional materials. The second sub-function involves the selection of digital materials—with a multiple select function, teachers are able to select multiple instructional materials and classify them. The ICSS will present the selected instructional materials based on their classification and reduce the required search time. Once the search for all keywords is completed, it exports the complete course packages for the selected instructional materials in the fourth step (see Figure 8).

As demonstrated in Figure 8, the results show that the topic links to the initial material storage. This approach can promote the utility of instructional materials and provide the records needed for the CSS to analyze the usage of instructional materials.

3. Methodology

A measurement of teachers’ satisfaction was taken after ICSS implementation. Doll and Torkzadeh’s satisfaction scales (regarding system content, system accuracy, report format, ease of use, and system timeliness) [34] are widely used to measure user satisfaction. However, the scales of Doll and Torkzadeh were developed in the context of end user computers, focusing on the interaction between human and machine. In the context of the World Wide Web, the knowledge-sharing via a virtual community involves frequent interactions between individuals. Considering that the ICSS is focused on the sharing of instructional material resources, this study integrated community building with Doll and Torkzadeh’s [34] satisfaction scales to measure ICSS user satisfaction. All of the scales were refined by three experts who are experienced with the management of school CSS and three teacher users who are familiar with the ICSS context. The scales were designed as seven-point Likert scales, ranging from 1 for “strongly disagree” to 7 for “strongly agree”.

Sample data were collected from the members of the ICSS, who were invited to participate in a web-based survey. Among the 262 received questionnaires, 253 completed questionnaires were used for the analysis.

4. Reliability and Validity

Reliability was evaluated by internal consistency, such that a factor’s Cronbach’s alpha with a value of greater than 0.7 means good reliability [41]. The reliability of each factor is as follows (see

Table 1. Reliability and construct validity.

Dimensions	Measured Items	Factor Loading	Eigen-Values	% of Var.	Cronbach’s α Value
System content	SC1: Does the site’s digital content meet your needs?	0.918	4.499	78.034	0.904
	SC2: Does the site provide the precise content you need?	0.883
	SC3: Does the site provide help that seemed to be exactly what you need?	0.874
	SC4: Does the site provide sufficient instructional materials?	0.856
System accuracy	SA1: Are the searching functions of the site accurate?	0.964	2.391	92.863	0.923
	SA2: Are you satisfied with the accuracy of the site?	0.964
Report format	RF1: Do you think the instructional materials of this site are presented in a useful format?	0.964	2.425	92.442	0.918
	RF2: Are the site instructional materials presented in a clear format?	0.959
Ease of use	EU1: Is the site easy to use?	0.966	3.252	92.567	0.919
	EU2: Is the interface of the site user friendly?	0.958
System timeliness	SI1: Does the site provide up-to-date instructional materials?	0.966	2.605	89.067	0.877
	SI2: Can you access the instructional materials you need quickly on the site?	0.958
Community building	CB1: I have a sense of belonging when using the site.	0.887	6.028	67.273	0.901
	CB2: I am interested in a club for the site’s users and the service provider.	0.862
	CB3: I feel a sense of kinship with other teachers who use the site.	0.855
	CB4: I have met really nice people through the site.	0.801
	CB5: The site provides a message board forum for teacher-to-teacher/site manager communications.	0.773
	CB6: Teachers can share experiences about the services or instructional materials obtained from the site with other teachers who also use the site.	0.733

Global Cronbach’s alpha = 0.969.

): the global Cronbach’s alpha is 0.969, system content is 0.904, system accuracy is 0.923, report format is 0.918, ease of use is 0.919, system timeliness is 0.877, and community building is 0.901. Therefore, the instrument of this study was found to have good reliability.

The basic qualitative criterion concerning validity is content validity. Since there were three experts of digital material design and three teachers who assisted in refining the questionnaire, the instrument of this study can be said to possess content validity. Further, this study adopted exploratory factor analysis (EFA) to verify the construct validity using a varimax rotation. The results of the EFA (see Table 1) revealed that each of the variables demonstrated sufficient construct validity, because the factor loading of each item in the construct is above the recommended cut value of 0.5— the eigen-value of each variable is greater than 1 [41]. Overall, the results provided empirical support for the scales used to test user satisfaction with the ICSS.

5. Finding

As the teacher characteristics relate to the teaching performance and IT use, the six demographics were examined in relation to their impact on teachers’ satisfaction with the ICSS. An ANOVA test was conducted, using demographics as the independent variable, and total ICSS satisfaction and each component satisfaction as the dependent variables (i.e., system content, system accuracy, report format, ease of use, system timeliness, and community building). The results for all of the ANOVA tests are shown in

Table 2. ANOVA tests of ICSS satisfaction by the p values of demographic variables.

Satisfaction Components	Age	Effect Size (Age)	School Level	Effect Size (School Level)	Experienced in Teaching	Effect Size (Experienced in Teaching)	Position	Effect Size (Position)	Gender	Effect Size (Gender)	Education Degree	Effect Size (Education Degree)
System content	0.007 a	0.012	0.000 a	0.012	0.008 a	0.024	0.109	0.188	0.438	0.215	0.229	0.007
System accuracy	0.013 b	0.013	0.002 a	0.007	0.068	0.005	0.900	0.105	0.360	0.139	0.147	0.003
Report format	0.196	0.007	0.023 b	0.010	0.274	0.005	0.105	0.227	0.725	0.081	0.260	0.004
Ease of use	0.001 a	0.008	0.019 b	0.002	0.080	0.013	0.120	0.135	0.824	0.048	0.253	0.005
System timeliness	0.042 b	0.010	0.001 a	0.015	0.134	0.005	0.605	0.204	0.737	0.085	0.148	0.005
Community building	0.008 a	0.010	0.007 a	0.004	0.018 b	0.009	0.643	0.207	0.629	0.134	0.296	0.004
Total satisfaction	0.005 a	0.004	0.001 a	0.004	0.029 b	0.011	0.043 b	0.231	0.718	0.080	0.169	0.009

^a Significant at the 0.01 level. ^b Significant at the 0.05 level.

. It appeared that four of the six demographics—age, school level, experienced in teaching, and position—had a significant effect on the total ICSS satisfaction. The factor of gender has no significant association with satisfaction. The possible reason is as follows: the purpose of ICSS is to assist teachers in preparing teaching materials and provide tools for sharing knowledge and experience. In Taiwan, the gender of most of the teachers in primary school and junior high school is female. The study respondents were 34.75% male and 65.25% female. These results imply that each teacher has basic information literacy and the ability to use the ICSS. The means of each demographic classification related to the six ICSS satisfaction components are listed in

Table 3. Mean analysis of ICSS satisfaction.

Demographics	Categories.	Frequency (%)	Means of ICSS Satisfaction
			Content	Accuracy	Format	Ease of Use	Timeliness	Community Building	Total Satisfaction
Age	Below 26 years	7.63%	5.13	5.24	5.43	5.26	5.06	4.95	5.18
	26~30 years	16.67%	4.94	5.07	5.14	4.65	4.84	4.68	4.89
	31~35 years	24.29%	5.03	4.92	5.17	4.90	4.98	4.83	4.97
	36~40 years	16.1%	5.21	5.33	5.29	5.19	5.11	4.91	5.17
	41~45 years	20.34%	5.47	5.43	5.54	5.42	5.37	5.17	5.39
	46~50 years	8.76%	5.62	5.48	5.42	5.27	5.19	5.13	5.35
	Above 50 years	6.21%	5.56	5.66	5.66	5.77	5.64	5.50	5.63
School level	Primary school	64.69%	5.40	5.33	5.43	5.25	5.29	5.08	5.29
	Junior high school	24.86%	4.83	4.95	5.05	4.80	4.81	4.72	4.86
Experienced in teaching	Senior high school	10.45%	5.08	5.30	5.38	5.12	4.80	4.81	5.08
	Below 2 years	12.15%	4.88	5.06	5.16	4.93	4.90	4.80	4.95
	2–5 years	14.69%	5.22	5.15	5.28	4.97	5.05	5.04	5.12
	6–10 years	21.47%	5.23	5.14	5.31	4.91	5.12	4.95	5.11
	11–15 years	19.77%	4.97	5.06	5.15	5.09	4.94	4.66	4.96
	16–20 years	18.36%	5.49	5.52	5.51	5.38	5.26	5.14	5.38
	21–25 years	8.19%	5.62	5.57	5.66	5.41	5.57	5.23	5.51
	Above 25 years	5.37%	5.39	5.42	5.47	5.66	5.34	5.38	5.44
Position	Senior high school	10.45%	5.08	5.30	5.38	5.12	4.80	4.81	5.08
	Administrative teacher	46.89%	5.32	5.31	5.46	5.24	5.27	5.09	5.27
	Pure teacher	53.11%	5.14	5.17	5.21	5.03	4.99	4.85	5.07
Gender	Male	34.75%	5.28	5.31	5.301	5.15	5.09	5.00	5.19
	Female	65.25%	5.19	5.20	5.34	5.12	5.14	4.95	5.15
Education degree	5-year junior college	1.41%	5.35	4.90	5.20	5.00	5.40	5.27	5.19
	University	66.95%	5.29	5.31	5.40	5.20	5.20	5.01	5.23
	Master or doctor	31.64%	5.08	5.08	5.19	4.96	4.95	4.85	5.02

Sample size: 253.

, and the significant components are discussed in greater detail below.

5.1. Age

Age was significantly associated with total ICSS satisfaction at the 0.01 level. Age was also found to be significantly associated with system content, ease of use and community building at the 0.01 level, and with system accuracy and system timeliness at the 0.05 level. However, the association between age and report format was insignificant. Table 3 further shows the means for different age groups associated with each ICSS satisfaction component. Generally, the results indicate that older teachers have higher ICSS satisfaction levels than their younger counterparts. This finding may be due to the greater opportunity young teachers have to experience and use information technology in daily life. Therefore, young teachers have higher expectations of the ICSS, which explains their lower satisfaction levels. In summary, the ICSS can capture the integrated contributions for older teachers but is yet to adequately capture the needs of younger teachers.

5.2. School Level

School level was significantly associated with total ICSS satisfaction at the 0.01 level. Under further analysis, school level was found to have significant associations with system content, system accuracy, system timeliness and community building at the 0.01 level, and with report format and ease of use at the 0.05 level. Table 3 shows that those respondents teaching in primary schools had higher ICSS satisfaction levels. One reason for this is that primary school teachers face less pressure to promote their students to enter high schools than do teachers in junior or senior high schools. They can give more attention to prepare the instructional materials, which in turn increases their satisfaction with the support offered by the ICSS. Another reason is perhaps that the ICSS has, to date, provided more teaching resources for primary schools than for junior or senior high schools. This finding implies that the ICSS should strengthen its service for teachers of junior or senior high schools.

5.3. Experienced in Teaching

There was a significant association identified between teachers’ teaching experience and total ICSS satisfaction at the 0.05 level. The teaching experience was significantly associated with system content and community building. Simultaneously, insignificant associations were found between teachers’ teaching experience and system accuracy, report format, ease of use and system timeliness. Further, the more experienced teachers were more satisfied with the ICSS than were novice teachers. These results imply that experienced teachers are more aware of the support provided by integrated instructional materials and the sharing of teaching activities.

5.4. Position

Although a significant association was found between teachers’ positions (administrative teacher vs. pure teacher) and total ICSS satisfaction, there appeared to be no association between teachers’ positions and any ICSS satisfaction components. As demonstrated in Table 3, the administrative teachers have higher satisfaction levels with the ICSS than do the pure teachers. This result shows that no position differences among teachers affected satisfaction with the ICSS, but that the ICSS can effectively help busy administrative teachers to reduce the time needed to prepare instructional materials.

Table 2 provides the significant result and Table 3 helps us to deeply understand some discrepancies in the different personality traits. According to the Bonferroni correction results, the demographics of age present significant discrepancies in the following factors: ease of use—those aged between 26 and 30 years old have a significantly associated with those aged between 41 and 45 years old (p = 0.006); the aged between 26 and 30 years old is significantly associated with those above 50 years old (p = 0.003); the aged between 31 and 35 years old is significantly associated with those above 50 years old (p = 0.022). For system timeliness, the age between 26 and 30 years old has a significantly associated with those aged above 50 years old (p = 0.049). In terms of community building, those aged between 26 and 30 years old have a significantly associated with those aged above 50 years old (p = 0.016), and those aged between 31 and 35 years old have a significantly associated with those above 50 years old (p = 0.027). The demographics of school level present significant discrepancies in the following factors: the primary school level is significantly associated with the junior high school level in the factors of system content (p = 0.000), system accuracy (p = 0.000), report format (p = 0.001), ease of use (p = 0.018), system timeliness (p = 0.000) and community building (p = 0.004).

6. Conclusions

While the type of content plays an important role in the design of online courses [21], instructional materials on the Internet can help teachers to enrich their design of teaching activities. However, an excess of Internet resources that are not integrated could confuse teachers when searching for materials to assist in the design of their teaching plans. This study implemented an ICSS to assist teachers in searching for their required instructional materials from a distributed ESLN using an integrated approach. Meanwhile, depending on teaching requirements, teachers can use the ICSS to personalize the presentation format for their selected instructional materials.

As the ICSS can benefit teachers by offering them instructional materials in a convenient location, exploring users’ satisfaction with the ICSS is important for its continued improvement and promotion. Current mature models of user satisfaction are invariably developed from the environment of end-user computers. For use in online professional discussions and the sharing environment of instructional materials, this study refined a popular satisfaction model proposed by [34]. By adding a new construct of community building, this study developed a measurement instrument and then embarked on an investigation of the satisfaction patterns of teachers from primary schools, and junior and senior high schools.

The results reveal that some demographics, such as age, school level, experience in teaching, and position, have significant effects on total ICSS satisfaction. Among those demographics that impact each satisfaction component, school level was found to have the strongest effect, followed by age and teaching experience. Comparing the three different school levels, we found that primary school teachers were more able to accept innovative teaching activities than junior and senior high school teachers. Therefore, the ICSS must provide users from different school levels and differently aged teachers with different system functions, system contents and knowledge-sharing models according to their needs.

The ICSS can be used to provide an online environment that facilitates interaction among teachers and creates a professional virtual community for them. This community will enable teachers to share and develop their professional knowledge [7]. At the same time, the teachers’ responses in this community will allow ICSS developers to gain a better understanding of teachers’ preferences toward ICSS features-knowledge that can be applied to continuing improvements or enhancements. This will in turn increase teachers’ satisfaction with the ICSS, and thereby increase their acceptance and continued use of the national course service system.

7. Limitations and Further Research

In Taiwan, the instructors have the pressure of teaching progress in junior and senior high school. Teachers often use their past teaching experience to conduct course activities, so the flexibility of using ISS to support teaching is limited. On the other hand, booksellers often take the initiative to support the teaching materials and teaching plans of the junior and senior high school teachers. Therefore, teachers who are more passively adding teaching materials have a lower willingness to use ICSS. They are more inclined to actively use ICSS teachers in the selection of subjects. The subjects of the study may be more inclined towards the active use of ICSS teachers. The ICSS has provided teachers with the functions of searching for teaching materials. In the future, the ICSS can customize system interfaces and related functions by personal habits and then enhance their satisfaction. The study can also promote passive teachers to use ICSS, and further conduct the qualitative analysis of ICSS based on the teacher’s teaching effectiveness.

With the advancement of technology, teaching aid systems must be more flexible to support teaching activities. Therefore, a customized system with a customized interface and flexible system function, which is stable and fast, will be the goal of future research efforts.