Investigations into EFL Students’ Pragmatic and Grammatical Awareness through Peer Collaboration

Abstract

Grounded in sociocultural theory, collaboration has been recognized as a useful teaching method. In English teaching, while a number of studies have examined the effectiveness of collaboration on grammar, research on pragmatic awareness is scarce. As such, this study explored how collaboration influenced, firstly, grammatical and pragmatic awareness and, secondly, error correction: the two major components in language competence. In this study, 32 Taiwanese English major juniors and seniors with CEFR B2 were recruited. These participants were required to work individually and collaboratively to identify pragmatic/grammatical errors, to rate the severity of the errors, and to correct these errors in a discourse completion task (DCT). The results revealed that peer collaboration achieved higher scores on error identification, severity ratings, and error correction than did individual work. The facilitative effects of collaboration may have been attributable to collaborative and expert–novice interaction patterns that showed a high degree of mutuality and equality during discussions. In conclusion, the findings showed that collaboration echoed the spirit of sociocultural theory, and could serve as a useful approach in English instruction, to create a sustainable learning environment in an EFL context such as Taiwan, where the national bilingual 2030 policy has been launched to require learners to develop life-long learning of English.

1. Introduction

The current trend in education is to foster sustainable skills: that is, skills which not only enable students to cope with their studies, but also allow them to function properly in their future jobs and lives. From the 1990s onwards, organizations such as UNESCO and OECD have proposed conceptions about 21st century skills for sustainable development of life; these organizations argue that the skills fall into three major areas: learning and innovation skills; digital literacy skills; and personal and professional life skills. In the area of learning and innovation skills, the core competencies for having a sustainable life include the 4Cs: communication; collaboration; critical thinking; and creativity [1]. It is recommended that the core competencies be incorporated into subjects. In terms of English learning, it is evident that the paramount goal is to cultivate students’ competence at different linguistic levels (e.g., grammar, writing). However, in many Asian countries, such as Taiwan, where English is a foreign language, English teaching is more focused on grammar and vocabulary memorization. Little attention has been directed to pragmatics or language use, in relation to contextual factors. Hence, to equip students with sustainable communication abilities through English, the implementation of instructional approaches that equally value grammar and pragmatics deserves more attention.

One of the pivotal instructional approaches is collaboration. Most importantly, collaboration is one of the core competencies that enables students to sustain their development after they leave higher or further education [2]. In the field of education, collaborative learning has been a crucial teaching method, supported by theoretical frameworks such as the sociocultural theory [3]. Learners can create collaborative dialogues, defined as “dialogue in which speakers are engaged in problem solving and knowledge building” [4] (p. 1), to scaffold one another to gain cognitive development in their potential area of growth. This approach has resulted in considerable impact on English language teaching, and has generated a vibrant body of research on grammatical performance, vocabulary acquisition, and collaborative writing [5,6,7,8,9,10,11,12]. In terms of grammar, previous collaborative learning research has mainly been conducted on advanced proficiency learners in ESL environments [6,8,12], and the general findings have shown its effectiveness for learning [5]. However, although collaboration can enhance grammatical performance, scholars have also mentioned that the extent to which collaboration can lead to accurate decisions on grammatical use is still uncertain [5]. Compared to grammar, scant research has been conducted on the influence of collaboration on pragmatic performance. Unlike its effect on grammatical performance, whether collaboration can yield a positive influence on pragmatic performance remains inconclusive [13,14,15,16,17]. Such mixed results may be due to differences in proficiency [18] or L1 [14] between members in pairs. Furthermore, the majority of research has focused on either pragmatic awareness [15,17] or production alone [13,16,19,20]. Little is known about how collaboration influences pragmatic competence in a more comprehensive way. Taken together, although collaboration is the key competency for life-long development, its effect on grammar/pragmatic learning still awaits exploration. As such, the present study investigated the extent to which collaboration can serve as an effective method in developing grammatical/pragmatic awareness and production.

2. Literature Review

2.1. Sociocultural Theory and Peer Collaboration

One major goal of achieving sustainability in learning is to emphasize the way one should view oneself in connection to others and the surrounding world. For language learners, this means that learning is not a discrete experience; instead, it involves others, so that learners can mutually increase and sustain their motivation to continue learning. For language teachers, it means that they need to combine an interaction-oriented approach with a theoretical explanation, so that they can perform in different learning environments, and thus enhance language abilities and improve learning sustainability. One widely-used theory—the sociocultural theory—has provided useful and clear concepts that endorse the use of peer collaboration in fostering sustainable learning. According to the sociocultural theory, human mental functioning is a process mediated by cultural artifacts (e.g., language) and activities [3]. Furthermore, the cognitive development of humans is rooted in their interaction with society and their material environment. Specifically, the sociocultural theory encompasses major constructs—including mediation, internalization, and the zone of proximal development [21]—which have laid the foundation of various approaches in education, such as peer collaboration in L2 learning [5,6,18,22]. To illustrate, mediation refers to utilizing concrete tools or signs (e.g., language) to mediate the relationship between individuals and the physical environment. The major means of mediation are object-, self-, and other-regulation: object-regulation means occasions where the environment affords cognition (e.g., using a dictionary to look up a word); self-regulation refers to individuals relying on internalized forms, such as language, to complete a task; and other-regulation refers to the assistance provided by other people (e.g., assisted performance of grammatical production by a participant in the collaboration). Internalization describes the process of an individual internalizing ideas from their surroundings. In other words, an individual can progressively decrease the level of reliance on external mediation, and increase the level of reliance on internal mediation (e.g., becoming more capable when solving problems in peer collaboration). Finally, regarding the concept of the zone of proximal development (ZPD), Vygotsky [3] defined it as “the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem-solving under adult guidance or in collaboration with more capable peers” (p. 6). Most importantly, it is believed that the creation of a ZPD is an essential precondition for learning. In the case of peer collaboration, it means that learners will have an area in which they fulfil their potential when they receive assistance from their peers.

2.2. Peer Collaboration and L2 Grammar

In recent decades, a considerable number of studies on peer collaboration in language competence have been conducted, to explore its effects on the development of grammar and collaborative writing [5,6,8,9,10,11,12]. Research has mainly focused on the examination of grammatical production via various types of tasks, such as cloze exercises, text reconstructions, essays [5,8], short texts based on graphic prompts [6], and multimodal compositions [12]. Regardless of the task types, the general findings derived from this line of research have indicated that students working collaboratively outperform those working individually. Several reasons have been proposed to account for the positive effects. Collaboration affords learners more time on the tasks, and results in more revisions and stronger motivation [5]. It also offers opportunities to receive feedback from peers, develop a sense of joint responsibility, and pool linguistic resources [6,8]. However, despite the positive impacts of collaboration on grammatical accuracy, scholars have still advised caution regarding the interpretation of collaborative learning. Close comparisons among the literature have indicated the possible existence of several potential variables, such as the type of grammatical form, proficiency level, and task type. For instance, regarding the type of grammatical form, Storch [5] examined the effects of pair and individual work on grammar-focused tasks (e.g., a cloze exercise, a text reconstruction, and composition). She reported that it could not be conclusively claimed that collaboration would lead to correct decisions on grammatical forms, because some grammar, such as the articles ‘a/an’ and ‘the’, did not have simple rules to follow. With respect to proficiency, in an attempt to investigate the extent to which pair and individual work influenced fluency, accuracy, and complexity in L2 writing produced by 23 ESL intermediate and advanced-level learners, Storch [6] showed that while some learners commented that collaboration improved their grammatical accuracy, a few learners had reservations about correcting their peers because they lacked confidence in their own language skills. Furthermore, Wigglesworth and Storch [8] compared differences in accuracy, fluency, and complexity among 48 pairs of advanced-level learners and 48 advanced-level learners who worked individually when writing an argumentative essay. Given that the participants’ proficiency was fairly advanced, the attention to language-related episodes was primarily directed to lexis rather than to grammar. With respect to task types, Kim and Kang [12], who investigated what Korean learners of English focused on in the course of making a collaborative multimodal composition project, also showed that the learners’ focal attention was not predominantly on grammar. It was the content that received the most attention during the interactions. Based on these previous studies, it is speculated that, although collaboration can generate a positive effect, learners’ grammatical performance in pairs might vary because of other factors.

2.3. Peer Collaboration and L2 Pragmatics

Peer collaboration in L2 pragmatics has attracted researchers’ attention in recent years [13,14,15,16,17,18]. Overall, the research findings have suggested that the effect of collaboration on L2 pragmatic learning remains inconclusive, presumably because the designs of the studies varied in some respects. Some studies focused on the heterogeneity of participants in collaboration, such as differences in proficiency or L1 language. With regard to the level of proficiency, Ajabshir and Panahifar [18] conducted research on the production of requests, apologies, and refusals to explore and compare the effectiveness of teacher scaffolding, peer collaborations, and self-work among 119 learners. Prior to the treatment, all the learners took a pre-test in the form of a written discourse completion task. The peer collaboration group was further divided into symmetrical and asymmetrical pairs according to their pre-test scores. The written discourse completion task used as a post-test indicated that the teacher scaffolding group and the peer collaboration group outperformed the control group. Compared to teacher scaffolding, peer collaboration worked even more effectively. In addition, asymmetrical pairs outperformed the symmetrical groups in the production of target speech acts. The findings thereby suggested that the combination of a novice and an expert in collaborative activities could lead to better L2 pragmatic performances. In terms of L1 language background, Kim, Lee, and Kim [14] examined the effects of collaborative writing tasks on the development of Korean honorifics among Korean learners in the United States. The participants were divided into heritage language learners (HLLs) and foreign language learners (FLLs). The students completed a pre-test, two collaborative writing tasks, an immediate post-test, and a delayed post-test during the same semester. The participants were required to complete a discourse completion task and an acceptability judgment test. Fourteen HLL–FLL dyads and 9 FLL–FLL dyads were recorded and analyzed in terms of pragmatic-related episodes, defined by Taguchi and Kim [19] as “any part of language production where learners talk about the pragmalinguistic forms they are producing and the sociopragmatic factors they are attending to” (p. 419). The results showed that, over time, all the learners improved their scores in both discourse completion tasks and acceptability judgment tests. Moreover, the HLLs in HLL–FLL dyads were more advantageous than FLLs in FLL–FLL dyads in the immediate post-tests, which implied the importance of the co-existence of FLLs and HLLs in using collaborative learning for pragmatics.

In addition to individual differences in research design, other studies utilized different tasks to illustrate the effect of collaboration when learners exhibited their productive or receptive performance. In terms of productive performance, Kim and Taguchi [13] used collaborative dialogues to examine the impact of task complexity and pragmatic situational demands. They divided 49 Korean junior high school students into a simple group (n = 25) and a complex group (n = 24). The participants needed to co-construct requests based on drama scripts. The simple group was given details regarding the interlocutor relationship and the size of imposition of the scenarios, while the complex group did not receive the same information and had to negotiate to complete the task. The collaborative dialogues of both groups were recorded and analyzed in terms of pragmatic-related episodes. The results indicated that collaboration was beneficial to generate rich metapragmatic discussions on sociopragmatic (e.g., power, distance, and imposition) and pragmalinguistic elements (e.g., bi-clausal structures such as “I’m wondering if…”) for speech acts. Collaboration also helped learners construct their pragmatic knowledge, as they provided chances to negotiate meaning and scaffolding for peers.

Aside from manipulating the complexity of the script reconstruction task, Lin, Chen, and Wu [16] examined 11 pairs and 11 individuals to compare their pragmatic performances and cognitive processes when working on a written discourse completion task through verbalizations. The individual groups’ private speech and the collaborative groups’ dialogues were elicited and analyzed. Before writing the written discourse completion task, the individuals tended to plan the general direction of their writing. On the other hand, the collaborative pairs would start to plan the details at the very beginning. However, there was no statistically significant difference in the overall performance of the written discourse completion task between the collaborative and individual groups.

Also adopting a discourse completion task, Taguchi and Kim [19] studied the effects of peer collaboration on the speech act of request with three groups of Korean English learners (control, collaborative, and individual). Explicit metapragmatic instruction was provided to the collaborative and individual groups prior to the completion of the task, but not to the control group. The results showed that the explicit metapragmatic instruction had a positive effect, as both treatment groups outperformed the control group in the immediate post-test. In addition, the collaborative group produced more target-like request head acts than the individual group, but the two groups did not differ significantly in the use of modifications on the immediate post-test, suggesting that collaboration did not exert a greater impact on the use of requests.

Another production task that was used was e-mail, which Cho and Kim [20] adopted, to compare learning outcomes between collaborative and individual task groups. Thirty-two learners of Korean at a high beginning level, provided with indirect written corrective feedback, were randomly assigned to either a collaborative or an individual group to complete e-mail writing tasks with the use of four types of Korean honorifics. The researchers designed written description tests to investigate the short-term and long-term learning of Korean honorifics, and to analyze the suppliance and accurate production of each target feature from the students’ responses to teacher feedback, according to three categories: correctly resolved, incorrectly resolved, and unresolved. Both groups performed much better on the immediate and delayed post-test than on the pre-test. However, there was no significant difference when collaborative and individual task groups produced target features during task performance.

In terms of receptive performance, Chen and Lin [15] investigated the effects of peer collaboration on EFL learners’ comprehension of conversational implicatures among 33 EFL learners. Adopting a multiple-choice discourse completion task, the findings showed that, although the collaborative group gained higher scores on the multiple-choice discourse completion task than did the individual group, the difference was not significant. Chen and Lin [15] also utilized a multiple-choice discourse completion task of two isomorphic versions to compare the differences in the comprehension of six apology and request items between individual and collaborative work. Twenty intermediate Chinese learners of English completed the first version independently and the second version in 10 pairs. The collaborative dialogues were recorded as they worked together. The results indicated that the task performance of the collaborative work was better than that of the individual work. Furthermore, based on the categories of pragmatic-related episodes, the participants focused on politeness, repair, interlocutor relationship, tone of voice, relevance, clarity, and clarification when negotiating the correct answers collaboratively.

To sum up, the effect of collaboration on pragmatic learning appears to be inconclusive. On the one hand, studies such as Chen [17], Kim, and Taguchi [13], and Kim, Lee, and Kim [14], indicated that peer collaboration worked better than individual work. On the other hand, studies such as Taguchi and Kim [19], Chen and Lin [15], Cho and Kim [20], and Lin, Chen, and Wu [16] found that collaborative work did not seem to differ significantly from individual work in L2 learners’ pragmatic productions. Furthermore, the mixed results of the effect of collaboration were affected by individual differences (e.g., proficiency, L1 background). In addition, these studies on speech acts mainly examined either production or perception alone. Therefore, given these reasons, more studies are needed to investigate the extent to which collaboration influences pragmatic awareness, together with production, with a more homogenous group of participants. Furthermore, since the primary goal of achieving greater sustainable language learning is to emphasize the way learners view themselves in relation to others (i.e., collaboration), studies exploring the effect of collaboration are valuable, because they may illuminate the importance of collaboration, and confirm its effectiveness via the examination of different learners’ characteristics and aspects of their language abilities.

2.4. Pragmatic and Grammatical Awareness

The development of pragmatics and grammar in L2 learners’ interlanguage is a complex issue [23]. Although grammatical and pragmatic competence operate in a complementary and interactional manner [24], the level of grammatical competence does not guarantee a corresponding level of pragmatic competence [25]. To examine the relationship between pragmatics and grammar, Bardovi-Harlig and Dörnyei [25] designed an audio–visual questionnaire-based instrument to assess L2 learners’ pragmatic and grammatical awareness with respect to four speech acts: requests; apologies; suggestions; and refusals. To assess learners’ awareness, the researchers designed 20 scenarios, in which the learners needed to identify whether the last utterance of a conversational exchange was pragmatically appropriate or grammatically correct. This task was utilized because noticing is closely associated with pragmatic awareness [26]. Next, to avoid rejection bias, if the utterance was judged as inappropriate or incorrect, the learners needed to rate the severity of the errors on a 6-point Likert scale. An EFL group with 370 Hungarian students, 25 Hungarian teachers, and 112 Italian pre-service teachers, and an ESL group with 173 students and 28 teachers in the United States, were recruited to complete the questionnaire. In terms of error identification, the major findings revealed that the EFL group identified more grammatical errors than pragmatic infelicities, whereas the ESL group identified more pragmatic infelicities than grammatical errors, suggesting the importance of language exposure in this aspect. With regard to severity rating, the EFL group regarded grammatical incorrectness as more severe than pragmatic inappropriateness, while the ESL group considered pragmatic infelicities as more severe than grammatical errors, suggesting the need to emphasize the significance of pragmatics for EFL learners in instructions. Furthermore, although this study elucidated some aspects of L2 pragmatic awareness, the author noted that high pragmatic awareness did not equally translate to appropriate production. In other words, the findings of this research only reflected a portion of L2 pragmatic competence. Hence, further studies are recommended, to assess both awareness and production, in order to capture a more comprehensive picture of learners’ pragmatic competence.

Several subsequent studies have employed Bardovi-Harlig and Dörnyei’s instrument in its original or revised version, to examine pragmatic and grammatical awareness from different perspectives, including ESL/EFL learning contexts [27,28], length of residence [28,29,30], and learners’ proficiency [27,29,31,32]. In other words, previous studies have exclusively emphasized the extent to which individual-related factors affected pragmatic competence. Moreover, while the majority of the studies focused solely on awareness, a few have examined both pragmatic awareness and production, as reviewed in the following paragraphs.

The first individual-related factor was the learning context. For instance, Niezgoda and Röver [27] replicated Bardovi-Harlig and Dörnyei’s [25] study, and obtained results unlike those of the original study. The EFL context did not prohibit the learners from developing pragmatic sensitivity. An EFL group with 124 Czech learners tended to recognize a higher number of pragmatic infelicities and grammatical errors than their 48 ESL counterparts in the United States. Additionally, the EFL learners regarded both types of errors as more severe than the ESL learners. The researchers concluded that the environment was not the most crucial factor accounting for learners’ pragmatic awareness: rather, the key to pragmatic and grammatical awareness was based on “an interaction between exposure to pragmatic and grammatical input and individual learner characteristics, specifically the degree to which learners actively attend to input” (p. 77). Another study focusing on learning contexts was conducted by Schauer [28], who replicated Bardovi-Harlig and Dörnyei’s [25] study by using a similar research design and instruments. She added post-hoc, semi-structured, one-to-one interviews, and observed the effects of a longitudinal stay in the target language environment. Three groups—17 German EFL learners, 16 German ESL learners living in the U.K., and 20 British English native speakers (BNS)—were compared in terms of pragmatic and grammatical awareness. In line with Bardovi-Harlig and Dörnyei [25], Schauer found that pragmatic development and grammatical awareness were conditional on the learning environment. The ESL learners had higher pragmatic awareness, as they were more responsive to pragmatic errors, and their perceptions of pragmatic infelicities were similar to those of BNS. On the other hand, the EFL learners were more able than the ESL learners to detect more grammatical violations, and to perceive these errors to be more salient.

The second individual-related factor was the length of residence. Bella [30] investigated the relations between residence in the target language country and the development of pragmatic awareness, by comparing 140 Greek advanced learners who had lived in Greece for 1.6 years and 3 years. The two groups examined pragmatic and grammatical violations, and rated the errors. The results revealed that both learner groups recognized grammatical errors accurately, and judged them to be more severe than pragmatic infelicities. Furthermore, those with longer residence times had greater awareness of grammar. However, the relationship between the length of residence and pragmatic awareness was not certain. On the other hand, Schauer [28] also looked into how length of residence impacted ESL learners’ pragmatic awareness. Schauer compared the test results of German EFL learners in Germany, and German ESL learners living in the target language country for one academic year. The results revealed that, with a nine-month stay in the host environment, the ESL learners’ pragmatic awareness increased significantly, and their pragmatic awareness was approximate to that of the English native speakers.

The third individual-related factor was proficiency. For instance, Xu, Case, and Wang [29] worked with 126 ESL graduate and undergraduate students who pursued their degrees in the United States. They were divided into four groups according to proficiency and length of residence. Generally, the graduate students had longer stays in the target culture, and exhibited better language proficiency than the undergraduates. The researchers adopted the original version of the questionnaire, but added a section in which the students needed to revise inappropriate or incorrect utterances. The graduate students were found to outperform undergraduate students in error identification and error correction, regardless of the length of residence. In addition, the graduate students considered pragmatic appropriateness as being more important than did the undergraduate students. Moreover, overall L2 proficiency was found to have a stronger impact than the length of residence. In another study, Xu, Case, and Williams [31] longitudinally explored 60 EFL students’ pragmatic and grammatical developments in mainland China. The researchers modified the original questionnaire by adding an error revision section. The students needed to recognize error types, rate their severity, and correct the errors. Both the EFL learners and the native speakers rated pragmatic violations as more severe than grammatical errors. Nevertheless, the Chinese students’ pragmatic productions were different from those of the NS in terms of content, address forms, and grammar. They improved significantly in L2 pragmatics on error judgment and correction, when their grammar improved. This study showed that EFL learners developed pragmatic competence and grammatical ability over time. The findings suggested that these students developed pragmatic proficiency when they received multiple exposures to the target language input, and had chances to put it into practice in their local contexts. In other words, pragmatic development could take place as the learners’ overall proficiency progressed, whether or not they lived in the target language environment.

Conversely, some researchers [27,32] have found that proficiency did not have an effect on pragmatic awareness. Niezgoda and Roever [27] divided ESL and EFL learners into high and low proficiency groups, based on their scores on placement and achievement tests. Proficiency did not correlate with pragmatic awareness, as the low-proficiency learners significantly recognized more pragmatic than grammatical errors than did the high-proficiency learners. In addition, the low-proficiency learners rated pragmatic errors as significantly more severe than grammatical errors, whereas the high-proficiency learners rated the two error types similarly. Likewise, Sorour [32] compared the pragmatic awareness of 67 Egyptian ESL learners with their grammatical awareness. They were grouped in high and low proficiency levels. Proficiency did not seem to have an impact on their pragmatic awareness, for the high-proficiency group displayed a higher level of grammatical awareness than the low-proficiency group. On the other hand, the low-proficiency group were able to recognize more pragmatic errors than grammatical ones. This indicated that possessing a high proficiency level did not guarantee a higher level of pragmatic awareness. Therefore, Sorour recommended integrating pragmatics into L2 instruction and assessment, to reinforce L2 learners’ pragmatic development.

In summary, the above literature has provided some major insights, which have paved the way for the current study. Firstly, the effects of peer collaboration on grammar learning have generally been positive; however, studies have cautioned that collaboration might not necessarily lead to correct decisions, implying a need for more studies. Secondly, the effects of peer collaboration in L2 pragmatics are inconclusive, for research has shown that students working collaboratively do not necessarily perform better than those working independently, probably because the influences of individual variables were included in the previous studies, or because past research targeted only awareness or production. Thirdly, previous research on pragmatic and grammatical awareness has predominantly scrutinized the impact of individual-related factors, presenting a relatively static picture of L2 pragmatic development. To shed more light on L2 pragmatic instruction, it is assumed that investigations into how the implementation of instructional approaches—such as peer collaboration—affect pragmatic awareness and production would provide more practical insights in terms of teaching material design. Hence, this study used Bardovi-Harlig and Dörnyei’s [25] instrument to examine whether L2 learners’ pragmatic and grammatical awareness and production would be influenced by collaborative and individual work. It is hoped that this investigation will shed light on sustainable language learning, as its goal was to enable learners to understand the significance and benefits of including others in the learning processes. The current research was guided by the following research questions:

• Is there any difference between collaborative and individual work in error identification of pragmatics and grammar?
• Is there any difference between collaborative and individual work in severity rating of pragmatics and grammar?
• Is there any difference between collaborative and individual work in error corrections of pragmatics and grammar?

3. Methodology

3.1. Participants

The participants were recruited by convenience sampling. Prior to the study, the researchers explained the purpose and procedure of the study. They were informed that their identities would not be disclosed. Oral consent was obtained face-to-face, and the procedure was video-recorded. As shown in

Table 1. Participants’ demographics.

Gender	Male: 9	Female: 23
Year of Study	Junior: 8	Senior: 23
Age	Mean = 21

, the participants of this study were 32 Taiwanese learners of English (23 females and 9 male students) on a four-year course based in the Department of Applied English, University of Technology, in central Taiwan. There were eight juniors and 24 seniors, and the average age was 21 years. Their English proficiency was at vantage level B2, based on the Common European Framework of Reference for Language (CEFR).

3.2. Instrument

The major instrument used in this study was the Bardovi-Harlig and Dörnyei’s [25] Discourse Completion Task (DCT). This task consists of 20 scenarios and four speech acts, including requests, apologies, suggestions, and refusals. The task is in English. In this task, there are eight pragmatically inappropriate, eight grammatically incorrect, and four perfect responses to the scenarios. In addition, this task is accompanied by video clips to provide rich contextual clues to the learners.

To suit the research needs, we divided the 20 scenarios into two isomorphic versions. Another reason for the division of the scenarios was that the task had been adopted by various previous studies, which allowed the researchers to compare the previous results with the current research. To be equivalent, each version consisted of ten scenarios: four pragmatically inappropriate, four grammatically incorrect, and two perfect responses. For the pragmatic error correction, the number ranged from two to four. Conversely, there was only one error in each grammatical correction item. As with the original task, the learners were required to judge the appropriateness/correctness, and to rate the severity of, the last utterance to a given scenario. In this task, however, the learners were asked to further identify if the errors belonged to pragmatics or grammar, and to provide appropriate/correct responses, c.f., [29]. The reason why error correction was included in the tasks was that, according to Bardovi-Harlig and Dörnyei [25], “higher pragmatic awareness does not necessarily translate into appropriate pragmatic production” (p. 254). Therefore, error correction allowed researchers to further assess students’ pragmatics and grammar in terms of production. In addition, one sample item reads as follows: “In this scenario, the interaction was conducted by the teacher and the student. Given that the social power of the teacher was higher than that of the student, Peter’s refusal should have been more courteous. Hence, the first box should be “No” and the second box should be “Pragmatics”. The correct response based on the semantic formula should be “① I’m sorry. ② I’d love to help you. ③ But I am working late at the supermarket today. ④ Could I do it tomorrow?”, which shows an apology, an expression of positive opinions, excuses, and a promise of future acceptance.

Scenario: The teacher asks Peter to help with the plans for the class trip.

Teacher: OK, so we’ll go by bus. Who lives near the bus station? Peter, could you check the bus times for us on the way home tonight?

Peter: * No, I can’t tonight. Sorry.

Is the last part appropriate/correct? □ Yes □ No.

What type of error do you think it is? □ Pragmatics □ Grammar.

How bad do you think the error is? Not bad at all ____:____:____:____:____:____ Very bad.

How would you revise the error? _________________________”.

3.3. Procedures

The entire data collection was conducted via Google Meet, due to the COVID-19 pandemic, during August and September of 2021. To avoid potential issues such as inattentiveness in the discussion online, each pair, and the individuals, separately made appointments at their chosen times with the first author, to complete the tasks. Furthermore, given the facts that the number of items was small, and that the participants chose times that suited them the best, it was assumed that fatigue would not exert a serious impact on their performance. The participants were asked to complete the tasks in a quiet room so that they could avoid disturbances.

Prior to the actual experiment, a training session was provided to the learners. First, they watched an introductory video in Mandarin with detailed instructions. The concepts of pragmatics were also included in the video instructions. Next, three sample items—one with pragmatically inappropriate, one with grammatically incorrect, and one with perfect utterances [33]—were also presented to the learners, to familiarize them with the task.

The formal experiment began in the two subsequent weeks. In the first week, the learners self-selected their own peers and constituted 16 dyads. They needed to discuss and complete Version A collaboratively in pairs. They were allowed to use L1 or L2 for discussion. No time limit was given for the collaborative and individual work. Each of the scenarios in this version was shown once. The utterance to be evaluated was the last response to a given scenario. The targeted utterance that the learners needed to evaluate was conspicuously indicated by a flashing exclamation mark (!) in the video, and was also highlighted by the same mark in the task. Each scenario was shown to the learners twice: they were asked to watch and listen to the scenario the first time, and to respond to the last utterance the second time. They were not informed of the number of corrections they could make. However, for grammatical error correction items, each item contained only one mistake. One week later, the learners followed the identical procedures for Version B, but this time they were required to work independently.

3.4. Data Analysis

To answer the research questions, the task was analyzed in terms of error identification, severity rating, and error correction, each of which will be described in detail below. Basically, the participants obtained a score when working individually. However, when working collaboratively, the raters allocated the same score to each individual in a given pair for error identification, severity rating, and error correction.

In terms of error identification, the learners earned one point by judging whether the last utterance in each scenario was pragmatically appropriate/inappropriate or grammatically correct/incorrect. If the utterance was pragmatically appropriate or grammatically correct, one point would be given if the learners ticked the “Yes” box. On the other hand, if the utterance was pragmatically inappropriate or grammatically incorrect, one point would be granted if the learners ticked the “No” box. In addition, they would obtain another point by accurately identifying whether the error was pragmatic or grammatical. Since there were four pragmatically inappropriate, four grammatically incorrect, and two perfect utterances in each version, the participants could earn as many as 18 points in total for each version.

Next, the severity rating, ranging from 0 to 6, was used to indicate how the learners felt about the errors in the utterances. Zero meant that no error was made (not bad at all), while 6 meant that the learners thought that the error in the utterance was severe. When the learners ticked the “YES” box for a pragmatically appropriate or grammatically correct utterance, they did not need to rate the severity level of the given utterance. However, we would key in 0 (not bad at all) in the cell, for computation in the Excel file.

Finally, the learners were asked to correct pragmatic or grammatical errors if they found the last utterance to a given scenario inappropriate or incorrect. For the pragmatic errors, scores were given by calculating the semantic formulas in the corrections, c.f., [34,35,36]. For example, the revised utterance “① I’m sorry, ② I’d love to help you. ③ But I am working late at the supermarket today. ④ Could I do it tomorrow?” to a refusal scenario was granted four points, because there were four semantic components. We involved two raters. One was the first author, who was a non-native speaker of English, and the other was an American English native speaker, who was pursuing a Ph.D. in Taiwan. The reason for recruiting raters with different language backgrounds was to minimize different degrees of lenience regarding error correction [16,37]. The raters were given a training session first, and then scored all the corrections in both versions independently. The raters scored the answers against the rubric (see Appendix A). Given that the semantic formulas in the two versions were different, and resulted in different full scores, normalization of the scores was needed, to compare the differences between the two versions. To normalize the scores, we adopted a percentage system: we divided the participants’ scores by the full score of a given version, and multiplied it by 100. The interrater reliability was 0.99, as computed by intraclass correlation (F = 204.8, p < 0.001).

As for the grammatical errors, one point was given if the learners fixed the problematic part. For example, if the participant deleted “on” for the utterance I’m sorry, I just can’t. I’m very tired. I couldn’t sleep on last night, then he/she would be given one point.

4. Results

4.1. Error Identification

Table 2. Between-group comparison of error identification.

	Overall	Pragmatics	Grammar
z score	−3.31	−2.649	−2.463
Individual (median)	12 ***	6 *	6 *
Collaborative (median)	14 ***	6.5 *	6 *

Note: * p < 0.05, *** p < 0.001.

displays the between-group comparison of error identification. The Wilcoxon Matched-Pairs Signed-Rank test showed that collaborative work led to significantly better performance than individual work, with medium effect sizes in overall (Mdn = 14, z = −3.31, p < 0.001, r = −0.414), pragmatic (Mdn = 6.5, z = −2.649, p < 0.05, r = −0.331), and grammatical error identification (Mdn = 6, z = −2.463, p < 0.05, r = −0.308). For the within-group comparison,

Table 3. Within-group comparison of error identification.

	Individual	Collaborative
z score	−0.943	−0.317
Pragmatics (median)	6	6.5
Grammar (median)	6	6

shows that there was no significant difference between pragmatic error identification (Mdn = 6) and grammatical error identification (Mdn = 6) when working individually: z = 0.943, p > 0.05, r = 0.118. There was also no significant difference between pragmatic error identification (Mdn = 6.5) and grammatical error identification (Mdn = 6) when working collaboratively: z = −0.317, p > 0.05, r = −0.04.

4.2. Severity Rating

Table 4. Between-group comparison of severity rating.

	Overall	Pragmatics	Grammar
z score	−3.697	−3.578	−1.753
Individual (median)	24 ***	12 *	9 *
Collaborative (median)	30.5 ***	16 *	10 *

Note: * p < 0.05, *** p < 0.001.

shows the between-group comparison of severity rating. The Wilcoxon Matched-Pairs Signed-Rank test showed that the learners working collaboratively rated the errors significantly more severely than those working alone, with medium effect sizes in overall (Mdn = 30.5, z = −3.697, p < 0.001, r = −0.462), and pragmatic (Mdn = 16, z = −3.578, p < 0.05, r = −0.447), and with a small effect size in grammar (Mdn = 10, z = −1.753, p < 0.05, r = −0.219). With regard to the within-group comparison, the results in

Table 5. Within-group comparison of severity rating.

	Individual	Collaborative
z score	−2.836	−4.795
Pragmatics (median)	12 *	16 ***
Grammar (median)	9 *	10 ***

Note: * p < 0.05, *** p < 0.001.

show that both solo individuals (Mdn = 12, z = −2.836, p < 0.05, r = −0.354) and collaborators (Mdn = 16, z = −4.795, p < 0.001, r = −0.599) significantly rated pragmatic infelicities more severely than grammatical errors. Furthermore, the effect size of the individual rating was medium, but the effect size of the collaboration rating was large.

4.3. Error Correction

The pragmatic and grammatical competence of the 32 Taiwanese EFL learners was further examined by analyzing their written production on the DCTs. As stated earlier, the analysis was based on the presence of semantic components. The rubric normalized scores were used to compare the pairs’ and individuals’ work. The Wilcoxon Matched-Pairs Signed-Rank test was computed between the percentage means in pairs and individuals, as shown in

Table 6. Between-group comparison of error correction.

	Overall	Pragmatics	Grammar
z score	−2.994	−2.656	−2.794
Individual (median)	58.82 *	59.09 *	50 *
Collaborative (median)	67.5 *	71.43 *	75 *

Note: * p < 0.05.

.

In terms of the between-group comparison, the Wilcoxon Matched-Pairs Signed-Rank test showed that collaborative work led to significantly better error correction than individual work, with medium effect sizes in overall (Mdn = 67.5, z = −2.994, p < 0.05, r = −0.374), pragmatic (Mdn = 71.43, z = −2.656, p < 0.05, r = −0.332), and grammatical correction (Mdn = 75, z = −2.794, p < 0.05, r = −0.349). As for within-group comparison,

Table 7. Within-group comparison of error correction.

	Individual	Collaborative
z score	−0.758	−3.215
Pragmatics (median)	59.09	71.43 *
Grammar (median)	50	75 *

Note: * p < 0.05.

shows that the participants performed significantly better on grammatical error correction (Mdn = 75) than on pragmatic error correction (Mdn = 71.43) when working collaboratively, and that such a difference had a medium effect size, z = −3.215, p < 0.05, r = −0.402. However, there was no significant difference between pragmatic (Mdn = 59.09) and grammatical error correction (Mdn = 50) when working individually—z = −0.758, p > 0.05, r = −0.095—implying that their grammatical and pragmatic productive abilities were quite the same.

5. Discussion

The primary goal of sustainable language learning is to help learners recognize the relationships between individuals and others in the learning process. Hence, this study aimed to investigate the pragmatic and grammatical awareness of Taiwanese EFL learners in the completion of a DCT through collaborative and individual work. The findings showed that peer collaboration appeared to differ significantly from individual work in terms of error identification, severity rating, and error correction. Firstly, more errors were identified by collaborative work than by individual work; secondly, while collaboration led to more severe ratings than individual work, all the participants regarded pragmatic infelicities as more severe than grammatical errors; finally, collaboration led to better error corrections than individual work, especially in grammar corrections.

In terms of the better performance of collaborative work in error identification (research question 1) and correction (research question 3), the general findings may seem to be consistent with previous studies [15,18,19] conducted in EFL settings; however, it should be noted that the designs of those studies were different from that of the current study, and therefore caution should be exercised in the interpretation. For instance, Chen and Lin [15] utilized a multiple discourse completion task, in which the participants’ responses might have been affected by the options or test-taking strategies. Taguchi and Kim [19] included metapragmatic instructions before their experiment; hence, the results may have been influenced by the instructions, rather than by the effect of collaboration alone. Adjabshir and Panahifar [18] paired their participants with those of different proficiency levels; therefore, their results were not completely comparable to those of the current study, which recruited participants of relatively homogenous proficiency. To interpret the better performance of collaborative work in error identification and correction, one plausible explanation may be the simple adage that “two heads are better than one”. Previous research has shown that when working collaboratively, learners were able to employ more resources by pooling linguistic knowledge to complete a task [6]. They were also more motivated, spent more time, and made more revisions on a task than their counterparts working individually [5]. Therefore, with these benefits, collaboration could enhance the learners’ performance on the tasks. Another plausible explanation is that the pair-work created collaborative dialogues, with collaborative and expert–novice interactional patterns (Collaborative means a pair in which both parties contribute to all work, engage with each other’s ideas, and accept resolutions. Expert–novice means one party acts like an expert and encourages the other (the novice) to participate in the work.) that represented a high degree of mutuality and equality in discussions, as the members in each pair were self-selected, had known each other for more than two years, contributed equally to the tasks, and agreed with each other’s opinions, according to the researchers’ observations. These two patterns of interactions were found to facilitate the transfer of knowledge, and hence to lead to better error identifications and error corrections [38]. In addition, although there may have been slight variations between them, their proficiency was generally high; therefore, the degree of equality was generally high between the two parties. The combination of high–high or high–low, in terms of proficiency, was thereby assumed to enable the learners to form collaborative and expert–novice patterns. In addition to transferring more knowledge, these combinations also contributed a positive effect, because they created more language-related episodes that focused on forms in collaborative dialogues when completing grammatical production tasks [39]. Furthermore, we speculate that the use of L1 may have promoted the occurrence of collaborative and expert–novice patterns because it gave these learners opportunities to understand the meaning of the awareness task, to establish goals to be achieved, and to externalize and vocalize their thoughts without any barriers during interactions. In addition, using L1 provided additional advantages, such as eliciting more production as needed, when working together [40], making meaning of the text, generating ideas to solve linguistic problems [41], and effectively managing a task [42]. In sum, pair-work induced collaborative dialogues and interactional patterns, whose mechanisms echoed the concept of the zone of proximal development (ZPD): namely, the idea that cognitive development is attributed to the collective scaffolding of others. It was also postulated that the learners’ proficiency and the use of L1 were not the only two potential reasons why the collaborative and expert–novice pattern of interactions happened; it also brought about accompanying advantages, such as the greater focus on forms, and more production than the tasks required.

In addition to the stronger facilitative effects of peer collaboration, the study further found better performance on grammar corrections than on pragmatic ones. Such a finding may be a result of the grammar-translation teaching method, and the rigid, exam-oriented conditions in the English education system in Taiwan [43]. Another possible explanation may be related to the nature of the task. To correct grammatical errors, the learners’ focal attention was primarily on linguistic aspects (e.g., a double marking of the past tense, “I didn’t brought it”). However, to correct pragmatic errors, they needed to consider multifaceted dimensions simultaneously (i.e., sociopragmatic and pragmalinguistic usages). For instance, when making a request, the participants needed to take distance, power, and imposition into consideration first, to determine which aspect (syntactic or lexical level) should be corrected [34]. With the limited amount of processing capacity, and a relatively greater load of cognitive demands [44], grammatical error corrections were by nature less challenging than pragmatic ones. Hence, the learners obtained higher scores on grammatical error corrections.

With regard to error severity rating, the learners rated pragmatic and grammatical violations more seriously when working collaboratively than when working alone. One possible explanation may be associated with the results of error identification. In the present study, the participants first needed to recognize if an utterance was erroneous, so that they could subsequently proceed to the next severity rating step. If a participant did not treat an utterance as an error in the first place, then he/she did not have to provide a severity rating score. As stated previously, the participants identified more errors in peer collaboration. As such, it was reasonable that the collaborative pairs, who had more opportunities to provide their severity ratings, gained higher scores in this dimension. Another plausible explanation was the effect of metapragmatic discussions. However, this explanation was not directly supported by further analysis, such as using a different yet equal number set of items for the individuals and the collaborative group in the current study, and hence caution should be exercised in the interpretation. In a study on the effects of metapragmatic discussions on the comprehension and production of requests, Takimoto [45] also discovered that this approach could motivate and compel students to be more attentive to sociopragmatic (e.g., distance, power) and pragmalinguistic features (e.g., past tense). Furthermore, Tullis and Goldstone [46] investigated the effect of peer discussions on the accuracy of multiple-choice questions. The authors concluded that peer discussions elicited more information processing resources, facilitated greater motivation, and created new and multiple perspectives of knowledge. Therefore, it is postulated that the participants, when engaging in pair discussions, would generate diverse aspects and direct more attention to both grammatical and pragmatic errors, which in turn led to greater severity ratings.

Besides the between-group differences in severity ratings, violations of pragmatics were viewed as more severe than those of grammar in both individual and collaborative work. Although some past research has shown that EFL learners generally rate pragmatic infelicities as less severe than grammatical ones [25], the current findings were consistent with Niezgoda and Röver, who also sampled English majors in an EFL context [27]. Such findings may suggest that the learning environment can be a determining factor with respect to severity ratings. In other words, even though the participants were in an EFL context, their learning background was crucial. More precisely, the participants in this study were English majors. Therefore, they were situated in a learning context that enabled them to use English in a more contextualized way, instead of learning specific linguistic elements in a less contextualized manner, such as memorization of grammatical rules. These reasons may help explain why they rated the pragmatic errors more severely than the grammatical errors.

In short, the current research extended the scope of Bardovi-Harlig and Dörnyei’s study [25], by comparing the effects of collaborative work and individual work on error identification, severity ratings, and error correction of speech acts. The results indicated that peer collaboration significantly enhanced the participants’ performance (error identification and error correction), because of the facilitative effects of collaborative interaction patterns, the same proficiency level among the participants, and the use of L1. In addition, when working collaboratively, the participants had a greater ability to correct grammatical errors than to correct pragmatic errors, because grammatical errors were less cognitively demanding for these learners. Furthermore, the findings revealed that the collaborative pairs ascribed greater severity to pragmatic and grammatical errors than did the individuals, because of the sequence of the task (i.e., error identification before severity ratings), and possibly the effectiveness of the metapragmatic discussion, which generated focal attention on the pragmatic infelicities and grammatical incorrectness. Finally, regardless of the working modes, pragmatic errors were rated more severely than grammatical ones, suggesting that learners could also develop high pragmatic awareness in an EFL environment. To conclude, these findings suggest that collaboration is a useful method of creating a sustainable EFL learning environment, by showing the significance of others in the learning process. Students can benefit from this approach, and improve their grammatical/pragmatic awareness and production.

6. Conclusions

Collaboration, as one of the essential 21st-century skills, has played a tremendous role in the sustainability of students’ academic achievement and future career success [47]. Therefore, it is paramount for teachers to equip students with the ability to work effectively with others. Specifically, it is suggested that learning approaches can be oriented to foster the willingness for assistance, the flexibility to make compromises to achieve a shared goal, and effective listening to decipher meaning [1]. In the spirit of exploring how to create a sustainable language-learning environment, the present study adopted collaboration as the focal point of our investigation. The findings have proved that L2 language learners could perform better, with respect to error identification and error correction, through collaboration.

Several implications can be derived from the findings. From the national policy perspective, in the ELF context of Taiwan, the government has launched the Bilingual 2030 policy, aiming to bilingualize education and boost English ability in the hope of cultivating international talents and running bilingual policy and native language policy in parallel before 2030 [48]. Evidently, this policy shows the need to create, and the importance of creating, a sustainable EFL learning environment, so that learners can form a lifetime commitment to English learning. Therefore, incorporating collaboration would help achieve this goal, by making students learn better, and creating a sustainable EFL learning environment in the future. From theoretical perspectives, this study contributed to the existing body of socio-constructivist research that endorses the benefits of collaborative learning. By extending its scope from the traditionally predominant domain of L2 writing [6], the findings further showed this approach to be useful for less-researched areas such as L2 pragmatics. Specifically, this study also demonstrated that learners could jointly assess and produce pragmatics and grammar, which provided them with opportunities to be engaged in meaningful discussion that served as a useful method of judging pragmatic (in)appropriateness/grammatical (in)correctness, and producing more correct responses based on errors. From the methodological perspective, the addition of error corrections to the task in this study has shown that learners’ receptive ability does not directly correspond to their productive ability. Hence, in practice, there should be an equal focus on awareness and production to promote sustainable learning. With regard to pragmatics instruction, the results revealed that collaboration could serve as a useful technique to enhance the effectiveness of raising pragmatic awareness and producing more appropriate responses, and to create a more interactive atmosphere that helps learners to co-construct their knowledge. In practice, teachers can assign students to work in pairs so that they can share different ideas for discussions [49], retrieve information, and recognize and modify gaps in their understanding [50]. Similar impacts can be brought about by peer collaboration, to lead students to produce more correct and appropriate speech acts and grammar. For instance, when engaging in discussions, learners can be encouraged to write down possible pragmatically appropriate and grammatically correct answers, exchange their ideas, and provide reasons to justify why a certain response is better than another. Finally, given that collaboration was found to be beneficial for grammatical/pragmatic error identification, severity rating, and error correction, it is assumed that incorporating the function of collaborative learning would be helpful for designers who wish to develop applications for language learners.

Despite the abovementioned contributions, the study has several limitations. Firstly, the current study only recruited students of high proficiency. However, given that proficiency levels are an influential factor in pragmatic comprehension [51], future studies are encouraged to sample low-proficiency learners to examine whether there are differences in the effectiveness of working collaboratively and individually. Secondly, the present study only focused on speech acts. However, given that pragmatics consists of other components, researchers can explore the extent to which collaboration/individual work influences learners’ comprehension and production in different dimensions, such as routines and implicatures. Finally, the number of the participants and the number of items given were fairly limited. Future studies could recruit more participants, and provide them with more items, to better generalize the findings to a larger population.