Synthesizing the Attributes of Computer-Based Error Analysis for ESL and EFL Learning: A Scoping Review

Abstract

An error analysis (EA) is the process of determining the incidence, nature, causes, and consequences of unsuccessful language acquisition. Traditional EA for English as a second language/English as a foreign language technique lacks an orderly investigation due to human errors. Consequently, computer-based error analysis (CBEA) was introduced into EA in linguistics to achieve accuracy and instant analysis. Although many studies have concluded that CBEA holds numerous strengths, other studies have found that CBEA has certain limitations. However, the strengths and limitations of the CBEA were not clearly synthesized and outlined. Accordingly, this review aims to explore the strengths and limitations of CBEA to provide areas for improvement of computer applications toward an efficient EA procedure. This work also aims to synthesize the strengths and limitations of CBEA mentioned in a variety of articles into a single review to sustain its efficiency and serve as a guide for teachers to benefit from the strengths and gain awareness of CBEA’s limitations. Stakeholders can access broader perspectives on developing application software capable of addressing the deficiencies in EA. By doing so, we can sustain CBEA’s efficiency for the benefit of all. For this purpose, Arksey and O’Malley’s procedure of a scoping review and the PRISMA framework were adopted to guide the filtering and selection of relevant previous studies. Sixty-two articles were selected through the processes of identification, screening, eligibility, and inclusion. Although the findings showed six strengths and seven limitations of CBEA, CBEA can only perform the diagnostic part of EA. Human intervention is still required to perform the prognostic part to accomplish an efficient EA.

1. Introduction

Conventional teaching strategies are catching up to digital learning as teachers increasingly use technology to create an innovative and motivating learning environment for students to learn English. Describing this, Brock [1] explained that a computerized text analysis program or computer-based error analysis (CBEA) has been employed since the 1980s, has evolved through various improvisations, and is now commonly employed for error analysis (EA) in the linguistic field. Brock [1] highlighted that CBEA is known by various names, such as writing analysis programs, text evaluation programs, grammar checkers, spellcheckers, text analysis programs, text checkers, text analyzers, style checkers, and error checkers. This tool can analyze and investigate language production to detect errors, thus providing recommendations to correct any errors. Researchers have demonstrated the various strengths of CBEA and have stated that it allows students to learn in an inquiry-based, constructive learning environment while reducing teachers’ workload. For instance, Chukharev-Hudilainen and Saricaoglu [2] described the functions of CBEA as straightforward, time-saving, and beneficial to increasing enthusiasm for learning English. Furthermore, as stated by Lee and Briggs [3], CBEA is thought to benefit learners and teachers, as learners can see deficiencies in their language use, and teachers can focus on pedagogical development. Teachers of English as a second language (ESL)/English as a foreign language (EFL) have a strong behavioral intention to use technology, because they believe that it will aid them in optimizing their teaching with the most recent material, thus improving the teaching quality [4]. Teachers can employ CBEA for EA to enhance their pedagogical knowledge and provide an effective remedial to sustain ESL/EFL learning, because teaching and learning have been digitized in the past decades.

Although it has been demonstrated that CBEA has numerous strengths, certain limitations have also been discovered. Additionally, CBEA’s outcome is claimed to have been impeded by imperfections. CBEA is further claimed to have failed in detecting certain errors, such as content and organization errors in learners’ writing [5]. In addition, CBEA can only detect lower-level errors, whereas higher-level errors, such as errors in figurative language and complex sentences, are not detected [5]. Although CBEA can recognize error types and recommend how to alter the incorrect words, it cannot add to them. The strengths and limitations of CBEA must be identified to improve and sustain its efficiency in the future.

Despite the fact that the strengths and limitations of CBEA have been discussed separately in individual studies, reviews on this topic are limited, and the strengths and limitations of CBEA are scattered across different articles. We were highly motivated to carry out a scoping review on previous studies of CBEA and researchers’ findings on its effectiveness to identify the strengths and limitations of CBEA and present them in a single review. Therefore, this scoping review aims to explore the strengths and limitations of CBEA and determine the efficacy of CBEA in accomplishing EA processes. This work also aims to provide some insights into the additional features of the software, such as suggestions for improvements that can help learners more efficiently learn the language. Moreover, this work can serve as a guide for teachers to benefit from the strengths while gaining awareness of CBEA’s limitations. Thus, this review addresses the following review question: What are the strengths and limitations of CBEA in the ESL/EFL learning context, and how efficiently can CBEA accomplish error analysis (EA) processes?

2. Background: Evolution of CBEA with the Advent of Technology—Three Strands of CALL Theory

Technology has evolved in EA, and computer-assisted language learning (CALL) was first introduced in the 1960s and evolved with time. When computer-based error analysis was introduced in the early 1990s, error analysis was given a new dimension. When human resources are scarce, the number of documents is large, and the need to analyze learners’ errors is pressing, human error analysis becomes inconvenient. CALL initially focused on language input and feedback. Traditional EA techniques lack an orderly investigation of errors, because errors are manually analyzed, and human errors are likely to occur due to unusual monotonous tasks [6]. The first phase of CALL was developed in the 1950s and put into practice in the 1960s and 1970s. It was based on the Behaviorist Theory, which placed an emphasis on repetitive language exercises [7]. This theory emphasizes language proficiency. Grammar or focus-on-form exercises received the majority of its attention. Behaviorist CALL claimed that a computer can perform repetitious actions with the same linguistic material without growing weary or making mistakes while also providing immediate feedback. Such material could be presented individually by the computer, enabling learners to work at their own pace and ensuring their success.

Following Behaviorist CALL, a popular educational theory in the 1970s and 1980s was Communicative Theory [8]. John Underwood, the main proponent of the Communicative CALL, laid down a set of principles that are linked to CBEA [8]. The first principle stressed that CBEA should allow learners to concentrate more on how to use forms than on the forms themselves. The second principle asserted that learners learn grammar implicitly rather than explicitly. Accordingly, the third principle enabled and encouraged learners to construct their own sentences rather than modifying prefabricated language. The fourth principle stated that CBEA should not evaluate or judge every action taken by the students, nor should it offer them encouragement, and the fifth principle of the theory stated that CBEA should be open to a range of student responses and should refrain from telling the students that they are wrong.

The Interactive and Integrative Theory of CALL refers to a perspective that aims to better incorporate technology and language skills [9]. Developed by Warschauer, Pennington, and Garrett, it is based on two recent technological advancements: namely, the Internet and multimedia, which enable students to browse at their own pace [9]. This theory describes the interaction between two devices: one to produce audio or images and the other to control them.

A significant number of software is available today for language learning. Every software has its own range of operations and is designed for a specific purpose. Among these, Computational Natural Language Learning for instance, collaborated on a grammatical error correction task aimed at automatically detecting and correcting grammatical faults in essays. Automatic writing evaluation (AWE) is a tool that can identify learners’ writing errors and provide feedback to improve text revision success. Proofreaders are tools for detecting incorrect words, missing commas, capitalization, and verb errors. Automated speech recognition is used for English speech recognition, whereas automated computer scoring systems are used for spell checking. Thus, computer software is used for nearly all linguistic purposes, such as proofreading, grammar checking, language translation, spell checking, and writing styles, to suit the goals of usage.

Analyzing the errors of second-language learners can be used for identifying the level of proficiency in the targeted language. Education has evolved as a result of the development of various teaching methods to meet the varying demands of different generations. Teachers can utilize numerous online educational technologies to enhance the efficiency of their teaching and learning processes. Language teachers and researchers have employed computer software in their field and found it to have been a great help in instantly and accurately detecting learners’ errors [10]. An error-detecting software has been developed to not only analyze various writing errors but also provide feedback to writers or to immediately correct spelling in texts [5]. CBEA can potentially achieve efficient outcomes with higher accuracy and is a promising avenue for future research. Consequently, research persistently examines, develops, and regularly updates the efficacy of CBEA to reduce its limitations.

3. Materials and Methods

This scoping review was aimed at exploring the strengths and limitations of CBEA. A scoping review is a suitable method for identifying specific characteristics or ideas in studies, as well as surveying, reporting, or discussing these characteristics or concepts [11,12]. This scoping review was able to synthesize the strengths and limitations of CBEA from various articles into a single review.

3.1. Search Strategy

Certain criteria of inclusion and exclusion were followed. The EA concentrated only on ESL/EFL learners’ errors. Owing to the rapid advancement of technology, the chosen articles were from the past 7 years, ranging from 2016 to 2022. The articles explaining theories and definitions from prior years were included to provide a thorough overview. All of the articles included were related to CBEA, and articles not related to CBEA were excluded. Articles that did not fit the aforementioned criteria were excluded.

Table 1. Summary of the inclusion and exclusion criteria.

Variables	Inclusion Criteria	Exclusion Criteria
Population	ESL/EFL Learners	Non-ESL/EFL learners
Publication Year	2016–2022	Before 2016
Focus	Empirical studies related to CBEA	Studies not related to CBEA and review articles

describes the selection criteria.

3.2. Charting the Results

This review adopted the Preferred Reporting of Items for Scoping Review (PRISMA) flowchart as a guideline for the related article and journal searching procedure. The following four phases from PRISMA, as described by Moher, Liberati, and Tetzlaff [13], were employed to collect data for this study: the articles were carefully identified, screened, checked for eligibility, and included in the review. Figure 1 shows the PRISMA flow diagram explaining the steps of identifying the relevant studies involved in this review.

• The first phase was “identification”, which required the selection and acquisition of materials from databases, such as EBSCOhost, Google Scholar, ERIC, and Scopus. Error analysis for ESL/EFL, second language errors, artificial intelligence for ESL/EFL error analysis, computer software for ESL/EFL error analysis, computer-aided ESL/EFL error analysis, technology for ESL/EFL error analysis, Grammarly for error analysis, and grammar checkers ESL/EFL were the terms used to search the related articles and journals. Owing to the enormous number of references generated by the search technique, the inclusion and exclusion criteria were used to eliminate references that were irrelevant to the study [12].
• In the second phase, i.e., “screening”, articles related to CBEA for the ESL/EFL errors published from 2016 onward were screened by reading the abstracts. This process was aimed at ensuring that the results were trustworthy, as technology is constantly evolving, and CBEA frequently produces new results. The population chosen was mainly ESL/EFL learners’ language production by employing software to analyze the errors. Given that this review required authentic information on CBEA, only empirical publications were included, and review papers were excluded.
• In the third phase of the selection, i.e., “eligibility”, full-text articles were reviewed for the eligibility of the findings and information presented in the retrieved resources. Considering the ambiguous data, some publications were removed, because they did not offer information, as raised by Munn and co-authors [12]. Some studies were unclear about the software that they employed, and their conclusions did not reflect the use of CBEA.
• The fourth phase, i.e., “inclusion”, ended the process with a qualitative and quantitative synthesis of the articles to include the most appropriate resources. After evaluating the citations, full articles for those publications that were the “best fit” for the study question were retrieved [14]. Sixty-two papers (full texts) were chosen for inclusion in the review from the original 1839 references (mainly abstracts). Some articles could be ruled out simply by looking at the title or abstract.

  Table 2. Summary of information from the selected articles.

  	Database	Year	Location	Research Design	Method and Software	Findings (Excerpts from the Articles)
  [2]	Eric	2016	USA	Quasi-experimental	Stanford CoreNLP	Less agile, and it takes a good evaluation mechanism to identify the issue. Effective in identifying technical errors. Effective feedback also needs to be “nonjudgemental”, “contextualized”, and “personal”, which is much more difficult to achieve, as it requires a level of teacher presence.
  [3]	Eric	2020	Korea	Quantitative	Machine Translation	Tool for accuracy in L2 writing. MT should not be regarded as a replacement for the traditional language learning classroom.
  [5]	Google Scholar	2020	Korea	Quantitative	Grammarly	Reduces teachers’ workload. Unable to detect sentence-level errors. Incorrect suggestions and insufficient explanations. It has a long way to go before it can be fully developed.
  [6]	SCOPUS	2020	Saudi Arabia	Quantitative	Grammarly	Able to detect errors missed in manual analysis. Users can repeat the process as many times as they want. CEA seamlessly integrates into the workflow with ease of use. Provide detailed and immediate feedback. A larger amount of data can be analyzed.
  [10]	ERIC	2020	Taiwan	Quantitative	Grammarly	Instant analysis and analyze a large amount of data. Can perform the first three steps of the procedure, although researchers may need to enumerate and analyze errors to complete the process.
  [15]	Google Scholar	2021	Korea	Quantitative	Grammarly	High accuracy. Fails to detect tense shift and sentence structure errors. Teachers should make judicious decisions regarding how and when to use Grammarly, being fully informed of both its strengths and limitations.
  [16]	Eric	2021	France	Quantitative	Moodle	Students can self-analyze their writing. It produces an accurate output.
  [17]	Google Scholar	2016	Saudi Arabia	Quantitative	Criterion	The Criterion® system has great potential for tracking progress and generating individualized student portfolios, including areas of strength and weakness.
  [18]	Scopus	2020	Canada	Quantitative	Microsoft Word, Grammarly, Virtual Writing Tutor	Can accurately identify mechanical and grammatical errors. The system is unable to detect every error. Cannot be relied upon alone.
  [19]	Google Scholar	2022	Saudi Arabia	Quantitative	Grammarly	Not suitable as an independent assessment tool, only as a complementary tool. Achieves high accuracy compared to human raters. Grammarly cannot detect all errors although it does offer valuable suggestions; thus, it is critical to be aware of its strengths and weaknesses.
  [20]	Google Scholar	2020	Japan	Quantitative	COCA	Helps learners make appropriate adjustments to correct their errors. Analyzes a large amount of data.
  [21]	Eric	2016	USA	Quantitative	CyWrite and Criterion	The performance of CyWrite detection on the four target error types, quantifiers, subject-verb agreement, articles, and run-on sentences, outper-formed Criterion.
  [22]	Google Scholar	2019	Saudi Arabia	Mixed-method research	Padlet Web 2.0	Reveal more errors. Help students develop competency in writing. Various features, such as autocorrection, and smart prediction.
  [23]	Google Scholar	2020	Malaysia	Quantitative	AntConc (Version 3.5.8)	Corpus study is the solution for inaccuracies in analysis and avoids overlooking certain errors. Analyzes a large amount of data.
  [24]	Scopus	2019	China	Mixed-method research	Pigai	Clear and immediate feedback is provided, which saves time. The AWE system can only comment on grammar errors and basic word collocations. It cannot meet the requirements of the evaluation for the composition of the text structure, content logic, and coherence.
  [25]	Google Scholar	2021	Spain	Mixed-method research	Grammarly	Able to categorize errors and provide clear explanations. Occasionally presented errors related to hypercorrection. Over-flags feedback thus making it more useful to the learner.
  [26]	Google Scholar	2022	China	Quantitative	WordSmith Tools 6.0	Easy to categorize all errors more accurately. Encourage autonomous learning. Helps to design new pedagogical tools.
  [27]	Google Scholar	2016	Australia	Qualitative	Grammarly	Easy to use and enhance learners’ confidence in writing and understanding of grammatical concepts. Incorrect suggestions and hard to understand.
  [28]	Google Scholar	2019	Pakistan	Quantitative	AntConc software	Corpus leads to wide data analysis. Easier, efficient, and, more objective.
  [29]	Scopus	2019	USA	Quantitative	Gamet	Unable to capture complex errors that may occur across phrases or clauses within sentences, the semantics of missing words, or redundant words, which is a difficult task for rule-based software.
  [30]	Google Scholar	2019	Australia	Mixed-method research	Grammarly	Provides prompt feedback, reduce teachers’ workload. Improves students’ language learning.
  [31]	Google Scholar	2018	USA	Quantitative	UAM CorpusTool program	Easier, quicker, and more consistent than annotating by hand. Allows searching for examples of errors easily.
  [32]	EBSCO host	2021	Philippines	Quasi-experimental	Grammarly	Provides feedback Enhance students’ writing performance. Can be systematically integrated into the teaching of writing.
  [33]	Google Scholar	2018	Bahrain	Qualitative	Write & Improve	Less agile and effective in identifying technical errors. Effective feedback is much more difficult to achieve, as it requires a level of teacher presence.
  [34]	Google Scholar	2021	Egypt	Quantitative	Write & Improve	Provides support for apprehensive EFL writers. Provides immediate feedback.
  [35]	Google Scholar	2019	Australia	Mixed-method research	Grammarly	Provides immediate feedback, reducing teachers’ workload. Promotes greater autonomy in students. Tends to be multifarious and contentious. Inaccurate suggestions can be made relating to the use of the passive voice.
  [36]	Google Scholar	2021	Canada	Quantitative	Criterion	Possible to check the number of times students revised their papers. Fails to capture some errors.
  [37]	Google Scholar	2021	China	Mixed-method research	Pigai	Helpful tool but there are some flaws in identifying collocation errors suggesting syntactic use.
  [38]	Google Scholar	2018	Canada	Quasi-experimental	n-gram/LSTM	Can generate accurate results and runtime performance.
  [39]	Eric	2018	UK	Quantitative	Write & Improve	The system is used in an iterative fashion as envisaged.
  [40]	Google Scholar	2020	USA	Quantitative	e Inverted Spellchecker, Pattern+POS	Outperforms the inverted spellchecker; analyzes a larger dataset.
  [41]	Google Scholar	2020	China	Mixed-method research	Pigai	Convenience and immediacy are its merits and it also reduces teachers’ workload.
  [42]	EBSCO Host	2020	Malaysia	Mixed-method research	AntConc (Version 3.5.8)	Corpus study is the solution for inaccuracies in analysis and avoids the overlooking of certain errors. Analyzes a large amount of data.
  [43]	Google Scholar	2021	Russia	Quantitative	Inspector	Does not always provide the best solutions. Encourages self-editing and improves learning.
  [44]	Google Scholar	2018	Malaysia	Quasi-experimental	Grammarly	Overcomes the problem of delayed feedback. Helps school students with assessment and grading of their essays.
  [45]	Google Scholar	2019	Belgium	Quantitative	CYWrite	Although timely feedback has been argued to be most useful this is not clearly reflected in the revision patterns nor users’ satisfaction.
  [46]	Google Scholar	2020	Greek	Quantitative	Greek Grammar Checker	Help students regulate their learning. Cannot track all mistakes.
  [47]	EBSCO Host	2018	Indonesia	Qualitative	Grammarly	Easy to use. Very helpful to in minimizing the need for teachers to provide corrections on students’ essays. Students actively participate in the teaching–learning process.
  [48]	EBSCO Host	2019	Iran	Qualitative	DIALANG	Learners can benefit from the affordance of computer-mediated dynamic assessment in overcoming their developmental errors.
  [49]	Eric	2019	Ecuador	Quantitative	Grammark and Grammarly	Improves learner writing performance. Human guidance is important to compensate for the limitations of AWE programs.
  [50]	Google Scholar	2019	Taiwan	Qualitative	Pigai	Identifies vocabulary, collocation, and common grammatical errors. Provides immediate feedback and error corrections.
  [51]	Eric	2022	South Korea	Quantitative	Grammarly	Successful at identifying local-level errors. High-stakes testing results in more risk-taking with vocabulary and sentence complexity, which come at the cost of readability (i.e., clarity).
  [52]	Google Scholar	2020	Indonesia	Qualitative	Grammarly	Students believe that Grammarly is easy to use. Corrects errors automatically.
  [53]	Google Scholar	2021	Mynmar	Qualitative	ICALL	Reduces challenges regarding time constraints. AI only detects surface-level errors, whereas teachers’ feedback covered lower and higher-level errors; integration of both types of feedback is required.
  [54]	Eric	2018	Spain	Qualitative	CEA (N-gram)	Errors that are homophones (e.g., your versus you’re) or otherwise real words (were versus where) are missed by generic spellcheckers, accounting for 16% of spelling errors in the corpus.
  [55]	Google Scholar	2020	USA	Qualitative	Grammarly	Supplemental tool to facilitate lower-order concerns.
  [56]	Google Scholar	2018	Indonesia	Mixed-method research	Grammarly	Detects grammar, spelling, and punctuation errors. Can also detect the addition and omission of some syntactical items in a sentence. Can be misleading when it comes to long phrase, passive voice structure, and question structure.
  [57]	Google Scholar	2021	Myanmar	Mixed-method research	Grammarly	Has pedagogical potential as a tool that can facilitate teachers’ identification of surface-level errors.
  [58]	Google Scholar	2021	Philippines	Quantitative	Grammarly Premium and Flesch Kincaide Tools	With Flesch Kincaide Reading Ease tools that can be instantly integrated with the Microsoft Office Word and Ubuntu programs, students can understand the level of their writing’s readability and their vocabulary and grammar competence. Likewise, suppose that teachers can obtain a Grammarly premium subscription. In that case, other aspects of students’ writing errors can be analyzed with regard to the correctness of punctuation, tone, clarity, engagement, and delivery of words.
  [59]	Scopus	2020	India	Quantitative	Grammarly, Ginger, ProWritting Aid,	All apps fail to identify sentence structure errors. One cannot completely trust these apps for the identification and correction of grammar errors.
  [60]	Google Scholar	2017	Nigeria	Quasi-experimental	Shallow Parser	The efficiency of operation of each of these systems varied widely. The scale of the operation is still too small, limiting its ability to tackle fundamental linguistic phenomena.
  [61]	EBSCO Host	2019	Malaysia	Quantitative	Paper Rater.com	With regard to which application should be used, this depends on the users’ preferences and needs. Learners can independently check their errors and correct them.
  [62]	Google Scholar	2021	Korea	Mixed-method Research	Grammarly	Feedback from online grammar checkers is not always accurate. A balance may be found, whereby students focus on micro-level writing errors, and teachers focus more on macro-level errors, such as organization and idea development.
  [63]	Google Scholar	2018	Indonesia	Quasi-experimental	Grammarly	More effective in reducing errors in relation to three indicators (diction, language use, and mechanics). Has less of an effect on content and organization and cannot detect whether or not the content is appropriate for the topic.
  [64]	Google Scholar	2021	Taiwan	Quantitative	Error Taxonomy	Several errors, such as spelling, punctuations, and even word choice, might automatically be corrected by the software.
  [65]	Eric	2016	USA	Quantitative	MyAccess	In this study in particular, it missed 60.4% of errors that should have been attended to according to human judgment. The choice to use this software could be made as part of a combined pedagogy.
  [66]	EBSCO Host	2021	Malaysia	Qualitative	Google Docs	Google Docs auto-corrects users’ grammatical errors while users are writing.
  [67]	Scopus	2020	Malaysia	Mixed-method Research	Social Networking Sites (SNS)	The auto-correct feature of the profile can help learners to correct their errors/mistakes automatically.
  [68]	Google Scholar	2021	Indonesia	Qualitative	Grammarly	One student’s perspective: “Each time I am typing, auto correct will automatically appear.”
  [69]	Scopus	2020	Malaysia	Mixed-method Research	Social Networking Sites (SNS)	The auto-correct feature from the profile can help learners to correct their errors/mistakes automatically.
  [70]	Google Scholar	2021	Indonesia	Quantitative	Grammarly	Effectively detects errors but requires human involvement to find the causes of the errors.
  [71]	Eric	2016	Sweeden	Qualitative	Word (Spell Checker)	Allows self-regulation and enhances learning. Occasionally, the spellcheck function gives rise to unnecessary corrections.
  [72]	Google Scholar	2018	Turkey	Quantitative	CEA	The nature of EA could accommodate both diagnostic and prognostic features.

  provides a summary of the findings from the selected articles.

4. Results

Sixty-two articles were reviewed to identify the strengths and limitations of CBEA. To answer the first review question, the results show the seven most found strengths and the six most found limitations of CBEA. The strengths and limitations of CBEA were identified by examining the actual patterns of use through a concordance system and statistical charting of the results. Most articles discussed the functions of the software for error analysis and corrective feedback, because the software can detect errors in addition to providing feedback. The studies revealed more than one strength, such as accurate and instant analysis, or more than one limitation, such as being unable to detect long and complex sentence errors and content and organization errors of the writing. Accordingly, the total number of articles supporting the strengths and limitations varied as a function of the total number of articles reviewed. The subsequent discussion of CBEA’s strengths and limitations was exclusively based on the findings of the articles chosen for this review.

On the basis of the results illustrated in

Table 3. Number of studies supporting the strengths of CBEA.

Strengths of CBEA	Number of Studies (n=)	Percentages (%)
Provide solutions	15	23.07%
Accurate and precise	14	21.54%
Instant analysis	10	15.39%
Reducing teachers’ workload	8	12.31%
Ease of use	6	9.23%
Enable iteration	6	9.23%
Analyze big data	6	9.23%

and Figure 2, the strengths of CBEA found in the articles were categorized into different categories, and the results of the 15 studies found that CBEA can provide solutions for errors (23.07%), 14 studies indicated that the analysis of CBEA is accurate and precise (21.54%), 10 studies proved that CBEA is able to provide instant analysis (15.39%), 8 studies found that CBEA is able to reduce teachers’ workload (12.31%), 6 studies showed that CBEA is easy to use (9.23%), 6 studies showed that CBEA enables iteration (9.23%), and 6 studies found that CBEA can analyze big data (9.23%). Each strength is explained in detail below with the supporting articles and their points.

Based on the results illustrated in

Table 4. Number of studies supporting the limitations of CBEA.

Limitations of CBEA	Number of Studies (n=)	Percentages (%)
Inability to analyze higher-level errors	15	28.30%
Inability to identify content and coherence errors	11	20.75%
Misleading feedback	10	18.87%
Autocorrection	7	13.21%
Need for various software packages	6	11.32%
More diagnostic than being prognostic	4	7.55%

and Figure 3, the limitations of CBEA found in the articles reviewed are categorized into few categories, and accordingly, the results of 15 studies (28.30%) indicated CBEA unable to analyze higher-level errors, 11 studies (20.75%) showed CBEA unable to identify content and coherence errors, 10 studies (18.87%) proved the CBEA providing misleading feedback, 7 studies’ findings (13.21%) showed CBEA may miss certain errors by autocorrect them, 6 articles (11.32%) highlighted about the need of various software packages to conduct a complete CBEA, and 4 articles (7.55%) found CBEA is more diagnostic than being prognostic. The limitations are explained in detail below with the supported articles and their points.

5. Discussion

5.1. Strengths of CBEA

• Accurate and Precise: Accuracy and precision are among the prominent features of CBEA, and they are vital for EA. The majority of the articles searched for this review concurred with this statement. CBEA was employed in the form of Grammarly, Moodle, and Criterion software in studies achieving a high level of precision [15,16,17]. Moreover, CBEA excelled in traditional strategies in terms of accuracy [18,19]. Given that accuracy is one of the vital parts of EA, the reliability of the CBEA of the COCA software can be trusted [20]. CyWrite produces fast and accurate analyses and provides technical assistance for academic writing [21,22]. Furthermore, studies employing Grammarly and AntCont (Version 3.5.8) proved that CBEA is able to detect the errors missed by humans while analyzing large amounts of text [9,23]. The main strength of CBEA is accuracy, as frequently mentioned in most of the studies reviewed. The findings of such studies indicated that the number of errors recognized by CBEA was substantially higher than that found during manual analysis.
• Ease of Use: CBEA as employed in the form of Grammarly, COCA, Pigai, and Word Smith tools (6.0) was found to be user-friendly by researchers, requiring less effort to operate the software [20,24,25,26]. Given that written texts were computerized prior to the CBEA processes, the software could automatically analyze the errors, minimizing the effort needed [27,28].
• Manual EA requires teachers to carefully check through the learners’ writings, and CBEA can alleviate this load when correctly handled.
• Instant Analysis: CBEA can provide instant analysis as opposed to manual EA, which requires teachers to identify learners’ errors individually, which is a time-consuming and tedious task. Respondents of the studies that employed Grammarly, CyWrite, and Pigai software expressed their satisfaction with regard to time spent for the analysis and agreed that CBEA can produce instant analysis [29,30]. CBEA is designed to detect each error and deliver instant analysis with the appropriate response alternatives [31,32]. Accordingly, if immediate results are required, then the teacher can provide them to learners by adopting suitable tools to analyze the errors [33,34]. This action creates a positive effect on the learners and motivates them to initiate corrective action and improve their language use [30].
• Reducing Teachers’ Workload: Teachers viewed tensions or discrepancies within classroom practices and beliefs due to contextual factors, such as time constraints, high-stakes examinations, and prescribed curricula [73]. Studies on Grammarly and Criterion showed that CBEA saves teachers time and allows them to concentrate on further actions on the basis of the EA results [35,36]. This phenomenon occurs because teachers typically devote extra time to carefully examining students’ errors to ensure that none are ignored [19]. Furthermore, teachers must devote a significant amount of time to analyze a large number of samples. Language teachers can use software to help them in effectively managing the work of analyzing students’ writings [5]. Given that CBEA can reduce teachers’ workload and save time, teachers can devote additional time to the preparation of teaching materials that are appropriate for correcting and improving learners’ errors.
• Enabling Iteration: In the event of a questionable circumstance, teachers and students can reiterate the EA procedure for clarity or identify the linguistic part where the greatest errors are made, and the response purportedly utilized to correct those areas [6]. A teacher stated that CBEA as employed in Criterion is helpful in monitoring her students’ written work by encouraging them to repeatedly identify errors as a means to allow students to amend their work [36]. Teachers attempt to avoid making repetitive analyses due to time constraints [37]. Moreover, teachers can repeat CBEA as employed via n-gram/LSTM to generate more trustworthy and concrete results that can be used to determine errors [38,39]. CBEA enables teachers and learners to obtain precise data on their errors and their causes by going over text as many times as necessary.
• Analysis of a Large Amount of Data: CBEA can analyze large datasets in a short period because the software is designed to handle large amounts of data [40,41]. CBEA takes less time to complete, and the analysis process can be completed more quickly than manual EA; thus, teachers can instantly move on to the next dataset [10] A corpus-based study used the AntConc software to analyze large datasets and completed the analysis in a short time, and this capability has substantially aided ESL/EFL research [23,42].
• Providing Feedback: CBEA provides not only detailed information about each of the writing errors but also extra writing judgments according to a set of writing objectives. Teachers and learners agreed that CBEA as employed in Moodle, Grammarly, and Inspector software is actually useful because it allows them to verify their grammatical mistakes, thereby instantly correcting them [43,44]. Software packages, such as spell check, grammar check, electronic translators, and machine translation (MT), have helped learners autonomously analyze and revise their written work [44,45,46]. MT can assist learners with individualized feedback that they can relate to their second language translations to aid interpretations and paraphrases throughout the editing process [3]. Grammarly and DIALANG software enhanced learners’ involvement with their tasks and reduced learners’ struggle to overcome errors [47,48]. Long-term usage of CBEA software for EA can enhance learners’ language competency because they can recognize the reasons for errors, as well as solutions to improve them; it can also be the best way to help learners successfully learn their second language autonomously [49,50]. The authors of [51,52] found that learners experience convenience and confidence when using Grammarly to correct their errors, thus improving their writing quality.

5.2. Limitations of CBEA

• Inability to Analyze Higher-Level Errors: CBEA is unable to detect errors in long and complex sentences. Such a limitation adds difficulty for teachers in explaining the problem to their students. Studies on CBEA as employed in Grammarly, Gamet, and Pigai software demonstrated that it cannot easily detect semantic issues in texts and can only recognize surface-level errors while failing to cover major issues [37,53]. Findings from a study on CEA (N-gram) software indicated that errors that were homophones were overlooked, accounting for 16% of spelling errors [54]. CBEA failed to identify incorrect multiword units or collocations [55,56]. The Grammarly and Pigai systems were confused by long and complex sentences, such as those involving the use of idioms and collocations and a passive voice in sentences [25,35]. CBEA is only effective at detecting errors and providing feedback at the surface level [19,57]. Thus, teachers should change the grade and manually recognize the student’s creativity in these situations.
• Need for Various Software Packages: Another limitation of CBEA is that one application is insufficient to detect all errors in a document. This condition proved that spellcheckers are meant to check spelling errors and not grammatical errors. Avila et al. [58] supported that one software was not enough to obtain the required data for their study. When evaluating the performance of different grammar checking tools, Sahu [59] found that they were unable to detect errors in sentence structures. Crossley et al. [29] highlighted that it is impossible for software to analyze the overall errors, since it is rule-based software. Each program has restrictions in terms of analyzing errors because it is designed to detect particular types of errors [60]. The authors of [10] explained that no single feature set can predict a skill across all second-language writing datasets. CBEA can be carried out with limited purpose. The authors of [61] asserted that users should be aware that CBEA serves a variety of purposes, and learners should carefully select the appropriate tool recommendations according to their specific goals.
• Inability to Identify Content and Coherence Errors: CBEA is unable to identify content appropriateness and the sentence movement in each paragraph, regardless of whether or not the paragraph is coherent [62]. The reliability of CBEA as employed in Grammarly, Pigai, and Criterion was questioned with respect to the content and organization of learners’ writing [10,17,24]. Sahu [59] highlighted that CBEA remains in a development stage, because it cannot properly evaluate the text structure, logic, or coherence. Results of studies on Grammarly, CYWrite, MyAccess, and Write&Improve demonstrated that human involvement for CBEA is necessary to identify errors such as disconnection between the topic and the content, since they will be attentive if the text lacks cohesion [63,64,65]. CBEA only detects programmed errors, whereas anything that is not in the program will not be detected. Consequently, false detection deviates from the purpose of EA [66].
• Autocorrection: Certain errors are automatically corrected by the software without the author’s knowledge, resulting in erroneous analyses. While explaining reasons for not employing software for EA, Shirban Sasi [67] mentioned that the software may automatically rectify various problems, such as spelling, punctuation, and even word choice, without the researcher’s knowledge. A study by Barrot [32] employing Grammarly required learners to turn off the autocorrect feature as protocol to prevent the software from prescreening the text. Grammarly highlights errors in red, and students can simply click on these errors for Grammarly to correct them [52]. “CBEA software has various features, among which autocorrection allows learners to autocorrect their errors” described Ziad [22]. The autocorrect features in software for ESL/EFL learning help learners correct their erroneous written text [68,69]. In studying the perspective of students using software to learn, Yunus and Hua [70] mentioned that the autocorrection feature can help learners correct their writing errors; however, it may not be suitable for error analysis. Given this condition, the autocorrection feature can be a strength for ESL/EFL learning. Nevertheless, this feature is a limitation for the EA process where the analyst is unable to collect a genuine result.
• Misleading Feedback: CBEA as employed in grammar checkers often provides corrective feedback to correct erroneous words or sentences. However, researchers employing Grammarly, CYWrite, and Write&Improve found that, on certain occasions, the feedback given could divert the meaning of the sentence; this situation occurred when the suggested answers were not suitable with respect to the intentions of the sentence [33,45,67]. The feedback provided by Grammarly is not in line with the intentions of the users [27,42,68]. Shelvam and Bahari [69] claimed that, on certain occasions, the software provides misleading feedback that needs improvisation in the future. Furthermore, students must be aware of the need for the sentences because the suggested answers can be accepted or dismissed by users according to the need of the sentence [35,69]. Systems are often confused about the difference between American spelling and British spelling, whereby some words can be detected as erroneous or correct [58]. Additionally, Musk [71] asserted that, although the spelling is always correct, it depends on the language setting, which isn’t always the case.
• More Diagnostic Than Prognostic: The authors of [72] described the characteristics of EA, including diagnostic and prognostic aspects. CBEA has its own set of limitations that are said to be more diagnostic than prognostic. Considering that CBEA analyzes learners’ errors as a whole, it often provides the types of errors [72] but fails to identify the causes of errors. Im and Can [62,72] employed a human specialist to interpret the causes of the errors that were detected by CBEA, highlighting that human involvement is vital to completing CBEA processes. CBEA can accomplish the first three steps of Corder’s three-stage process, namely, collection, identification, and description; however, researchers may need to identify the causes of the errors and explain them to complete the process.

5.3. Review Question 2

To answer the second review question, CBEA could not accomplish all five stages of Pit Corder’s EA process. CBEA can efficiently accomplish the first three stages, namely, collection (collecting dataset), identification (identifying errors), and description (classification of errors). However, CBEA is unable to complete the fourth and fifth stages of the EA process, namely disclosure (identifying the causes of the errors) and evaluation (providing remediation to overcome the errors). The feedback from CBEA can be used to correct erroneous writing, but justification for the correction can only be provided by teachers [25,49]. Teachers can observe the learners’ language use, identify the causes of the error, and provide a remedy. Given this situation, human intervention is needed to accomplish the last two stages. Thus, this review revealed that CBEA needs human intervention to complete the EA process.

6. Pedagogical Implications

According to the results, CBEA was found to be a helpful method in conducting EA in terms of accuracy and ability to produce an instant analysis of large datasets. Teachers have time constraints, and error analysis can be a tedious and burdening work for them. Although CBEA was found to be ineffective in identifying higher-level errors such as errors in passive voice, homophones, content reliability, and complex sentences, it can accurately detect lower-level errors, such as mechanical and grammatical errors, which account of a large portion of ESL/EFL learning. Despite its limitations, CBEA can still be a helpful tool for teachers in reducing their workload by analyzing errors and providing feedback that enables them to focus on identifying the causes of errors and providing remedial actions. Similarly, Behaviorist CALL argued that a computer can perform repetitious actions within the same linguistic material without growing weary or making mistakes while also providing immediate feedback. Haong and Kunnan [66] explained that analyzing errors and providing feedback are intended to transform errors into learning opportunities by exposing students to potential errors in their writing and offering them language skills as they progress through the feedback. The principles of Communicative CALL can be used to explain this statement, as learners are able to learn grammar implicitly through the feedback provided. Learners, therefore, begin to focus on the use of forms, hence using their language skills authentically rather than fabricating them via the use of software.

The best option is an integrated pedagogy in which teachers serve as facilitators and students conduct the CBEA to understand their errors and resolve them using the software. For instance, teachers can provide students with an explanation of the causes of the errors and the proper method to correct the errors, whereby teachers can ask students to work in groups to generate ideas before writing to overcome content issues. Accordingly, teachers can encourage critical thinking among students in addition to improving their language skills. CBEA can also be a tool for autonomous learning. Teachers should encourage learners to employ CBEA in their learning. Teachers and learners should be aware of the strengths and limitations of CBEA when conducting EA and guide learners in independently analyzing their errors, while being mindful of the limitations, to sustain ESL/EFL learning. However, teachers must realize that CBEA cannot detect everything, especially authentic errors, and they should not exclusively depend on it to accomplish the EA process. These limitations may not stop users in employing CBEA. However, awareness of the strengths and limitations can serve as a guide for gaining advantages and avoiding higher-level errors. With the evolution of technology, stakeholders should look into the strengths and limitations of CBEA and develop software to overcome the current limitations. Lawmakers should make an effort to introduce CBEA in a school context such that teachers and learners can benefit from it. Thus, CBEA is recommended at all levels of academia to sustain ESL and EFL learning.

7. Conclusions

Unlike manual EA, CBEA is a computer-assisted analysis, and some researchers have claimed that it is not as efficient as expected in sustaining language learning. In light of this finding, a scoping review was conducted on 62 studies to determine the strengths and limitations of CBEA. This scoping review focused on studies that employed various software. The results showed that CBEA can assist teachers in a variety of ways, the most essential of which are saving time and reduced workload. Moreover, CBEA can produce a more accurate and precise result, as well as conduct real-time analysis. Furthermore, the availability of diverse software packages for specific purposes allows teachers to select the appropriate software on the basis of their teaching. However, some limitations exist, the most important of which is that teachers should not rely solely on the CBEA to complete tasks. Moreover, teacher participation is critical to avoid false results.

CBEA can perform the diagnostic aspect of EA. However, human intervention is required to perform the prognostic aspect, thereby producing an absolute analysis report. The inability of CBEA to determine the causes of errors leads to an incomplete EA process, thus necessitating human involvement to identify the causes of the errors.

Although CBEA has been in ESL/EFL language production for some time, developments aimed at achieving greater accuracy and detailed analysis have been beneficial for teachers. Researchers have limited the usage of CBEA in schools where English is taught as a second or foreign language. Therefore, ESL/EFL teachers and students are not fully aware of the efficacy of CBEA in detecting learners’ writing errors. A CBEA study in schools can be an eye-opener for teachers to adopt CBEA in their teaching practice. Thus, this tool is important for enhancing the pedagogical knowledge of teachers, particularly in understanding and tackling errors incurred by their learners.