Synergies in Developing Pre-Service Teachers’ Language Assessment Literacy in Ukrainian Universities

Abstract

Developing the language assessment literacy (LAL) of pre-service foreign language (FL) teachers is thought to be beneficial both to trainee teachers themselves and to their future students and schools. However, it also presents a number of challenges to teacher educators involved in training these cohorts. Ensuring that pre-service FL teachers are fully confident and competent in assessment requires the design of a preparation programme that is detailed and relevant to their needs. While Taylor has argued that different stakeholders have different language assessment literacy needs, there will also be different needs within groups according to their specific assessment responsibilities. Thus, for language educators, the needs of pre-service teachers are likely to be different from those of in-service classroom teachers, and this should be reflected in the content of their training. However, research has shown that across Europe, training practices vary considerably, often with no training provided in teacher training programmes at all. In this study, I address this issue through a discussion of current approaches to the pre-service development of language assessment literacy across Ukrainian universities. Due to the principle of organizing the learning process at Ukrainian tertiary institutions in blocks of disciplines of general and professional preparation and practical training, students acquire knowledge and develop skills in the field of assessment gradually, through studying various subjects, and not only within one course. This interaction of different subjects with each other and with school placements presents synergies that can enhance LAL, if the corresponding programme takes into account these connections. In order to have a full picture of Ukrainian pre-service teachers’ language assessment development, the current study presents a description of the training system and its LAL-related components retrieved from educational and professional training programmes and discipline curricula of the Ukrainian universities providing relevant preparation programmes. I provide a list of components and discuss didactic conditions that aid the acquisition of LAL by future teachers. I argue that these synergies in the Ukrainian system may provide a useful example of practice for other universities in the European context.

1. Introduction

Language assessment literacy is a type of professional competency acquired over a lifetime. It is typically developed at the undergraduate and postgraduate levels of teacher training and then when in service. The level of competency in language assessment required by teachers depends on multiple factors and constitutes an important aspect of the teacher-assessor identity. Naturally, assessment needs vary depending on a teacher’s work context [1]. For example, secondary school teachers have specific assessment responsibilities and needs, which differ from primary school teachers. As a consequence, they should ideally carry out their training in LAL through a customized programme.

One of the main problems concerning the language assessment community is formal training for teachers. Hasselgreen et al. (2004), Vogt and Tsagari (2014) and Kvasova and Kavytska (2014) claim that foreign language (FL) teachers are often underprepared for carrying out assessments [2,3,4]. Their research identified little or no training at all for practicing teachers in different European countries. A question is then raised about pre-service teachers, particularly how we might design an appropriate preparation programme, which can reflect the full needs of these future assessors. One of the limitations of designing a comprehensive programme is the shortage of time which can be allocated by universities to teaching a course on assessment. In spite of this limitation, such a preparation can be made feasible by spreading and integrating the components of the target competency, across different courses throughout all years of study. This approach has been termed a “synergetic approach”.

The aim of the present research was to investigate how pre-service teachers are currently prepared in Ukrainian universities to carry out assessment at secondary schools. It was anticipated that this would permit a clearer idea of the way in which a synergetic approach might work effectively. To achieve this aim, I examined curriculum documents of existing education programmes for teaching assessment.

2. Literature Review

The synergetic approach has been widely discussed in research in various fields—philosophy, politics, sociology, biology and didactics. Synergies can be found at different levels of teacher training, either pre-service or in service. Generally, synergies are viewed as the interdisciplinary approach or interaction of trainees and their teachers, administrators, researchers and schools. Grabar and Engler (2011) applied the synergetic approach to teaching a foreign language at undergraduate technical institutions [5]. Lu et al. (2014) used it for enhancing research-related skills and knowledge of graduate students and pre- and in-service teachers [6]. Gossin (2017) relied on synergies between research, education and teaching pre-service teachers [7]. Asonitou and Englezos (2022) presented a private–public sector synergy project to enhance business student learning and prepare them for real professional world requirements by equipping them with professional soft skills and digital skills [8]. Budayasti (2022) researched the synergy between teachers and supervisors in professional development to make the learning process more focused and measurable [9]. Banegas and Pinner (2021) dealt with the motivational synergy arising from learner–teacher interactions, which included giving feedback to language students to strengthen their self-confidence [10]. Gumilet et al. (2021) called the interaction of mentors and teacher-trainees a type of synergy [11]. Desimone (2009) used synergies of frameworks for better evaluation of teachers’ professional development [12]. Costa and Bracken (2019) also argued that shared approaches to curriculum and qualification development and recognition in Spanish teacher training allow for synergies between competency-based frameworks to meet the learning requirements of all students [13]. Thus, there are a number of methodologies to implement the synergetic approach in teaching FL, but there is not any specific method proposed to teach LAL. My assumption is that its potential is far-reaching, and its concepts can be usefully addressed to develop pre-service teachers’ language assessment literacy in the system of higher education.

The analysis of theoretical views on didactics shows that the interdisciplinary approach has evolved into the synergetic approach. The mere integration of disciplines does not contribute to the comprehensive development of professional competency. A system of knowledge, skill and abilities emerges from combining nonlinear relationships. This system is more flexible in our dynamic and uncertain reality when, to solve a problem, a single method is not enough. The components of this system interact and, under various circumstances, evolve.

Grabar and Engler (2011) describe the synergetic effect with the formula 2 + 2 = 5′, which means that within a synergetic system, an ‘educational surplus’ emerges ([5] p. 92). By applying Grabar and Engler’s interpretation of the synergetic approach, I will describe the process of developing language assessment literacy as follows: students interact with each other and with their teachers to accomplish some educational objective by applying knowledge and skills obtained in different courses; thus, their joint activity surpasses their individual efforts and, therefore, improves the whole process of learning by interweaving their activities and knowledge. The synergetic approach can allow for taking into consideration not only the basic components of assessment but also its broader anchoring in the teaching process.

3. Materials and Methods

In the present research, I approached language assessment preparation of pre-service teachers by studying the organization of the educational process in Ukrainian tertiary institutions.

The data set comprised educational and professional training programmes, discipline curricula and syllabi, which were made available for public view at the websites of 38 tertiary institutions where students major in FL teaching. The list of universities and the links to their websites are available at https://osvita.ua/vnz/guide/search-17-0-0-60-0.html (accessed on 11 December 2022).

Before proceeding with the description of the methods, it is necessary to clarify what curriculum documents are in order to avoid a misunderstanding of terminology. They are all standard documents and must be publicly available on universities’ websites.

An educational and professional training programme is a standard document, which describes a system of educational components of a specialty at bachelor or master’s level, which determines requirements to training, the level of previous compulsory preparation, the list of core/compulsory and elective disciplines to study and their logical sequence of studying and the number of credits and learning outcomes (competencies).

A discipline curriculum is a standard document, which determines the place of a discipline in the system of professional training, describes the aim of the discipline, its objectives, its learning outcomes and competencies to develop, the number of credits, distribution of time across different forms of training, topics, topics for individual work, assessment criteria, assessment of learning achievements and recommended literature.

A syllabus is a standard document, which is an abridged version of a discipline curriculum.

I searched for standard documents in the following disciplines: Pedagogy, Psychology, Information Technologies, General FL Practice, Methods of FL Teaching and Language Testing/Assessment. It should be recognized that not all universities uploaded these documents to their websites. Thus, the data set comprised 77 sets of curriculum documents. The corpus of the documents is presented in

Table 1. Corpus of the documents.

Documents	Number of Documents
Educational and professional training programmes	12
Discipline curricula and syllabi on Pedagogy	9
Discipline curricula and syllabi on Psychology	8
Discipline curricula and syllabi on Information Technologies	6
Discipline curricula and syllabi on General FL Practice	7
Discipline curricula and syllabi on Methods of FL Teaching	17
Discipline curricula and syllabi on Language Testing/Assessment	18

.

A qualitative and quantitative analysis of the data was undertaken. This involved several steps:

• I isolated standard documents, which concerned assessment and/or language assessment. Primarily, I searched for stand-alone courses on language testing/assessment. Furthermore, I targeted such disciplines as Pedagogy, Psychology, Information Technologies, General FL Practice and Methods of FL Teaching based on my knowledge of the preparation of student teachers due to my 20-year experience of working in a university and teaching various courses at different levels.
• I carried out content analysis of the isolated materials to identify instances of assessment knowledge, skills and abilities. The decision was made to focus assessment and not only language assessment since fundamentals are common for different disciplines, for example, ethics of assessment, fair assessment, test administration, bias, washback and formatting test papers. Language assessment literacy is a sub-competency of a more general assessment literacy. My assumption was that this would allow me to avoid repetition of the material in the course dedicated to language assessment and testing since its aim is to expand what has been acquired before. Thus, I scanned the documents for mentions of ‘evaluation’, ‘assessment’, ‘testing/tests’, ‘feedback’, ‘progress’, ‘achievements’, ‘competency/skills’, ‘marks’, ‘notes’. I performed this step manually since there are parts of documents which do contain some information related to assessment but which do not contain the key words mentioned above. The word search could have missed these cases.
• I categorized the retrieved knowledge, skills and abilities by stage of assessment activity because this is how I suggest classifying constituents of language assessment literacy: planning, designing and administering assessments, interpreting assessments results, applying the results of assessments, discussing them with parents and learners.
• I counted the universities which provide stand-alone courses on language assessment/testing based on the availability of a corresponding discipline curriculum and/or syllabus at their website.
• I carried out content analysis of the discipline curricula and/or syllabi of stand-alone courses on language assessment/testing. The parameters of the analysis were as follows: the level of studies (bachelor or master degree programme), whether it is a compulsory or elective course, the number of credits allocated for this course, the range of knowledge and skills to be developed, the language it is taught in and the FL of those students who take this course.

Below, analysis of the educational and professional training programmes, discipline curricula and syllabi is presented to gain insight into how Ukrainian future teachers are currently prepared for assessment activity.

4. Results

A general feature of the data under analysis is that they are inconsistent since not all universities provided comprehensive descriptions of their courses. Moreover, there is occasional misuse of terminology pertaining to the language testing/assessment area. This makes providing exact results difficult.

The programmes for teacher preparation vary across universities in Ukraine in terms of their scope and content. There are blocks of compulsory disciplines with set numbers of hours but, also, there is an elective component. Universities are free to offer disciplines of their choice, targeting various professional competencies. One notable issue in the data was that Pedagogy, Psychology, Information Technologies, General FL Practice and Methods of FL Teaching are compulsory disciplines at all 38 universities, which means that their students acquire some fundamentals of assessment. These disciplines belong to the block of professional preparation. Their content is more or less the same. The number of credits allocated for their acquisition is the same across universities.

I identified constituents of (language) assessment literacy in the content of various disciplines taught at undergraduate level from the retrieved discipline curricula and syllabi available on the universities’ websites.

Table 2. Constituents of assessment literacy in various disciplines.

Disciplines	Constituents of Assessment Literacy
Pedagogy	pedagogical assessment, basics of fair assessment (concept of assessment, types and functions of assessment, interrelation of learning and assessment, the role of a teacher as an assessor), administering assessments, ethics of assessment, assessees’ needs, test standardization
Psychology	construct, ethics in topic selection, bias, standardisation of rater behavior, giving feedback, individual age characteristics of assessees
Practical course in General FL	construct, task formats, test taking strategies, feedback
Methods of FL Teaching	needs analysis, defining construct, task selection, selecting and adapting (audio)texts, designing tasks, writing rubrics, using rating scales, marking written tests, assessing oral performances
Information technologies	Google forms for needs analysis of testees, searching and saving materials which can be used input for tests, formatting test papers, tools for constructing automated assessment, applications for analysis of test results, distant synchronous/asynchronous testing, messengers for peer reviewing of items and rating scales
Research methods	statistical analysis

summarises the findings of the content analysis.

Data obtained from the analysis of the standard documents revealed that a course on language testing/assessment is taught in at least 11 universities (

Table 3. Stand-alone courses on language testing/assessment provided at Ukrainian universities.

University	Course Title	Level	Type	Language of Provision
Dragomanov Ukrainian State University	Technology of testing knowledge of foreign language	Bachelor	Elective	Ukrainian
H. S. Skovoroda Kharkiv National Pedagogic University	Testing technologies in education	Master	Compulsory	English, French
Kryvyi Rih State Pedagogical University	Subject testing	Bachelor, Master	Compulsory	No information publicly available
Kyiv National Linguistic University	Modern approaches to testing in teaching English/French/Spanish in secondary schools	Bachelor	Compulsory	Ukrainian
	Modern approaches to testing in teaching German in secondary schools	Bachelor	Elective	Ukrainian
Lesya Ukrainka Volyn National University	International practices of testing and evaluation	Master	Elective	No information publicly available
Mukachevo State University	Introduction to language testing	Master	Elective	English
Rivne State Humanitarian University	Language testing and assessment (for German and French future teachers)	Master	Elective	Ukrainian
Sumy Makarenko State Pedagogical University	Basics of language assessment	Bachelor	Elective	English
Taras Shevchenko National University of Kyiv	Assessment and evaluation of learning achievements	Bachelor	Elective	Ukrainian
	Training in item-writing	Master	Elective	English
Vasyl Stefanyk Precarpathian National University	Language testing and evaluation	Bachelor	Compulsory	English
Volodymyr Vynnychenko Central Ukrainian State Pedagogical University	English language testing and evaluation	Master	Elective	English

), which is encouraging because a previous investigation I performed in 2018 (Ukrayinska 2018) showed that there were only 7 [14]; moreover, this course is compulsory in some universities and elective in others. It is taught either for bachelor or master students, or at both levels. In most of the universities, the course is taught in Ukrainian to students majoring in different FL. In some universities, it is taught in English, and in two universities, in French. A list of common target assessment skills developed during these courses is presented below:

• Selecting an appropriate assessment format;
• Writing specifications;
• Selecting assessment tasks;
• Adapting assessment tasks;
• Creating assessment tasks;
• Administering speaking and writing assessment;
• Assessing oral performances;
• Marking written tests;
• Providing feedback.

Though one of the objectives of many of the courses is to teach students to construct their own tasks, the publicly available information suggests that there is no dedicated practical component in the syllabi. Many programmes just repeat the content of the topic of assessment taught in line with the Methods of FL Teaching. These courses mainly familiarize future teachers with test task formats, their advantages and disadvantages, international proficiency language tests and ZNO (the proficiency test taken by school leavers aiming to apply for a university).

The discipline Methods of FL Teaching in all the universities includes a module on language assessment and testing, which covers principles, functions, types, forms, techniques of assessment and constructs.

Apart from stand-alone courses, there are various elective courses for master’s students on methods of teaching FL, where one of the topics is language testing, namely, digital tests, test-taking strategies and formats of international tests.

School placements are compulsory everywhere in the fourth and fifth years of study. In some universities, taking part in the joint pilot project of the Ministry of Science and Education and the British Council school placements start in the second year, but they presuppose only observation. Thus, students have an opportunity to observe in-practice teachers conducting assessments as well as trying their hand at it.

Even those universities which do not provide stand-alone courses on language testing/assessment offer topics concerning language testing for research projects and diplomas on methods of teaching FL.

Figure 1 summarizes how language assessment literacy is developed at Ukrainian universities.

My hypothesis is that developing language assessment literacy at the undergraduate level must mean interweaving already acquired general FL competency and professional competencies with multidisciplinary skills and specific assessment skills in the last year of study. Ukrainian future FL teachers acquire necessary knowledge and skills when studying Pedagogy, Psychology, Information Technologies, Foreign Languages and Methods of FL Teaching, when undergoing school placements as well as from their teachers. Moreover, learners’ autonomy presupposes learning from various resources at an individual pace, for instance, when they write their course work research projects or diplomas. Some students already work as teachers at secondary schools, language schools and give private lessons. Thus, their language assessment literacy develops, on the one hand, consistently in line with their education programme and, on the other hand, chaotically, acquired not systematically from random resources. Hence, I insist that teaching language assessment literacy should be based on the synergetic approach, which can ensure that appropriate language assessment skills have to be mastered as a precondition for the successful professional activity of an assessor.

5. Discussion

Despite the fact that some findings that have emerged in this study are inconsistent, they are still notable because they stir an idea of relying on synergies in developing pre-service teachers’ language assessment literacy.

My idea of the notion of ‘synergies’ implies that inside the system of acquiring language assessment at different moments, various pieces of assessment knowledge and skills make contact with each other in order not only to fecundate their own potential but to improve the potential of the whole. Grabar and Engler (2011) argue that the intrinsic and extrinsic synergetic effects are achieved if FL teaching is organized properly [5]. However, the question remains as to whether it works for language assessment literacy development. The intrinsic synergetic effect arises from the individual need to develop one’s own language assessment literacy, while the extrinsic synergetic effect is achieved in the learning environment through methodological interaction with peers and teachers in response to the education programmes. Co-operative learning and methodological interaction build synergistic relationships of multiple teacher-assessor behaviours, developing their assessment strategies. The sum of intrinsic and extrinsic synergetic effects pushes the system to a higher level of quality, where the system itself in feedback enriches and improves each of its constituents. The paradigm of lifetime learning language assessment has to be organized as the mediation of various formal (compulsory or elective courses, school placements) and supplementary sources (individual professional experience if any, books, conferences) to actualize the proficiency that students can achieve depending on the context of their potential workplace. For a future FL teacher, it is important to develop personal and professional qualities that contribute to their readiness for fair assessment.

The aim of developing language assessment literacy in Ukrainian tertiary institutions requires acquiring skills to plan, to develop and to administer assessments at secondary schools, to mark and score tests and to interpret and apply results of assessments, including a range of corresponding subskills. These skills and subskills start to be developed in Pedagogy, Psychology and FL Practice courses, and they are further intensified during Methods of FL Teaching and broadened when taking the course on Language Assessment and Testing. At master-degree level, this teaching is followed by more specific material on item writing and validation. Proficiency in language assessment is reached in practice during seminars, when doing assignments and during school placements. And, as Grabar and Engler (2011) remark, with finishing one’s formal education, the need for further learning does not diminish [5].

One limitation of this approach is that it poses a challenge to determine content input. The issue is that the discipline curricula should not only contain the final outcome aimed by them but also show students what else they might need to acquire in this field to become a competent assessor. For example, students are taught about needs and the individual psychological characteristics of their potential assesses—secondary school children—but in the future, they could teach and assess primary school or high school learners (in Ukraine, high school refers to the last two years at school). Or, as another example, Ukrainian universities do not prepare students to design inputs for listening tests but to use ready-made recordings, but under some other circumstances, teachers might need and have opportunities for producing recordings themselves and, for this, they need to be aware of the corresponding procedure and specific requirements. Thus, these professional assessment variables should be foreseen, and students should be taught to develop the skill to find information for themselves in the future.

The structure of language assessment literacy and objectives of the subject make it possible to create the integrated development of the target competency at universities, given its provision is organized under the following didactic conditions:

• Content analysis of curricula of disciplines taught to future teachers with the aim of identifying aspects concerning assessment;
• Structuring language assessment literacy based on future teachers’ needs;
• Decomposition of language assessment literacy into components specific to a particular relevant discipline;
• Excluding repetition of the same material in the study of various disciplines;
• Time coordination of disciplines, where each of them relies on the previously acquired assessment knowledge and skills and prepares students for successful acquisition of the concepts of the subsequent discipline;
• The need to ensure continuity and consistency in development of language assessment literacy: assessment skills must continually develop from subject to subject, be filled with new content and enriched with new connections, thus ensuring implementation of interdisciplinary links;
• Close cooperation of teachers of Pedagogy, Psychology, Information Technologies, General FL Practice, Methods of FL Teaching and language testing/assessment, and those responsible for school placements and supervision of course works and diplomas;
• Implementation of a unified approach to the organization of the learning process in all components—interpretation of learning outcomes, methodological recommendations for implementation of courses, tasks for self-study, assessment criteria, banks of assessment tools, list of references;
• Treating a student as an independent agent, encouraging their self-education, self-organisation and self-assessment, with the aim of their self-realisation (self-development);
• Interaction between schools and higher-education institutions.

I argue that synergies can be perceived as promising in terms of expanding content input by offering a larger scope for professional development and better targeting the students’ needs. The earlier preparation starts, the more students can acquire. The synergetic approach makes it possible to re-discover new learning techniques to better address the complexity of language assessment literacy. Course providers should recognize the significant role of randomness in the development of their students’ LAL because higher education is capable of self-organisation being complex, open and non-linear.

6. Conclusions

Organizing teaching of a new discipline naturally raises questions for teachers and administrators. For Ukrainian tertiary institutions, language assessment remains a rare discipline to teach, though language assessment literacy needs to be developed with FL teachers, taking into consideration the role assessment plays in FL acquisition. There is a need for a fuller understanding of how to optimize developing crucial assessment skills in a way that is not at the expense of the development of other skills. Knowing what to teach and how to effectively organize requires studies like this one to be undertaken in other educational contexts. There are several possible ways to enhance LAL of pre-service teachers. One idea is that this may be achieved through the engagement of teachers of different disciplines and implementation of interdisciplinary learning.

The present study reveals the application of the synergetic approach to the organization of the educational process at Ukrainian universities, when links between disciplines concerning assessment represent an interdisciplinary synthesis of assessment knowledge and skills. For this, I investigated how language assessment literacy is taught at Ukrainian universities by analysing standard documents.

Following the suggested didactic conditions of provision of synergies can result in more effective training of FL pre-service teachers in Ukraine and other contexts.