Exploring the Perceptions of Estonian Teachers’ Data Use in School Development

Abstract

Educational data use practices have a positive impact on evidence-driven school development. Although schools have an abundance of data available to make evidence-driven decisions, it might not be used to its full potential. This paper aimed to explore Estonian teachers’ perceptions of data use that support evidence-driven school development. In this qualitative research, semi-structured interviews were conducted with 21 teachers from six different schools. Data were analyzed deductively using a qualitative content analysis. The findings led to two themes, school level and teacher-level factors, that affected the perceptions and practices of data use for evidence. Collaboration was emphasized by both these factors. The results revealed that teachers did not see much interplay between the data they collected and used in their own classrooms for improving teaching and the data used at school-level developments. They evaluated their own data literacy skills as low. Researched schools did not have a systematic approach to data use, and teachers were left without support regarding how to use data and create meaning concerning data school development processes.

1. Introduction

Owing to the ongoing changes in education, the role of teachers has gained more influence in supporting different school development processes [1,2,3]. Teachers are expected to learn from their teaching experience, act as innovators of their teaching methods, and carry out ongoing inquiry of their own work. Furthermore, the recent, rapid shifts to distance-learning periods at schools due to the COVID-19 pandemic means that there is a need for quick solutions for personalized teaching and monitoring different learning processes, which are also relevant in promoting self-regulation in learning. Although changes in education may take time and pose numerous challenges, such teacher inquiry contributes to new pedagogical practices and benefits overall school development [4]. Due to personal characteristics or data use skills, teachers tend to overuse either intuition or professional judgement when making decisions, rather than using formal data or research [5].

The attitude toward data is impacted by how meaningful it is for teachers and has remained somewhat equivocal to educators. On the one hand, there is some ambiguity of the various terms, such as ‘data’, ‘information’, and ‘evidence’, that are used when discussing this topic and that might affect using data to its fullest. On the other hand, the quality of data plays a role as well. If it is of low quality or is not easily accessible, it might become an inhibiting factor [6,7]. Furthermore, the degree to which teachers are able to collect and analyze data, or be data literate, can be another impeding aspect in using data [6,8]. Previous studies have shown that data use may seem too complex and time-consuming, or that it is viewed simply as the latest trend in education [9]. However, when teachers understand the relevance and meaningfulness of data use in their work, their attitude tends to be, generally, more positive and constructive [10]. These arguments provide a strong incentive for further research on how Estonian teachers in particular understand and perceive data use in school settings.

One aspect of the problem is that data use in European schools is still relatively limited [3]. Similar to other countries, Estonian schools have an abundance of data at their disposal (e.g., national level satisfaction surveys, final graduation exam results at the end of 9th and 12th grade), which should help schools measure, evaluate, and improve a range of key processes and outcomes [11,12]. However, it might be argued whether different types of data are effectively put into use either in planning school development or classroom instruction, as the value of data lies not in its production, but its use. Schildkamp et al. [7] argued that schools may have enough data available, but they might lack adequate data systems or might encounter other problems, such as missing data, trouble accessing data, dealing with data overload, or having problems with the validity of relevant data. Moreover, research shows that teachers continue to struggle with data use at the classroom level [13]. Even though schools have increasingly more data available, teachers tend to rely heavily on either assessment data or on informal data, such as observations and discussions [5,14]. All these possible issues should be addressed as potential factors that affect the perceptions and practices of data use by educators.

Last, successful reforms at school mean that teachers are involved and that collaboration has become central in supporting change in a sustainable manner. Abubakhar et al. [15] argued that learning new things requires participation, and that collaboration has a positive impact on the performance of the institution. Successful collaboration, among other aspects, is dependent on certain personal characteristics, school support, time, and shared vision [3,16]. Despite these challenges, evidence-driven school development and the recent distance-learning situation have further heightened the need for teacher collaboration. However, in Estonia, collaboration has remained a challenge due to organizational and/or structural matters [17]. The exhaustive studies among Estonian teachers in 2016 and 2017 concluded that, to sustain development in schools, a shift from an individualistic to a more collaborative approach is required (Kooliuuring 2017, 2019). On a similar note, Estonian schools have not yet reached parity in their practices and learning outcomes, and some schools are still struggling with the stigmas from practices of vertical leadership.

Overall, the attitudes toward and the development of data use practices among teachers is both an individual and collective endeavor [12]. In order to explore the aforementioned gaps, this research aimed to explore how Estonian teachers perceive the use of educational data that supports evidence-driven school development. The findings of this research provided the knowledge and means to support effective data use through an understanding of what teachers think of data and data use, how meaningful it is for their work, and what might be the enabling and inhibiting factors that affect data use for school development.

2. From Data to Evidence-Driven School Development

There are numerous factors that affect evidence-driven school development processes. First, there needs to be a clear understanding of the concepts that are used when talking about educational data. Second, the types and purposes of data need to be clarified in order to use data in a meaningful way. Third, as decisions and actions that are made are based on fully optimizing data when executed with the involvement of all teachers in a school, the importance of collaboration also comes into play.

The term data encompasses ‘…information that is systematically collected and organized to represent some aspect of schooling’ [18] (p. 21). Only when data is analyzed and interpreted can it be called information. This process of interpreting data can be called data use. Unlike data, information has ‘meaning, relevance and purpose by contextualizing, calculating, connecting and summarizing data’ [8] (p. 482). This information can further be used as evidence: a type of information that helps assess the truth or the validity of any possible claim [19]. In Estonian, however, these terms are often used interchangeably, which may cause confusion.

In a school context, different types of data are available, such as context (parental expectations, school culture), input (socio economic indicators, student characteristics), process (work in class), and output (test results, student satisfaction) [8,9,16,20]. The aim of school data is to identify and bridge gaps in student achievement, to improve teacher quality, to develop curriculum, to share the best practices, or even to communicate with key stakeholders, such as parents, the local government, and etcetera [21]. Using data has benefits for both student learning and teacher professionalism, which leads to school development in that teachers learn how to solve different educational problems and enhance their skills simultaneously [3,7,9].

Data can also help schools make better-informed decisions on student academic achievement or when planning instruction. Such decisions are referred to as data based decision-making [13,21]. This enables educators to prioritize the time spent on instruction, to target individual student needs, and refine instruction, all of which support school development [12]. Effective data use by teachers and school leaders impacts school development regarding the increase of student achievement, teacher professionalism, and job satisfaction [16]. However, teachers might face difficulties when analyzing data; there is evidence that teachers may conform to using informal data instead, which causes them to focus their attention on what they already expect to see which, subsequently, means they might miss some relevant data that would complicate their previous assumptions [5,14].

As data use skills are more likely to be developed in a professional learning community (as cited in [22]), such as a teacher team, school leaders would benefit greatly from supporting teacher development by creating an analytics-supportive environment that advocates professional learning capability and data literacy of the staff, as effective data use requires collaboration [3,5,11]. Thus, the pivotal role that teachers play in school development processes cannot be underestimated. Drawing from research, instruction can be improved when data is used as a basis for school-related decisions and advocates sustainable data use practices [9]. The processes of improving instruction and the role of teacher collaboration in supporting that became prominent during the distance-learning period.

Schildkamp et al. [3,6] highlighted that effective data use requires collaboration, and certain teacher and organizational characteristics. Collaborative or team learning is not a new concept. It is based on the assumptions of postmodern philosophy that posits that learning should be an active process and each member of the learning community contributes to the production of new learning and knowledge through social exchange [23]. According to post-modernistic philosophy, reality (and our knowledge of it) is seen as socially constructed [24]. Although studies on collaborative learning seemed to initially focus on how individuals function within a group, the group itself has become the incentive for academic research [25]. The first scientific works on the subject of collaborative learning emerged at the beginning of the 20th century, though they have become more prominent in the last few decades [26]. Collaboration is closely linked with a constructivist learning theory which draws from the works of John Dewey and Lev Vygotsky. According to them, learning is an active experience wherein learning by doing and reflecting about one’s own learning should be implemented [23]. Vygotsky further stressed the importance of social interaction, language use, and the critical role that culture plays in the process of cognition [23]. Senge [27] emphasized that, as the world does not include separate and unrelated forces, learning organizations are meant to let people expand their competence in unity, which ultimately helps the participants gain an increase in the development of professionalism. People reach understanding through an iterative process, after attempts to explain processes in social interaction, which means that new ideas surface between people as opposed to within them [28,29]. Therefore, collaborative learning can be defined as a strive to enhance organization-related matters through knowledge and understanding [29]. In order to do this, teams should develop a clear vision of their current situation, goals, and the methods of how to attain those set goals [30]. Some noteworthy benefits of such collaboration among team members include better development of social and communication skills and better rapport and relationships within the team, which ultimately lead to a higher level of self-esteem, work satisfaction, and capacity for problem-solving [26].

3. Conditions for Data Use

In order to promote teachers’ input to support school development, to refine instruction and, ultimately, to impact students’ learning outcomes, certain conditions need to be met [9]. Drawing from previous research, two main levels for data use in schools were highlighted: teacher and school level (see Figure 1).

School-level factors are, for instance, having a school leader that helps create and sustain an environment that supports data use in school. The role of the school leader is also to enhance teacher collaboration by allocating time and other resources that allow for it to happen. On a teacher level, the factors that influence data use are mostly to do with personal characteristics, such as having a willingness to collaborate and develop professionally. However, data use skills play an equally important role.

3.1. School Level Factors for Data Use

The range of how frequently and successfully data use is implemented by teachers is directly dependent on the school leader and the environment that they have helped create in that school. Organizational culture also plays a significant role in both collaboration and data use [31].

Firstly, in a school setting, the person who can help maintain such an environment is a school leader, although teachers can also contribute positively to its sustainability. The most successful institutions are those who ‘…prioritize staff motivation and commitment, teaching and learning practices, and developing teachers’ capacities for leadership’ [11] (p. 163). School or team leaders are considered key figures in creating conditions that are vital to engage its members and to pass on team-learning behaviors in that they guide teachers to understand reasons behind different processes and to make them agree with them [26]. However, they frequently face impeding factors, such as dominating team members, problems with obstructive authority, or when members feel a cognitive overload when performing poorly-composed tasks [26]. To overcome these obstacles, school leaders should strive to define clear team goals and to provide members support and means to collaborate [26]. School leaders can also contribute by providing time for teacher collaboration and by demonstrating their enthusiasm for inquiry and data use.

Second, to promote teacher collaboration in data use, there needs to be a safe environment. A safe and supporting school culture is a precondition for collaboration to occur [32]. School leaders play a significant role in cultivating an open climate and positive, trusting relationships that encourage the teacher’s willingness to become involved in decision-making processes [1,33].

Third, Brezicha et al. [1] supported the idea that when teachers are presented with opportunities to partake in meaningful decision-making on school-wide matters, they feel ownership of and commitment to their profession. Therefore, the role of the school leader is to work towards offering teachers means to take on leadership roles and ascertain that their voices are heard. These activities raise commitment in teachers and support their data use practices, which can improve the quality of education provided by individual teachers or schools and can lead to improved student achievement [34]. Thus, the role of the school leader is significant in offering means to engage teachers and to support continuous data use.

3.2. Collaboration in Data Use

Teacher collaboration when using data can be a fruitful ground for professional learning since teachers are vast sources of experience and knowledge. Teachers can offer deeper insights into the profession and provide continuous learning at the workplace by critically reflecting on different teaching practices, by offering collegial support, sharing knowledge, creating teaching materials together, and giving peer feedback [35]. The same applies to developing personal data literacy skills, as it is much easier to do so in collaboration and by learning from colleagues. Teacher teams who have a strong link to teaching practice are more willing to change their personal beliefs about teaching and learning, which makes them more willing to collaborate [35].

School development is a collaborative process wherein data plays a key role. Data is used for making decisions and its systematic use helps support school improvement. Incidentally, although collaborative data has many benefits, no one can really be taught how to collaborate. However, the factors that could promote collaboration use are, first, the enthusiasm and example from the school or team leader and, second, by creating a supportive and safe environment wherein teachers are expected to use collective inquiry. When there is a wish to incite collaboration, the relationship between the team members should be promoted. Since collaboration occurs among colleagues, they should share the same norms and goals, bearing in mind that knowledge creation processes raise the competitive advantage of the team and the whole school [15]. Furthermore, a school leader should find means to allocate extra time for teachers to make collaboration happen because it can be challenging to fit it into the normal timetable of busy teachers.

Therefore, to help collaboration thrive, certain characteristics of a teacher come into play. It is possible that teachers who feel less certain about their teaching practice and have little experience with collegial collaboration, or who consider their data literacy skills low, also underestimate the importance of sharing ideas and experiences [35].

3.3. Teacher Level Factors for Data Use

When the school leader is encouraging and supportive of collaboration and provides teachers extra time, teachers can use data for meaningful purposes. Collective data use complements and broadens the individual expertise of teachers, which subsequently leads to higher quality in instruction [36]. Subsequently, the school benefits in terms of its learning outcomes and gains a considerable competitive advantage. Furthermore, teachers can offer a considerable input to maintain an inquiry-based and supportive school culture.

Using reflection can also help teachers determine their strengths and weaknesses and become more willing to change their behavior and implement different and innovative practices [34]. Moreover, teachers tend to possess a more positive and constructive attitude to data use when they can see the direct links with the classroom and how to improve classroom instruction [10]. Thus, teachers need to see the meaningfulness of the data and be objective about their data use skills.

Thus, teacher inquiry needs to be systematic, intentional, contextual, self-critical, practical, action oriented, planned, evidence-driven, evaluative, and shared [4]. Although there are gaps in teachers using data for the purposes of increasing professionalism, such activities occur best in a team as they contribute to the shared understanding of collecting, sharing, and using data [4].

In this research, the focus was on the teacher’s perceptions of data and the knowledge of how to use data from its purposeful and systematic collection to its use for school development. The latter involves teachers making data-based decisions on improving their instruction as well as their connection with broader school data that is used outside their classroom. Taking the above into consideration, this paper aimed to explore teachers’ perceptions of data use in a school development process. The following research questions were developed: (1) what are teachers’ understandings of data and data use in a school context; (2) how do teachers perceive the meaningfulness of data use in connection with their teaching practices and with school development processes; and (3) how do school level factors support teachers’ data use?

4. Methods

4.1. Research Context

In Estonia, schools plan and monitor their own progress. On a national educational policy level, it is stated that it is obligatory for schools to have a development plan and to conduct internal evaluations [37]. Based on national strategies, one of the goals for schools is to implement an evidence-driven approach when making decisions on school development [38,39]. To support schools in their evidence-driven practices, different data sources are available, such as the results of school satisfaction surveys or graduation exams that are made available by the government via a national data base.

As stated by OECD [17], school leaders in Estonia are rather autonomous when it comes to supporting the professional development of teachers and developing the pedagogical aspects of the school. Thus, school leaders have a strong impact on the school creating and sustaining an environment that is collaborative, transparent, and evidence based. A further positive aspect to collaboration is that it increases teacher professionalism and creativity, which help sustain educational change [40]. In such school cultures, there is collective responsibility for problem solving and student learning outcomes.

In order to support the evidence-driven development of a school, a school-university joint program, Future School, was set up by Tallinn University [41]. The aim of this program was to support up to six schools throughout the academic year to enhance instruction and overall school culture. Each school appointed a team to work on the chosen school development plan, and they were further assisted by consultants from the university. The team comprised the school head, teachers, and members of the administration, but the team leader was always a teacher. University experts organized monthly meetings with school teams to offer them theoretical and practical tools that helped them carry out development plans. There were three pillars for this program. First, help foster teacher leadership as the team leader is a teacher, not the school head. Second, the school development activities are planned in collaboration between the teachers of the school and the university experts. Third, the schools are encouraged to use evidence-driven decisions by finding means to measure their progress throughout the academic year.

4.2. Participants

A phenomenological study was carried out with 21 teachers from the six different schools that entered the university-school joint school development program in autumn 2019. A phenomenological study includes ’lived or experiential meaning and attempts to describe and interpret these meanings’ [42] (p. 614). The way teachers talk about data, specifically what language they use, is an important part of phenomenological research.

Purposive sampling was used to select participants for this study (see

Table 1. Overview of the Schools and Participants.

School Type	School Size	Number of Teachers in the Study
Public upper-secondary school (Grades 10–12)	Teachers: 40 Students: 389	5
Public basic school (Grades 1–9)	Teachers: 22 Students: 211	3
Private primary school (Grades 1–4)	Teachers: 23 Students: 215	3
Public secondary school (Grades 1–12)	Teachers: 107 Students: 1240	4
Public basic school (Grades 1–9)	Teachers: 42 Students: 460	3
Private basic school (Grades 1–8)	Teachers: 68 Students: 560	3

). The schools varied in size and were located both in urban and rural areas. Although most schools were public schools funded by the state, there were also two private schools in the sample.

Altogether, 21 teachers from six different schools and with different backgrounds were interviewed. With one exception, all the interviewees were women. All participants gave their informed consent for inclusion before they participated in the study.

4.3. Data Collection and Analysis

Data were collected by conducting interviews and interview questions were structured around the concepts in the theoretical framework (see Figure 1) with a focus on the perceptions and current practices of data use among the interviewees as well as the conditions behind them. The main themes of the interviews were: (1) perceptions of data use (how do you understand data in a school context?); (2) teacher’s data use practices (share your experience of an instance where data use has helped you in your work); (3) teachers’ data literacy skills (how do you evaluate your data literacy skills); (4) collaboration among teachers (give examples how you have analyzed data and made decisions on it together with your colleagues); and (5) the role of the school leader in data use (how important is the role of the school leader in data use at the school).

The individual interviews were conducted from March 2020 to May 2020. For the interviews, teachers belonging to the same school were sent an email that further explained the conditions for the interviews that were conducted in Estonian via Google Meet.

Overall, the interviews lasted between 20 min to over one hour. The audio files and transcriptions of data were marked with codes for the sake of anonymity, and the authors were the sole users of the secure folders. The names of schools and teachers were coded with letters and numerals so as to further ensure the anonymity of the participants of the study: S1-S8 for schools and T1-T21 for teachers. The interviews were recorded and later transcribed verbatim by using a speech transcription system [43]. For the sake of this paper, the quotes from the interviews were translated into English.

A qualitative content analysis of the interviews was conducted by the web based interactive software QCAmap. A deductive approach was used in that conclusions were deduced based on a sequence of steps from general to particular [44,45]. The analysis was based on the school and teacher-level factors (see in Figure 1).

The validity and transparency of the data analysis was achieved by first analyzing data independently and then by administering discussions to reach consensus [44].

5. Findings

This study aimed to discover how teachers perceive data and what factors influence those perceptions. The findings are introduced in three sections based on the research questions starting with how teachers understand data, followed by how they perceive the meaningfulness of data use in their work and school development, and, finally, what school-level factors support or hinder data use of teachers.

5.1. Teachers’ Understanding of Data and Data Use

Overall, the majority of teachers viewed data as information that they had about a student. They agreed that, in a school setting, there was an abundance of data available, perhaps even too much to be able to navigate. Interestingly, one teacher pointed out that data could be a type of sensation or intuition that we notice, which indicated that, sometimes, the border between intuition and data or evidence is a blurry one. There was only one teacher who mentioned that data could originate from different sources, meaning that data can be either formal or informal.

• Data is information that comes from different parties, so that I guess data can be formal, such as from a questionnaire to parents or students. At the same time, data can be informal, such as oral feedback from students. I think it is also important. (S4, T14).

In terms of what purposes data is used for by teachers, the respondents mentioned several sources of data, such as satisfaction surveys and distance-learning feedback questionnaires for teachers, students, and parents—student feedback being the relatable one for teachers.

• Data could be various feedback that we get by using different methods, such as a questionnaire. What appeals to me most are observations as they truly give me information about what is going on in class. The same goes for discussions with students. (S2, T6).

One teacher mentioned that the school-university joint program made her contemplate that data was not only useful for her own teaching practices. She was the only teacher who said that there should be even more data available for use.

• Before the program, data (especially quantitative) stood separately from what was happening in my class. However, because of this program, I understood that all data are not only relevant for my job, but also for the whole school; and that there should be even more data collected and available, not only by one teacher in his or her class. (S5, T17).

The interviewees did not point out that there was also research data available, such as previous studies or educational books that could be used as evidence in decision making, when reflecting on one’s work, or when interpreting the behavior or feedback from students. One teacher discussed this aspect, which demonstrated a deeper understanding of different data available for schools.

• In general, most improvements in our school derive from a lot of reading. I cannot stress enough the amount of reading we do. The ideas behind our school concepts and values have come from and rely on different research and previous studies. I believe that if we do not study something then we cannot change anything. I feel as if being a teacher researcher is part of school culture. We take research seriously and we collect a lot of data, such as feedback. Data should be collected when there is a problem /…/there is no need to collect it only for the sake of collecting it—it does not help attain our goals. (S1, T3).

Overall, teachers saw data as related to students and their progress, rather than evidence they could also get from previous research or studies. They seemed to be far more interested in finding out how students had progressed or what was going on in their classroom rather than in how to improve their teaching.

5.2. The Meaningfulness of Data Use

One of the aims was to see if there were patterns of interplay between data use in the school overall and its relatedness to classroom practices as seen by the teachers.

The majority of respondents talked about classroom-related data, which might indicate that teachers tend to use data almost exclusively for instructional purposes, such as planning classwork, choosing teaching materials, or conducting tests without seeing the wider context of data. Some teachers pointed out that they were hesitant about how others, such as the management team, would use and benefit from the classroom data. One teacher mentioned that she did not have a clear understanding of school-related data and how it could impact her or the school in general.

• I have not really thought about data that much because I am not sure what kind of data would be available and I do not have a clear understanding of how to use them. Perhaps the school management team could help with this more—to share different data that the school has and maybe then I could think how it could be useful for me or my work. (S1, T2).

Furthermore, one teacher mentioned that this evidence-driven approach in education was something that researchers have imposed on teachers, and that data could even have a negative connotation and confronted the researchers.

• Well, I can see that I am talking to a researcher and you want that everything is evidence based. So that I would take the data analyses and would draw conclusions based on these. But yes, before that you had this nice term “gut feeling” and like to think of it this way that I know my students very well and to be honest, I could already tell in September what kind of grades they will get in spring. (S2, T7).

The way schools utilize data affects teacher attitudes and how well they understand data. When teachers do not feel connected with data collection and analysis or are not part of discussions based on data, they do not feel that the whole process may be meaningful for their own classroom practices.

• Our school is rather big. We do not have such a culture in our school that we analyze data together. Yes, we do have many data collections in school and the leadership team analyzes and presents the results to us. However, no discussion follows the presentation and data use does not seem very systematic. (S4, T14).

Moreover, teachers can feel detached from data use because they do not understand it.

• Basically, the school presents numbers to us, there is less of an emotional side to it. They tend to speak in the language of numbers. I know school management uses data, but they do not open the topic too much and after presenting the data to teachers, they move on. (S4, T15).

One respondent indicated that the assessment data the teacher collected was used by management as well. As its actual purpose was not known to the teacher, it might imply that school data use, in particular, is not transparent and therefore could be used for ‘blaming and shaming’.

• A class teacher needs to write down all grades at the end of the trimester /…/. We have been doing it for years. I do not know why exactly /…/. I do not know what it will give to our department head or what the school is doing with this information. When I fill in the table, it will give me an overview of the grades and /…/ that makes me reflect on my work. But yes, I do not know how beneficial it is for someone else. I even think that this data is not analyzed at all. (S4, T12).

Another teacher from a different school pointed out that there was a tradition to come together and analyze data together with other teachers, and that everyone could be part of the process, hence being able to relate to school data.

• After data is collected and initially analyzed by our head teacher, we come together and discuss them together. We ask what could be inferred from the result. Yes, we always analyze them together or in small groups and everyone can state their opinion and provide reasons behind it. (S6, T21).

To summarize, the degree of the meaningfulness of school-related data to teachers and their teaching practices depends on how data is collected and talked about in school and how it is used for making decisions. Teachers who felt there was something lacking in these areas perceived the connection and meaning to school data was weak.

5.3. School Level Factors of the Data Use

Finally, we looked at the aspects that might be either hindering or promoting in terms of data use on both the school and teacher level. Different characteristics when analyzing these factors, such as data literacy skills of individual teachers and questions about school culture, e.g., teacher involvement in decision-making processes, collaboration, and the role of the school leader, were found.

School culture seemed to affect teacher attitudes toward data both positively and negatively. First, teachers seemed to hold the school leader accountable for creating a collaborative and inquiry-driven environment.

• The school leader can set an example. The leader can lead teachers to gather more data and to analyze them by demonstrating interest in that. I think the school leader plays an immense role here. (S4, T 14).

Next, how the school leader could affect the creation of a collaborative environment was related to how involved teachers could be made to feel overall in decision making. Apart from making the job more interesting, one teacher pointed out that being involved could be a chance to develop their own work environment and that it helped make different processes more meaningful for teachers.

• I think that being part of school and decision-making processes is naturally a very important part to a teacher. It shows that my opinion is valued and that I am also a creator of the school, in a way. And that makes me feel that I have more ownership of the running of the school. (S3, T10).

The hindering factor of not involving teachers in decision-making was addressed. For this teacher, the leader did not provide enough support in involving staff.

• I do not think that teachers are so involved. Rather we feel that our management stands separately, they are doing their own thing. It reminds me of a top down approach. (S4, T14).

Interestingly, one teacher did not perceive the sole responsibility of the principal as helping create and sustain a collaborative school environment but saw that teachers themselves were a significant part of the whole school development processes.

• For me, the teacher is a leader in the school because he/she leads all the main processes. What happens in my classroom is number one priority for me, and for my school and the management level. And when I am aware of it and take responsibility, then I should be willing to invest to achieve the big picture. Yes, that does depend on the school leader a lot—if we are given the time and means to express our opinions, work together, make decisions, and put in the extra effort. (S1, T3).

Apart from school culture and its factors that might either support or inhibit data use, data literacy skills play an important role. The majority of teachers confessed having rather reserved or even negative attitudes towards data analysis for different reasons, such as being insecure about one’s skills, or that they did not remember how to analyze data or that they have never been taught to do so.

• Sometimes I do compile questionnaires to add to my observation of the class, and the questionnaire helps me confirm my beliefs. However, this is not structured, systematic and scientific. I would not know if I had the skills to analyze data. (S2, T6).

One teacher admitted that their data analysis skills were satisfactory but data in general was not very appealing; another teacher who mentioned that was confident in data use. The idea that teachers might need some help with data use at the school level to create ownership and meaning was emphasized by one teacher.

• I would say that there is a lot of data, the question is how to interpret it. I do not believe in numbers so much, because behind each number there is a person. (S1, T3).

Generally, teacher data use and their attitudes are affected by personal and school level factors. On the one hand, the majority of teachers felt that the way school leaders talk about and use data leads the way for the whole school in terms of how data is analyzed and how involved teachers are in decision-making processes. On the other hand, teachers were also affected by their own personal limitations, such as lack of data literacy skills.

6. Discussion and Further Implications

This research aimed to explore how Estonian teachers view data in their work. Specifically, the perceptions that teachers have of data, how meaningful data is for them, and the factors that inhibit or enhance data use to support the implementation of an evidence-driven school development.

First, we found that teachers viewed data mostly with regard to the information that they had about the student, feedback being the most prominent source of data. However, the findings revealed that research evidence was mentioned as related to school development, though not seen as evidence for improving teaching. This is similar to earlier findings by Schildkamp [5] and Vanlommel et al. [14], who noted that teachers tend to underuse formal or research data and rely mostly on intuition or professional judgment. The latter is then used to handle data to, usually, get feedback so as to prepare new material, to determine the pace of the studies, or to see how well students have acquired the topic. This brings forth a question on whether and how schools could help spread different educational research amongst teachers to support their professional development and to help make better-informed decisions in the classroom.

We analyzed how teachers perceived the meaningfulness of data use; the findings demonstrated that teachers viewed data as being strongly connected to the classroom level, rather than to the overall school development. The findings also indicated that teachers and schools did collect ample data about their students and their progress. Nevertheless, what the school did with the data and how it was used for school development was not clear to teachers. The more meaning data has to teachers, the more relevant they see it to their work. [10]. However, the prevailing attitude of teachers in terms of school data is hesitant, if not negative. Teachers may be unaware of what kind of data is available at school, which implies that they are not involved in data collection nor in the analysis process. This, in turn, is connected to the lack of collaboration that seems prevalent among Estonian teachers [17]. Furthermore, teachers seem to be hesitant in how the data they collect in their classrooms could help support the whole school. Not seeing the connection between classroom and school data leads to mixed or negative feelings about data in general and to the fear that data could be used for blaming and shaming purposes. Some teachers feel that data use is the latest trend and is too complex or time-consuming [9]. This means that if teachers do not understand and the school culture does not support their involvement in discussion about data, then data remains detached and will be seen simply as numbers without meaning.

Another finding indicated that schools lacked a systematic approach to data collection and data use at the leadership level, hence the somewhat hesitant attitude to its meaningfulness by the teachers. This is supported by research on how meaning is created by being involved in the discussions so that understanding can be reached [28,29]. Support to schools is needed to: overcome struggles with data overload; promote systematic and meaningful use; and inform how to communicate it clearly to the whole school, as it has benefits for whole-school development in general, and successful school reforms require the participation of all teachers.

Finally, we analyzed how school-level factors support teacher data use. When what aspects might either enable or impede data use by teachers were considered, a striking contrast between teacher perception of how comfortable they felt about data collection in their classroom and their actual data literacy skills became evident. Surprisingly, the majority of teachers felt insecure about their data literacy skills, which indicated that, although teachers seemed to believe that they used various data in their classroom, they were actually struggling with its systematic and purposeful collection and analysis [13]. This leads us to propose that teachers do need help with analyzing data, and that the schools are in need of extra support regarding how to create a sense of ownership in teachers. Here, universities could have an important role, first at the teacher-training level as well as when planning different programs for schools. As previous research demonstrates data literacy skills develop more efficiently when done in collaboration [3,5,11], this practice should be considered to support teachers.

Different patterns were found in data use addressed by teachers. Data use cannot happen successfully without having the necessary environment. This is where the immense role that the school leader has comes into play. Drawing from the interviews, the attitude to data tended to be more negative in schools that seemed to have a vertical leadership practice, as the development of the school as a whole was impeded. Teacher involvement also tended to be low, and data was not meaningful for teachers outside their own classrooms. The school leader can do a lot towards creating and sustaining a collaborative school culture when involving teachers in school development processes. Even though collaboration time was allocated to all schools that took part in this study, it was not enough to incite meaningful work; the school leader can set an example in this respect.

This study had some limitations. There were some limitations with the sampling, as the teachers that participated in the study were a representative sample of their schools, and they did not represent the perceptions of wider teacher communities. On the one hand, the choice of purposive sampling added to representation, meaning that the sample was chosen for the specific needs of this research and added more depth to the study as the school took part in the school-university joint program and perhaps some had had more contact with an evidence-driven school development. On the other hand, it lacked broader representation in the wider population. However, although the sample was convenient for the purposes of the researchers, this choice was deliberate. Self-reported perceptions also posed some limitations that might have affected the analysis of the results, such as when the subjects felt pressured to or somehow involuntarily adapted their answers to the interviews for some reason. Another aspect that is both supportive to the study as well as a limitation was that the participants worked in schools with very different cultures and leadership styles. However, determining the patterns and relations between teachers’ understanding of data and the school leadership style was not in the scope of this study. The scope of the research also did not include comparators between different schools or between instances wherein data were used and wherein they were not. Further research could draw upon how teachers use data in schools and compare the difference in situations wherein data were used and wherein data were not used. Another approach could be to conduct a quantitative research program with a focus on a broader view on Estonian teachers and their perceptions of data in schools. Moreover, the relations between leadership styles, school culture, and teachers’ perceptions of data could be researched more closely.

Some practical implications that emerged from the study were that school leaders need to find ways to involve teachers in systematic and meaningful data collection and analysis. It is not important how much data is gathered, but rather what meaning is given to it. By setting an example, school leaders can help teachers find meaning and ownership in data and how it could be used for school development purposes. This would strengthen the interplay between classroom and school data for teachers. One option to sustain data use would be to enhance teacher collaboration so that they could learn from each other. Furthermore, school leaders and teachers need support in terms of how to understand data and how to analyze it. For data literacy skills to improve, universities could offer personalized courses for schools or training.