Using the Visible Learning Research to Influence Collaborative Leadership

Abstract

John Hattie’s Visible Learning research has been read, dissected, and implemented by educators and leaders around the world. Whether it’s the effect sizes or the influences that challenge the thinking of the status quo, the Visible Learning research has taken a place in many educational conversations. Over the last three years, the author of this paper has been researching and implementing the Visible Learning research, and used some of the findings to create a leadership workshop that focused on six of the influences that Hattie has been studying for years. The point of using the six influences he focused on was that based on the author’s knowledge of leadership through his experience as a teacher and building leader, as well as his understanding and research around school climate, there were six areas that all leaders should understand deeply if they are to have a positive impact on student learning.

1. Do School Leaders Really Understand Educational Research?

Close to five years ago the author left his position as a school principal in the USA, and began working as a Visible Learning trainer for John Hattie. Fairly early on he had a strong desire to connect Hattie’s research to a practical context for leaders, teachers, and instructional coaches. Part of the issue that he saw was that everyone wanted to use Hattie’s name to build credibility around what they were trying to implement in classrooms around their schools or districts. These leaders touted the research, but many had strong misperceptions about the research, and they were not clear on how to make it practical.

One of the issues that began surfacing is the fact that leaders and teachers read the research and even take part in Visible Learning trainings that are facilitated by seasoned trainers, but those leaders and teachers did not always know where to begin implementing the work. A second issue that rose to the surface was that of leaders and teachers who felt knowledgeable about Hattie’s research, but often had a misunderstanding. Hattie is not the first researcher who had this happen.

We need not look any further than researchers such as Carol Dweck [1], who had to clarify her work around the growth mindset in Education Week; Howard Gardner [2], who had to clarify his multiple intelligences work in the Washington Post; and Carol Ann Tomlinson [3], who had to clarify her work on differentiated instruction in Education Week, to understand that these researchers were forced to clarify because educators and leaders were not using it correctly.

Some other misunderstandings involved not knowing the effect sizes were averages from the synthesis of the meta-analysis collected by Hattie. Other times it had to do with implementation, and leaders told teachers not do use some strategies in the classroom because Hattie’s research did not support those strategies, which is not entirely correct. The point of Visible Learning is to understand the research, but to also gather evidence to support whether strategies of learning are working in the classroom, regardless of what the present research that was collected may say.

There are other debates when it comes to looking specifically at the Visible Learning research. One of the biggest criticisms is the disagreement over the synthesis of meta-analysis that Hattie took in his research. Although the synthesis of the meta-analysis approach can be very beneficial for leaders to use in their conversations with teachers in their buildings, other researchers question the approach.

We know that meta-analysis are large studies that involve individual smaller studies that all come with effect sizes. Hattie combined those studies together in a synthesis of meta-analysis approach, which became popular in 2009 when he published Visible Learning: A Synthesis Of Over 800 Meta-Analyses Relating To Achievement [4], and that is where the criticisms begin. Some researchers feel that we should respect the smaller scale studies and not combine them together because something will get lost in translation. However, the synthesis approach can help in that translation. Why this is important is that the synthesis approach provides readers with a list of the important common themes that Hattie found when looking at the research. For example, when looking at hundreds of studies on feedback, Hattie [5] found through looking for common themes that the most effective feedback included instructional feedback at the task, process, and self-regulation level (2012).

There is another important element of the synthesis approach Hattie took to meta-analysis. There are a handful of teachers in each school building that may look at small scale studies and say, “Our students are different,” than those students in the study, even if they do not know that to be true. However, it is much more difficult to have that reaction if the teachers are handed a synthesis of meta-analysis that includes thousands of students.

2. Methods

This paper will consist of two separate approaches to learning what works best for educators where the Visible Learning research is concerned. The reason for highlighting two different approaches is to explain where the author’s work originated, and the then the lessons learned after the work began being taught in workshops around North America and the United Kingdom.

First, the author read, and learned from, narratives given by participants after Visible Learning workshops over a two-year period. Over that two-year period, the author looked at narrative comments on the Visible Learning evaluations to help him better understand how to facilitate more effective workshops. In those evaluation narratives the author found that there were six areas from the Visible Learning research that were of particular interest to workshop attendees. These narratives aligned with the author’s understanding of leadership given his experience as a school building leader over 8 years in the United States. At that time of understanding, the author began referring to those six areas as Collaborative Leadership: 6 Influences That Matter Most. In this paper, the author will further explain the importance of those six influences.

For the Collaborative Leadership work, the author used a mixed-methods approach to understanding the impact of the content used to help explain the six influences. In this mixed-methods approach used in this paper, it consists of quantitative research addressed through a Likert Scale approach used in evaluations completed after Collaborative Leadership workshops in North America and the United Kingdom. The evaluations were completed by participants in one-day, two-day, three-day or four-day workshops immediately following the workshop.

Additionally, a qualitative approach was taken in understanding the Collaborative Leadership work using participant narratives completed after one-day, two-day, three-day or four-day workshops immediately following the workshop.

3. Filling the Leadership Gap

Additionally, the author found that that there was a gap between what leaders took as part of their understanding of the Visible Learning research, and how they were actually supposed to implement it. Part of that gap involved where to begin implementing the work.

Having been a school leader for eight years, and in a building filled with teachers, staff and students that he respected, the author has a deep appreciation for the role and want for those in it to go deeper with their practices. After all, in over 420 studies that Hattie collected on school leadership, the overall effect size is a 0.33, which is below the hinge point of 0.40. Leadership plays an important part in student engagement but many leaders are distracted by duties that come with the role, which makes it difficult for leaders to choose where to begin.

Leadership is not easy, nor is it for the faint of heart. Schools are places that have to balance between offering rigorous academic curriculum, social-emotional learning, work with students who experience trauma, and do it at a time when they have to work through, and successfully meet, mandates and accountability measures imposed from the national level. The author understands that this is a monumental undertaking by school leaders and staff.

Too often what complicates the process is that leaders feel they have to know it all before they begin having teachers implement changes, and then they ultimately tell teachers what they have to do to accomplish all of these tasks. This leads to the enabling of teachers rather than the empowering them. It causes teachers to meet every question of “What do you think about how to move forward” by a leader with and answer of, “Just tell me what it is you want me to do,” by the teacher. This is not conducive for deep learning.

What we understand about all of the required changes taking place in school is that leaders cannot do it alone, which is why the author began exploring the concept of collaborative leadership using Hattie’s Visible Learning research. Collaboration is not easy for leaders because they often provide opportunities for teachers to collaborate, which is important, but what they forget is that the leaders also have to take part in the collaboration.

Collaboration is a technique we want our students and teachers to do, and the word collaboration is part of the 4 C’s in those 21st century skills we have all been focusing on for well over a decade. A collaborative mindset is necessary if we are truly going to delve into collaboration with teachers and students, which means we need collaborative leaders. DeWitt [6] (2017), writes,

Collaborative leadership includes the purposeful actions we take as leaders to enhance the instruction of teachers, build deep relationships with all stakeholders through understanding self-efficacy (0.63) and building collective efficacy (1.57) to deepen our learning together

(p. 10)

Unfortunately, unless we have a deep understanding of collaboration, we can truly believe that we are collaborating when in reality we are just trying to push our own agenda. A criticism of any mindset is that we often want the people around us to do it, without exploring whether we are doing it ourselves. When leaders do that, they put themselves and their initiatives at risk of failure. We no longer should work as silos in education because our issues are too large to do alone, and we learn a great deal by working with others. Ideas are not created by schools. Ideas are created by people who work in schools, and that takes collaboration.

The author found there are six areas that leaders, teachers, and students need to work on in order to create collaborative environments where they can learn from one another. Those six influences are instructional leadership, collective efficacy, professional learning and development, assessment capable learners, and family engagement [7] (p. 6).

4. 6 Influences that Matter Most

As a Visible Learning trainer, the author found through follow-up presentation requests and feedback from participants that there were six of Hattie’s influences where leaders should be focusing their time. As you read, the author will build the case that each influence is directly related to the other influences.

In his mixed-methods research, the author has come to find that there are leaders who do not go into classrooms to see the learning taking place, and they do not make learning the center of conversations in their school. Therefore, they lack the ability to become instructional leaders. Where this is an issue is when that same leader completes a teacher observation and provides feedback to the teacher being observed. What ultimately is at risk of happening is that the relationship trigger goes off for the teacher because they do not believe in the credibility of the leader providing feedback. The author hopes this one example illustrates how one of the six influences has an impact on another.

The following are the descriptions of each of the six influences on learning that the author has researched and uses in workshops, which has been adopted at the state level in the United States used by districts and school boards in the US, Canada, and Australia, and has been adopted by universities in the U.S., UK, and other parts of the world.

Instructional Leadership (0.42)—Overall, in Hattie’s research he found that leadership has a 0.32 effect size (combining instructional and transformational), which just falls short of the 0.40 hinge point that has been found to represent a year’s worth of growth for a year’s input. However, when we look at just instructional leadership the effect size is 0.42.

Robinson [8] (2011) found that instructional leadership includes such important elements as establishing goals and expectations, resourcing strategically, ensuring quality teaching, leading teacher learning and development, and creating a safe and orderly school environment.

Where Robinson cites leading teacher learning and development is where the author will circle back to an earlier point about understanding the research they are touting in their school. An instructional leader is one who reads the research and has a deep understanding of it. They read current information as much as possible, like when Dweck, Gardner and Tomlinson had to all clarify their research. Instructional leaders keep abreast of those important pieces of clarification so they can best use them in discussions with their teachers.

Additionally, the author sees instructional leadership as being under the umbrella of collaborative leadership because it’s important to understand teacher learning, content, and instructional strategies to offer authentic teacher observations focusing on feedback, and faculty meetings that offer opportunities to engage in deep learning. However, leaders also have to build trust in order to properly collaborate with the individuals who work in school.

Collective Teacher Efficacy (1.57)—Collective is most likely more important now than it has been in education. Given the challenges that schools face such as increased accountability, mandates, budget cuts, students experiencing trauma, mental-health issues, and the increased focus on academic rigor, the days of teachers working in solitary confinement should be in the past. School leaders need to promote and support the development of collective efficacy among their staff, while at the same time understanding the importance of self-efficacy when bringing in new initiatives, especially those initiatives that involve Visible Learning.

Bandura [9] (1997) found that, self-efficacy “Refers to beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments.” It comes down to a person’s confidence in given situations. To further support Bandura’s research, Tschannen-Moran and Gareis [10] (2004) found that, “Self-efficacy beliefs are context-specific, however, people do not feel equally efficacious for all situations.” What this means from a system’s perspective is that school leaders will bring in initiatives that they feel will help enhance learning or increase achievement among students, but not every teacher who is on the receiving end of the initiative will feel efficacious enough to follow through on implementing it in the classroom.

Where Visible Learning is concerned, there will be many educators in a school who may not feel efficacious when it comes to any one of the influences on learning that Hattie studied. In the end, it is important for leaders to build collective efficacy. Tschannen-Moran & Barr [11] (2004) define collective teacher efficacy as, “the collective self-perception that teachers in a given school make an educational difference to their students over and above the educational impact of their homes and communities.”

Collective efficacy is vitally important to the improvement process because the bringing together of diverse-minded individuals and setting up opportunities for them to learn. At the same time protocols for them to follow through the collaborative process will not only help strengthen the collective efficacy among the group, but it will also increase likelihood that the level of self-efficacy among each individual will be elevated as well.

Professional Learning and Development (0.51)—In many countries, due to mandates and accountability, professional learning and development is compliance based because districts and divisions of education want to build fidelity in the practices of their teachers. Hargreaves and Fullan [12] (2017) found that in order to build authentic learning experiences for the adults in schools, leaders must offer both professional learning and professional development opportunities.

Hargreaves and Fullan (2017) found that, “Professional Learning is often like student learning—something that is deliberately structured and increasingly accepted because it can (to some) more obviously be linked to measurable outcomes.” Additionally, they found that, “Professional Development involves many aspects of learning but may also involve developing mindfulness, team building and team development.”

Both professional learning and development are necessary to the improvement process, and it takes collaborative leadership to help foster these two types of development. When thinking of initiatives around Visible Learning, teachers not only need to understand what outcomes are necessary when implementing any of Hattie’s initiatives in the classroom, they also have to have a level of mindfulness around why these are necessary. It is one of the major reasons why Hattie and Zierer [13] wrote about the 10 mind frames of learning and implementing Visible Learning in the classroom.

Feedback (0.75)—When it comes to teacher instruction and student learning, Hattie developed 3 levels of feedback teachers can use with students, which are task, process, and self-regulation. Each level of feedback is used at specific times in the learning process. For example, task level feedback is provided to students when they are experiencing brand new learning. Process level feedback is provided when students have a deeper proficiency around a topic.

However, collaborative leadership is more than just a focus on student level feedback. It is vitally important for leaders to be able to offer deep feedback to teachers. In the work of Stone and Heen [14] (2014), they found that we all experience feedback triggers when receiving feedback from others. Collaborative leaders understand that at any given time those on the receiving end of feedback may get upset with the feedback they are receiving. Stone and Heen found that there are three feedback triggers. One trigger is the truth trigger, which is initiated when the receiver of feedback finds that feedback to be unhelpful or untrue.

The researchers also found that there is a relationship trigger, where the receiver of the feedback gets upset because they do not believe in the credibility of the person giving them feedback, even if the feedback is true. This feedback trigger is an important one to highlight because although the feedback given may be correct, the receiver doesn’t believe in the credibility of the person giving it, which allows us to circle back to the first influence of instructional leadership. If a leader does not get into classrooms, and put a strong focus on learning in the classroom, they will most likely lack credibility with their teachers, and therefore are at risk of seeing the relationship trigger go off in their teachers.

Last in the feedback triggers that Stone and Heen researched is that of the identity trigger. This trigger goes off when the feedback the giver is providing does not match up with the identity the receiver created for themselves. From a Visible Learning standpoint, this is important because many of Hattie’s influences on learning challenge the preconceived notions of teachers when it comes to what works in the classroom. It is also a great time for leaders and teachers to delve into microteaching, which Hattie’s research shows to have an effect size of 0.88.

When utilizing the microteaching influence, teachers video themselves and have to view it three times. This will allow them from viewing it at the surface level which is when they get caught up in how they sound on video or the clothes they wore that day, to the second viewing which allows them to understand what strategy they were using in the classroom and how effective it was for the learning process, and then they move on to the third viewing which allows them to think of other teaching strategies that may be more impactful in the classroom.

Assessment Capable Learners (1.44)—Hattie has changed the language around assessment capable learners a few times. It started out as student expectations and then evolved into assessment capable learners. Some schools call these students self-directed learners. The bottom line is that, no matter the ability level of our students, we need to help them understand where they are, how they got there, and where they’re going to next (Hattie) so that they know what to do when they don’t know what to do.

However, what the author also understands it important is not just the academic impact teachers and leaders have in school, but also the social-emotional impact they have in school. Students need an emotional connection to their school community, which means that schools have to offer curriculum, novels, and images on their school walls that are representative of all of the students in the school. Additional to just offering materials that are representative of the students, those materials have to offer a positive representation of all of those students, especially those students who have been previously marginalized in other school settings.

Family Engagement (0.49)—What makes collaborative leadership a bit more effective is that families are included in the dialogue around school, as opposed to just instructional leadership which seems to focus on everything within the school walls. The author feels as though they can work in partnership with the leader and school community. Too often we give parents the message we want them to have after the decision is made. Instead, we have to do a better job of bringing them in and giving them opportunities to share their voices.

5. Using the Research in an Impactful Way

During 2018, the author reviewed the evaluations sent out directly after each Collaborative Leadership workshop. Over 1000 people responded through the workshop evaluation that was created by the author’s publisher Corwin Press.

Additionally, the author began formulating questions that could be sent out to participants from collaborative leadership workshops to get a deeper understanding of the effectiveness of the collaborative leadership work than the initial workshop evaluations offered. Those surveys were created by the author and then sent out to other researchers to provide feedback on the effectiveness of the questions.

Those Collaborative Leadership workshops that provided the best feedback ranged from one-day to four-day competency-based workshops. The author created four-day Collaborative Leadership workshops that were competency-based because, following Hattie’s lead, the author found through interacting with participants and getting to a deep sense of learning, that it was important for leaders to gather evidence of impact for each of the six influences under the collaborative leadership framework.

In the workshops, participants learn about each of the six influences on learning, and then are provided with time to gather evidence and sit in groups of two or three where they go through a process of sharing their evidence and receiving feedback and deeper questions for understanding by their partners in the groups.

The author sent the questions using Survey Monkey, and the surveys were sent to groups that included instructional coaches, teacher leaders, as well as building and district leaders. The questions included in the survey were as follows:

• If you have not utilized any of the 6 influences that matter, please choose the reason that prevented you from moving forward down below.
• Which of the 6 influences of collaborative leadership have you utilized in your teacher leader/instructional coaching or building/district leadership practices?
• On a scale of 1 to 5, (1 being no change and 5 being a significant change), how has collaborative leadership helped impact your teacher leader/instructional coaching or building/district leadership practices?
• On a scale of 1 to 5, (1 being no change and 5 being a significant change) how has collaborative leadership impacted student learning?
• On a scale of 1 to 5, (1 being very different and 5 being very similar) how has the collaborative leadership work matched up with your previous knowledge of teacher leader/instructional coaching or building/district leadership?
• What evidence have you collected to determine that the influence you focused on has had an impact on student learning?
• What have been the challenges of practice the influence/s?
• Is there one influence that determined more challenging than others?
• Is there an influence that was less challenging to practice?
• What is the least effective influence and why?
• Have any teachers, students or families remarked on your change in leadership? If so, what are some commentaries you have heard?
• On a scale of 1 to 5, (1 being very little and 5 being a well prepared) how well did the collaborative leadership workshop you attended prepare you to lead in your school or district?

Participants in the survey remained anonymous. One limitation that surfaced after the surveys were sent out was that the author did not include a question asking whether the respondent took the one day, three day or four day competency-based workshop. This is an important distinction because the one-day workshop is not competency-based due to the fact that participants do not always enter into the workshop knowing the six influences. The other multi-day workshops are competency-based.

6. Results

Out of 1000 participants providing feedback on Corwin Press’ workshop evaluations provided by the author directly after the conclusion of each workshop, over 80% of the participants responded that they strongly agree that the workshop was impactful and 17% answered that it was impactful. Additionally, when asked if they will be able to use the work in their school, over 95% of participants answered likely or highly likely.

When sending out the second round of surveys through Survey Monkey to a smaller group of respondents from the first round of evaluations, the author received important feedback from respondents. 95% of the close to 100 respondents felt prepared or well prepared to go back to their schools and implement the work. However, one piece of feedback in particular that was important to the author was that of the challenges participants face when trying to implement the influences from the collaborative leadership work.

Participants cited that a lack of support from their supervisors as one of the leading reasons why they had not been able to implement any of the influences. They reported that their leaders provided them with other work that took them away from being able to implement the influences.

Other respondents stated that they had been transferred to another position, which meant that they had to learn a brand new skillset. And lastly, time was a major factor as to why some respondents could not follow through on their learning.

On the other side of the spectrum were those respondents who were able to carry out the work. They stated that they came to the workshop with a level of understanding around the collaborative leadership work, and created a goal at the beginning of each session with the author which allowed them to focus in on the current reality of their position as well as the practical suggestions offered by the author to help them meet their goal.

7. In the End

Hattie’s Visible Learning research has had a profound impact on educators and students around the world. The research has provided educators and leaders with the opportunity to have deep conversations around their practices in the classroom and school, and has inspired many to look at those practices and reflect on whether they are having the deep impact on learning they think.

The author found early on that teachers were not going to get the opportunity to do the Visible Learning work to the fullest extent if the leaders within those schools didn’t foster an environment that focuses on learning. In order to do that those leaders needed to reflect on their own practices. More than that, they needed to develop their own tendencies as collaborative leaders.

The six influences written about in this paper are those influences that are necessary to focus on learning in schools. Leaders have an opportunity to find our best leadership skills within and foster growth among all of the instructional coaches, teachers, students and staff that work in school.

The benefit learned early on when delivering workshops is the influences are important for all school staff to learn and not just school leaders. However, one of the lessons learned through reviewing workshop evaluations and asking for deeper feedback from those engaged in the workshops is that leaders have to find ways to prioritize their workload and find time to work on one of the influences at a time. When trying to implement all of the influences at once it is easy to understand how leaders may become overwhelmed.

This level of learning from the evaluations and feedback around collaborative leadership is not unlike the lessons we can learn from the implementation of the Visible Learning work as well. Hattie has always said it’s not about the top ten influences. What all educators trying to implement this work need to do is understand their current reality, and choose the influence that will be most beneficial. After doing that, they will be able to move on to another influence and have a deeper impact on the learning in their school.