Physics Teachers’ Perceptions about Their Judgments within Differentiated Learning Environments: A Case for the Implementation of Technology

Abstract

There is a national shift in the new Indonesian curriculum towards employing differentiated learning approaches in addressing the diversity of students’ needs and abilities. A teachers’ judgment evidently corresponds to the duty required of physics teachers to monitor their students at an individual level. Within the context of Indonesian physics education research (PER), empirical study addressing this subject is still lacking. To fill this gap, eight Indonesian physics teachers’ experiences and limitations about their judgments within differentiated learning environments have been investigated through phenomenological study. Physics teachers were voluntarily recruited after they declared their endorsement and personal consent to participate in the study. Our participants were distributed over several teaching experiences, geographic regions, and information and communication technology (ICT) experiences. The latter experience might be taken into account since, through this study, upcoming developmental research will be projected on engaging recent technological approaches to address the limitations of teachers’ judgments. Online semi-structured interviews (~50 min) were conducted by the first author to all physics teachers involved. Other authors contributed in reviewing the interview protocol and training the first author’s pilot interview. The model of teachers’ thought and action, from Clark and Peterson, was implemented to uncover physics teachers’ experiences and limitations in making judgments within a diverse group of students. The findings revealed that physics teachers have conceded that they should dynamically adapt the learning process in order to meet heterogeneous students’ performances. Personal observation has mainly informed teachers in identifying students’ differences. After students have been identified, the teachers creatively designed learning transformations to accommodate the wide spectrum of students’ abilities. Nevertheless, several limitations were discovered as being encountered by physics teachers, particularly in terms of judgments’ equity and accuracy, as well as the teachers’ workloads. To overcome this, the teachers indicated various and supportive attitudes about technological implementations that would assist their judgments. Ideas for technological development were provided to address identified obstacles during the teachers’ judgments.

1. Introduction

One pedagogical competence that should be performed by teachers is to measure the extent to which students have achieved their learning outcomes [1,2,3,4]. Teachers’ assessments associate with their judgments of their students [5]. A teacher’s judgment is the main source of information on how a student’s learning has been progressing. It will guide teachers’ decisions in preparing learning treatments such as instructional strategies, learning sequences, content complexities, and the difficulty levels of the assessments that they will administer [6]. Teachers’ judgments are determined by early identifying differences in students’ abilities in the classroom. Students who are judged as more capable will be assigned more advanced learning than lower-performing students [7,8,9]. Therefore, teachers’ judgments must be further acknowledged, particularly within heterogeneous students’ abilities and backgrounds.

Facilitating diverse students’ characteristics is one aim that is currently being resonated within the physics education research (PER) community [10]. This study hopes to celebrate diversity in physics during the creation of more inclusive physics learning strategies to motivate each individual student. Our PER scholars have established several learning reforms to promote this intention [11]. Recent study by Dunleavy, et al. [12] has developed multimedia resources in the form of one-minute videos and short text summaries that have made students able to organize their learning pathways according to their own mental frameworks. Moreover, the well-known scaffolding approach has been widely implemented within the PER community in several parts of the world [13,14,15]. The scaffolding approach assists students’ learning, through a self-determined pace, to plan, to monitor, and to measure the extent to which they have been progressing in their physics learning. Diversity in physics has been promoted through other studies, including the adaptive tutoring system (ATS) [16,17], the computerized adaptive test (CAT) [18], and employing machine learning (ML) algorithms [19,20,21].

Recently, Indonesian PER scholars have also been encouraged, through a national call, to promote differentiated learning that is recommended by the educational policy of the new Indonesian curriculum [22]. This approach is considered capable of nurturing different students’ backgrounds. Differentiated learning will lead teachers to make judgments on recent individual students’ performances [23,24]. Teachers’ judgments have vital roles in the implementation of differentiated learning. This approach requires information about the diversity of both students’ prior backgrounds and their current learning performances that should be observed through teachers’ judgments. Teachers’ judgments associate with the extent to which teachers are able to monitor their students’ learning progressions at an individual level [5]. Students’ developments at the individual level is a key feature of the implementation of differentiated learning in physics classrooms.

Within the Indonesian educational context, diversity in physics can be represented by zoning-based students’ admission policies in basic and secondary education [25]. Students are only permitted to enroll for the school within the closest proximity to their home. This selection scheme has made Indonesian physics learning more heterogeneous than the former circumstance: the previous admission policy had often caused brighter students to register for their most preferred school in a certain Indonesian region. When the brighter students were gathered in the most preferred schools, homogeneous patterns emerged in each school, from the most preferred to the least preferred schools. Eventually, clear distinctions formed between the students’ backgrounds. These distinctions implicitly influenced the former physics classes. Zoning-based policies exist to ensure equality between schools, and they must accommodate all of the students’ prior backgrounds. Accordingly, physics teachers might adjust their physics learning to facilitate ability spectra from high- to low-performing students in each school. Differentiated instruction is recommended by the Indonesian government to address these emergent diversities. Nevertheless, to the best of our knowledge, limited resources exist to help in understanding how Indonesian physics teachers have approached differentiated learning in their physics classrooms. Moreover, corresponding teachers’ judgments are largely unexplored within an Indonesian PER context. Therefore, this study would contribute to revealing physics teachers’ experiences in undertaking judgments and approaching differentiated instruction in the context of Indonesian education.

Judgment begins with monitoring students’ progressions throughout the learning process. This task requires teachers to continuously observe their students. In practice, teachers’ observations of students typically reveal limitations [26]. To comprehend the aforementioned issue, limitations experienced by teachers should be contemplated when undertaking judgments through differentiated instruction. It is imperative that this knowledge is studied, since there are developing digital technologies such as machine learning (ML), educational data mining (EDM), and learning analytics (LA) [27,28] as potential teachers’ support systems that can assist them in overcoming obstacles using data-driven decision systems. EDM and LA are the emergent educational fields most developed by computer scientists to explore the large volume of potential data within learning environments [28,29,30]. Those fields employ the robust state of ML, which can predict the future conditions of learning process. As mentioned above, teachers’ judgments associate with teachers’ activities to continuously monitor and form expectations of the students’ learning. It associates with the extent to which our physics teachers are aware of potential educational data that should be understood to support their judgments within differentiated learning. Worthy of reflection and remembering is the recent online learning transformation throughout the COVID-19 pandemic [31,32,33,34]. At the same time, opportunities may be observed from these past experiences that online learning managed through the learning management system (LMS), which was a very large channel to collect educational data during the learning processes. Data are important resources for teachers’ considerations in forming judgments [35,36,37,38,39,40,41,42]. Previous methods of monitoring students’ learning encounter several drawbacks, and ML-enhanced methods of teachers’ judgment might be offered using the opportunity to employ these data.

This article is categorized as preliminary research, since authors initiated the effort to investigate physics teachers’ experiences and limitations in making judgments as a basis for future developmental study. Phenomenological study has been used to investigate the extent of physics teachers’ experiences and limitations in making judgments over time. To achieve the goal of investigating physics teachers’ experiences and limitations, this study is guided by the following research questions:

RQ1.

What are the physics teachers’ experiences in judging the diversities of students’ performances?

RQ2.

To what extent are limitations that teachers encounter involved in forming judgments on individual students?

The findings in this study contribute to an argument in favour of future technology developments for assisting teachers’ judgments. Teachers’ experiences discussed in this article should inform the best practice of teachers’ judgments for the general audience in the educational field. This study also addresses the context of the new Indonesian curriculum that is being recommended, as well as recent predictive technology that is potentially considered for use in forming teachers’ judgments. We anticipate that our findings may correspond to the new Indonesian curriculum and the corresponding challenges that arise in addressing the students’ diversities. Finally, we provide future direction to assist in developing the recommended technology, discussed throughout the paper, as a support system to help teachers overcome limitations in making judgments.

2. Methods

2.1. Study Design

Teachers’ judgments are pedagogical routines that should be managed by our study’s physics teachers. We believe that they possess the professional competence to maintain monitoring activities during students’ learning processes. Teachers’ judgments involve monitoring tasks at the individual level [43,44]. Qualitative study is considered as the appropriate research design to explore physics teachers’ experiences and limitations in making judgments about their students. The experiences and limitations of Indonesian physics teachers in approaching differentiated instruction with their ways of forming judgments are extracted as our collective basis of technological development which will address discovered limitations [45]. Our focus to these aims motivates us to use a phenomenological study lens in this research. In particular, phenomenology is a qualitative approach that aims to find the essence of one’s experience of certain phenomena [46]. The findings revealed in this paper are intended to open an exploration about how physics teachers experience their judgments within differentiated learning environments rather than to make generalizable information to any sort of population available as grounded by a positivist paradigm [45,47,48].

2.2. Participants

Eight secondary school physics teachers (three males and five females) were interviewed. They lived in various different parts of the Indonesian archipelago. The majority of participants were recruited from the Javanese people (n = 5), since this island hosts the percentage of the Indonesian population (40%) based on the recent census [49]. Hence, it is reasonable to specify a larger proportion here than in the other regions. Seven teachers were junior physics teachers who possess fewer than 10 years of teaching experience. There was one teacher who possesses more than 10 years of teaching experience, as well as several experiences in developing computer-enhanced learning technology. The amount of ICT experience possessed by participants associated with our study helps in discovering the ways in which teachers overcome their limitations, one of which is through current digital technology. A recent study by Guillén-Gámez et al. [50] reported that ICT competence could be influenced by pedagogical experiences that directly affect how our physics teachers approach their perceptions.

Participants for this study were recruited voluntarily by the first author through a private platform. Research networking was used as the main channel in tracing possible participants willing to join this study. Participants who have the opportunity to be explored in this study must meet certain criteria, including working a as tenure physics teacher for at least two years, and possessing insights about teachers’ judgments in physics learning. After the candidates were listed, we privately offered the teachers acceptance to participate in the study. If they declared that they were compliant to join as participant, the researcher then scheduled an online semi-structured interview (~50 min) which was conducted and recorded through the virtual meeting platform.

The number of participants can be viewed as an arguable position through the positivist paradigm. Nevertheless, we argued that it might be more relevant, or more fair, to consult several references from the qualitative paradigm to understand how constructivist views shape our interpretivist thoughts. As explained above, we attempt to understand rather than to generalize the essence of physics teachers’ experiences during investigations of their judgments and limitations. To support our argument, we may consult theoretical reviews from the literature [51,52,53], as well as the best practices of phenomenological study from previous works published in high-impact journals within the physics education research (PER) community [54,55,56,57] or within the educational sciences in general.

Moreover, the additional context of how many students each Indonesian physics teacher has in their classroom and the teaching methodology used by each of them would be beneficial to more knowledgeably interpret their experiences. As mentioned above, our participants possess tenured status, hence they are ruled by national standard. Since the further descriptions on this subject become quite technical, interested readers are encouraged to consult Hapsari’s policy review [58] for further details.

2.3. Data Collection

The first author was the main interviewer in the data collection process. To maintain consistency in understanding each teacher’s experience, one interview was scheduled per day. As soon as the interview had finished, we directly transcribed the recent conversation. This was performed to form our initial understanding and record of what had been discussed. It also constructed our understanding and consideration about physics teachers’ experiences that were being investigated. With an early understanding of what had just been discussed, we could more easily evaluate the interview result. If there are accidentally forgotten questions or missing topics that had not been explored, but are imperative to answer the research questions, a fresh record of the conversation would be of value to our research. For further additional inquiries, we requested the participants’ permission to seek answers to follow-up questions through an instant messaging application.

The interview protocol was designed by the first author and reviewed by the other authors. The second author is a Physics Education professor, focusing particularly on assessment, possessing more than 20 years of teaching experience. The last author is an associate professor in the field of Educational Measurement. They were involved in evaluating the list of questions in the interview protocol draft and providing advice to the first author’s pilot interview. We realized that, in its development, the interview protocol was not consistent. It was always adapted continually, dealing with emergent participants’ experiences in making judgments of their students. Prior to the real interview, the first author conducted a pilot interview with the second author to evaluate any technical issues possible to occur in the interview. The results of this training were then observed by the second author to evaluate the interview technique that had been performed by the first author.

In this study, the teachers’ thought and action model, created by Clark and Peterson [7], was assumed as our theoretical framework to reveal physics teachers’ experiences and limitations in making judgments within differentiated learning environments. This theoretical lens was proposed to help explain the teacher’s role as decision maker during the learning process. Therefore, in addition to reporting the experience, we endeavored to inquire further about teachers’ efforts to overcome the challenges presented to them. Teachers’ judgments should be in accordance with their decision making in the classroom [59]. There are two main domains in this theoretical framework: teachers’ thoughts and teachers’ actions in learning environments. These two domains appear to correspond to both constraints and opportunities in classroom settings (the diversity of students (RQ1) or limitations during the formation of judgments (RQ2)). The physics teachers considered always encounter constraints and opportunities in the designing and conducting of physics learning for their students. Thus, they should contemplate and analyze decision making or judgment formation and take actions to overcome their constraints and meet their opportunities.

To address RQ1, teachers’ experiences were investigated, specifically how they perceive the diverse characteristics of students and how they accommodate them.

Table 1. Interview protocol designed from the model of teachers’ thought and action.

No.	Theoretical Domain	Question Example
1	Constraints and opportunities	A zonation-based enrollment rule is being implemented within Indonesian education. In your opinion, has this caused any changes towards your physics instruction and learning?
2	Teachers’ thought	Have you ever heard of differentiated learning? So far, what are your opinions regarding this matter?
3	Teachers’ actions	The heterogeneity of students in the classroom is a form of diversity that exists in the learning process. How do you manage students’ diversities during the learning process?

presents a sample interview protocol designed through the theoretical framework. We commenced the interview by investigating teachers’ observations of heterogeneous student characteristics due to zone-based enrollment policies within Indonesian education. Thereafter, the teachers confirmed and shared their knowledge about differentiated learning and their undertaken actions to adapt to physics instruction with the discovered students’ diversities.

Henceforth, our RQ2 was purposed to deepen our foregoing investigation which has just been discovered by RQ1. In working towards this aim, the next questions investigated physics teachers’ experiences with several limitations during their “actions” mentioned in answering RQ1 (particularly on point 3 in Table 1), and how they transformed physics learning toward their respective diverse students’ abilities. Therefore, the same theoretical framework should also be appropriate to use in exploring RQ2. As was accomplished prior in RQ1, the teachers’ actions undertaken to address their obstacles were also investigated. Particularly, we considered how teachers could access the newest digital technology that might be employed, according to the teachers’ knowledge and experience, when making judgments regarding their students. Therefore, during the interview regarding RQ2, we focused on the recent employment of machine learning technology in addressing obstacles in teachers’ judgments. The technological knowledge extracted from this study would anticipate use as an introduction for upcoming developmental studies, which would design teachers’ support systems using predictive learning technology.

2.4. Data Analysis

In each interview, recorded conversation data were consistently maintained as audio files. Indonesian transcription engines and the corresponding regional languages, however, are still currently limited. In fact, our participants speak two languages at minimum, both Indonesian and their respective ethnical languages. Possible shortcomings could be introduced if we rely solely on a transcription machine. Therefore, we decided in favor of transcribing them manually. Our transcripts were maintained in text format (.txt). Content analysis then was conducted through an RQDA (R-based Qualitative Data Analysis) package [60] within R language [61] to assist in the coding management of each transcript’s segment. This open-source library is no longer available in the recent version of R language 4.1.3 (One Push Up) [61]. Fortunately, we could still employ the portable version of RQDA in which the suitable R version for RQDA was embedded. We were able to download this resource via developer’s GitHub [62].

The first stage of qualitative data analysis was identifying the attributes present among our participants. We classified our participants within several criteria: teaching experience, geographic region, and ICT experience. ICT experience was considered because it associates with the findings and implications that we put forward about technologies that teachers might consult in forming judgments. Second, open coding was analyzed by iteratively reading and interpreting every segment of the participant’s experiences from the transcripts. Third, a collection of coding that had been marked was then interpreted, in depth and repeatedly, to sort it into categories regarding physics teachers’ experience and limitations in making judgments. In this step, we yielded a codebook as a researchers’ basis to assist in answering our two research questions. Through several focused group discussions with two authors, the codebook might be improved upon, based on suggestions given by other authors. In addition to the coding agreement between the authors, we also conducted member checking with the participants. Transcripts were returned to the participants, and they were permitted to verify and to comment, if needed, on how accurately the transcript reflected their thoughts and experiences. Analysis results and manuscript drafts of publication were also consulted among the participants. These proved the concept of trustworthiness as our internal or external validity within qualitative inquiry [48,63]. Thus, it may be anticipated that our findings have correctly represented the essence of the interviewed teachers’ experiences and limitations during the formation of judgments.

Furthermore, the model of teachers’ thought and action has extracted physics teacher’s experiences in making judgments and their limitations in supporting students’ needs. In the next section, our answers to the proposed research questions are presented within three categories based on this theoretical framework. Then, each category is elaborated upon through several subcategories of the participants’ conversation samples. In the first domain of constraints and opportunities, we explored physics teachers’ experiences, with particular focus on how they identified heterogeneous students’ abilities (RQ1) and what limitations that they experienced in observing diversity (RQ2). In the second domain of the teachers’ thoughts, we investigated how teachers perceived the differentiated instruction approach that is recommended by the new Indonesian curriculum (RQ1) and their existing knowledge about technologically enhanced efforts to overcome their limitations in making judgments (RQ2). In the second stage, the main investigator persuaded the physics teachers to describe their thought processes regarding their roles as evaluators of individual students in order to make physics physics more tailored towards students’ needs. In the last domain, they explained the action processes in the monitoring of students that have been carried out as the teachers’ efforts to adapt to diversity during the physics learning process (RQ1) and how they overcame shortcomings during judgments in the physics learning process (RQ2). The organization of the categories presented below represents the teacher thought and action model of Clark and Peterson [7] and how physics teachers responded according to their experiences. Evidence from each category is introduced by the inclusion of a corresponding conversation segment. The participants’ names (Desi, Fika, Fitri, Hendro, Narti, Tinah, Yoga, and Yono) have been anonymized arbitrarily to maintain and respect participants’ privacy as a key part of our ethical clearance in this study.

3. Results

3.1. Physics Teachers’ Experiences in Judgments within Differentiated Instruction (RQ1)

In answering RQ1, phenomenological study has extracted three main categories driven by the model of teachers’ thought and action [7]. The physics teachers were aware of their constraints present when adapting physics learning to meet students’ diversity. The teachers’ judgments were considered mandatory in the process of adjusting their teaching environments to meet students’ abilities observed by the teachers’ judgments through several channels, such as personal observation, task observation, or peer observation. Inspired by their observations, teachers designed several learning reforms. Our results confirmed that the teachers typically already possessed sufficient understandings of the concept of differentiated instruction. In addition, they realized that there was potential data to mine that would assist and contribute to their judgments. The teachers’ institutions have also provided support that the teachers acknowledge. The institutions have directed teachers to deal with the physics students’ diversities.

3.1.1. Category 1: Constraints and Opportunities

In this category, the physics teachers conceded that students’ differences are tangible learning constraints, yet challenges that might be confronted and resolved at the same time. As declared by the results of the interviews that would be discussed in this category, the physics teachers had awareness of students’ diversities which might encourage teachers to adapt their learning throughout students’ changing needs. The teachers also confirmed that schools have taken actions to advance their professional development processes in approaching differentiated instruction. The physics teachers emphasized that judgment is their main task in learning management. Therefore, they conceded that this task is important in order to support individual students during the learning process and students’ assessments.

Students’ Differences

The physics teachers underlined that there is a gradation of students’ abilities during physics lessons. Hendro notes that physics learning might be able to adapt to students’ needs across whole spectrum of abilities. Hendro admitted: “Teaching is unique. Compared to the old days when I was student, my learning was quicker. However, it evolves now. I observe that I tend to pay more attention to low-performing students than high-performing students. I sometimes even wonder whether these students feel cared for or not. In their heart, they may be feeling bored because my learning has no significant progress”. Ideally, teachers’ judgments might be more effective when they could reach a whole breadth of students’ abilities.

Data Availability

The teachers assumed that their judgments could be decided through the assessments that they administered routinely. Information availabilities that have been presented through their assessment processes should enrich teachers’ attentions about students’ developments at the individual level. As Narti suggested: “My second suggestion might be [that] our assessment model should be continuously designed. It is because our learning should ascertain student’s development through continuous assessment. Subsequently, assessments should be aimed for facilitating us to judge students one by one in large class size[s].”

Institutional Support

Teachers’ knowledge about how to form judgments was not only obtained after they pursued academic degrees and teachers’ training. The physics teachers confirmed that they received pedagogical support for knowledge development through school programs or their teachers’ communities. This program should have previously been discussed with the local offices of education as well as regional teachers’ forums. Fitri stated: “What’s that… it’s reflective teacher program, but not all of them participate. Moreover, zoning policy for the first time was implemented during [the] pandemic outbreak. Teachers’ training in accordance with the government’s destiny was hampered. I heard that other teachers also had participated in program[s] about learning modules or worksheets development for students’ activities.”

3.1.2. Category 2: Physics Teachers’ Thoughts

In this category, the researchers encouraged physics teachers to argue about differentiated learning for physics education; the teachers discussed how teachers’ judgments could support students, why teachers should consider this approach, and implications for physics learning and assessment practice. The results of the interviews highlighted that most of our teachers had been familiar with this learning approach, even though they did not claim it as their teaching practices. The monitoring of students was recognized by the physics teachers as an important task in facilitating diverse students’ abilities. Differentiated instruction requires teachers’ competence in discovering and judging current students’ progressions. The results of judgement are also considered as having an impact on their assessment practices.

Knowledge about Monitoring and Prediction Tasks

Fitri declared: “For students’ ability, it implies that, without actually following the assessment process, we actually have been intuitively informed about student’s prior background[s]. Since we began to teach, we actually already analyze[d] our student’s background[s], prior abilities, and their limitations.” Fitri’s statement implies that teachers have recognized their roles in making judgments during the learning progress. Judgments might be supported by information obtained through the monitoring process; hence, teachers were able to judge predictions. Prior ability is one factor that teachers noticed from their students’ backgrounds within physics classrooms. This factor could conceivably impact the teachers’ judgments towards the students. Teachers typically have recorded expectations about their students’ retentions during the physics learning.

The Importance of Teachers’ Judgments

The physics teachers perceived that their judgments represent their pedagogical task as decision makers. Yono narrated: “In my opinion, our judgment is highly impactful, actually. From this judgment, we then actually determine what treatment will be approached afterwards. If, for example, there was [a] disoriented pupil during physics learning. From this thing, it informed us that physics should be discussed more simply.” Physics learning management might be tailored to better suit the students’ characteristics as they are discovered. The proper decision obviously led to a robust positive effect on the students. Teachers’ judgments then would produce learning effects on students after participating in physics learning.

Teachers’ Judgments for Assessment

In addition to influencing the learning treatment that is approached by physics teachers, the teachers in this study argued that judgment was essential for the assessments’ consideration. Within the context of the new Indonesian curriculum, Hendro discussed that “…the new curriculum design recommends the differentiated assessment. It represents that, for the brighter student, the passing grade should be higher, for example, ninety-five points, arbitrarily. In case of [a] lower-performing student, sixty points then should be considered as their peak. Therefore, assessment is more tensile. As we know in the current context of [the] assessment system, successful students might have to obtain [a] certain grade point, typically, for instance seventy-five. I have heard… [this] from several resources”. Through the institutional support, as explained in the aforementioned category, physics teachers have been informed that the new assessment paradigm would be adjusted to adapt responsively based on students’ abilities which were observed by formed judgments. From Hendro’s experience, his judgments to the students could influence how teachers create expectations towards their assessment standards.

3.1.3. Category 3: Physics Teachers’ Actions

In this category, the physics teachers discoursed about their actions’ experiences in providing support for their students’ diversities which have been mainly discussed above, both by designing responsive learning transformations and by the implementation of three monitoring channels. Various learning transformations have been creatively designed by our physics teachers, particularly in fostering cooperative group discussion. Their efforts might be determined through exhaustive teachers’ observations that are collected via three main channels, namely: personal observations during the class, task observations that have been assigned by the teacher, or peer observations through informal discussions with other quantitative subjects’ teachers.

Learning Transformations

Heterogeneous students’ abilities are spread across an ability spectrum that urges teachers to provide the correct learning assistance for their students. Cooperative group discussion was deemed the appropriate avenue through which to address this subject. Students with higher abilities could serve as tutors in delivering physics learning to low-performing students. Hence, physics education has attempted to meet both high and low students’ abilities alike (differentiated learning). As recounted by Fika: “For the management of students’ diversity, I sometimes approach students’ groups that [were] designed to adapt learning throughout students’ diversity. Although it might be imperfectly implemented, my efforts have been made so that this diversity can be accommodated by the formation of students’ groups.”

Personal Observation

To supervise the learner developments, physics teachers typically conducted personal observations through various channels. Tinah confirmed: “If you just observe the students, actually, it’s not difficult for me. Fortunately, I’m assigned as a class advisor in my school and I have some occasions that enable me to privately talk to students. I then take note about their difficulty, like that. It can be explored more deeply. However, this case is subjectively in my experience.” Observation is a mandatory task that is routinely undertaken by physics teachers. Tinah’s experience articulates that teachers feel no difficulties in making personal observations if they are supplemented with additional information from outside the classroom. The information obtained from their observation serves as the teachers’ basis for judging students’ diversities and designing the appropriate learning adaptations.

Task Observation

Students’ behaviors could typically be studied through their assignments given by the teachers. Teachers’ expectations to students could be influenced through this channel. They could argue that assignments are part of the assessment point towards the final exam. Submissions of students’ assignments are typically related to their performance throughout the rest of the semester. Fitri described: “Sometimes I solely considered assignment submittance from students. For example, I gave four assignments. I could clearly make judgments when there were students who only submitted as many as three assignments. Even my warning was neglected and they still left the learning. From this, my prediction conclude that he/she was inadequate on physics. You can already predict what the students will be like.”

Peer Observation

The social competence of teachers could be reflected via this method of observation. Collaboration is needed to create effective physics learning [64]. Not only in attempting personal or task observations do teachers rely on one another, but teachers also often share their observations with other teachers, either within the same subject or different subjects, through peer discussions outside the classroom. Desi told that: “Usually it was not just on physics. For instance, on mathematics or other quantitative lessons, our students’ ability is lacking or still tends to be low.” Information obtained from peer discussion could serve as reinforcing information for the teachers’ judgments. Therefore, other supporting evidences have augmented teachers’ understandings about characteristics of their students.

3.2. The Limitations of Physics Teachers’ Judgments within Differentiated Instruction (RQ2)

In studying this second research question, we enriched our knowledge from RQ1 by more deeply investigating physics teachers’ limitations when making judgments within differentiated learning environments. Among the three components in the model of teachers’ thought and action [7], the physics teachers confirmed their most common experiences with constraints when physics teachers dive into students’ learning, particularly how they should identify students’ needs at the individual level. Furthermore, we then oriented the discussion towards discovering how teachers’ knowledge presents to overcome their constraints during the teachers’ judgments. Particularly, we explored their understandings about the existence of recent developments in artificial intelligence such as machine learning (ML) and educational data mining (EDM) that would be cited as implications of this article for upcoming developmental research. Experience in this category might also associate with the data availability subcategory in RQ1 above, since data is the knowledge base in building the ML algorithm. As mentioned earlier, the context of teachers’ ICT experiences clearly influences how teachers understood what technology could be approached to overcome their limitations. In order to more accurately capture our teachers’ experiences, we also investigated teachers’ practices that have been cited as their actions in strengthening the accuracies of their judgments in physics learning.

3.2.1. Category 1: Constraints and Opportunities

We maintained the same theoretical framework in understanding teachers’ limitations when making judgments about their students within differentiated learning environments. The teachers greatly confirmed their constraints during their observations of their students’ characteristics at the individual level. Our findings revealed that, to date, physics teachers still tend to employ informal data collection when making judgments even when they have adequate pedagogical knowledge discussed above. They realized the weaknesses in terms of accuracy and equity of teachers’ judgment using this method. The limitation in the form of teachers’ workloads was also still presented through our discovery. Observing students at the individual level and maintaining teaching schedules and other additional duties were relatively difficult to manage simultaneously. In addition, curriculum shifts led to future challenges for their educational routines in teachers’ judgments. This would also implicitly impact the available data that should be considered through the learning reforms. Therefore, a readaptation of teachers’ judgments should be attempted.

Informal Data

Hendro admitted that his observations have never been carried out based on sophisticated data analysis. He confirmed: “For your information, I have never really done that way. We’re just marking students. For example, there are only four most diligent students in tenth grade. There are ten middle students. Then, a total of fifteen students with lower abilities, just it is. And then, those students should follow my remediation program. For the advanced calculation, it has never been implemented. Just mark students. Student A, Student B, Student C, as you know.” Hendro’s experience illuminated that students’ judgments were made informally by counting students’ characteristics within the range of the cognitive ability scale.

Limited Data Resources

Physics teachers required a support system to assist them in observing students’ learning progresses at the individual level. In the context of Indonesian education, the e-report system that has just been implemented proved to be inadequate in providing information about students individually. The previous educational report system was recognized as having more advantages which helped monitor students at the individual level. As Tinah stated: “If it is like the old assessment report (I remember when I was in elementary school). If I am not wrong, the elementary school report is handwritten, right? And the description of each student was handwritten by the teacher. So, we can find out students’ developments one by one. However, for the e-report nowadays, it cannot be one by one, it cannot be.”

Curriculum Shift

The physics teachers suggested that we might pay attention to the curriculum change that is currently being disseminated prior to forming teachers’ judgments. Yoga said: “Yes, considering about data availability, let us follow (the curriculum) first. Because it is actually still continuously updating. You should not decide that it would A-B-C-D. You should try to learn what the government wants first and where will their policy go next”. Curriculum shifts could reorganize some of the previously available data with different assessment systems. Yoga recommended following the dissemination directed by the national government.

Judgment Accuracy

The physics teachers argued that observational assessment, or teachers’ judgment, is solely the preliminary of their assessment processes. Yono explained: “So, I see that attitudinal observation is only the first step of my criteria actually. For example, we implement self-assessment or peer assessment between students, then we still have to examine it by considering their cognitive abilities. Maybe from the assignment or the final exam.” Teachers, however, doubted their judgment accuracies if that was only derived from their personal observations of students. The inferences of their personal observations should be verified through other measures, such as self-assessments or peer assessments based on students’ participation. Yudi also believed that cognitive tests should be calculated.

Judgment Equity

This subcategory associates with Hendro’s statement in which he wondered what his students perceived about their physics learning in which he, as a teacher, currently tends to accommodate low-performing students (see Category 1 of RQ1 above). This constraint implied that physics teachers were unable to warrant whether their personal observations of students’ differences have been ideally adequate for all students. There was an impression conveyed by Fika that brighter students frequently complained to her since low-performing students were lacking involvement. She was afraid of the event that their learning will be out of control, since the high performers were learning more quickly than the slower students. Fika confirmed: “My experience was the high-performing students complained [to] me that their low-performing friends were not actively involved in class. Hence, the higher students’ learning preceded [that of] their slower friends.”

Teachers’ Workload

Ultimately, the teachers described their serious concerns about their high workload as a stumbling block preventing them from ideally approaching the concept of differentiated learning. Desi suggested: “But there are a lot of students, right? So, my coverage might be in the limited range that would be judged immediately. I imagine best practice might be in several meetings since it would be hard to monitor individual students at every single day and, you know, we have numerous students that should be managed.” The high teachers’ workload was highlighted in this subcategory. Teachers have many responsibilities beside making judgments. The number of classes and students that might be observed by each teacher could threaten judgment accuracy and equity. This finding should imply that teachers require a support system in making judgments.

3.2.2. Category 2: Physics Teachers’ Thoughts

In this category, novel information might be revealed from the results of this phenomenological study. To the best of our knowledge, there is no similar research on recent technology development, particularly machine learning (ML), educational data mining (EDM), or learning analytics (LA), that considers qualitative investigation prior to the developmental agenda. Suggested technology in the discussion below was addressed to overcome a forecited teacher’s constraints when making judgments. We have explored physics teachers’ knowledge to approach possible ways to design technology to address their challenges. Therefore, ICT experience and teaching experience, as previously explained, would greatly influence how teachers perceived those experiences. Yoga is one of our participants who has 10 more years of teaching experience than the others. He was frequently involved in digital activity. He has developed numerous products and industries, such as three-dimensional animation, Android-based software, and internet of things (IoT) platform on physics laboratory. From his experience, he possessed a quite cultivated understanding of how technology could enhance physics learning and instruction. Recently, he has been developing a computer-based test to administer online assessment for thousands of high school students at 11 public high schools during the COVID-19 constraint. Developments of artificial intelligence (AI) technology such as machine learning (ML), educational data mining (EDM), and learning analytics (LA) have also been explored during his spare time.

Technological Insights

Among our eight physics teachers, few of them have discovered machine learning and routinely learn it independently, excluding Yoga. The presence of ICT experience might be considered to more carefully interpret their approach to this technology in making judgments. In the interview, Fika had just discovered that machine learning could be approached for educational purposes. Fika narrated: “I am literally slightly familiar with that. But if it can be implemented for learning, I have no experience. Instead, this may be the first time I have heard that artificial intelligence could be applied for teaching and learning decision[s].”

Technological Attitudes

Supportive attitudes have been exhibited by teachers towards the recommendation of technological resources. However, some notes were proposed by Hendro that should be taken into account for our machine learning development to better assist teachers’ judgments. He argued that the construct validity of the learning outcomes measurement tools should be examined among the schools and teachers. Each school has different backgrounds that implicitly affect teachers’ spaces. Hendro said: “The data is based on the students’ performance. A student obtains [a] certain score, for example eighty point[s] at a topic. And then, he/she gets eighty-five at other topic, then machine learning predicts students’ performance in the end of semester. For the information, we usually administer unstandardized items, you know. It was different from college scholars who provide research-based assessments that are valid, reliable, or whatever. Hence, their measures might represent students’ ability. Moreover, I admit that most of the teachers occasionally design physics problems about what comes into our heads and we assign it to our students. Several books typically were also adapted then we engage university enrollment test[s] also. We typically set this kind of psychometric analysis aside”. Hendro’s notes might be utilized within teachers’ communities in fostering cooperative efforts in these issues.

Technological Resources

Yoga is merely one talented teacher with certain ICT experiences. Yoga gained sufficient knowledge through independent inquiries outside of his position in education. As a physics educator, he had the awareness to follow recent developments of digital technology during this time, including machine learning development. However, the aforementioned high teachers’ workload, admitted by Yoga also inhibits him from exploring more about machine learning technology in teachers’ judgments. For the initial effort, we appreciated his attempt to approach technological resources for his knowledge. Yoga declared: “I have heard machine learning and I have found it on Google. You (author) are planning to study so it is kind of intrigued me. It is just because I have a lot of teaching schedules and so on. For now, I have no clues, but I see it on Google cloud platform. It was interesting to be learned. But the resources were not in Javanese, hahaha. So, I have to explore a bit more about that, you know.”

3.2.3. Category 3: Physics Teachers’ Actions

Constraints that have been observed by physics teachers in the first category above were addressed by their actions’ experiences in this category. Limited technological resources that have been made available to them, however, presented future challenges to our aims for this article. However, this had been considered based on our discoveries on the teachers’ existing knowledge about machine learning technology. We conveyed true appreciation to our physics teachers for their actions in conducting other cognitive assessments and associating it with the previous observational results in making judgments.

Cognitive Test Consideration

To improve judgment accuracy discussed in the first category, physics teachers experienced that they might reconcile former results of personal observations and other measurement methods such as cognitive tests. Information support through cognitive tests was considered to enable them to strengthen the teachers’ previous subjective judgments. Yono said: “I think that attitudinal assessment or observations were my preliminary assessment but still have to be proven by cognitive assessment.”

Data Matching from Learning Experience

Several learning transformations in their actions to adapt to students’ abilities were considered as imperative factors for adjusting teachers’ judgments. Narti has implemented laboratory activities and project-based learning for diverse students. Narti described: “There were sometimes those who are talented in experimenting or making projects. There are students who are inactive in classical interactions then evolved more passionate[ly] on project-based learning. Therefore, at that time, the former judgment should be adjusted. Projects, well, that is what makes them more interested in physics. Other case on demonstration tasks and they were found more active. If my learning was only one direction, sometimes, make them passive. Something like that. Not the same situation, different. For example, it fosters process skills.” This result implies that, in one case, students may underperform because they have different characteristics that are not in line with the current learning approach. However, through other learning approaches, students tend to become more involved.

Following Class Observation

To address the aforementioned high workload of our physics teachers, Tinah suggested conducting additional observations outside learning hours to strengthen judgment evidence. The situation is favorable since Tinah was assigned as class advisor. Students would usually be more comfortable sharing with their advisor. Thus, students’ problems should be more accurately identified by teachers. Tinah confirmed: “For the case, not every class. We sometimes have spare time for certain occasions. For example, we wonder [about] students that seem like less focus[ed] during the class. Then, I try to privately talk to them. We will talk outside of class. I conduct out[-of-] class sharing for enriching information for judgment.”

4. Discussion

Teachers’ judgments on the learning process were carried out to improve the quality of learning. This activity is closely connected to teachers’ monitoring processes of the students’ progressions, including how teachers expect and predict students’ performances during the collective learning process. In this study, the model of teachers’ thought and action has explained how three components could explain physics teachers’ experiences and limitations when making judgments within differentiated instruction environments. Students’ diversity is a topic that was recently considered within the PER community [65]. Physics learning should be inclusive and it might be effective to create more widely accessible physics learning [10]. In this article, we have found that our physics teachers have recognized the constraints and opportunities during students’ learning. There are three main channels that teachers often employ to identify students’ differences in physics learning, namely: personal observation, task observation, and peer observation. Teachers’ judgments are important to ensure Indonesian physics learning could meet students’ developments. In addition to these methods, previous studies reported that teachers also often conduct students’ monitoring processes through students’ journals [66,67]. However, there were some limitations that we underline in the study results, namely intuitive and informal results which sometimes threaten the accuracy and equity of teachers’ judgments. Observations in this way might be easier to follow, but the obtained judgment could not be generalizable because several limitations are still present [68]. The practice of teachers’ judgment discovered several drawbacks, such as accuracy, equity, and teachers’ workloads, which could interfere with their opportunities to pay attention to each individual student.

As a teacher’s support system in making judgments, data-driven monitoring processes should be an alternate way to reduce previous teachers’ constraints. To improve the quality of judgments that teachers have made, we could consider the opportunities of online learning habits that have just recently become available. The COVID-19 pandemic since the beginning of 2020 has forced education around the world to transition into online learning [31,32,33,34]. This challenge implicitly offered opportunities for our physics teachers to explore their potential in overcoming their limitations on teachers’ judgments. Learning management systems (LMS) are very large channels for collecting educational data such as assignments, quizzes, projects, activities, laboratory works, or even prior grades that have been obtained before coming to class. As mentioned earlier above, data is the main tool that teachers must use to make judgments about their students’ diversities. Therefore, our physics learning should benefit from the challenge of LMS applications.

Despite the current limitations of pandemic learning, there has been a great deal of global discussion on remote or hybrid learning. The mixed-method study published by Herodotou et al. [28] has implemented predictive learning analytics (PLA) in a large-scale open university in the United Kingdom. This research was a series of developmental projects that the authors have carried out since 2017 to augment hybrid learning, which is the main feature of the learning process provided by their institution. Authors reported that the implementation of PLA by teachers could significantly affect students’ performances. The more often teachers involved PLA in the learning process, it was proven that they were more able to provide greater support to students. Based on the qualitative phase, the authors suggested providing ongoing support to teachers in the PLA implementation phase and especially in translating the information. From this multi-method study, the author concluded that PLA is a source of information that is able to provide resources to improve learning, especially for distance learning.

The technology that we suggest in this article could be assumed as the implication of our study. We recommend recent developments of artificial intelligence (AI) to address limitations discovered in teachers’ judgments. One of the fields of AI study, machine learning (ML), is a predictive model that has recently received great attention in the assessment of physics learning [19,20,21,69,70]. The implementation of ML studies for educational purposes, namely educational data mining (EDM) and learning analytics (LA), could lead to employing ML models to develop predictive systems to monitor students’ learning. Hence it would serve as a teacher’s support system to overcome judgment accuracy and equity [71,72,73,74]. Therefore, this ML and EDM technology should be involved in teachers’ judgment. The literature has widely applied this idea to be implemented in physics learning. However, to the best of our knowledge, previous works that have addressed ML implementation in teachers’ judgments are still lacking. There is rooms for developmental study to implement ML for secondary school physics learning, because some aforementioned studies were mostly developed for higher education institutions.

This research has been entirely constructed through the theoretical framework offered by Clark and Peterson [7]. We are aware of the possible limitations from this decision, since the teachers’ perceptions of using and implementing the suggested technology could be unanswered from this theory. To address this gap, we suggest that future researchers could approach another theory that is widely implemented in the development of digital technology: the theory of acceptance model (TAM) [74]. This theory was proposed to model how the technology implementation could be accepted by users of certain occupations. TAM emphasized more specifically whether technology development is able to understand external variables such as user attitudes towards their benefits and ease of use.

According to the previous literature inquiries, studies in our subject have been dominated by examining quantitative measures of judgment accuracy [44,75,76,77,78,79], for which qualitative investigations through the TAM lens are still limited. Therefore, there is an open opportunity to explore a more comprehensive teachers’ experience regarding the teacher’s perception of ML application to assist teachers’ judgments. Future researchers could also investigate factors that influence teachers’ opinions through the TAM framework. This investigation is vital because our teachers are the real users in this technology development to overcome teacher’s limitations in making judgments.

The results of the study reported in this article assume that teachers already understand the concept of differentiated learning and how to provide support to other teachers. Teachers’ limitations in making judgment reported in this article are potentially addressed as a baseline for other researchers to conduct further research to overcome them. We suggest that it is time for us to promote data-driven technology that could offer more accurate and fair judgments. Teachers’ workloads are also expected to be reduced through the introduction of this teachers’ support system, although, for now, there is no policy that requires teachers to employ machine learning in students’ judgment. For this reason, the teachers’ opinions extracted in this study have implications for giving recommendations to stakeholders in the development of teachers’ support systems.

5. Conclusions

The phenomenological study has extracted Indonesian secondary school physics teachers’ experiences and limitations during making judgments to facilitate differentiated students’ needs in the physics classroom. The model of teachers’ thought and action has described that there are three domains of the shared teachers’ experience. First, the teachers considered that students’ diversities are opportunities to transform physics learning. Through the support of data availability and school programs, teachers have implemented their actions to provide support for diversity among students in the context of differentiated learning. On the other hand, our study reported several teachers’ limitations encountered while making judgments.

Second, the most prominent limitations of teachers’ judgments were most recognized by teachers to be teachers’ workloads, judgment equity, and judgement accuracy. The teachers still considered that their judgment was still an informal decision so that they might be adjusted from other channels, such as involving adaptive technology to support judgments. Digital technology is suggested as the main implication of this research, as it is considered capable of becoming a teacher’s support system in making judgments. The implications of developing predictive technology for teachers’ judgments suggested in this study are expected to encourage upcoming research projects to address this issue more comprehensively. Providing support to teachers is our proposed first step to improving the sustainability and diversity of physics learning.