Generative AI for Culturally Responsive Science Assessment: A Conceptual Framework

Abstract

In diverse classrooms, one of the challenges educators face is creating assessments that reflect the different cultural backgrounds of every student. This study presents a novel approach to the automatic generation of cultural and context-specific science assessments items for K-12 education using generative AI (GenAI). We first developed a GenAI Culturally Responsive Science Assessment (GenAI-CRSciA) framework that connects CRSciA, specifically key cultural tenets such as indigenous language, Indigenous knowledge, ethnicity/race, and religion, with the capabilities of GenAI. Using the CRSciA framework, along with interactive guided dynamic prompt strategies, we developed the CRSciA-Generator tool within the OpenAI platform. The CRSciA-Generator allows users to automatically generate assessment items that are customized to align with their students’ cultural and contextual needs. We further conducted a pilot demonstration of item generation between the CRSciA-Generator and the base GPT-4o using standard prompts. Both tools were tasked with generating CRSciAs that aligned with the Next Generation Science Standard on predator and prey relationship for use with students from Ghana, the USA, and China. The results showed that the CRSciA-Generator output assessment items incorporated more tailored cultural and context assessment items for each specific group with examples, such as traditional stories of lions and antelopes in Ghana, Native American views on wolves in the USA, and Taoist or Buddhist teachings on the Amur tiger in China compared to the standard prompt assessment items within the base GPT-4o. However, due to the focus on nationality in the pilot demonstration, the CRSciA-Generator assessment items treated the countries as culturally homogeneous, overlooking subcultural diversity in these countries. Therefore, we recommend that educators provide detailed background information about their students when using the CRSciA-Generator. We further recommend future studies involving expert reviews to assess the cultural and contextual validity of the assessment items generated by the CRSciA-Generator.

1. Introduction

“Sometimes it’s not the student who is failing the assessment—it might be that the assessment is failing to fully assess the abilities of the student”—

[1]

In today’s diverse science classrooms, one of the challenges educators face is cre-ating science assessments that genuinely reflect the cultural backgrounds of every student [2]. K-12 science education plays a vital role in equipping students to tackle social and cultural issues like climate change, public health, and technological advancements. Consequently, K-12 science assessments, guided by frameworks like the Next Generation Science Standards (NGSS) [3,4], seeks to cultivate a profound grasp of fundamental and culturally scientific concepts and the capacity to utilize this knowledge in addressing real-world challenges [5,6]. This goal encourages the teaching and assessment of science to engage students in practices that are meaningful to their cultural experiences as assets in their learning processes [6,7]. There have been initiatives, such as culturally responsive assessment (CRA)) to connect science concepts to students’ cultural experiences an asset [8]. Studies on culturally responsive science assessments (CRSciAs) indicates that they make learning more relevant, engaging, and improve achievements among student [9]. However, implementing CRSciAs in K-12 classrooms remains challenging to educators. Educators continue to face difficulties due to increasing classroom diversity and the dominance of traditional, west-ern-centric assessments, which often disadvantage migrant, historically marginalized, and Indigenous students [10]. Furthermore, CRSciA practices, though essential, are time-consuming and complex, making them hard to scale effectively [11,12].

The recent development of Generative Artificial Intelligence (GenAI) offers a promising solution to CRSciAs [13,14,15]. GenAI can provide multimodal learning opportunities [16], handling a variety of cultural contexts, such as adapting to different languages, symbols, and local knowledge, more often and more effectively than what a single classroom science teacher could manage alone [17,18]. Recent studies indicate a significant rise in the use of GenAI, particularly ChatGPT, is used by educators to automate assessment generation. This trend reflects the potential of GenAI to address the traditionally time-consuming and labor-intensive aspects of assessment creation [18].

Nonetheless, the rapid adoption of GenAI for this purpose also presents challenges, as it does not yet fully assist teachers in achieving culturally responsive science assessments. Concerns persist about cultural and contextual misrepresentations and inaccuracies, which may stem from the reliance on the generic, large datasets used to train GenAI systems, as well as the imperfections in prompts crafted by teachers or users [19,20,21]. For instance, Kıyak [22], in their review study on GenAI and automatic assessment, highlighted the critical issue of GenAI-generated assessments as they lack contextual knowledge that relates to languages and Indigenous knowledge. Moreover, Chan et al. [23], in their study exploring ChatGPT’s potential in question generation, also identified cultural biases concerning race, ethnicity, and religious beliefs in the generated questions [24]. These studies, among others, recommended the need for datasets and frameworks that can guide GenAI to prompt engineering to consider the diverse cultural backgrounds of students as assets in assessment generation. These recommendations emphasize the urgent and critical importance of addressing these issues, as without careful consideration, there is a potential risk that GenAI could hegemonically reinforce the very inequities they are intended to overcome in education through assessment generation [25].

In response to these challenges, this study seeks to develop a conceptual framework that brings together the capabilities of GenAI and the core tenets of CRSciA for K-12 science assessments. Specifically, we first develop a GenAI-CRSciA conceptual framework by identifying the key concepts of culturally responsive assessments with GenAI and articulating the relationships between them [26]. This lays the foundation for the CRSciA-Generator within the OpenAI model. We then integrate the conceptual framework into the GPT model to create the CRSciA-Generator. This involves an interactive guided dynamic prompt method based on the CRSciA-Framework. This prompt approach dynamically initiates conversation and further interacts with users to generate assessment items tailored to their students’ cultural and context-specific needs based on the information provided by the user [27]. Finally, we pilot the CRSciA-Generator by comparing it with base GPT 4o and standard prompt within the cultures of the US, Ghana, and China. The findings show that the CRSciA-Generator has the potential to automatically generate cultural and context-specific science assessments. This has the potential to acknowledge students cultural backgrounds as an asset to scientific literacy.

2. A Literature Review

There are few studies related to CRA, in general, and, consequently, to CRSciA. This section provides an overview of the impact of standardized assessment, which has traditionally dominated science education. It also examines the discourse about CRSciA, the challenges that persist in implementing CRSciA, and finally, the capabilities of GenAI in addressing these challenges.

2.1. Impact of Standardized Assessment

One of the key impacts of standardized assessment (traditional standardized assessment) is the high possibility of creating an “achievement gap”. Achievement gap discourse is prominent, particularly in countries like the US with diverse student populations [28,29]. This term refers to the disparities in standardized test scores between Native Indigenous, Black, Latina/o, recent immigrants, and White students [30]. While this is a concern, Ladson-Billings [28] further posited that even a focus on the gap is misplaced. Instead, we need to look at the “education debt” that has accumulated historically over time. She draws an analogy with the concept of national debt, which she contrasts with that of a national budget deficit to argue the significance of historical inequities in education. Moreover, historically marginalized students, including Native Indigenous, Black, and Latina/o, have had accumulated disadvantages, limited opportunities, and a lack of access to equal education for generations [31].

Beyond the supposed “achievement gap”, CRSciA aims to address the educational debt in science. This implies that the lack of representation of cultures in science assessments has profound and far-reaching effects across every aspect of life in wider society. A biased science assessment that affects a particular group of students can influence their career goals and limit their contributions to society [32]. Studies show that unfair assessments have broader societal impacts, such as contributing to higher school dropout rates [33]. Students who are unfairly assessed may become disengaged in the educational system and may be pushed out (drop out) from school [33]. This, in turn, can result in a larger number of unproductive citizens, which negatively affects society by increasing the burden on social services and reducing overall economic productivity [34,35,36]. However, recent research shows that, unlike high-stakes assessments, culturally responsive assessments motivate students and promote authentic, life-long learning [37]. Furthermore, students who possess a well-developed understanding and awareness of cultural issues are ready to engage in the CRSciA [38]. GenAI’s ability to generate personalized educational content demonstrates its potential to continue addressing the diverse needs of students through CRSciAs.

2.2. Culturally Responsive Assessments in Science Education

Even though there has been an initial perception that science education is unsuitable for culturally responsive assessments, this misconception has been cleared [39]. Recent studies have shown remarkable practices of CRSciAs and even expanded to include Science, Technology, Engineering, Arts, and Mathematics subjects. This proves the value of culturally responsive assessments across all disciplines and educational levels [39]. Moreover, the Framework for K–12 Science Education articulates a broader set of expectations to ensure that by the end of 12th grade [3], all students would possess sufficient knowledge of science to engage in their everyday lives, continue learning about science outside of school, and have the skills to pursue careers of their choice. They emphasize the phrase “all students” throughout this framework to provide equitable opportunities, including assessment, for all students to succeed in science [40].

2.3. Challenges of Implementing Culturally Responsive Assessments in Science Education

The implementation of CRSciAs facessignificant challenges including teachers’ biases, limited cultural knowledge, and personal identities that influence their engagement with diverse student experiences. Additionally, the misalignment of CRSciAs with established curricular standards, inadequate institutional support, and insufficient professional development hinder effective adoption.

One major challenge arises from the limitations stemming from classroom teachers’ identities and biases, as well as their limited knowledge of diverse cultural contexts [41]. Teachers’ personal identities, including their race, ethnicity, and cultural background, can influence how they perceive and engage with CRSciAs [42]. For example, in science assessments, a teacher who lacks familiarity with the cultural experiences of their students may unintentionally introduce biases into the assessment process, either by favoring certain cultural narratives or by overlooking others.

Another challenge lies in the integration of CRSciA within the Framework for K-12 Science Education [3]. Though this is a reflection of a broader struggle within educational systems to effectively align CRSciAs with established curricular standards, it critically applies to science assessment [11,43,44]. Ladson-Billings [39] critiques this misalignment of CRSciA and =Framework for K-12 Science Education by highlighting how top-down initiatives to implement CRA often miss the mark; As countries, States, districts, and professional organizations attempt to address cultural issues of assessments through various frameworks and guidelines, their efforts frequently fall short of the theory’s original intent [39]. More to the challenges of CRSciAs is the issue of inadequate resources and lack of institutional support. The lack of institutional support and inadequate resources also makes it difficult for science teachers to effectively incorporate cultural responsiveness into their assessments, and with the necessary tools, guidance, and resources, educators are unable to fully leverage CRSciA practices [45].

These challenges such as limited teacher’s knowledge and misalignment of CRA with curricula standards call for continuous professional development, yet, studies indicate that there is currently a inadequate continuous professional development available for teachers regarding CRSciAs [46]. For instance, studies by Harris et al. [7] surveyed teachers across 18 states in the US and revealed that 86.36% of K-12 teachers viewed the integration of CRA with NGSS positively but suggested more robust teacher training programs to enhance awareness and effective adoption of both NGSS and CRA in science classrooms.

2.4. Generative AI and Culturally Responsive Assessment

There is currently limited research on the integration of GenAI into CRAs. However, this study draws upon the documented potential of GenAI in existing literature to establish its capabilities in addressing CRSciAs. Generative AI refers to advanced computational techniques that create new, meaningful content such as text, images, and audio from existing data [14]. These capabilities of GenAI can address significant challenges that teachers face in designing CRA, such as the time-intensive nature of creating materials that are both culturally relevant and pedagogically sound [47,48]. Furthermore, the interactive nature of GenAI-based assessments allows for real-time feedback and adaptation, providing teachers and students with immediate opportunities to learn and correct misunderstandings [49,50]. For instance, GPT-4’s ability to interpret and generate multimodal content, including visual data like graphs and chemical diagrams, enhances its utility in crafting assessments that are aligned with cultural contexts and could engage students in ways that traditional automatic text-based assessments cannot [16]. This multimodal approach, grounded in the theory of multimedia learning, can help overcome the limitations of traditional AI, which has been largely text-bound, by incorporating a broader spectrum of human experience into the assessment process [14,16,51].

3. Generative AI Framework for Culturally Responsive Assessments in Science

The framework of this study was grounded in cultural tenets that are central to shaping individuals’ identities and learning experiences in science as well as GenAI capabilities. Currently, GenAI has the ability to translate over 50 languages, an essential feature that plays a significant role in assessment generation. Studies also show that, with proper prompts, GenAI could reduce race and ethnicity biases [52,53,54]. Furthermore, GenAI is noted for demonstrating respect for religious and cultural differences and has the potential to enhance Indigenous knowledge [55,56,57,58]. These qualities of GenAI largely influenced the selection of these cultural tenets [59]. We acknowledge other cultural tenets, such as socio-economic status and gender, and advocate for future frameworks as the field of GenAI evolves [60]. The framework specifically focused on such prevailing tenets as Indigenous language, Indigenous knowledge, ethnicity/race, religious beliefs, and community and family (See Figure 1).

3.1. Indigenous Language

Indigenous language plays a crucial role in CRA within K-12 education, particularly in science classrooms where students from diverse linguistic backgrounds are increasingly prevalent [43]. Wright and Domke [61] highlight the significant role of language in the NGSS, noting how language can contribute to science learning and achievement among students’ K-12 students. Kūkea Shultz and Englert [62] provide a compelling example of culturally and linguistically responsive assessment with the development of the Kaiapuni Assessment of Educational Outcomes (KĀʻEO) in Hawaii. The Kaiapuni Assessment of Educational Outcomes (KĀʻEO) was developed to address the inequities faced by students in the Hawaiian Language Immersion Program (Kaiapuni), who were historically assessed with tools that were culturally and linguistically inappropriate. The assessment demonstrated significant outcomes by providing culturally and linguistically valid measures that better captured the academic abilities of Kaiapuni students. In the context of linguistically responsive assessment, the application of GenAI in educational settings has shown significant potential in addressing the diverse linguistic needs of students. A notable example is the study by Latif et al. [63], which introduced G-SciEdBERT (German Science Education BERT), a specialized adaptation of the standard German BERT (G-BERT) model. G-SciEdBERT was developed to overcome G-BERT’s limitations in scoring written science responses. By pre-training G-SciEdBERT on a substantial corpus of German-written science responses and fine-tuning it on specific assessment items, the researchers demonstrated a marked improvement in scoring accuracy, with a 10% increase in the quadratic weighted kappa compared to G-BERT. This finding highlights the role of GenAI models in creating linguistically and culturally responsive assessments.

3.2. Religion

Religious beliefs are a critical factor to consider in the development of CRA in K-12 science education. Mantelas and Mavrikaki [64] highlighted that the intersection of religiosity and scientific concepts, such as evolution, presents unique challenges for educators in CRSciA. Their study demonstrates that students with strong religious convictions may struggle with certain scientific ideas, which can negatively impact their academic performance if assessments do not account for these beliefs. This means CRSciA should allow students to demonstrate their scientific understanding without forcing them to choose between their religious beliefs and academic success [65].

Barnes et al. [66] further stress the importance of considering religious backgrounds in CRA, particularly for students of color who may rely on religion as a critical support system. Their research shows that strong religious beliefs can influence students’ acceptance of scientific theories like evolution, which, in turn, affects their academic success in science-related subjects. Culturally responsive assessments must, therefore, be designed to account for these factors, ensuring that they do not inadvertently disadvantage students whose religious beliefs differ from mainstream scientific views.

Moreover, Owens et al. [67] contribute to this discussion by advocating for a “pedagogy of difference” in teaching science, which could be extended to assessment practices in K-12 science education. This pedagogical approach encourages students to explore the relationship between their religious beliefs and scientific concepts, fostering an environment where multiple perspectives are acknowledged and valued. In light of this, Sumarni et al. [68] proposed a holistic model for integrating religious and cultural knowledge into STEM education, which can serve as a foundation for CRA practices. Their RE-STEM model emphasizes the importance of bridging the gap between religion, culture, and science, suggesting that assessments should be designed to reflect this integration.

Although the intersection of GenAI and religious contexts remains underexplored, GenAI’s potential for engaging in nuanced discussions about religious concepts is promising [69]. For instance, GenAI models like ChatGPT have shown the ability to participate in theological dialogues, offering responses that respect religious traditions [58]. This capability allows for the development of assessments that honor students’ religious beliefs. Nonetheless, ethical considerations are paramount, as Ashraf [70] advocates that GenAI applications must be carefully monitored to avoid infringing on religious freedoms, bias, and disrespect in digital interactions.

3.3. Indigenous Knowledge

Indigenous knowledge plays a pivotal role in shaping CRSciA in K-12 education, offering a means to create an equitable achievement [71]. Trumbull and Nelson-Barber [72] explored the challenges and opportunities in developing CRA for Indigenous students in the US, highlighting the limitations of standardized assessments that often disregard Indigenous knowledge systems. They argued that these assessments could be ineffective and even harmful, as they fail to engage Indigenous students or accurately measure their knowledge. This affirms Muhammad et al.’s [31] assertion that the traditional assessments and curricula often overlook the historical and cultural contexts of Black and Brown children in the US, as highlighted by the National Assessment of Educational Progress (NAEP).

Therefore, the concept of “culturally valid assessment” was proposed by Trumbull and Nelson-Barber [72] to incorporate Indigenous ways of knowing and to be responsive to the cultural and linguistic diversity of the students. This approach is crucial for creating assessments that support students’ academic success while also preserving and respecting their cultural identities [73]. Furthermore, Jin [74] systematically reviewed educational programs aimed at supporting Indigenous students in science and STEM fields, revealing the positive impact of integrating Indigenous knowledge with Western scientific assessment. This review shows that culturally responsive assessment approaches in these programs lead to improved educational outcomes, as they allow Indigenous students to draw connections between their cultural heritage and the scientific concepts they are learning.

GenAI has the potential to challenge dominant Eurocentric narratives and promote the inclusion of Indigenous perspectives in K-12 science assessment [75]. For instance, GenAI tools can help ensure that science assessments honor and reflect Indigenous cultural identities. However, it is essential that GenAI-generated content is contextually accurate and respects the complexity of Indigenous cultures. Castro Nascimento and Pimentel [76] emphasized the need for GenAI models to be trained on diverse cultural datasets to avoid perpetuating narrow perspectives. The deliberate integration of Indigenous knowledge into GenAI models can significantly enhance the cultural relevance of science education.

3.4. Race and Ethnicity

CRSciA requires a thorough understanding of how race and ethnicity shape students’ learning experiences and outcomes. Atwater et al. [77] emphasized that traditional science assessments often overlooked the diverse cultural backgrounds of students, particularly those from African American, Latino, and Asian American communities. They, therefore, advocated for science assessments that were inclusive and reflective of the race and ethnicity within classrooms. Similarly, Wells [78] called for strategic cross-sector collaboration between K-12 and higher education to address the sociocultural factors affecting diversity in education, reinforcing the need for assessments that are sensitive to the varied cultural contexts of students.

The importance of factoring race and ethnicity into science teaching and assessment was further highlighted by Riegle-Crumb et al. [79], who found that inquiry-based learning was associated with more positive attitudes toward science among students from diverse racial and ethnic backgrounds. When assessments are designed to reflect the competencies of students’ cultural backgrounds, they allow them to demonstrate their understanding through exploration, critical thinking, and problem-solving in STEM [80].

However, a study by Choudhary [81] highlights the prevalence of racial bias in GenAI tools, including ChatGPT, underscoring the necessity to mitigate these biases. In the context of CRA, GenAI tools must be designed to promote fairness and inclusivity, particularly in the assessment of students from diverse racial and ethnic backgrounds. Warr et al. [82] provided evidence of racial bias affecting GenAI evaluations, demonstrating that racial descriptors could influence GenAI-generated assessments. This underlines the importance of developing transparent and fair GenAI tools that can assist teachers to develop assessments that address the diverse cultural background of all students.

3.5. Family and Community Engagement

While family and community might not be directly involved in the creation of assessment items, the role of family and community experts in validating cultural factors of assessments is crucial in the CRSciA framework. Family and community involvement are elements providing essential context and resources that directly impact students’ academic performance and engagement. Denton et al. [83] emphasized the significance of community cultural wealth in STEM education, particularly in K-12 settings. They argued that an assets-based approach, which recognized the diverse forms of capital, such as familial, linguistic, and social, that students from nondominant communities bring, was essential for developing assessments that truly reflected students’ backgrounds. In K-12 science assessments, this means creating assessment tools that account for the cultural and social capital that students acquire from their families and communities.

Gerde et al. [84] provide insight into the specific ways that families contribute to science learning at the K-12 level. Their study shows that the resources and opportunities families provide, such as access to science-related books and toys, and community experiences like visits to parks and science centers, vary widely based on parents’ self-efficacy and beliefs about science. This is an indication that CRSciA at this level should not only measure what students know from formal education but also integrate the informal learning experiences that occur within their family and community contexts. Soto-Lara and Simpkins [85] further elaborated on the role of culturally grounded family support in the science education of Mexican-descent adolescents, focusing on the K-12 educational context. Their findings reveal that parents provide support through both traditional means, such as encouraging their children, and nontraditional methods, like Indigenous cultural practices and leveraging extended family networks.

Family and community involvement is a cornerstone of student success, particularly in science assessment. Garbacz et al. [86] stressed the importance of involving families and communities through ecological support. GenAI-driven tools can bridge the gap between school and home by providing culturally relevant resources and information, fostering a supportive learning environment. However, Saıd and Al-Amadı [87] noted challenges in engaging families, particularly in areas with limited digital literacy and technology access. GenAI has the potential to address these challenges by creating accessible communication channels between schools and families. The goal is to leverage GenAI not only for assessment but also to strengthen the connection between students’ educational experiences and their broader family and community contexts, enhancing overall educational support and involvement.

4. Developing the CRSciA-Generator

The OpenAI platform offers the opportunity to configure and customize GPT models as specialized ChatGPTs (such as. Code Tutor) for particular use cases [88]. This configuration allows users to tailor the model to meet specific needs, making it more relevant and effective for specialized tasks based on context. This process is essential for this study because base GPT models lack domain-specific focus and are not fine-tuned for specialized tasks or contexts, making them more prone to biases or hallucinations [89].

The development of the CRSciA-Generator follows a three-step process: Configuration and Customization Process; Prompt Engineering; and Final Output. These processes were designed to ensure that educators could engage seamlessly with the CRSciA-Generator tool and generate CRSciA items for their class without facing inaccurate outputs. With these steps, the model was made more user-friendly, allowing educators of all prompt skills to follow through and obtain the best possible output from the system (See Figure 2).

4.1. Configuration and Customization

The CRSciA-Generator is grounded in rigorous configuration and customization processes, starting with the integration of the GenAI-CRSciA frameworks and SNAP items. These documents are uploaded to the model as files. A critical aspect of customization involves optimizing the uploaded files for efficient processing while staying within the token limits of the GPT model. Exceeding the token capacity could have resulted in incomplete data processing. To mitigate this, we choose to upload PDF files instead of DOC/DOCX files, primarily due to their smaller file size. PDFs are generally more compact because they are designed for viewing rather than editing, allowing for content compression without a loss in quality [90,91]. The use of PDF is particularly important for SNAP items, which contain both text and visual components.

To support the creation of dynamic and interactive assessments, the configuration involves the incorporation of advanced tools such as web search, DALL·E image generation, and a code interpreter. The web search feature enables the generator to access relevant and up-to-date scientific information, ensuring the assessments remain accurate and aligned with the latest developments. The DALL·E image generation feature provides customized diagrams or visuals to accompany certain assessment questions, adding an interactive and visual dimension (multimodal ability), which is especially useful for science subjects requiring illustrations. Additionally, the code interpreter enhances the generator’s functionality by allowing for the creation of programming-based assessment items, particularly in STEM subjects.

4.2. Prompt Engineering

The customization involves a interactive guided dynamic prompt (dynamic prompts) strategy (proactive user prompts) based on the GenAI-CRSciA framework to support teachers and users in generating CRSciA items. The interactive guided dynamic prompt strategy specifically involves two strategies: conversation starters and dynamic replies. The conversation starters are designed to initiate interaction by asking key questions at the outset [92]. For example, the model prompts the user with:

“Welcome! I am your culturally responsive science assessment generator (CRSciA-Generator). I am here to help you develop science assessment items that meet the diverse cultural and context-specific needs of your students. Would you like assistance in developing a culturally responsive science assessment for your students that aligns with the NGSS? Please type ‘Yes’ or ‘No’ to proceed.

Moreover, the dynamic replies simplify user interaction with suggested inputs and follow-up questions, such as “Yes” or “No”. For example, a “yes” to the conversation ensures that users have the flexibility to tailor the assessments based on specific needs and preferences. An example might be as follows:

“Great! I can help you create an assessment aligned with NGSS standards. Would you like me to use the SNAP questions from the Stanford NGSS Assessment Project? Please type ‘Yes’ or ‘No’.”

(See Appendix A)

To use the OpenAI-powered API (like GPT-4) within the OpenAI Python library, see the example of how to adapt the interactive guided dynamic prompt (see

Table 1. Python Code for Dynamic Prompt in the CRSciA-Generator.

Component	Code Snippet
Import Libraries	import openai
API Key Setup	openai.api_key = “API Key”
Get Response Function	def get_openai_response(prompt, model = “gpt-4”):   response = openai.Completion.create(    engine = model,    prompt = prompt,    max_tokens = 150,    temperature = 0.7,    n = 1,    stop = None  )
Return Response	return response.choices[0].text.strip()
Conversation Starter Function	def conversation_starter():   starter_prompt = (    “Welcome! I am your culturally responsive science assessment generator (CRSciA-Generator). “    “I am here to help you create assessment items that meet the diverse cultural and context-specific needs of your class “    “that align with the NGSS. Let’s begin with a few questions to tailor the assessment for your class.\n”    “What science topic or NGSS standard would you like to cover?”  )
User Topic Input	user_topic = input(get_openai_response(starter_prompt) + “\n”)
Return User Topic	return user_topic
User-Prompted Pathway Function	def user_prompted_pathway():   language_prompt = “What are the dominant languages your students can read and write in for science?”   cultural_prompt = “Would you like to include any culturally specific knowledge or context in the assessment? (Yes/No)”
Get Responses from User	language = input(get_openai_response(language_prompt) + “\n”)   cultural_relevance = input(get_openai_response(cultural_prompt) + “\n”)
Cultural Context Check	if cultural_relevance.lower() == ‘yes’:   context_prompt = “Please provide some details about the cultural context you’d like to include.”   context = input(get_openai_response(context_prompt) + “\n”)   return language, context
Return Language and Context	else:   return language, None
Main Function to Generate Assessment	def generate_assessment():   topic = conversation_starter()   language, context = user_prompted_pathway()
Display Summary	print(“\n--- Assessment Summary ---”)   print(f”Science Topic: {topic}”)   print(f”Language: {language}”)   if context:    print(f”Cultural Context: {context}”)   else:    print(“No specific cultural context included.”)
Run the Generator	print(“\nYour assessment will be tailored based on the information provided.”) generate_assessment()

). In the provided Python code, parameters such as $temperature=0.7$ and $max\_tokens=150$ are set by default to control the randomness and length of the model’s output [89] (See Appendix B). This was to ensure consistency with the CRSciA-Generator parameters within the OpenAI platform, preventing any divergent behavior in the model’s response generation.

4.3. Piloting the CRSciA-Generator

In the application to determine the efficacy of the CRSciA-Generator demonstration was conducted to compare it with standard prompts within the base GPT 4o. The objective was to identify the aspects where the GenAI-CRSciA framework intersected with standard prompt strategies and to assess any differences in outcomes. Standard prompt offers a simple and direct approach for instructing ChatGPT by specifying a particular task for the model to execute, often using a format like “Generate a [task]” [93]. Given that SNAP items require students to complete specific tasks, standard prompts are an ideal choice for comparison in this context. In contrast, dynamic prompts provide a more flexible and adaptive approach, adjusting based on user inputs and the specific context of the task. This responsiveness makes interactive guided dynamic prompts particularly useful for generating content that aligns with the user’s needs. In the CRSciA-Generator, for example, interactive guided dynamic prompts start with a conversation starter, facilitating a deeper and more interactive engagement with the user [94,95].

4.4. Use Cases of the CRSciA-Generator and Prompts

To apply the GenAI-CRSciA framework in practical examples, we adapted the NGSS Life Science questions to prompt the base ChatGPT 4o (See

Table 2. Original Questions.

NGSS standard for MS-LS2-2 [96] NGSS standard for MS-LS2-2 [96] NGSS standard for MS-LS2-2 [96] NGSS standard for MS-LS2-2 [96] MS-LS2-2_024_ Wolves and moose interactions Ecosystem Interactions Construct an explanation that predicts patterns of interactions among organisms across multiple ecosystems. Activity: Wolve and moose interactions (ID# 116-04-Y03) Construct an evidence-based argument about whether there exists a predator and prey relationship between wolves and moose. From: Page 1 Wolves and moose interactions (ID# 116-04-Y03) (concord.org) Source: This resource is part of the Concord Consortium’s Middle Grades (6–8) project Task Wolves and moose interactions (ID# 116-04-Y03) Wolf and moose images are licensed under the public domain.

) using the standard prompt (See

Table 3. Standard Prompt with base GPT 4o.

Standard Prompt: Prompt: Create a culturally responsive science assessment question based on the NGSS standard MS-LS2-2, which involves ecosystem interactions and predator–prey relationships. The question should be suitable for middle school students in Ghana, the USA, and China. The question should ask students to construct an evidence-based argument about a predator and prey relationship, using examples that are relevant to their cultural context. Output Culturally Responsive Science Assessment Question: NGSS Standard: MS-LS2-2 Concept: Ecosystem Interactions Grade Level: Middle School Target Regions: Ghana, the USA, China Question: In ecosystems around the world, animals interact as predators and prey. Think about a predator and prey relationship that is familiar in your cultural context or from your region (Ghana, the USA, or China). Construct an evidence-based argument about the relationship between these two organisms. Consider the following in your response: Identify the predator and the prey in this relationship; Describe the ecosystem where these interactions occur; Provide evidence from your knowledge or experiences that shows how the predator and prey populations affect each other; Explain how this relationship might be similar or different from other predator–prey relationships around the world.

) and interactive guided dynamic prompt within the CRSciA-Generator (see

Table 4. CRSciA -Generator Conversation Starter and Interactive guided dynamic Prompt.

Prompt: Following CRSciA-Generator Conversation Starter and Interactive guided dynamic Prompt Strategies Objective: Create a culturally responsive science assessment question that is scientifically rigorous and tailored to the cultural, linguistic, and educational contexts of students from [insert country/region, e.g., Ghana, the USA, China]. The question should be based on the [insert specific scientific concept or standard, e.g., ecosystem interactions, photosynthesis, forces, and motion]. Instructions: Use the following guidelines to structure the assessment question: Language: ○ Use [insert language, e.g., English, Mandarin, Twi] as the primary language for the question; ○ Ensure the language used is appropriate for the educational level and cultural context of the students; Indigenous Knowledge: ○ Incorporate local or Indigenous knowledge relevant to the students’ cultural backgrounds; ○ Use examples that resonate with the student’s experiences, such as [insert specific cultural examples related to the scientific concept, e.g., traditional ecological practices, folklore]. Race/Ethnicity: ○ Respect and reflect the ethnic and cultural identities of the students; ○ Include scenarios that are geographically and culturally relevant to the student’s environment; Family: ○ Encourage the student to engage with their family or community members to gather insights or stories related to the scientific concept being assessed; ○ Ask the student to incorporate these familial insights into their response; Religious Beliefs: ○ Allow for the exploration of religious or spiritual beliefs that might influence the student’s understanding of the scientific concept; ○ Ensure the question does not conflict with or marginalize any religious beliefs. Standards Construct a question that requires the student to engage with the scientific concept by analyzing data, making predictions, or constructing an evidence-based argument.Ensure the question aligns with the [insert specific scientific standard or concept, e.g., NGSS standard MS-LS2-2]. Provide clear instructions for the student on how to incorporate cultural elements into their response.

CRSciA-Generator  Questions

Ghana: Language: English Context: Consider how local knowledge and traditional stories in Ghana describe predator–prey relationships, such as those between lions and antelopes in the savanna ecosystem. Think about how the knowledge passed down through your family or community elders might help you understand similar relationships in other parts of the world. Task: Based on the graph and your understanding of predator–prey relationships, write an argument that shows whether you agree or disagree with Jason’s explanation; Your argument should include: ○ A statement about whether you agree or disagree with Jason’s claim. Make sure to include Jason’s claim if you agree with him or write your own claim if you disagree; ○ A statement about what is accurate or inaccurate in Jason’s evidence and reasoning; ○ Examples from the graph to show why Jason’s evidence is accurate or inaccurate; ○ Reflect on traditional Ghanaian stories about predator–prey relationships and discuss how they support or challenge Jason’s reasoning. ○ USA: Language: English Context: Reflect on how Native American perspectives describe wolves as spiritual beings and key regulators in ecosystems. Consider how this Indigenous knowledge might influence your understanding of the predator–prey relationships shown in the graph.         Task: Based on the graph and your understanding of predator–prey relationships, write an argument that shows whether you agree or disagree with Jason’s explanation; Your argument should include the following: ○ A statement about whether you agree or disagree with Jason’s claim. Make sure to include Jason’s claim if you agree with him or write your own claim if you disagree; ○ A statement about what is accurate or inaccurate in Jason’s evidence and reasoning; ○ Examples from the graph show why Jason’s evidence is accurate or inaccurate; ○ Incorporate Native American perspectives on wolves and discuss how these perspectives might support or challenge Jason’s reasoning. ○ China: Language: Mandarin Context: Consider how traditional Chinese views on predators like the Amur tiger can help you understand predator–prey relationships in ecosystems. Reflect on how your family’s stories or the teachings from Taoism or Buddhism might influence your interpretation of the graph.         任务: 根据图表以及你对捕食者与猎物关系的理解, 写一篇论述, 说明你是否同意 Jason 的解释. 你的论述应包括: ○ 关于你是否同意 Jason 观点的陈述.如果你同意他的观点, 请务必包含 Jason 的论点; 如果你不同意, 请写下你自己的观点. ○ 关于 Jason 的证据和推理中哪些内容是准确的或不准确的陈述. ○ 从图表中举例说明为什么 Jason 的证据是准确的或不准确的. ○ 结合中国传统文化中对捕食者 (如东北虎) 的看法, 讨论这些文化视角如何支持或挑战 Jason 的推理.         (Translation: Task: Based on the graph and your understanding of predator–prey relationships, write an argument that shows whether you agree or disagree with Jason’s explanation; Your argument should include: ○ A statement about whether you agree or disagree with Jason’s claim. Make sure to include Jason’s claim if you agree with him or write your own claim if you disagree; ○ A statement about what is accurate or inaccurate in Jason’s evidence and reasoning; ○ Examples from the graph to show why Jason’s evidence is accurate or inaccurate; ○ Incorporate traditional Chinese perspectives on predators like the Amur tiger and discuss how these cultural views might support or challenge Jason’s reasoning.) ○

). Find below the original and generated assessment items for both CRSciA Generator and standard prompts within the base GPT for use with students from Ghana, the USA, and China.

In evaluating the outputs, the standard prompt was found to generate suggestive approaches to developing questions that could meet the GenAI-CRSciA framework, particularly suggesting teachers or users to “Think about a predator and prey relationship that is familiar in your cultural context or from your region (Ghana, the USA, or China)”. This makes it limited in its ability to fully embrace individual cultural nuances, which constrains the specificity of the output.

In contrast, the output from the CRSciA-Generator comprehensively addresses the GenAI-CRSciA framework by generating different sets of questions for each cultural group. For example, regarding language, the CRSciA-Generator translates certain keywords into the specific languages of each country (e.g., Akan for Ghana, Mandarin for China), ensuring linguistic accessibility and cultural connection for students. It further generates relevant examples and visuals that align with local species and ecological relationships, enhancing student engagement by resonating with their racial and ethnic contexts. Moreover, the CRSciA-Generator acknowledges the religious components within the framework, providing culturally specific examples, such as an antelope, a wolf, or an amur, that address beliefs or stories unique to each country.

Nevertheless, one observation about the CRSciA-Generator output is that it generalized the content for each of the countries rather than specifying distinct regional or subcultural contexts within each country, which could potentially introduce bias by overlooking the rich diversity within subcultures. This limitation is likely related to the amount of background information provided, as the input is more of country-specific levels.

5. Discussion

The alignment of the CRSciA- Generator with the unique cultural contexts of Ghana, the USA, and China involves careful adaptation across several key tenets. In terms of language, the questions are presented in English for both Ghana and the USA, where it is the dominant language of instruction. While in China, the questions are delivered in Mandarin [97,98]. This ensures that students can engage fully with the content in their native or dominant instructional language. For instance, the efficiency of the GenAI-CRSciA framework becomes particularly evident when addressing the challenges faced by students in linguistically diverse environments, such as China and Ghana. In China, despite policies aimed at incorporating English as a medium of instruction, research shows that Chinese students often struggle in predominantly English-medium contexts [99]. The GenAI-CRSciA framework effectively mitigates these challenges by tailoring content to specific cultural and linguistic contexts, such as translating content into Chinese and aligning it with the students’ cultural backgrounds and linguistic needs [100]. Similarly, in Ghana, where the language-in-education policy mandates the use of mother tongue (L1) as the medium of instruction in lower primary grades (KG1 to Primary 3) and English (L2) from Primary four onwards, the CRSciA-Generator demonstrates its cultural awareness by generating science assessments predominantly in English. This aligns with the reality of Ghana’s educational system, where English remains the dominant medium of instruction and assessment, particularly in science education [101,102].

Indigenous knowledge is also thoughtfully integrated, with traditional Ghanaian stories and proverbs highlighting the relationship between lions and antelopes, Native American perspectives shedding light on the spiritual and ecological significance of wolves and moose, and Chinese history and folklore contextualizing the interaction between Amur tigers and sika deer. For example, in Ghana, the framework integrates culturally significant symbols, such as the lion and deer, which are deeply embedded in local folklore and traditions. The ‘Aboakyire Festival’ celebrated by the people of Winneba involves the symbolic hunting of a deer, an event that carries profound cultural and spiritual significance [103]. Similarly, among the Akans of Akyem Abuakwa, animal symbolism, including that of the lion and deer, plays a crucial role in cultural identity and educational practices, conveying lessons and wisdom that are essential to the community’s heritage [104]. Likewise, in China, the framework demonstrates its effectiveness by incorporating ecologically significant animals such as the Amur tiger and sika deer into educational content, aligning the assessments with the local knowledge and values of communities in Northeast China [105].

Further adaptations reflect the importance of race and ethnicity, where the symbolic significance of lions is considered within various Ghanaian ethnic groups, differing views on wolves and moose are acknowledged among Native American tribes and European settlers in the USA, and the cultural heritage of Amur tigers is recognized within both the Han Chinese majority and ethnic minorities [106]. Moreover, religious beliefs are explored, with Ghanaian students reflecting on traditional animistic beliefs, American students considering how spiritual beliefs shape their views on wolves and moose, and Chinese students examining Taoist or Buddhist concepts of balance and harmony as they relate to the relationship between Amur tigers and sika deer [107,108]. This comprehensive approach ensures that the assessments are scientifically and deeply rooted in the cultural realities of the students. The findings demonstrate GenAI’s potential to revolutionize 21st-century assessments, which aligns with the conclusions of Owoseni et al. [109], who highlight GenAI’s capability to enhance summative assessments by automating tasks like question generation and grading.

Again, the findings of this study suggest that customization and interactive guided dynamic prompt engineering will be the future of AI literacy. This evolution highlights the need for further research and the integration of prompt engineering into education. This suggestion aligns with those of Knoth et al. [110] and Arvidsson and Axell [111], who emphasize the crucial role of prompt engineering in optimizing GenAI across different fields, including education and requirements engineering. Both works highlight that the quality of GenAI outputs is directly influenced by the precision of prompt engineering, underscoring the importance of GenAI literacy Arvidsson and Axell [111] and further pointing out how domain-specific guidelines can improve the accuracy and utility of GenAI for specialized tasks. Furthermore, the study suggests that teachers need to improve their knowledge to be able to use AI as an adopter, collaborator, or inventors [112]. Most educators, including teachers, pre-service teachers are not prepared with the knowledge, so they need professional learning, such as prompt engineering [113].

6. Conclusions and Future Directions

This study presents a novel approach to automatic generation of cultural and context-specific science assessments for K-12 education using GenAI. We first developed a GenAI-CRSciA framework that establishes the relationship between CRSciA and GenAI, by incorporating key cultural tenets such as indigenous language, Indigenous knowledge, ethnicity/race, and religion. Using the GenAI-CRSciA framework and interactive guided dynamic prompt strategies, we developed the CRSciA-Generator tool within the OpenAI platform. The interactive guided dynamic prompt based on the GenAI-CRSciA-framework to automatically guide educators to prompt the generate assessments following the conversational information from users. The interactive guided dynamic prompt strategy addresses the challenges associated with prompt engineering skills, as users do not require advanced prompt skills or iterative refinements, while ensuring high-quality GenAI outputs. We further conducted a pilot demonstration comparing the CRSciA-Generator with standard prompt in generating cultural and context specific science items.

The CRSciA-Generator produced assessment items that incorporate more local knowledge, traditional stories, and cultural perspectives, making it more culturally responsive to students in regions such as Ghana, the USA, and China than the base GPT4o with standard prompt did. The assessment specifically features examples such as traditional stories of lions and antelopes in Ghana, Native American views on wolves in the USA, and Taoist or Buddhist teachings on the Amur tiger in China. These culturally grounded examples illustrate the potential of the CRSciA-Generator to create equitable and inclusive assessments tailored to the cultural and contextual experiences of diverse learners.

However, the pilot test was overgeneralized by focusing on broad national contexts, treating entire countries as culturally homogenous. Therefore, we recommend that teachers provide sufficient cultural and context background information about their students to enable the CRSciA-Generator to produce more contextually accurate assessments. Additionally, we believe that the pilot demonstration does not fully validate the model’s efficacy. Future studies involving human experts are necessary to review and verify the cultural and contextual specificity of the generated assessments. We recommend conducting empirical studies in diverse contexts to further use and validate the tool’s overall effectiveness.

Furthermore, the results highlight customized GPTs as potential tools for future learning management systems (LMS). Customized GPTs in education will be impactful when powered by cultural frameworks, like those embedded in the CRSciA-Generator, as they ensure education remains equitable, inclusive, and culturally relevant. However, the argument could be strengthened by addressing how these tools integrate with existing LMS infrastructure and by providing specific examples of their scalability in diverse educational settings.