Using Motivation Theory to Design Equity-Focused Learning Analytics Dashboards

Abstract

Learning Analytics applications, and their associated dashboards, are frequently used in post-secondary settings; yet, there has been limited work exploring the motivational implications of their deployment, especially for under-served student populations that are more susceptible to (perceived) negative messages about their academic performance. In this paper, I argue that Situated Expectancy-Value Theory (EVT) is well-positioned to serve as a useful lens when developing and evaluating learning analytics dashboard designs and their future development. Used in this way, SEVT can help the learning analytics community to ensure that student experiences with learning analytics are adaptively motivating, both in general and for underserved student populations more specifically.

1. Introduction

Close to ten years ago, the 2014 EDUCAUSE Horizon Report stated that learning analytics would provide “ways...to improve student engagement and provide a high-quality, personalized experience for learners” [1]. Eight years after the fact, this objective has been pursued through the use of learning analytics applications that have reached far into post-secondary institutions and are approaching ubiquity within them. Applications for the design of learning analytics have ranged from support for student information-seeking behaviors with a focus on supporting how they learn [2] to large-scale implementations designed to support students seen as being “at-risk” with respect to academic challenges [3]. Meanwhile, there have been calls for learning analytics to focus on “information justice based on an ethics of justice and care” [4], i.e., by ensuring that learning analytics applications are designed and implemented in a manner that is respectful of student privacy and supports their agency. Despite the near-decade of learning analytics as a field, it is notable that a core dimension of the “personalization” argument (i.e., that learning analytics can use student data to provide individualized learning experiences to students while helping to understand the within-student variance of those experiences) is largely unchanged. It seems clear that a core dimension of learning analytics is personalization as means of supporting student learning.

Many contemporary applications of learning analytics research have enacted this core affordance of learning analytics by focusing on helping those students who are most at risk of facing academic challenges [5]. Considering that the concept of “at-risk” students suggests the potential for intervention to prevent undesirable outcomes (e.g., academic failure), there have been calls to understand how academic information is communicated through learning analytics dashboards (LADs). Indeed, the role that LADs play with respect to fostering, limiting, or otherwise affecting students’ academic motivation and self-regulated learning is a growing area of study (e.g., [6,7,8,9,10,11,12]). Yet, while the field has indeed moved towards personalizing the experience of students (e.g., [13,14,15]), predicting which students might need extra support (e.g., [16,17]), and communicating those insights via LADs, recent work has shown that learning analytics has a blind spot when it comes to issues of diversity and equity [18,19,20]. In this paper, I argue that motivation theory (i.e., Situated Expectancy-Value Theory [21]) can be a bridge that links the field of learning analytics with respect to LAD design to issues of educational equity. Specifically, I hold that if the role of learning analytics is to support student success though the lever of engagement, then it is important for insights communicated though LADs to be grounded in motivation theory in order to ensure a better understanding of how interventions academically motivate, demotivate, or differently motivate students.

2. Expectancy-Value Theory

Student academic motivation has been linked to how they engage with their courses [22]; if, when, and how they seek help [23]; whether or not they choose to cheat [24]; and whether they choose to persist despite struggles [25]. Thus, if LADs have the potential to influence student motivation, and student motivation plays a key role in learning, then the motivational implications of LADs warrant examination. While there has been work in this regard within the learning analytics community (e.g., [5,12,26]), that work has largely focused on the role played by Achievement Goal Theory [27,28,29] in how students respond to different LAD designs. Adding additional theoretical approaches, then, is a critical step toward determining how students make sense of the information they receive via LADs. Otherwise, the field risks using an overly narrow conception of motivation when studying students’ sense-making practices and whether those practices result in adaptive or maladaptive forms of academic motivation.

I extend the discussion of how motivation theory can expand equity focused discussions of of LAD design by using Situated Expectancy-Value Theory (EVT) as a theoretical frame. As its name implies, SEVT [21,30,31,32,33] seeks to explain student motivation by answering two questions about courses of action: “Can I?” and “Do I want to?” The former is conceptualized as “expectancies,” and the latter is captured by the multidimensional construct of “value”, in turn composed of intrinsic, utility, importance, and cost components. Each of these components, moreover, is situated in a particular context that, by necessity, attends to the individual dimensions of learners. Equity plays a central role in this, as SEVT accounts for individual motivational differences that might otherwise be collapsed into larger aggregate constructs. I examine each of the components of EVT below, and discuss how understanding each construct can be relevant to the design of learning analytics dashboards with equity in mind.

2.1. Expectancy

For students, expectancies serve as de facto self-evaluations of the likelihood of success or failure when a task is attempted. Students can have a range of expectancies for both general domains (e.g., math), and specific tasks (e.g., a math exam). Importantly, expectancies have a temporal dimension that can be proximal (i.e., a task requiring immediate or near-immediate attention) or distal (i.e., a task to be completed at a time in the future). A student, for example, could pose an expectancy question that takes the form of: “What is the likelihood that I will successfully accomplish X,” where X is an academic task that a student is considering undertaking. To help a student answer this question, a given LAD can use known data about students, e.g., past information, as it relates to their performance on similar tasks, as well as information from empirically matched peers who have successfully accomplished the same task in the past. The inclusion of both peer information and information that might be gathered through psychometrically validated surveys can enable LADs to attend to individual differences [5] while incorporating information to ensure that students are not treated in a way that is “color-blind” or otherwise ignores their identities [18].

Being able to use data about a student’s performance and that of their contemporary or historical peers can enable learning analytics applications to provide feedback to the student in a manner that is both personal and data-driven. In this way, a students’ expectancies can be grounded in tangible information and historical trends. How this information is displayed and what kind of information is displayed is both a pragmatic concern and an ethical one, as institutions need to walk the line between heavy-handed paternalism and providing students with information that could encourage positive outcomes [34]. One such concern includes providing students with access to information about their peers. While this information can be aggregated and anonymized to address privacy concerns, there may be possible effects on student motivation [26]. Such effects should be theorized prior to deploying LADs, as well as empirically examined after that fact to validate a priori hypotheses. The results of such investigations can enable iterative (re)design processes that encourage adaptive motivation among the students they serve.

Similarly, it is important to not overgeneralize historical data to a particular student or population of students. The promise of learning analytics is to personalize learning experiences and suggest course-corrections during moments of struggle; the possibility of boxing students in should be avoided. The danger of learning analytics doing this is insidious; being data-driven does not necessarily suggest that the available data that are driving learning insights represent all data on student or group of students. If this is assumed, subsequent inferences might become reified in designs or practice. Such concerns have been raised with regard to potential racism being inherent in collected data [35], resulting in its subsequent reification [36]. Instead, expectancies should be tied to domain-specific standards in order to promote equitable outcomes [37].

2.2. Value

2.2.1. Attainment Value

The “attainment” value is defined as the “personal importance of doing well on a task” [31]. Students learning algebra, for example, might assign a high attainment value to their performance on an upcoming algebra exam because they have developed a strong affinity towards math, such that a dimension of their identity is being “a good math student”. A student who self-identifies as “a good writer” may not assign nearly as much attainment value to the same algebra exam. Attainment values, then, represent what is important and salient to a student during demonstrations of competence via various academic tasks. Questions such as “What does it say of me if X is accomplished, and do I care?” are questions that speak to a student’s identity in relation to a specific task, and are well suited to informing the design of LADs. By definition, LADs are designed to use information that is unique to each student; thus, the individuality of a student is already foregrounded in a way that is both present and potentially more powerful, as it is better informed via the inclusion of student data during the generation of feedback. Course-taking behaviors, for example, can be used to introduce students to other courses of interest (e.g., [38]). While imperfect, such feedback mechanisms have the potential to expose students to courses and resources that are better aligned with their particular attainment values.

A potential objection to this approach is the notion that an interest-driven approach may lead students to where they would prefer to go (i.e., it is motivating); however, where a student wishes to go may actually be sub-optimal. While such concerns are valid, it is important to note that a core component of equity is not simply equity with regard to positive outcomes; there should be equity with regard to the processes (e.g., choices) that lead to good outcomes as well. A compromise is to ensure that optimal paths are made clear, rather than encouraging de facto tracking. This middle ground enables insights to not fall into the realm of a paternalism that denies students their agency through coercion.

2.2.2. Intrinsic Value

If attainment value is anchored on conceptions of self, then intrinsic value can be understood as a focus on internal processes. Specifically, a given task might evoke feelings of joy [33] or flow [39] as it is being performed. A student may simply be interested in the prospect of accomplishing a novel task, such as learning another language. Because intrinsic value is largely focused on internal perceptions of a task, the construct has been shown to be predictive of students choosing to master tasks rather than simply choosing easy ones (e.g., [40]). This is supported by other motivation theories, as mastery goals are generally characterized as being intrinsic [41].

To carry the above example forward, a student who assigns low attainment value to doing well on a math exam because they see themselves as a good writer and not as a student who is good at math may nonetheless assign high intrinsic value to the math exam because they simply enjoy algebra. This enjoyment need not be tied to notions of identity, and is usually temporally proximal to the student (i.e., what a student is working on now or will be working on soon). This is not always the case, however. Research has shown that female students may attach higher intrinsic value to obtaining higher education due to a “personal wellness” orientation, which is a move away from a more cautious orientation driven by avoiding the negative consequences associated with the pursuit of an advanced degree (such as limiting family planning decisions) [33,42]. Such students, then, attach intrinsic value to goals that remain somewhat distant. Understood in this way, intrinsic value is both complex in origin and dynamic in practice, and is conditioned on various possibilities and inclinations a student may possess or wish to posses.

Intrinsic value asks questions such as “Will I enjoy or feel engaged with the process that leads up to successfully accomplishing X?” This is another place in which LADs can inhabit a mediating role between antecedent questions a student may pose. This is especially true when considering all of the possible configurations in which information can be presented to students as well as which data are drawn on to generate a given representation. A graph showing a particular student doing poorly on an academic task in relation to their peers, for example, might have the unintended consequence of diminishing the intrinsic value that the student may have felt while engaging with the task [7,12].

2.2.3. Utility Value

Utility value relates to a student’s current and future goals. The above student may actually dislike algebra, for example; however, if he or she assigns high utility value to learning it, then he or she will nonetheless be motivated to pass the algebra exam because of the future purpose it serves. Consequently, utility value can depend either on current goals (such as passing algebra during a current term in order to take calculus the next) or distant goals (such as needing to pass algebra because it falls in a sequences of courses that ultimately lead to getting into medical school [33]. Whether or not these goals can be characterized as intrinsic or extrinsic, however, is a matter of interpretation; for instance, students’ goals may be internalized and intertwined with their aspirational identities (intrinsic), or may be anchored to the wishes of their parents (extrinsic).

In addition to shaping how students approach an academic task, utility values have been shown to predict farther-reaching decisions, such as the types of courses students choose to enroll in [43]. It is unsurprising, then, that utility values are the focus of intervention designs such as “relevance” interventions. Such interventions explicitly ask students to connect a given task to their own lives through a writing exercise in order to increase perceived utility value; these interventions have proved successful in both laboratory and classroom settings [44]. Interestingly, attainment value and intrinsic value as measured in reference to students accomplishing distant tasks (such as an undergraduate student’s goal of attending graduate school) have been shown to be more difficult constructs to separate from one another empirically [42].

Utility value questions such as “Will accomplishing X be useful to me now or in the future?” is where the ability for LADs to leverage institutional data becomes important. There is evidence, for example, showing that certain course-taking pathways are more fruitful for engineering students than others [45]. This information would be invaluable for utility decisions; if certain course “pathways” are shown to impose unneeded burdens on students, then students are able to avoid them. In addition, institutions could be prompted to evaluate certain sequences of courses to inform potential LAD designs. As learning analytics interventions continue to scale in higher education settings, LADs will be able to feed past information back to students. This can work well for specific study strategies, though it need not be limited in such a way.

2.2.4. Cost

Cost refers to an internal calculation of the opportunity costs associated with a given task, e.g., doing well on an exam or choosing to major in a STEM field. Students wishing to do well on an exam, for example, may need to delay their own gratification [46,47] and be attentive during instruction, whereas students pursuing a STEM career may have to factor in other long-term costs. In either case, doing well often comes at the cost of participating in enticing activities, such as talking to friends during instruction or choosing less rigorous field to major in.

Cost questions such as:“What will I lose if I accomplish X, and do the associated benefits for success outweigh those costs?” are well suited for learning analytics dashboard applications, and can inform students’ understanding and deployment of different resources. These resources include time (e.g., the number of credit hours attempted during a term), money (e.g., money spent on additional learning materials), and effort (e.g., time spent studying for a class), among others. Recent work in the self-regulated learning literature has shown the capability to predict and capture granular student behaviors that may make cost calculations easier for researchers to reflect back to students (e.g., [48,49]). This can be helpful to students, as cost management is important; students must manage various demands, as well as both their digital and analog resources.

This is especially important for under-served students, who need to make such cost calculations more often due to more limited resources. Knowing beforehand, for example, that taking a certain combination of courses will lead to the loss of time available for extracurricular activities or employment opportunities in the short term, even if it leads to a graduating sooner, is helpful for students to know as they register for courses, and can potentially help them to save money in the long term. While this information may already exist within peer networks or university handbooks, such information may not be as readily accessible to underserved student populations. Thus, while knowing the rate at which a particular combination leads to successful outcomes is not as readily available, it is possible with learning analytics, and can be continually updated to ensure that students make decisions based on the most recent information.

3. Discussion

It is obvious that learning analytics dashboards are well positioned to provide students with a platform to engage with their information in a manner that is personal, dynamic, and capable of answering particular questions that can influence both expectancies and task value. These data could consist of relevant information about students or about their learning contexts, e.g., historical trends in a class, their own learning trends, etc. Taking a thorough approach to which data best inform a set of LADs enables dashboards to be designed with equity in mind, as equity is predicated on attending to both the contextual and individual characteristics of learners.

This information is made salient through LADs, and is consequently more relevant to students as individual learners. Such an approach is a departure from providing students with information meant to help the “average” student, one who is often envisioned in a manner that decouples them from their core identity. The potential of learning analytics to influence motivation as conceptualized by EVT, then, is through mediation. Specifically, LA applications have the potential to help students make decisions that they are already contemplating by providing feedback and resources previously unavailable to them, or available only slowly via cumbersome channels. This feedback flows through a chosen representation.

Though their work predates much of the discussion of motivation within the learning analytics community, Makara and Karabenick [50] provide an example of how this process might occur within an EVT framework. They posit that help-seeking outcomes, such as soliciting help from a given source and/or utilizing help from available resources, are a product of student expectations of the source (e.g., expecting that the source will provide help) and the “value of the source,” such as the perceived accuracy and/or quality of the source. LADs have the potential to be valuable “sources” because they are adaptable in such a way that they can incorporate various sources of information. A necessary step for this, however, is for LADs to mindfully frame their designs in a manner that attends to the dimensions outlined above. Indeed, as shown in this paper, EVT is a theory designed to understand the relationships between students’ antecedent beliefs and attributions in and about various academic settings, behaviors, and outcomes. Framed in this way, the joining of these two fields provides a unique opportunity to bridge theory and practice via an instantiated design, namely, the LAD.

It should be noted that learning analytics applications can be understood as the instantiation of the goals of various stakeholders, including the institutions that implement them. Embedded in the design of a given application are various decisions regarding what sort of information may or may not help students, as well as the form that information takes. Rather than being distinct from decisions made in a one-on-one scenario (e.g., when a student meets an advisor), learning analytics applications automate this decision-making process and extend the decision-making powers of those who design them, enabling stakeholders to scale their decisions to thousands of students.

Work on motivation and LADs has shown that underserved students who are members of ethnic and racial minority groups interpret visualizations of academic information differently (e.g., [8,26,51]). Such work suggests that underserved students may have specific needs in this respect; such students may be sensitive to comparative information, though they may seek it out as well. This tension between the information underserved students are drawn to versus the information that may be most appropriate is an important one, and requires more empirical work and theorizing to determine the best approach. This is simply one example of special populations of students that might suffer unanticipated effects of LAD design, however. As has been reported elsewhere (e.g., [20], learning analytics has not focused sufficiently on students with disabilities, and it is important to examine any potential effects empirically. These two examples, taken together, suggest that there may be other blind spots worth investigating. Future work in this area would benefit from more focused investigations of specific populations.

4. Conclusions

As this paper has highlighted, LADs have the potential to help students manage university resources, adopt beneficial self-regulated learning strategies, and influence their academic motivation in adaptive ways. Such LAD designs can be improved by taking into consideration the potential benefits of information utilization in view of contemporary motivation theory and research. In this way, future learning analytics research and application can center the experience of students who are under-represented in their home institutions. Indeed, students’ academic motivation and information-seeking practices matter in all instructional contexts. The learning analytics community would benefit from further study into how LADs can provide adaptive motivational pathways. If LADs are to be designed in a way that informs students, it is essential to understand how they motivate students along many dimensions.