Increasing Personal Value Congruence in Computerized Decision Support Using System Feedback

The Theory of Universals in Values (TUV), a reliable and validated conceptualization of personal values used in psychology, is used to examine the effect of system feedback delivered by a Decision Support System (DSS) on personal values. The results indicate that value-based decision-making behavior can be influenced by DSS feedback to address value congruence in decision-making. User behavior was shown to follow the outcomes expected by operant theory when feedback was supportive and to follow the outcomes of reactance theory when feedback was challenging. This result suggests that practitioners and Information System (IS) researchers should consider user values when designing computerized decision feedback to adjust a system’s design such that the potential user backlash is avoided or congruence between organizational and personal values is achieved.


Introduction
The examination personal values influence have on Information System (IS) use and development has been studied in a variety of contexts for quite some time [1][2][3][4][5].In particular, a person's values can

OPEN ACCESS
have substantial impact on decision support and should be considered in the development of Decision Support Systems (DSS).Potential conflict could arise as organizational values may conflict with personal values.To better deal with these congruency conflicts, personal values could be incorporated into DSS through the use of targeted feedback.Feedback has been explored in great detail in the psychology, management and IS fields [6][7][8].Feedback incorporating user values in this study serve as decisional guidance as outlined by Silver [9].Thus a system can potentially influence user's decisionmaking as well as assessing the effectiveness of the DSS and their decision-making in within a value laden context [10][11][12].
As computing use has become more ubiquitous in business and personal situations, so too has decision support [13]; yet decisions and decision-making are no less complex [14][15][16].The tools in turn continue to become even more complex and difficult to use [17].Part of the difficulty in providing computerized support for value-based decision-making can arise from considering all factors that affect the decision process, which in turn affects decision-making behavior.Indeed, the DSS implemented today could have the ability to constructively influence decision-making behavior by incorporating a value aware component.
This study examines value-based decision support by using the Theory of Universals in Values (TUV) [18] to identify decision maker values and the use of targeted computerized feedback.The experiment presented here uses validated value constructs in an empirical study examining the interplay between system feedback with decision-maker values.The goal of this study is to answer the question, "how does user response to the effect of computerized decision feedback effect personal value and decision congruence?"The answer will provide a useful theoretical basis to study the incorporation of value congruence in DSS design.Understanding user response to the inclusion of value perspectives into a DSS may help alleviate system avoidance [19] as well as help developers to better customizes the DSS interface [20].While this study focuses on personal values in a low risk setting, future studies could examine the interplay of personal and organizational values, as well as potential ethical issues that could arise from value specific decision support.
The paper is organized as follows: First, a review of the relevant literature and theory is synthesized into the hypothesized effect of feedback on user decision-making behavior.Then, the methodology of the experiment and resulting analysis are presented.Finally, the discussion of results, the contributions and limitations of the study, future research directions and concluding remarks are presented.

Theoretical Development
DSS provide many benefits [21], which could increase further through the incorporation of value or ethical perspectives [22,23].The theoretical basis for incorporating user values into a DSS utilizes the Theory of Universals in Values (TUV) to identify a user's strongest value(s) then provide targeted system feedback to moderate subsequent decision behavior.The literature review presents the background for the theoretical basis of the propositions being tested.The background begins with the definition of values and how values subsequently guide behavior; the review of system feedback literature is provided.The theoretical mechanisms that potential describe decision maker behavior in response to the DSS is used to tie together the background concepts into two propositions tested in this study.

The Theory of Universals in Values
The primary component of value-based decision-making behavior is personal values.To understand the role of values in this process, we first define values in terms of the TUV and then the method by which values influence behavior is presented.
Values are defined as "trans-situational goals that serve as guiding principles in people's lives" ( [24], p. 256]).The trans-situational characteristics of values differentiate values from other beliefs, where a belief is a proposition used by a person to describe, evaluate, or prescribe/proscribe a specific action or object [25].A value's importance to a person's evaluation of the world around him/her is an integral part of an individual's personality and a factor in how a person responds to situations and the environment.
Schwartz [18] developed the TUV that includes ten value constructs (listed in Table 1) which can be measured for all individuals regardless of culture or country.The values can be presented as a circular structure along two axes (see Figure 1).Adjacent values have similar characteristics and importance to an individual, while values that are across from each other in the circle are unable to simultaneously exert strong influence on an individual.Tradition and conformity share a wedge in Figure 2 because these values share similar goals, with tradition falling further from the center because it is more resistant to change.Two dimensions are evident in this structure of values.The first highlights the difference between values aligned with "openness to change" (self-direction and stimulation) vs. those associated with "conservation" of tradition and social stability (security, conformity, and tradition).The second dimension groups the values supporting the individual or the "self-enhancement" axis vs. those values that go beyond the individual in the "self-transcendence" axis.Hedonism is denoted with a dashed line because it shares goals with the "openness to change" and "self-enhancement" axes.While the theory of universals in values proposed by Schwartz has been shown to hold across different cultures [26], the order of importance of the ten values can differ.The values have a typical order priority with benevolence, universalism, and self-direction ranking highest.Security, conformity, and achievement typically rank in the middle, while hedonism, stimulation, tradition and power are typically ranked lowest.This theory has proven robust to measurement method according to Schwartz et al. [27], since two different instruments, the Schwartz Value Survey (SVS) and Portrait Values Questionnaire (PVQ), have been used to validate the theory (each instrument has been tested on over 10,000 subjects).

Values, Attributes and Behaviors
Values can directly affect behavior without the mediating effect of attitudes.Maio et al. [28] successfully measured an increase in pro-value behavior when values were made salient through a series of questions.Verplanken and Holland [29] also found that when values were activated there was a strong direct influence on behavior.Sagie and Elizur [30] examine personal values in the context of organizational behavior.
In this study, the relationship between values and behavior are expected to follow the principle of belief congruity, which was developed and further refined by Rokeach [25,31].The principle states that individuals tend to prefer a belief (or value) system that is congruent, as measured by similarity and importance, with their own.The principle can be modified to use values in place of beliefs, since values occur as higher order beliefs ( [31], p. 282).Decision-making behavior is influenced by this desire for congruity by seeking to first achieve either similarity or importance, but ideally both.For example, consider two users who are using a decision support system to investigate possible solutions to a problem related to homelessness.A user who highly values self-achievement would likely lean towards alternatives in which the homeless are given training to work towards improving their own situation.On the other hand, a user who highly values benevolence is likely to be biased toward alternatives that incorporate more socially responsible action from the community to aid the homeless.A similar approach could be applied from an organizational perspective to achieve value congruence with organizational values.
The principle of value congruity has been empirically tested in Rokeach [25], Biernat et al. [32], and Verplanken and Holland [29].In each case there has been strong support for the value-behavior congruence.Decision makers are expected to behave according to the principle of value congruence, thus decisions will be made that align behavior with the most relevant strongly held values to the decision maker.Within the IS literature, Widmeyer [33] examined the design issue of optimization by aligning a decision maker's actions and values.The effect of various types of computer feedback on individual values has been studied in the psychology and IS literature [34][35][36].In an exploratory study, Hosack [37] found some initial support for a decision support system moderating the effect of values on decision-making behavior.

System Feedback
As computing power has become more ubiquitous and the availability of information has increased, so too has the need for decision support.Given the complexity of the decision-making environment, Silver [9] discussed the role of decisional guidance as a system component for aiding decision-making.He defined decisional guidance as "how a [decision support system] enlightens or sways its users as they structure and execute their decision-making processes" ( [9], p. 107).Parikh et al. [7] tested the effectiveness of decisional guidance, finding significant support for decisional guidance influencing the decision-making process and decision effectiveness.Feedback, in decision-making, is the process by which information on past decision results are returned to the decision maker from the environment to influence future decision strategies [6].Using decisional guidance as a framework, feedback can inform, educate, support, persuade and/or challenge the decision maker.Feedback is designed to cognitively activate thought processes in the decision maker, making values a salient and important part of the decision or decision process.Decisional guidance has been studied in the IS literature to support group decision-making [37] and to assess the influence of a query interface on outcomes [38].
Rokeach [36] studied the effects of computer feedback in the form of value self-confrontation to make subjects explicitly aware of how their individual rankings of values compared to others.The feedback was in the form of a computer printout, limited in design to printing average rankings of different groups of interest for subjects to consider and compare.Rokeach was able to show that individuals, when confronted with inconsistencies in their values compared to the values of others, made the decision to reorder their value ranks.Maio and Olson [39] challenged values as "truisms" and showed that ratings of values changed when cognitive support was provided.Maio et al. [28] were subsequently able to encourage pro-value behavior by providing reinforcing cognitive support of values.These studies show that feedback can alter an individual's behavior regarding his/her value preferences.
Cognitive feedback has been studied in IS, examining the effect on individual and group decision support [17,[40][41][42] and has been found to outperform other types of feedback under various experimental conditions.Te'eni [42] suggests feedback be considered as part of the system design.This study builds on Te'eni's suggestion by using decisional guidance as a model for empirically testing cognitive feedback, which can be operationalized as deliberate and suggestive guidance using Silver's [9] categorization.

Operant and Reactance Theory
Individuals respond to situational cues in many ways.This section discusses two theories which may explain value based decision-making behavior in response to feedback in a value-choice frame: operant and reactance theory.A decision maker's perception of feedback as supporting or challenging may create possible conflict for the decision maker which will need to be resolved.Theoretical arguments on how decision makers respond to computerized decision aids can provide insight to how systems and decision makers interact, which should be an interest for those who design systems that support decision-making.
Operant theory [43], often called operant conditioning, states that the consequences resulting from a certain behavior shape that behavior.The theory predicts behavior based on the type and the schedule of reinforcement.According to McElroy [44], the common types of reinforcement are either designed to increase the occurrence of a behavior, through positive or negative reinforcement, or discourage behavior, through extinction or punishment.The schedule of reinforcement, according to McElroy, can be of fixed or variable frequency and based on an interval of time or ratio of responses that is a function of the behavior.Finally, the number of reinforcement encounters can vary from receiving a single reinforcement to being given multiple reinforcements that continue over a period of time.Davis and Luthans [45] categorize the behavior resulting from operant conditioning as a function of the environment that provides the consequence rather than a function of the person.
Operant theory has been used in many different behavioral situations.Davis and Luthans [45] discuss components of operant theory as part of a social learning model of self-management.Rajala and Hantula [46] found support for operant conditioning explaining consumer behavior while browsing for compact disks in an on-line store.Lovata [47] discusses operant theory as a useful tool to be considered in information system design, but the theory has yet to be empirically tested in the IS literature.DSS feedback can represent punishment in the form of exception reporting, highlighting unfavorable performance and discouraging continuation of behavior and decision-making that resulted in the performance [47].
Reactance theory [48] suggests that user behavior may respond in way opposite of an intended challenge.If the challenge is perceived as constraining in nature, individuals may exhibit behavior contrary to that desired.For instance, the more a small child is told not to run out into the road, the more attractive that specific behavior becomes because the child resents the imposed limits on his/her freedom.Reactance theory in IS and decision-making literature is under-used, but it has been mentioned as a possible explanation of individual adjustment to information driven technologies [49].Within the management literature, reactance theory has been used in studies of scarce goods [50], used in studying entrapment [51], and proposed as a possible response to phantom alternatives in decision-making [52].
To illustrate reactance theory, consider the case of phantom alternatives [52].A phantom alternative is a decision solution that is known but unavailable because it is outside the bounds of the decision.For example, one type of phantom alternative is an ideal solution which is known, but not feasible at the current time.The decision maker knows of the phantom alternative which exists simultaneously with a feasible but less ideal solution.A decision maker surprised by the unavailability of the phantom alternative will react to having only the single non-ideal solution available in such a way as to attempt to reverse the decision process.The decision maker tries to remove him/herself from the bottleneck situation in which only one alternative is available.This sudden and uncharacteristic reversal is intended to reestablish the ability to choose between multiple alternatives as predicted by reactance theory, when the perceived freedom of choice is removed.
Decision maker behavior is expected to respond to feedback following either operant and/or reactance theory because both theories have been shown to have validity in previous research [53].The study is designed to contribute to the literature by illustrating how operant and reactance theory can be used to explain how a decision maker's behavior changes in response to the system.

Research Proposition
The following proposition is based on the ability to effect change in decision maker behavior for subsequent decisions (i.e., after the initial decision) in response to feedback.Values based on the literature analyzed here are expected to influence behavior in a manner congruent with the amount of cognitive support activated for a decision is based on the decision maker's values.Decisional guidance in the form of feedback, as discussed above, serves as a trigger to make the decision maker's values a more salient part of the decision process.Feedback could either support or challenge a user's value congruence.Supporting feedback was expected to increase value congruent behavior; whereas, challenging feedback was expected to decrease value congruent behavior following operant theory, while no discernable change in decision maker behavior was expected for the control or neutral feedback treatment.Therefore, cognitive system feedback was proposed to affect decision-making behavior.
System feedback will influence decision-making behavior in two ways: (1) Supporting system feedback will increase value congruence (2) Challenging system feedback will decrease value congruence Decision behavior is measured at multiple points in the study, leading to some of the proposed effects occurring at specific times.

Experimental Methodology
The web-based system in this study supports a fund allocation task that provides value specific feedback to decision makers.The "foundation" task was chosen because it provides value driven decisions, requires no previous experience and has been proven to work with students [34,54].For the purpose of this study, the task was modified to work with the TUV values through multiple rounds of pilot testing.The 30 items were identified that represented three sets of ten organizations that each aligned with a value in the TUV.The task was completed three times each with a different set of ten organizations, with feedback provided between Decision 1 and Decision 2, and between Decision 2 and Decision 3. The following directions were provided for the task: You have $100,000 to allocate to the organizations below.Each program is in need of the whole $100,000 but each can benefit from any contribution that you might make.The greater the contribution that you make to a particular program, the more likely it is that the chosen program will succeed.Enter the amount you want to give to the program or programs of your choice in the box to the right of the program description.Remember, you must allocate the full $100,000.If you need to know how much you have left to allocate, click on the "Show me my allocation total" button.When you are finished allocating the funds, click on the "Continue" button.
The DSS was developed to administer the survey instruments, supports the execution of the task and stores the collected data in a database with system feedback serving as decisional guidance after each decision.Each of the decision points contained a new set of allocations that were similar (i.e., aligning with a single value) but different.Table 2 lists ten example organizations used in the study with the corresponding value while Figure 2 shows an example of a data collection page and the feedback delivery screen.Two pilot studies were conducted to test the system functions, ensure the system administered the feedback treatments as planned, and accurately calculated and stored all measures collected in the study.Revisions were made to the system design after each pilot to resolve minor programming issues and to clarify procedures, but no changes with regard to the design of the research model were made.

Value Measurement
The Schwartz Value Survey (SVS) [18] and Portrait Value Questionnaire (PVQ) [27] were considered as alternative measurement methods in this study because they remove the complexity of ranking values by using a Likert based scale and both measure the constructs in the Theory of Universals in Values (TUV), building on previous value theory and instruments.Ultimately, however, the PVQ was chosen because it is one of the most current and straightforward instruments to administer.
Schwartz et al. [27] developed the PVQ to minimize the difficulty subjects had in evaluating the items associated with the SVS measurement of the TUV and to make the questionnaire further accessible for individuals across diverse cultures.The PVQ uses 40 third person "portraits" to target ten value constructs described by Schwartz [18].Each portrait item is a description of an individual that embodies a particular aspect of a value and loads on that value construct.Subjects rate the relevance of the portrait on a six point scale from "very much like me" to "not like me at all."The methodology of using third person statements is uncommon but instrument has been extensively tested and validated [24,26].
A decision maker's value profile taken from the PVQ was used to determine the decision-maker's strongest value or values.These values became target of the decision feedback.A decision-maker's values themselves were not the focus of this study, only how the decision-maker's behavior changed given different types of feedback based on their choices.Thus, the study tested behavior as a function of the principle of value congruity, where values directly influenced decisions in a manner where decisions are congruent with the decision maker's values.Thus, the IS literature is expanded through the examination of the human-computer interaction between a decision maker's values and a computer system designed to explicitly considered those values.

Feedback Treatment
Based on the subject's location in a specific treatment group, he/she received neutral message box to continue with "no feedback"), supporting, or challenging feedback.The feedback followed the subject's first and second execution of the allocation task and corresponded to his/her strongly held values.In the terms of operant theory, feedback followed a fixed schedule presenting multiple reinforcements based on the user responses to the task.The challenging feedback below for a decision maker whose strongest values were hedonism and stimulation used the opposing value construct (conformity and tradition, see Figure 1) to challenge the decision maker's previous value congruent decision.System generated items are indicated by italics based on the decision maker's value profile and previous allocation information: Based on your preferences that you provided to the system earlier you are interested in pleasing experiences for yourself and excitement and challenges in life.You allocated the most funds in support of Organization 3 and Organization 7.But you really need to consider providing more support for other organization(s) such as Organization 2 and Organization 8 in this next group, because they enforce society's expectations and/or stress the importance of culture and tradition.With this in mind, you may continue to the next page by pressing "Continue".
The following supporting feedback would be given to subjects who exhibited value congruent behavior.It was designed to encourage the decision maker to continue allocating the most money to the corresponding organization(s).Again, for a decision maker whose strongest values were hedonism and stimulation: Based on your preferences that you provided to the system earlier, you are interested in pleasing experiences for yourself and excitement and challenges in life.You allocated the most funds in support of Organization 3 and Organization 7. Keep up the good work and remember the more you allocate to organizations, such as Organization 3 and Organization 7 in this next group, the better the fit with your preferences.With this in mind, you may continue to the next page by pressing "Continue".
In the case of a subject having multiple strongly held values (i.e., tie scores on the PVQ, which occurred only 22% of the time), up to three organizations could be referenced in the feedback.
The feedback was designed to incorporate whether a subject made a value congruent decision.A value congruent decision occurred when the subject allocated the greatest amount of money to the organization whose description corresponded to the subject's strongest held value.For example, if a decision maker scored achievement as their strongest value and allocated the highest sum (i.e., $35,000) to the organization that corresponded to the achievement value then the decision maker was deemed to be behaving in a value congruent manner.In the case where a decision maker did not make a value congruent decision, the feedback was dynamically designed to encourage the subject to make a value congruent decision.This feedback was supporting or challenging depending on which treatment the subject was assigned.If the subject allocated the money in a tie, the decision was counted as value congruent as long as one organization was in alignment with one of the decision maker's strongest values.Feedback of similar structure has been used successfully in Wolford and Goodwin [55], and was modified in this study to use the supporting/challenging approach developed by Maio and Olson [39] and Maio et al. [28].The neutral feedback was value free and asked the participant to simply continue to the next allocation task.
The effect of feedback was measured using the change in magnitude of the dollar allocations between Decision 1 and Decision 2, and between Decision 2 and Decision 3 for the targeted value regardless of allocation size.For example if an amount was allocated to the organization which aligned with the decision maker's strongest value but the amount was not the largest, the magnitude would still be calculated and used to measure the effect whether it was positive, zero or negative.Thus, within the experiment the challenging, neutral, and supporting feedback types were tested.Subsequent analyses were also conducted on the value congruent vs. value non-congruent supporting and challenging feedback effects.

Procedure
Data was gathered by the system at six points in this study: during administration of the demographics form, the PVQ instrument, the three allocation tasks, and the posttest questionnaire.The decisions were compared using the change in the dollar amount allocated as a measure of the difference magnitude, not based on value congruence or dollars allocated.Thus, we could compare the difference of $0 to $10,000 with a change of $20,000 to $17,000 across subjects' decisions and see the direction and size of the change rather than comparing the actual allocations in the second decision of $10,000 to $17,000.
Subjects first completed the demographic form and created a subject ID.The consent form was available on the first page, and by submitting the demographic information the subjects consented to participation.The forty question PVQ instrument was completed next, and subjects were placed into treatments, where they allocated funds in the first task.Upon completing the first task the subjects received the appropriate treatment feedback and went on to the second task.This decision-feedback cycle repeated, and after the third decision task the post experimental debriefing questionnaire was completed to check the feedback manipulation and gather the subject's perceptions of the study.Questions were designed to determine what knowledge the subject had of the experiment's goals and whether they received supporting or challenging feedback.Finally, subjects were asked to what degree the feedback affected their decision process, and if it did not affect them, why they felt it was not influential.

Participants
One hundred and seventy student subjects from two southern United States universities played the role of decision maker successfully completed the study.Subjects were randomly selected into the Subjects from the larger university represented 58% of the sample with the remaining 42% of the sample coming from the second school.Subjects ranged in ages from 18 to 22 (90%), 8% were 23-30, and the remaining 2% were 31-40.Fifty-two percent of the subjects were female and 48% male.The reporting of ethnic origins was optional and 61% percent of 163 who reported subjects were Caucasian, 20% African-American, 10% Hispanic, 3% Asian, and 6% fell into the other category.Sixty percent of the decision makers were from various non-Management Information Systems Business majors, 5% were MIS majors, 3% were Liberal Arts majors, and 32% were from other university majors or undecided.The average time for completion was approximately 23.5 min with the shortest completion time at just under nine minutes and the longest at a little over 1.5 h.As an incentive, subjects were offered extra credit for their participation, and each subject who successfully completed the experiment was entered in a raffle to win one of six $30 gift certificates to Amazon.com.
A subject's strongest value(s) was selected based on the rank ordering of the mean value score.The use of rank ordering means for determining the importance of a value has been used in psychology and IS research [26,29,35,53,54,56].The distribution of values among the ten value constructs, indicate that only 38 out of the 170 subjects had two or more values tied and only eleven out of the 38 had three tied values.The tradition value was the only value not represented in this study as a strongest value.The MicrOsiris Smallest Space Analysis (SSA), a nonparametric test, was used to examine the value profile of the research subjects.The average centered score for each value was calculated, using the overall item average subtracted from an individual PVQ item average.The data indicated that the distinct value groupings of the TUV appear consistent for the decision-makers in this study, yielding a circular structure as seen Figure 3.This structure is the same as that identified by Schwartz [18].Table 3 contains the number of subjects and the descriptive statistics for each experimental treatment.

Experimental Results
The first test conducted on the feedback treatments was a Student t-test used to determine if there was a significant difference between the changes in monetary allocations for the neutral feedback treatment.The test was not significant (p = 0.222), which is consistent with not influencing the control subjects decision.Caution must always be taken when accepting a failure to reject is not equivalent to proving validity.
A one-way Analysis of Variance (ANOVA) model was run on the single factor, feedback treatment, when comparing the change in allocation behavior of each decision to determine if there was a significant effect (see Table 4).The model for the first change in decision behavior between Decision 1 and Decision 2 was not significant (p = 0.293), but the model for the second change in decision behavior between Decision 2 and Decision 3 was significant (p = 0.001).A preliminary indication that feedback became more effective as the decision process progressed, which lead to the post hoc analysis presented below.Paired comparisons were computed after the indication of significant effects, again comparing the specific feedback treatment between two decision points.Scheffe's and Fisher's Least Significant Difference (LSD) were used to compare each feedback treatment.The paired comparisons for the second feedback point are presented in Table 5 for change in decision behavior between Decision 2 and Decision 3.For the change between Decision 2 and Decision 3, there was a significant difference at a p < 0.05 between the challenging (CHL) feedback and the supporting (SUP) feedback while the neutral feedback (NEU) treatment was significant at p < 0.10 using Scheffe's method of comparison.The challenging feedback treatment created a change that was less than that of the supportive feedback treatment.Using the less conservative LSD method, the challenging feedback effect was significantly less at p < 0.05 level than both the neutral feedback and supportive feedback treatment effects and the neutral was significantly less than the supportive feedback at a p < 0.10 level.The findings for the change between Decision 2 and Decision 3 provide support that challenging feedback induced behavior which decreased allocations, supporting Proposition 2. While no significant support was found for Proposition 1, the supporting feedback produced an increase in behavior larger than the other two treatments.The change in mean allocation between Decision 1 and Decision 2, and Decision 2 and Decision 3 can be seen in Figure 4.

Post Hoc Analysis
The system feedback allowed it to adapt to the decision maker's allocation when a decision was not value congruent, allowing the decisions to be analyzed further for each specific instance of feedback and whether or not a decision maker was behaving congruent with his/her value profile.As noted above, if a decision maker did not make a value congruent decision then feedback was offered to encourage a more value congruent behavior (i.e., allocating the most money to an organization that aligns with the decision maker's strongest value) and this feedback was either supporting or challenging feedback based on treatment.Thus, an analysis could be conducted examining the effect of Value Congruent (VC) vs. Value Non-Congruent (NC) feedback of the Supporting and Challenging type.The number of subjects receiving a specific type of feedback (VC Supporting (SUP) or Challenging (CHL), NC Supporting (SUP) or Challenging (CHL) or Neutral Feedback (NEU)) is presented in Table 6.The NC feedback treatments and the VC Supporting feedback were all phrased to increase allocations to the organization which aligned with the a subject's strongest held value, while the challenging value congruent feedback was phrased to decrease allocations to a subject's strongest value aligned organization.The number of subjects making a value congruent decision could change from decision to decision; therefore, the specific feedback received would change to account for this difference.Thus, two separate ANOVA models were run to test the effect of value congruent and non-congruent feedback on decision-making behavior.Table 7 shows both decision ANOVA models were significant at an alpha of 0.05.Each model had a significant Levene's statistic (p = 0.001), indicating that the models did not have homogenous variances but both the Welch and Brown-Forsythe tests were significant (p < 0.01 across the two models).Model 1 analyzed the dependent variable change in decision-making behavior between Decision 1 and Decision 2. Model 2 analyzed the dependent variable change in decision-making behavior between Decision 2 and Decision 3. Feedback was further explored using Scheffe's test and Fisher's LSD test to conduct post hoc analyses.Table 8 includes the significant results for both paired comparison tests.For the change in allocation amount between Decision 1 and Decision 2, there was a significant difference between the challenging value congruent feedback and the neutral feedback and the supporting value-congruent feedback using Scheffe's comparison at the p < 0.05 level.Additionally, Fisher's LSD indicates a significant difference at the p < 0.05 level between supporting value congruent feedback and supporting non-congruent feedback, supporting non-congruent feedback and challenging value-congruent feedback, and challenging non-congruent feedback and challenging value-congruent feedback.The supporting non-congruent feedback was different from the neutral feedback at a p < 0.10 level.The change in allocation amounts between Decision 2 and Decision 3 exhibits a stronger response to the feedback.Using Scheffe's test, there was a significant difference at the p < 0.05 level between the supporting value congruent feedback and challenging non-congruent feedback as well as challenging value-congruent.For the supporting non-congruent feedback, there was a significant difference at the p < 0.05 level with the challenging non-congruent feedback and with the challenging value congruent feedback.The challenging non-congruent feedback differed at the p < 0.05 level from the neutral feedback and the neutral feedback differed from the challenging value-congruent feedback with a. Fisher's LSD, for the second set of comparisons, indicates an additional difference at the p < 0.05 level between the supporting value-congruent feedback and neutral feedback.
To summarize, the challenging value congruent feedback was significantly different from all other feedback treatments (except supporting value congruent feedback) for the change in decision-making behavior between Decision 1 and Decision 2, but its influence was the opposite of the hypothesized direction.For the change in behavior between Decision 2 and Decision 3, the supporting value congruent feedback treatment was significantly greater than all other treatments.The two non-congruent treatments indicated some support of significant difference from the other treatments as well, though in each case the tendency was opposite from the hypothesized direction.The difference in allocations due to value congruent and non-congruent feedback is shown in Figure 5. Two ANCOVA models were run to test the effect of the Attitude and Demographic variables on this post hoc analysis.However, none of these variables were found to be statistically significant.Table 9 presents the results of the hypothesis testing.

Discussion
The results of the study indicate feedback had a significant effect on decision-making behavior and the TUV adequately represents the values of DSS users.Feedback, regardless of congruence, had a significant effect in changing decision-making behavior between Decision 2 and Decision 3.This analysis indicated feedback was not successful in changing decision behavior between the first two decisions, but became more effective over time.This result could be due to the decision makers becoming more aware of and responsive to the feedback through repeated exposure.The post hoc analysis of the feedback partially supports this finding with regard to supporting feedback, but further research would be necessary to confirm this finding.
When the different treatments were compared for the change in behavior between Decision 2 and Decision 3, the effect of supporting feedback was significantly greater than that of the challenging feedback using Scheffe's comparison (see Table 5).The more liberal Fisher's LSD showed the effect of challenging feedback as being significantly less than the neutral feedback treatment and the supporting feedback being moderately greater than the neutral feedback treatment (see Table 5).The results indicate support for the hypothesis that supporting feedback would increase allocations for a decision maker's strongest held value, while allocations were typically less for the challenging feedback treatment.Figure 4 illustrates the differences between the treatments across the changes in decision behavior.It is also interesting to note the reversal in the change in behavior.While the allocation changes were not significant for the change in behavior between Decision 1 and Decision 2, a decrease in the challenging feedback treatment and an increase in the supporting feedback treatment are seen in Figure 4.
Supporting feedback appears to engage behavior that follows the predictions of operant theory.It acts as a significant positive reinforcement, encouraging the decision maker to continue the desired behavior-in this case increasing value congruent allocations between Decision 2 and Decision 3. It is evident that time may play a role in how effective feedback is on decision-making behavior because we observed a delay where feedback did not affect behavior until the second treatment.Predicting the effect of the challenging feedback is more difficult, because it seems to follow both operant and reactance theory in changing decision-making behavior.In the first change in allocation behavior, the challenging feedback shows a non-significant increase, which could be due to a negative reinforcement effect based on operant theory, or reactance behavior in response to the feedback as reactance theory would predict.In subsequent allocation behavior there is a significant decrease when using Fisher's LSD comparison, which may indicate that the challenging feedback could be perceived as a punishment under operant theory, thus discouraging behavior.Regardless, feedback was significantly effective in changing behavior.
The strong feedback effect in the model becomes especially evident in the post hoc analysis where the impact of feedback is very significant for both the change between Decision 1 and Decision 2, and between Decision 2 and Decision 3 (see Table 7).The effect of the value congruent challenging feedback treatment was in the opposite direction of the expected effect.The feedback was designed to decrease value congruent behavior, but two thirds of the challenging value congruent feedback subjects exhibited an increased congruency effect.The increase was significantly greater than all feedback types, except the supporting value congruent feedback treatment that was designed to increase value congruent allocations.This result seems to indicate behavior that is consistent with the predictions of reactance theory, since subjects are reacting in a direction opposite of that presented by the feedback.In contrast, the supporting value congruent feedback created the expected change in behavior, according to the predictions of operant theory with a positive increase in value congruent allocations.
Following this trend in positive increase, supporting value congruent feedback significantly increased the change in behavior between Decision 2 and Decision 3 consistent with the behavior predicted by operant theory for a positive reinforcement.The magnitude of change in aligned allocations across changes in decision behavior was stronger for the value congruent treatments (see Figure 5).However, the non-congruent treatments were not effective in generating the expected increase in value congruent behavior as expected.The design of these treatments may have been perceived as trying to force the decision maker along a single decision path in a way that seemed limiting and thus triggered a reactance response.
As the allocations changed from Decision 2 to Decision 3, an interesting change occurred where the positive reinforcement of the supporting value congruent feedback became a significant effect in increasing value congruent behavior.The supporting value congruent feedback's positive reinforcement significantly differed from all feedback treatments using Scheffe's comparison.This indicates that positive reinforcement substantially increased allocations for value aligned organizations corresponding to the decision maker's strongest value.Challenging value congruent feedback generated significant behavior in the opposite direction when using Fisher's LSD comparison, which again indicates decision maker reactance taking place in response to feedback.The effect is weaker, however, and less conservative methods have to be used when analyzing the data.A trend is also noted as the supporting non-congruent treatment has a positive increase.Though small and insignificant, it is nonetheless in the expected direction.In contrast, the challenging non-congruent feedback moves significantly further in the direction opposite of that expected.Future studies are necessary to determine if and how the predictive power of operant and reactance theories with regard to feedback can erode or strengthen over time.Comparing the change in behavior in Figure 5 illustrates the gradual fluctuation of allocation behavior in response to feedback type.The decrease in reactance continued for the challenging feedback, possibly causing subjects to "give-up" in response to repeated exposure to a behavior suppressive mechanism in a manner consistent with the findings of [48].In general, the results indicate that feedback significantly affects behavior, and even more so when examined in conjunction with value congruence, making these factors important to consider when designing systems that use feedback to support decision-making.Furthermore, operant and reactance theory are both useful in explaining how decision makers will respond to feedback.
The debriefing questionnaire provided an additional perspective on the perceptions given by a subject's response to feedback.For example, one subject states:

It [the feedback] didn't change my mind at all. I want what i [sic] want.
However, this subject responded by increasing his/her allocation in a manner consistent with the supporting value congruent feedback that was significant for Decision 2 and Decision 3. Additionally, comments provided by the subjects in the experiment further support this idea.For example, one subject suggested that considering values as part of the decision process allowed him/her to evaluate the resulting decision-making behavior: ... I realized that because of my preferences, I chose some options over others that were supported by different preferences.
Or, according to another subject, at the very least the feedback caused the decision maker to pause and consider other alternatives:

Not much [effect], the first feedback made me read the next recommended organization's description longer, but didn't convince me to give them money.
The debriefing questionnaire further illustrates the predictive usefulness of operant and reactance theory through subjects' perceptions of the feedback.One subject in the challenging feedback treatment indicating his/her displeasure by commenting: The feedback was pretty broad, I don't like how it told me which organization in the next set of charities I should give the most money to!Another subject (in the same treatment) expressed his/her feelings on the challenging feedback more succinctly:

[The feedback was] CRAPPY!
The results show that feedback was effective in altering decision-making behavior, and the perceptions of the subjects lend additional support to these findings.
This study was effective in answering the research question, "how does user response to the effect of computerized decision feedback effect personal value and decision congruence?" User's values followed the structure seen in the TUV.While rotated (as is common across different groups and cultures), the circular structure was maintained and the value order was intact.Decision makers were influenced by the feedback that targeted their strongest values.When decision makers were supported by feedback they continued making decisions congruent with their values illustrating that operant theory can predict decision-making behavior in response to feedback.However, when system feedback was not supportive decision makers reacted in a manner opposite of that proposed by the feedback indicating the predictive power of reactance theory.Indicating, possible congruency impacts to be considered.

Research Limitations and Contributions
This study was successful in its primary goal of measuring a significant feedback effect on value congruence.However, only 45% of the decision makers made one or more value congruent decisions, lending only partial support to the principle of value congruence as a mechanism for decision behavior.Yet the DSS was effective in helping guide decision makers towards more value congruent decisions via feedback.
This result could in part be attributed to the task items.While the items used in the study matched the value constructs in pilot tests, they could need further refinement to better align with the value constructs across different demographics of subjects.Likewise, the operationalization of the experimental environment may need further testing to ensure it correctly puts enough restrictive pressure on the decision makers to create an influential magnitude of consequence.Regardless of the limitations, the feedback treatment was successful in overcoming these potential obstacles.This study's focus on values and the use of a system that supports a personal value engaging task minimizes the concern for using student subjects, as all people possess values.The study targeted a population of southern U. S. college students, but increased external validity by incorporating students from two universities, representing multiple majors and grade classes.
While the effectiveness of feedback was established, further testing and replication of this experiment in other areas of design are necessary.The role feedback plays in targeting a decision maker's values can be an effective tool in improving decision-making.Dynamic cognitive feedback offers an alternative for system designers looking to provide guidance in complex decision environments.This study expanded on previous research aimed at considering values as part of computerized decision support.The study provides additional findings that support the use of operant and reactance theory when interpreting decision maker behavior in response to feedback.This research makes a contribution to the design of feedback in cases where decision makers' values, specifically their strongest value, are a part of the decision process.
This research makes a contribution to practice by testing the design of a system component that incorporates consideration of personal values in aiding a decision maker.It also provides explanatory mechanisms, by using the guidelines specified in operant and reactance theory, to help practitioners design feedback that can be effective in particular situations.By being aware of reactance, the practitioner can design and test for alternative approaches to overcome it.Similarly, the use of supporting feedback can be effective when positive reinforcement is needed.The PVQ provides a tool for practitioners to measure a broad umbrella of values that can be incorporated into a system's design.Issues such as privacy and accessibility fall within the value constructs measured by this theory.The value preferences of decision makers can be readily measured via the PVQ, incorporated into, and supported by a system.
This experiment also has implications for researchers.The TUV and the accompanying PVQ instrument were used for the first time in an IS study.By establishing the TUV as the standard definition of value constructs in IS research, future studies can be conducted and compared to draw connections across seemingly different streams of research, providing new insight and directions for further exploration.The broad characteristics measured by the PVQ and the TUV can be incorporated into a broad range of IS research, such as value sensitive design and other human-computer interaction applications.By using the TUV as common base for IS values research, studies examining a variety of IS issues can be compared and revealing new relationships and directions for research.Revising and updating the classic foundation task to work with the PVQ will ensure that this validated task will continue to be useful by using a current value instrument.

Future Research and Concluding Remarks
This research has extended the existing body of IS literature and offers several directions for future research.A longitudinal study of dynamic feedback would provide insight on how system feedback is incorporated into the decision process.The shift evident in Figure 5 indicates a longitudinal effect that could be taking place.While not significant, there was a larger magnitude of change for the feedback treatments for the second change in allocations when compared with the first.In addition to this trend, the responsiveness of low self-monitors to the system feedback could be explored further to examine how feedback exerts influence over time.Such a study would provide the next step in this stream of research, expanding on how feedback becomes a necessary or unnecessary guidance tool with continued use.The TUV could serve as a common foundation for the study of the effect of personal values in IS research, allowing for comparisons to be made across seemingly unrelated topics and to provide a common starting point in new areas of research on values in IS.
Another study could examine how to minimize the reactance of decision makers in response to feedback that does not necessarily tell them what they want to hear.Reactance must be accounted for in system design, as the tendency for most is to oppose conventions that challenge one's desired course of action.Leading to the question of how robust are particular values in response to feedback.For example, would we find that the tradition value is less likely to change than the hedonism value?If so, the research question becomes "Can a system be designed to target specific values in the decision process?"Finally, a study on the role values play in the design of systems and how best to incorporate and align user, designer, and organizational values is a necessary next step in this stream of research to provide a broader picture of how values shape this process.By conducting an analysis of how values affect all the stakeholders involved in the design of a system, an understanding of where values most influence the design process and how to best make values an explicit part of system design.
In conclusion, this experiment was successful in measuring the effect of feedback on value-based decision-making behavior.The contributions offer tangible benefits to practitioners and researchers interested in the impact of feedback and values on system design.The explanatory mechanisms of operant and reactance theory can be useful in guiding both practitioners and researchers who are creating systems that support human-computer interaction.The study also tested and validated instruments which can be used in future studies and to measure decision maker preferences.This study addressed an opportunity to design a system component that considers the value preferences of decision makers as part of the decision process and indicating that cognitive feedback can be an effective part of system design when trying to achieve value congruency.

Figure 4 .
Figure 4.The Effect of Feedback Type on Allocation Behavior between Decisions.

Figure 5 .
Figure 5.The Effect of Feedback and Value Congruency on Allocation Behavior between Decisions.
UNIVERSALISMUnderstanding, appreciation, tolerance and protection for the welfare of all people and for nature.(broadminded,wisdom,social justice, equality, a world at peace, a world of beauty, unity with nature, protecting the environment)BENEVOLENCEPreservation and enhancement of the welfare of people with whom one is in frequent personal contact.(helpful, honest, forgiving, loyal, responsible)

Table 2 .
Example Allocation by Value.

Table 4 .
One-Way ANOVA of Feedback Type.

Table 5 .
Significant Paired Comparisons for Feedback Type.
Note: Change in Behavior between Decision 1 and Decision 2 had no significant paired comparisons; (*) Statistically significant at the p < 0.05; (+) Statistically significant at the p < 0.10.challenging = CHL; neutral feedback = NEU; Least Significant Difference = LSD.

Table 6 .
Subjects per Feedback Treatment.

Decision 2 N Decision 2-Decision 3 N
Note: VC = Value Congruent and NC = Value Non-Congruent feedback.

Table 7 .
One-Way ANOVA of Feedback Type Considering Value Congruence.

Table 8 .
Significant Paired Comparisons for Change between Decisions by Value Congruency or Non-Congruency.

Table 9 .
Expected Feedback Effect.