Scale Development and Validation for Psychological Reactance to Health Promotion Messages

: According to the psychological reactance theory, psychological reactance is strongly associated with many adverse outcomes of health promotion messages. This is particularly pertinent when health messages are targeting young adults, as they resist freedom-threatening messages compared to other age groups. However, previous reactance measures either relied on the open-ended thought-listing procedure, or incorporated both antecedents as well as consequences of reactance and state reactance. This study aimed to develop and validate a comprehensive scale to measure the state of psychological reactance speciﬁcally toward health promotion messages. To this end, this study was situated in the context of an anti-binge drinking intervention targeting college students. A cross-sectional survey was conducted with 203 Singaporean undergraduate students. The dataset was analyzed by exploratory factor analysis, conﬁrmatory factor analysis, and item analysis. The ﬁnal 27 items were loaded on eight factors (anger, exaggeration, design derogation, authoritative tone, ine ﬀ ectiveness, know-it-all attitude, jadedness, and source motive) that accounted for 78.53% of the variance. Each factor showed satisfactory reliability and validity (discriminant, convergent, and predictive). This study speciﬁed cognitive reactions by multiple dimensions and examined how they are intertwined with the a ﬀ ective dimension, which is represented by anger. The scale proposed herein will help researchers and practitioners develop sustainable health interventions.


Purpose of the Study
Psychological reactance is a motivational state in which people become psychologically aroused by a threat to their behavioral freedom [1]. The underlying assumption of the psychological reactance theory (PRT) is that human beings are autonomous agents who refute persuasion attempts as a default state [2]. This explains why health intervention messages, which aim to produce a beneficial behavior (e.g., anti-binge drinking), sometimes produce the exact opposite behavior (e.g., excessive drinking). Ample studies have demonstrated that health messages, particularly those targeting young adults, can elicit the negative consequences of psychological reactance by focusing-either intentionally or unintentionally-on the message features that intensify reactance arousal (e.g., controlling language, negative or loss-message framing, and using graphic images) [3][4][5][6][7][8][9][10][11][12].
Along a similar line, several researchers have attempted to measure this concept of reactance [13][14][15][16], but few of these "one-off assessments" have been subsequently validated or applied in the health

Psychological Reactance: Antecedents, Outcomes, and Measures
The four fundamental elements of PRT are freedom, threat to freedom, reactance, and the restoration of freedom [25]. Psychological reactance occurs in a two-step process, wherein individuals must first perceive that their behavioral freedom is threatened. Once they perceive that their freedom is threatened, they will sequentially enter two states: (1) the reactance state, in which they are psychologically aroused, and (2) the reactance restoration state, in which they exhibit aversive cognitive or affective responses (e.g., message derogation) and behavioral outcomes (e.g., boomerang effects such as engaging in restricted or similar behaviors).
Since Brehm [2] originally proposed psychological reactance as an immeasurable state of motivation, early PRT research assessed the outcomes of reactance rather than directly measuring reactance. However, over the past 15 years, several studies in the health communication domain have attempted to directly measure state reactance using various self-report scales [17]. While previous scales measuring state reactance are limited to one-off measurements [13][14][15][16], health intervention research has widely used the self-report measure suggested by Dillard and Shen [26].
Dillard and Shen [26] conceptualized state reactance as consisting of negative cognition and affect, which are intertwined with each other. Based on this intertwined model, studies [10,11,22,27] have assessed state reactance by measuring (1) recipients' affect on a four-item anger scale ("annoyed; aggravated; irritated; angry"), and (2) negative cognitions using a thought-listing procedure. In the thought-listing procedure, recipients are asked to list their thoughts, and independent judges code their responses as positive, negative, or neutral; the total number of negative thoughts represents the negative cognition score.
Although many researchers have consistently used Dillard and Shen's measure [26], it presents challenges that are common for measurements involving cognitive tasks, and its applicability in various health intervention contexts remains unresolved. First, the thought-listing protocols must be scored by independent judges, so this raises reliability concerns [28]; scoring the listed thoughts involves the judges' subjective evaluation on the units, dimensions, and relevance of thoughts. Second, individuals may be unwilling or unable to record all their thoughts accurately [29]. Furthermore, individuals may report fewer thoughts with this method than with other methods [30], such as comprehensive questionnaires. Third, analyzing listed thoughts can also be arduous and time-consuming, which makes replicating this method in various settings difficult for researchers and practitioners [29]. Fourth, measuring reactance as an inseparable amalgam of anger and negative cognition requires a specific analysis method (e.g., Structural Equation Modeling), which may not be applicable for pilot studies or those with a small sample size. Furthermore, it requires further conceptual refinement because several researchers [6,7,31] have actually found support that the cognitive and affective components may represent separate factors.
Consequently, some researchers recently proposed a multiple-item scale to measure state reactance in the business context [32] and on cigarette-package warnings [33]. These studies are meaningful because they called for developing a more comprehensive and practical scale of state reactance. However, both scales have concerns about construct validity, because their items appear to measure antecedents of reactance (e.g., threat to freedom) rather than reactance per se [17]. Thus, based on the extensive literature on psychological reactance against health promotion efforts, this study aims to develop and validate a comprehensive scale with which to measure state reactance. The present study differentiates between and separates the antecedents (e.g., perceived threat to freedom) or consequences (e.g., reactance restoration) of reactance and state reactance in the measurement to improve the construct validity of measuring the state of psychological reactance. With this new scale, which offers convenient utility without the thought-listing process, we expect that psychological reactance can be better assessed and managed in various health intervention contexts.

Pre-Study
As an initial stage of developing a pool of psychological reactance scale items, a traditional thought-listing technique [29] was applied to collect broad negative responses to anti-binge drinking messages. In total, 156 undergraduate students (age: M = 21.2 years, SD = 1.30; gender: female = 78%; male = 22%) at a large Singaporean university participated in this open-ended survey. The participants were asked to read two anti-binge drinking campaign brochures and freely write down their thoughts while reading the brochures. The anti-binge drinking brochures from Health Promotion Board (HPB) Singapore contained information for college students about the risk of binge-drinking and tips on how to stay in control. Participants were asked to list their thoughts in several, separate boxes provided alongside the brochures, and they were rewarded SGD 5 for their participation. Two trained coders independently reviewed the collected thoughts to sort out negative responses from positive and neutral ones. Among the collections, the wordings of frequent responses were polished and converted into a full-sentence form for potential reactance items.

Survey Procedure and Sample
In total, 203 Singaporean undergraduate students (excluding those who participated in a pre-study) participated in this survey for a monetary reward of SGD 5 (The participant-item ratio was 7.5:1, meeting the minimum standard of 5:1 recommended by Carpenter [34]). The study took place at the university's computer lab, and participants were first asked to read a written consent form briefly describing the purpose and nature (e.g., length, voluntary participation, confidentiality, potential risks and benefits of participation) of the study. After the participants signed the consent form, they were asked to read the same set of anti-binge drinking brochures from HPB Singapore in a print version and complete the questionnaire online. After completing the questionnaires, the participants were debriefed, thanked, and dismissed. Removing two cases with a missing value in the responses, a set of 201 responses was analyzed. The proportion of females (68.2%) was higher than that of males (31.3%), and their ages were in the range of 19-35 years, with an average of 22.32 years (SD = 2.19). The majority of respondents were Chinese (82.1%), followed by Indians (8.7%) and Malays (4.1%).

Instrument
The survey instrument included two major sections: (a) measures of potential reactance items after the exposure to anti-binge drinking brochures from HPB Singapore, and (b) measures of relevant concepts. First, an initial pool of psychological reactance measures was created based on the pre-study results, and from consulting the wording of questionnaires from previous PRT studies and relevant literature. Because many responses obtained in pre-study were incomplete or included multiple meanings, the statements were prepared in a more objective and clearer language for the main study. For instance, "yet another attempt by HPB-It has a lot of campaigns these days" and "feels like another attempt to manipulate young people" were reworded to the following items in the final version of the potential reactance scale: "I am bored by yet another campaign focusing on health topics (JA1)." "I feel numbed by the many attempts to educate the public about healthy living (JA2)." and "I feel jaded by yet another health campaign (JA3)." The candidate items were carefully reviewed to minimize any overlap with other relevant measures, such as those for the perceived threat to freedom, reactance trait, and boomerang effects (i.e., reactance restoration). Particularly, the current study aimed to differentiate psychological reactance from its antecedents (e.g., perceived threat to freedom) and consequences (e.g., reactance restoration); therefore, statements related to those were excluded. For example, statements relevant to perceived threat to freedom ("I am aware of my health, there is no need for someone to tell me what to do." "There is no need for HPB to worry about my health. I am in control." "I do not need the brochure to tell me what to do.") and reactance restoration ("It made me think about drinking over the weekends. It created a sudden urge." "I'm still drinking anyway." "I've never tried drinking, now it is tempting." "Drinking alcohol this weekend.") were removed from the scale. In addition, following previous studies [26,35], four items were added to measure the level of anger ("when I was reading the anti-binge drinking brochures from HPB, I felt: annoyed; aggravated; irritated; or angry.'"). As a result, a batch of 27 items (4 items from the previous anger measure and the remaining 23 items from the pre-study) was used in the survey.
In the second part, some prior scales of relevant concepts were employed-namely, cynicism (7 items), issue involvement (3 items), trait reactance (11 items), perceived threat to freedom (4 items), reactance restoration (9 items), cognitive elaboration (3 items), and behavioral intention (3 items)-to examine the discriminant, convergent, and predictive validity. They were measured on 7-point scales, except for the behavioral intention measures, which were measured on a 0-100 scale (see the Appendix A for details). Concluding the survey, respondents reported their age, gender, race, and academic status.

Item Analysis
As part of the item analysis, the 27 items were analyzed using "correlated item-total correlation" and "squared multiple correlation" (see Table 1). The former refers to the correlation between an item and the sum of the remaining items (should be ≥0. 20), and the latter is an indicator of the association between an item and the remaining set of items [36]. Thus, if the squared multiple correlation approaches a value of 1, it is perfectly predicted by the remaining items, which means that the item has no unique contribution to the scale [36]. The scores of the correlated item-total correlation were in the range of 0.233-0.644 (except for the "angry" item at 0.073), and those of the squared multiple correlation were between 0.412 and 0.893.

Reliability of the Scale
The scale's reliability was investigated using Cronbach's α for internal consistency. Overall, the alpha scores of the eight factors were 0.915, 0.882, 0.939, 0.845, 0.907, 0.794, 0.876, and 0.669, respectively. Additionally, Cronbach's α for the whole scale was 0.896, indicating satisfactory internal consistency (see Table 1). Although a slight increase (to 0.899 or 0.897) could be made by deleting one of the two items (i.e., "angry" and "I am already aware of the information given in the brochure."), they were retained because the increase was not substantial, and their removal would produce a two-item factor, which is not recommended in scale development [34,36].

Validity of the Scale
The 27-item scale was correlated with other measures to test the discriminant validity, convergent validity, and predictive validity, as shown in Table 4. First, discriminant validity denotes whether two dissimilar constructs are distinct from each other. Cynicism showed significant correlations with two factors (authoritative tone: r = 0.163, p < 0.05; jadedness: r = 0.167, p < 0.05), and issue involvement was only significantly correlated with one factor (design derogation: r = 0.172, p < 0.05), supporting that psychological reactance is generally distinct from cynicism or issue involvement.
Second, convergent validity examines whether two similar constructs correspond with each other. In the current study, the eight identified factors were correlated with prior reactance scales and relevant concepts, such as the (a) reactance trait [37,38], (b) perceived threat to freedom [26], and (c) prior scale of psychological reactance toward health messages (e.g., [39]). These measures showed significant correlations with at least three factors of the proposed psychological reactance scale. In particular, reactance trait, which is considered a personal trait rather than situational state, appeared to be associated with authoritative tone (r = 0.246, p < 0.001), know-it-all attitude (r = 0.155, p < 0.05), jadedness (r = 0.250, p < 0.001), and source motive (r = 0.205, p < 0.01) factors. The perceived threat to freedom, as an antecedent of psychological reactance, showed strong significant correlations with the authoritative tone (r = 0.672, p < 0.001), anger (r = 0.382, p < 0.001), and exaggeration (r = 0.294, p < 0.001) factors. Prior scale of psychological reactance was correlated with five of the eight factors, but not with design derogation, ineffectiveness, and know-it-all attitude factors.  Third, predictive validity was tested by checking correlations with related behaviors and other consequences, such as reactance restoration (i.e., direct, vicarious, and related motivation), cognitive elaboration, and behavioral intention. The scale was most successful at predicting the level of cognitive elaboration and behavioral intention; six out of the eight factors showed significant negative relationships with cognitive elaboration and behavioral intention, meaning that greater psychological reactance suppresses cognitive elaborations and behavioral intentions to avoid binge-drinking. Moreover, most factors were significantly associated with three reactance restorations, confirming that psychological reactance to a message engenders boomerang effects. Among the eight factors, two (design derogation and know-it-all attitude) were consistently unrelated to the three dimensions of reactance restoration.

Discussion
Since its introduction in the mid-1960s, the PRT has provided health researchers and practitioners with a ground-breaking perspective as to why persuasion attempts will more likely fail than succeed, no matter the health message's beneficial nature. The theory has helped them become mindful of how health promotion attempts could create less reactance among recipients-shedding light on the importance of language tone, message frame, and delivery modality, among others, to improve the potential of persuasion [8,13,22]. However, a common issue for reactance studies is related to the fact that administrating cognitive procedures (e.g., thought listing and subsequent coding of listed thoughts) is time-and cost-consuming and has volatile reliability, mainly due to its dependence on coder competence. Furthermore, recent studies have suggested that, given that reactance may not be an inseparable amalgamation of anger and negative cognitions, a multiple-item scale would provide great benefits for clarifying the different dimensions of psychological reactance and improving practical applications in a given context [6,7]. Accordingly, this current study aimed to develop and validate a self-report scale to measure the multiple constructs of psychological reactance to health promotion messages.
The results of the current study indicate that the final scale (27 items in eight factors) demonstrated high internal consistency and validity. The eight factors included a wide spectrum of psychological reactance from negative emotions (e.g., anger) to negative responses to message argument (e.g., exaggeration) and authoritative tone, design, effectiveness, and sources. Furthermore, an individual's jadedness and know-it-all attitude also appeared to factor into psychological reactance, probably due to the widely conducted health campaigns in Singapore.
In summary, a major contribution of this study is the identification of the multifaceted nature of psychological reactance as it relates to its cognitive dimensions. This current scale improves the construct validity of the prior scales (e.g., [32,33]), which measure antecedents of reactance (e.g., threat to freedom) as a part of reactance per se. Also, previous scales have largely amalgamated the wide spectrum of cognitive dimensions as a single dimension by mostly focusing on negative responses to authoritative tones, controlling language, or exaggeration [17]. However, the present study extended the cognitive dimensions to other realms, such as design derogation, ineffectiveness, source motive, know-it-all attitude, and jadedness. In this regard, individuals may consider themselves "a superior message evaluator" who refutes health messages by criticizing design components, arguing for the ineffectiveness and source motive, and displaying jaded and know-it-all attitudes. Presumably, when faced with persuasive attempts, people may be triggered to exercise autonomy by self-persuading themselves that they are more knowledgeable and experienced regarding the subject matter. Grounded on these findings, future studies are expected to further investigate the underlying psychological mechanisms surrounding psychological reactance.
Besides, the contextualized nature of this study allows scholars and health practitioners to understand specified reactance responses. In other words, the scale developed in this study can identify how and which dimensions of psychological reactance lead to negative consequences in a given health context. For example, items such as anger, exaggeration, and source motive may be triggered for certain health issues (e.g., vaccination and social distancing to arrest the spread of COVID-19), whereas other health issues may trigger more nuanced dimensions of reactance such as ineffectiveness, jadedness, or a know-it-all attitude (e.g., binge-drinking). The present scale allows us to differentiate the varying magnitudes among the constructs that make up the psychological reactance depending on the relevant health issue.
Another interesting finding is that the "anger" factor was significantly correlated with only one (authoritative tone) out of the seven factors. This may be in line with Rains' meta-analytic review of reactance research that found a weak correlation between anger and negative cognitions [40]. In this sense, this study offers added empirical evidence that the cognitive and affective components of psychological reactance may work independently from one another. Homing in on this finding, future studies should further examine what moderators may influence the magnitude of different dimensions of psychological reactance in health communication. Previous studies have shown how moderators play a role in the cognitive versus affective dimensions of psychological reactance. For example, Kim and Shin [6] found that the fear appeal in sexually transmitted disease (STD) prevention messages increased the level of anger arousal, whereas the message source type increased the level of negative cognitions. LaVoie et al. [7] found that different moderators, such as graphic images and individuals' traits of reactance, influenced the magnitudes of the two indicators of psychological reactance differently. To this end, an examination of how different dimensions of psychological reactance are triggered by different message factors in various health contexts would be a meaningful extension to this research area.
We empirically tested discriminant, convergent, and predictive validity by examining the relationships with a wide range of relevant concepts. Specifically, an individual's broad cynicism and issue involvement were not typically correlated with the identified factors, which supports satisfactory discriminant validity. Second, to see the convergent validity, the commonly used reactance trait scale [38] and the perceived threat to freedom scale [26] were correlated with the identified factors and showed substantial correlations. Finally, to examine the predictive validity, the items were correlated with three dimensions of reactance restoration (i.e., direct, vicarious, and related motivation), cognitive elaboration, and behavioral intention. The vast majority of the factors (5 or 6 out of 8 factors) were statistically associated with the outcome variables, confirming the sufficient predictive power of the scale proposed in the current study. In particular, the greater reactance that individuals perceived, the less likely they would be motivated to enact health-promoting behaviors. Moreover, corresponding to greater psychological reactance, reactance restoration increased and cognitive efforts decreased. Once again, the results suggest that reducing one's psychological reactance is critical to improving a health message's intended persuasive effects and minimizing the boomerang effects, which enables sustainable health interventions.
Overall, the final 27 items showed a satisfactory level of reliability and validity, but there was a little concern found over the "angry" item in the item analysis; the correlated item-total correlation score was 0.073, which did not meet the conventional 0.20 criterion. The four-item anger measures (angry, annoyed, irritated, and aggravated) have been widely accepted and applied in PRT studies. Although the current study retained the "angry" item, corresponding to previous studies [26,35], it uncovered the potential that the "angry" state was not particularly induced by health promotion messages (M = 1.73, SD = 1.10), compared to annoyed (M = 2.00, SD = 1.30), irritated (M = 1.97, SD = 1.29), and aggravated (M = 1.81, SD = 1.16). However, because the result might be due to the uniqueness of the topic (anti-binge drinking) or the population (Singaporean college students), future studies should revisit this topic.

Conclusions
Three general conclusions may be drawn from this study. First, the present study is a direct response to Rosenberg and Siegel's call that scholars need to continue developing valid reactance measures that can examine the intertwined nature of psychological reactance [17]. Through our scale development process, the present study identified additional cognitive dimensions (e.g., design derogation, ineffectiveness, source motive, know-it-all attitude, and jadedness) that contribute to reactance arousal. With these new constructs in mind, further reliability and validity studies are necessary for the scale, and its utility and theoretical applicability should also be tested in various health promotion situations.
Second, it was found that the "anger" factor was significantly correlated exclusively with "authoritative tone." This finding of the study offers additional empirical evidence that the cognitive and affective components of psychological reactance may work independently from one another and that the relationship between anger and negative cognition may be less substantial than historically thought [41]. It is crucial that future studies examine this relationship further to dissect whether the anger-negative cognition path is context-specific or influenced by additional moderators [6,7].
Lastly, we recognize that this study involved undergraduate students in Singapore, therefore limitations exist in its generalizability. To this end, future research should consider retesting this measurement using both different demographics (e.g., age, education level, health-awareness, etc.) and different health contexts to not only ensure generalizability of the scale, but to also contribute to the test-retest process of scale development to further enhance the robustness of the measurements' validity and reliability.

Conflicts of Interest:
The authors declare no conflict of interest. Table A1. Measures of relevant concepts.

Concepts
Measures Sources Cronbach's α Cynicism "People will tell a lie if they can gain by it." "People claim to have ethical standards regarding honesty and morality, but few stick to them when money is at stake." "People pretend to care more about one another than they really do." "It is pathetic to see an unselfish person in today's world because so many people take advantage of him or her." "Most people are just out for themselves." "Most people inwardly dislike putting themselves out to help other people." "Most people are not really honest by nature." [41] 0.784 Issue involvement "To me, the issue of binge-drinking is . . . " irrelevant/relevant; uninvolving/involving; unimportant/important. [42] 0.892 Trait reactance "Regulations trigger a sense of resistance in me." "I find contradicting others stimulating." "When something is prohibited, I usually think, 'That's exactly what I am going to do.'" "I consider advice from others to be an intrusion." "I become frustrated when I am unable to make free and independent decisions." "It irritates me when someone points out things which are obvious to me." "I become angry when my freedom of choice is restricted." "Advice and recommendations usually induce me to do just the opposite." "I resist the attempts of others to influence me." "It makes me angry when another person is held up as a role model for me to follow." "When someone forces me to do something, I feel like doing the opposite." [38] 0.826 Perceived threat to freedom "The message threatened their freedom to choose." "The message tries to manipulate them." "The message tries to make a decision for them." "The message tries to pressure them." [26] 0.900 Prior reactance scale "The message triggered a sense of resistance in me." "I feel the message to be an intrusion." "I feel like resisting the attempts of the message to influence me." "I found contradicting the message stimulating." "While reading the message against binge drinking, I feel like doing the opposite." "While reading the message prohibiting binge drinking, I felt like 'That's exactly what I am going to do.'" "The message against binge drinking induces me to do just the opposite." [39] 0.897 Reactance restoration "Right now, I am . . . to get drunk." (direct) "Right now, I am . . . to be around others who are drunk." (vicarious) "Right now, I am . . . to do something totally rebellious."(related) unmotivated/motivated; not encouraged/encouraged; not inspired/inspired [35,43,44] Direct: 0.982 Related: 0.985 Vicarious: 0.988 Cognitive elaboration "The amount of attention I paid to the anti-binge drinking brochure was . . . " very little/very much. "The extent to which I thought about the content of the anti-binge drinking brochure was . . . " very little/very much.
"The extent to which I concentrated on its contents was . . . " very little/very much. [26] 0.950