Changing Pro-Environmental Behavior: Evidence from (Un)Successful Intervention Studies

Human behavior is the main driver of environmental degradation and climate change [1,2]. Preserving our standard of living requires behavior change, and the most successful attempts will be informed by a solid understanding of what causes behaviors that affect the natural environment. Intervention studies can further this understanding by studying the causal determinants of pro-environmental behavior through (quasi-)experimental manipulation. Researchers across the behavioral sciences have followed this approach and tested a variety of behavior change techniques and intervention strategies [3,4,5,6]. With this Special Issue, we aim to contribute to this impressive body of evidence by inviting intervention studies that might otherwise be lost to the file drawer.

For the intervention literature to provide reliable behavior change knowledge, it needs to be largely free of systematic bias. Unfortunately, studies are more likely to enter the published literature if they yield statistically significant results, which leads to inflated effect size estimates [7,8,9]. Recent meta-analyses show that this kind of publication bias also affects intervention research in the environmental domain (e.g., [10,11]). In addition, researchers often tend to selectively report analyses that provide statistically significant results, which further contributes to the inflation of effect size estimates and the prevalence of false-positive findings in the literature [12,13].

1. Valuable Null Results

In this Special Issue, we addressed these issues by promoting the unbiased and transparent reporting of pro-environmental behavior research. We explicitly encouraged the submission of non-significant results and replication studies and ensured that no submission was rejected because of null results or the lack of subjective novelty. Submissions were rejected, however, when they contained serious methodological or reporting weaknesses that rendered those papers less informative for readers. At the editorial review stage, we encouraged authors to report all the studies they conducted on their research question, to report their analyses in an unbiased way (e.g., including supplementary tables displaying correlations between all study variables), and to make raw data openly available.

In this issue, we are publishing eleven reports of empirical studies and one systematic review that are valuable additions to an unbiased research literature (see Figure 1 for a word cloud of published articles). All of the empirical reports included at least one intervention study examining the effect of an experimental manipulation on pro-environmental behavior or closely related outcomes. In total, the authors reported 16 experiments: four of them were conducted in the field, six in the laboratory, and six online. These experiments used a variety of methodological approaches examining different intervention techniques and measuring pro-environmental behavior.

The field experiments examined the effect of real-world interventions on observations of actual behaviors of environmental relevance. For example, Ramli [14] studied the effectiveness of a feedback intervention on water use, and Goodhew and colleagues [15] tested whether inquiries about a wall insulation scheme would increase following the presentation of images visualizing heat loss from uninsulated walls. Similarly, Sloot and colleagues [16] examined which kind of appeal (financial, environmental, communal) was most effective in stimulating students to request information or to take a flyer about an environmental initiative.

Other researchers searched for ways to study pro-environmental behaviors under more controlled conditions in the laboratory. Both Hahn et al. [17] and Buttlar et al. [18] made use of a mouse-tracking paradigm to study if information-based interventions could change (ambivalent) attitudes regarding packaged food and food with expired best before dates, respectively. Moreover, Brick and Sherman [19] tested whether making choices in public vs. private laboratory settings would increase donations to environmental organizations and self-reported preferences for pro-environmental products.

Similar approaches were pursued in some of the online intervention studies. While Hallez and colleagues [20] examined if eco labels could reduce the ecological footprint of hypothetical food choices, AlDoh and colleagues [21] tested the effect of dynamic norms on individuals’ self-reported interest to reduce meat consumption. Next to these cross-sectional studies, two online studies also used a longitudinal approach, measuring pro-environmental behavior at two or more time points and administering the intervention in between. Kesenheimer and Greitemeyer [22] examined whether daily text message appeals would increase pro-environmental donations and pro-environmental behavior on a multi-item self-report scale, and Tröger and colleagues [23] investigated the effect of reflective diary-writing on self-reported sufficiency-oriented behaviors.

All but one of these 16 experiments used a between-group design to compare pro-environmental behavior across (two to five) conditions. In their systematic review on community interventions, Biglan and colleagues [24] present interrupted time-series designs as an alternative experimental approach to the study of pro-environmental behavior. Rather than being varied between groups of individuals, an intervention could be (repeatedly) introduced, removed, or changed over time to study its effect on pro-environmental behavior. Lange and colleagues [25] used a similar approach by linking the refusal of plastic bags in a takeaway restaurant to different prosocial incentives implemented across time points.

Only three of the 16 published experiments presented statistically significant results for the main intervention effect. Even when results were statistically significant, they were subject to a number of critical limitations. In our view, none of the studies provides conclusive evidence for or against the effectiveness of a particular intervention approach. However, we think that it is vitally important for the success of our research field that these results are published after a thorough quality control process (e.g., peer review). This is particularly important for designing future field studies because little may be known about logistic and design challenges that face field work. Inconclusive individual studies can be valuable building blocks for a cumulative (and eventually conclusive) research literature.

2. Accumulating Conclusive Evidence

The most relevant questions of pro-environmental behavior research cannot be conclusively addressed in a single study but rather through the systematic and unbiased accumulation of results. For many interventions, we still lack practically useful estimates of their effectiveness and reliable knowledge about the behavioral, contextual, or individual characteristics that moderate this effectiveness. If we want to know which kind of intervention works best for the promotion of which type of behavior in which population, we need extraordinarily large datasets rarely found in individual studies. However, such knowledge can be obtained by pooling data across studies (e.g., with meta-analyses). For example, by aggregating information across multiple intervention studies examining the effect of incentives on pro-environmental behavior, Maki and colleagues [26] aimed to test whether the effectiveness of incentives differed between behaviors or depended on incentive characteristics. Unfortunately, they were not able to test some of their hypotheses (e.g., regarding the differential effects of positive and negative reinforcement) due to a lack of studies in the published literature. In the context of such meta-analyses, inconclusive intervention studies (such as the ones published in the present Special Issue) can make a valuable contribution to addressing relevant research questions, no matter if they found statistically significant results themselves. Moreover, these studies provide a wealth of exploratory, secondary estimates (e.g., of the relationship between predictors), and these estimates are also valuable for future accumulation.

We suggest that intervention studies must meet several criteria to contribute to a cumulative empirical literature (see also [27]). First, they need to be free of reporting biases. If only significant or hypothesis-compatible findings find their way into the literature, any meta-analysis of that literature will be biased. With this Special Issue, we illustrate one way this criterion can be achieved, similar to the Registered Report format: by publishing studies based on their methods, not their results. Second, methods and results need to be described in a comprehensive and transparent way to allow meta-analysts to identify potentially relevant commonalties and differences between studies. In our role as guest editors, we tried to promote such reporting practices and, together with the authors and reviewers, we tried to make every report as transparent and informative as possible, including open code and data where possible. Third, methods need to be comparable to a degree that allows meaningful aggregation in meta-analyses. When reviewing the submissions to this Special Issue, we observed that different authors gave different names to similar interventions or similar names to possibly very different interventions. Standardizing the operationalization of interventions in line with, for example, established taxonomies of behavior change techniques [28] may help to build a more cumulative research literature.

In a similar vein, we think that more standardization may be helpful on the level of behavioral assessment. Many intervention studies rely on non-validated ad hoc measures for the assessment of pro-environmental behavior or its assumed antecedents [29]. Increased use of psychometrically established measures and procedures may render results more comparable and thus more valuable for a cumulative science of pro-environmental behavior change.

We heartily thank all the authors and reviewers who contributed to this Special Issue. On a personal note, we both experienced the review and revision processes as very constructive and cooperative, guided by the common goal to create the most informative empirical reports rather than chase novelty or low p-values. We hope that this Special Issue illustrates the value of inconclusive results and improves our knowledge about the causal determinants of pro-environmental behavior.