Online Engagement with Memes and Comments about Climate Change

: Social media posts, and memes in particular, offer important opportunities for social media users and organizations to disseminate information about climate change. However, as this topic remains controversial, memes often elicit comments that may oppose (rather than support) the existence of climate change. In three studies, we ﬁnd that the position of the comments inﬂuences users’ engagement with the main post: when the user’s and the meme’s positions on climate change align, comments opposing the claim of the meme decrease users’ readiness to “like” the meme. We also examine social media users’ attitudes toward different comment moderating options, including disabling, deleting, hiding, or responding to comments.


Introduction
Climate change is arguably the greatest ongoing global challenge we face, with implications for public health, food, and safety [1,2] for both developed and developing nations [3,4]. Scientists and public policy experts agree it is critical to cultivate consensus around this topic to adopt the best solutions that can curb, and potentially reverse, the effects of climate change [5]. Thus, platforms that allow information to spread quickly and reach many people are especially important to use to create awareness and motivate proclimate change activism. As a result, social media platforms-such as Facebook, YouTube, TikTok, and Twitter-are often used by individuals, companies, advocacy groups, and other organizations to engage with the public on topics related to climate change [6,7]. Social media content is carefully created to encourage engagement [8]-in the form of "likes," "shares," comments, etc.-which both increases social media users' involvement with the post and raises the virality of the content, thereby reaching a larger audience. A commonly used type of social media posts-memes, which typically represent memorable images with superimposed phrases-have the potential to spread quickly and widely online, making them a particularly useful means of communication about climate change, and the main focus of this paper.
Despite the consensus about climate change in the scientific community, the public holds divergent views about the existence, causes, and impacts of climate change [9]. As a result, sharing information about it on social media-by, for instance, posting a meme-is likely to attract comments supporting as well as opposing the existence of climate change. The latter type of comments may undermine the original post's credibility, spread misinformation about the topic, or personally attack the poster [10], which may in turn demotivate individual efforts as well as at-large support for necessary climate change action.
In this paper, we focus on social media content regarding climate change and examine how the position of the meme (i.e., supporting vs. opposing climate change) and the position of the comments following the meme affect social media engagement in terms of "likes" and "shares" of the content. Unlike prior work on social media, which has largely viewed posts and comments as two separate elements of communication and which has focused almost exclusively on studying characteristics related to the main post, we suggest that the message communicated in the main post and the accompanying comments interact, particularly in the context of controversial topics such as climate change. Understanding the dynamics between these two elements of a social media post is important because comments are a hallmark feature of social media, and comment moderation strategies are a major concern for social media managers. We contribute to the literature by theorizing and demonstrating in three studies that comments opposing the claim of the meme decrease users' readiness to "like" the meme, when the user's and the meme's positions on climate change align. We further show that this effect holds not only in samples that include mostly supporters of climate change (i.e., social media users who believe climate change is real), but also in a sample that involves a balanced proportion of supporters and opponents (i.e., social media users who do not believe climate change is real). The results suggest that opposing comments can pose a significant problem for social media users and organizations trying to disseminate pro-climate change information. And while this result may well apply to posts regarding other controversial topics such as gun control or abortion, it is particularly concerning for climate change given its broad societal implications and urgent need for action. To provide further insights regarding how pro-climate change organizations should handle opposing comments, we also examine users' attitudes towards different techniques for moderating comments and conclude with practical implications and suggestions for future research.

Climate Change and Public Opinion
Climate change generally refers to long-term shifts in temperature and weather patterns [11]. In common usage, it references shifts in Earth's climate since the pre-industrial period due to human activities [12]. Although the scientific community largely agrees on the severity and consequences of climate change [13], the general public holds divergent opinions. For instance, while most American adults (72%) believe global warming is happening, a much smaller percentage of the population is convinced it is mostly caused by human activities (57%) or that it will harm them personally (47%; [14]). These data speak to the controversial nature of climate change, which is further exacerbated by political views [15], with liberals reporting stronger pro-climate change beliefs. However, experts suggest that "a massive audience speaking in one voice" can help establish the importance of this issue and prioritize actions aimed at reversing the detrimental effects of climate change [5,16].
Social media platforms can play a pivotal role in influencing public opinion about climate change due to the platforms' popularity and widespread accessibility. The Pew Research Center (2021; [17]), for example, estimates that over 80% of Americans aged 18-49 use at least one social media platform. As a result, social media users and organizations use these platforms to disseminate information, build communities, and expand their network of followers and activists [18,19]. In fact, the importance of social media continues to grow as younger generations embrace it as a tool for climate change activism. For example, recent polls indicate that 56% of Gen Z (individuals aged 7-22) report having seen content on social media about the need for climate action within the past few weeks, and 45% report having engaged (e.g., by liking, sharing, etc.) with such content [20].
But while pro-climate change activists use social media to disseminate information, so do users and organizations that oppose climate change. In fact, misinformation about this issue is rampant on social media [21] and often spreads more widely than does scientifically accurate information, especially when social media companies do not aggressively fact check climate change related posts [22]. Further, the "social" aspect of social media allows different users-both supporters and opponents of climate change-to respond to posted content, thereby further increasing the polarization of the issue. Thus, it is important to understand what influences a social media user's propensity to engage (e.g., "like", "share") with content about climate change. We examine this question next.

Engaging with Climate Change Information on Social Media
Social media platforms allow users and organizations to post content, with which others can engage by posting a reaction (e.g., likes, ratings), commenting (e.g., Facebook comments, Twitter replies), sharing it with others (e.g., Facebook shares, Twitter retweets), and posting user-generated content (e.g., product reviews; [23]). The more users engage with a post, the more likely it is that the post will be shown to others [24]; consequently, social media posts are specifically designed to encourage engagement [8]. In this paper, we primarily focus on "likes" of social media posts and comments, and "shares" of these posts with others as these are common behaviors across different social media platforms. We do not discuss or analyze sharing of comments as most platforms do not provide an easy way to share individual comments. In addition, comments are usually related to the main message of the post, making them less meaningful if shared in isolation.
Further, we examine users' engagement with memes-defined as images, phrases, or brief videos that have a humorous or memorable message that spreads widely online [25,26]. Memes allow for the communication of an idea or an opinion regarding a specific issue and have several unique features compared to other social media posts. First, as memes are widely spread online and can be modified by multiple users, they are considered anonymous in nature [27]-that is, it is often unclear who the creator of the meme is. While their format varies, memes generally feature a combination of images as background and superimposed text as foreground [26]. Further, a defining characteristic of memes is their simplicity-the message and the image deliver a clear and straightforward claim. These characteristics make memes a particularly powerful tool to influence social opinion, especially on controversial topics such as climate change. Interestingly, while prior work has looked at how contests, questions, videos, or images [28] elicit varying levels of engagement, memes have received much less attention [26,27,29], despite their wide use across social media platforms and their power to influence public opinion. More importantly, we are not aware of other work that has examined the interactive effect of the meme and comments on engagement in the context of climate change.

The Interactive Effect of Meme's Content and Comments on User Engagement
Despite the lack of research on the effect of comments on other forms of social media engagement (see [30] for a notable exception), several streams of research suggest comments may play an important role in how others interact with a post. This is particularly true for controversial topics such as climate change, which often elicit comments that disagree with the message expressed in the meme [10]. A meme affirming the existence of climate change may prompt comments opposing climate change, and vice versa. We predict that comments that support or oppose the meme will affect whether a user chooses to "like" or "share" it.
To investigate this topic, we assess users' personal stance on climate change and consider four main possibilities: (1) meme and comments support the social media user's stance on climate change, (2) meme supports and comments oppose the social media user's stance on climate change, (3) meme and comments oppose the social media user's stance on climate change, and (4) meme opposes and comments support the social media user's stance on climate change. We thus ask whether the user's willingness to "like" and "share" the meme will vary across these four conditions.
Consider first the situation where the meme and comments align with one's position on climate change. Here, the social media user may view the comments as indicating there is widespread agreement on the topic [31], which will in turn bolster their belief that the message of the meme is accurate, thereby enhancing engagement likelihood. Perhaps the Sustainability 2022, 14, 8900 4 of 19 more interesting cases, however, occur when there is a mismatch between the meme and the comments' position, as with conditions (2) and (4) above.
When the meme aligns with one's position on the issue, while the comments oppose it, there are two primary ways in which users may respond. First, users may feel compelled to defend the meme against the opposing comments. Consistent with this idea, research in marketing has found that negative comments about a brand on its Facebook page can inspire other users to respond (e.g., with a like to the original post, or with a positive comment) to bolster support for the brand [28]. Yet, such reactance is likely to transpire only when users have a strong connection to the disputed item [32]. Climate change-which, as discussed earlier, is not generally viewed as imminent and directly harmful by most Americans-may be unlikely to elicit such strong reactions.
Instead, we posit that the opposing comments will undermine one's confidence in the veracity of the meme and therefore limit one's willingness to "like" or "share" the meme on social media. Alternatively, the opposing comments might exert normative influence on the social media user, urging the user to refrain from engaging with the meme [33]. Thus, we propose that social media users would evaluate the meme and its comments as one whole, whereby the comments' position on climate change will color the meme's message. Similarly, prior work has shown that one may be reluctant to share information about highly controversial topics that elicit opposing views [34], particularly if one believes such information may not be useful to others [35]. Further, to the extent that the opposing comments may be perceived as misinformation, users may be more reluctant to share it online [36]. Building on this research, we propose that when the meme aligns with one's stance on climate change, opposing comments will decrease one's desire to engage with the meme.

H1:
A social media user will be more likely to "like" or "share" a meme when the positions of both the meme and its comments align with the user's position on climate change, compared to when the comments do not align.
Lastly, consider the context where the meme does not align with the user's views on climate change. We predict that, regardless of the position of the comments, social media users will be unlikely to "like" or "share" this meme. People do not generally like social media posts they disagree with [37][38][39], and memes are generally shared without the comments that follow them, so users have no incentives to share a post they disagree with, even if they support the comments. Further, when both the meme and comments oppose the user's beliefs about climate change, the user may perceive the information as less relevant and opt not to associate with it.
H2: When the position of a meme does not align with the user's position on climate change, the position of the comments will not affect the likelihood that a user will "like" or "share" the meme.

Study Overview
Next, we present the results from three studies designed to test the aforementioned hypotheses. In Study 1, participants, who are mostly supportive of the existence of climate change, report lower likelihood to "like" the meme when the comments oppose (vs. support) their stance on the issue; the effect of the comments is attenuated for memes that do not align with the user's stance. Study 2 replicates these results in a sample that involves the perspective of a balanced group of supporters and opponents of the existence of climate change, demonstrating that the effects hold even when participants are skeptical of climate change. Study 3 replicates these findings again with a pro-climate change sample and begins to investigate how organizations could respond to opposing comments. Specifically, besides supporting and opposing comments, we examine participants' liking and sharing intentions when no comments are allowed-and find that, in this condition, participants' engagement intentions fall between those of memes with supporting and opposing comments. We then extend our investigation by exploring participants' attitudes towards several moderation techniques (e.g., deleting, hiding, responding to comments) and find that social media users are most supportive of organizations responding to comments opposing climate change.

Materials and Methods
A total of 333 U.S. Amazon Mechanical Turk (MTurk) participants completed this study. We excluded 78 participants for poor data quality: six excluded participants failed the attention check ("If you are reading this, please select "Strongly agree" just for this question;" [40]), 26 excluded participants failed the meme condition manipulation check ("What did the meme you saw suggest?"; [41]), and 46 excluded participants failed the comments condition manipulation check ("What did the comments that followed the meme you saw suggest?").
Participants were informed they will see a meme that has appeared recently on a social media platform (see Appendix A for materials). Participants were then shown one of two versions of a meme (i.e., a meme claiming climate change is real or a meme claiming climate change is a scam), followed by one of two sets of comments (i.e., four comments supporting or four comments opposing the meme's claim about climate change). The stimuli were crafted to look realistic and approximate real-world exposure.
Next, participants indicated their intentions to "like" the meme ("If you saw this meme on your social media feed, how likely would you be to "like" this meme?"), "share" the meme ("If you saw this meme on your social media feed, how likely would you be to "share" this meme with your friends?"), and "like" the comments ("If you saw the comments that followed this meme on your social media feed, how likely would you be to "like" these comments?"), using 7-point scales (1 = extremely unlikely, 7 = extremely likely). To capture their stance on climate change, participants indicated whether they believed that global climate change is happening now (Yes/No). Several additional measures were captured in this and the following studies but are not discussed as they are not relevant to the current paper. Finally, participants completed the manipulation checks and demographic questions.

Results
Following Hydock, Paharia, and Blair (2020; [42]), we first recoded the meme and comments condition to denote whether each aligned (vs. misaligned) with participant's beliefs about climate change. For example, the meme aligned/comments aligned condition (N = 69) included participants who believed that climate change is happening now and saw a meme claiming that climate change is real, followed by comments that supported the meme's claim (N = 58) as well as participants who did not believe in climate change and saw a meme claiming that climate change is a scam, followed by comments supporting the meme's claim (N = 11). Thus, the study had a 2 (meme: aligned vs. misaligned with participant's stance on climate change) by 2 (comments: aligned vs. misaligned with participant's stance on climate change) design. Cell sizes ranged from 57 to 69 participants.
Intentions to "like" the meme. There was a main effect of meme (F(1, 251) = 124.92, p < 001) such that, as expected, participants were more likely to "like" the meme when it aligned (M = 4.13) than misaligned (M = 1.63) with their position on climate change. This effect was qualified by a significant interaction (F(1, 251) = 4.23, p = 0.04); when the meme aligned with participants' position, the propensity to "like" the meme was higher when the comments also aligned (M = 4.42) compared to when the comments misaligned (M = 3.84; ticipant's stance on climate change) design. Cell sizes ranged from 57 to 69 participants.
Intentions to "like" the meme. There was a main effect of meme (F(1, 251) = 124.92, p < 001) such that, as expected, participants were more likely to "like" the meme when it aligned (M = 4.13) than misaligned (M = 1.63) with their position on climate change. This effect was qualified by a significant interaction (F(1, 251) = 4.23, p = 0.04); when the meme aligned with participants' position, the propensity to "like" the meme was higher when the comments also aligned (M = 4.42) compared to when the comments misaligned (M = 3.84; F(1, 251) = 3.64, p = 0.06), supporting H1. No difference was observed when the meme misaligned with participant's position (1.46 vs. 1.79; F(1, 251) = 1.06, p = 0.30; see Figure 1), in support of H2. The main effect of comment condition was not significant (F < 1).

Figure 1.
Intentions to "like" the meme by condition in Study 1.
Intentions to "share" the meme. We again observed the expected main effect of meme condition such that participants reported higher intentions to "share" the meme when it aligned with their stance on climate change (M = 3.24) than when it did not align (M = 1.54; F(1, 251) = 56.54, p < 0.001). No other effects were significant (F's < 1; see Figure 2). Intentions to "share" the meme. We again observed the expected main effect of meme condition such that participants reported higher intentions to "share" the meme when it aligned with their stance on climate change (M = 3.24) than when it did not align (M = 1.54; F(1, 251) = 56.54, p < 0.001). No other effects were significant (F's < 1; see Figure 2). ticipant's stance on climate change) design. Cell sizes ranged from 57 to 69 participants.
Intentions to "like" the meme. There was a main effect of meme (F(1, 251) = 124.92, p < 001) such that, as expected, participants were more likely to "like" the meme when it aligned (M = 4.13) than misaligned (M = 1.63) with their position on climate change. This effect was qualified by a significant interaction (F(1, 251) = 4.23, p = 0.04); when the meme aligned with participants' position, the propensity to "like" the meme was higher when the comments also aligned (M = 4.42) compared to when the comments misaligned (M = 3.84; F(1, 251) = 3.64, p = 0.06), supporting H1. No difference was observed when the meme misaligned with participant's position (1.46 vs. 1.79; F(1, 251) = 1.06, p = 0.30; see Figure 1), in support of H2. The main effect of comment condition was not significant (F < 1). Intentions to "share" the meme. We again observed the expected main effect of meme condition such that participants reported higher intentions to "share" the meme when it aligned with their stance on climate change (M = 3.24) than when it did not align (M = 1.54; F(1, 251) = 56.54, p < 0.001). No other effects were significant (F's < 1; see Figure 2). Intentions to "like" the comments. The expected main effect of comments condition was significant (F(1, 251) = 88.72, p < 0.001) such that participants were more likely to "like" the comments when they aligned (M = 4.01) with their position on climate change than when they misaligned (M = 1.88). The effect of meme condition (F(1, 251) = 2.37, p = 0.13) and the interaction effect were not significant (F(1, 251) = 1.89, p = 0.17; see Figure 3 for means).
Intentions to "like" the comments. The expected main effect of comments condition was significant (F(1, 251) = 88.72, p < 0.001) such that participants were more likely to "like" the comments when they aligned (M = 4.01) with their position on climate change than when they misaligned (M = 1.88). The effect of meme condition (F(1, 251) = 2.37, p = 0.13) and the interaction effect were not significant (F(1, 251) = 1.89, p = 0.17; see Figure 3 for means).

Discussion
Study 1 offered some initial support for the proposed effects (H1, H2). First, we found that participants were more likely to engage with content (meme and comments) that aligned with their personal position on climate change. Yet, there was one important nuance: the propensity to "like" a meme that aligned with one's stance on climate change was attenuated when the comments misaligned (rather than aligned) with one's position on climate change. In other words, opposing comments decreased participants' readiness to "like" a meme that was aligned with their views on climate change. Further, the pattern of results was similar, although attenuated, for the propensity to "share" the meme. Yet, the predominantly pro-climate change sample in Study 1 makes it unclear whether the effects will hold in a more balanced sample that includes a higher number of opponents of climate change. We address this limitation in the next study by making a targeted effort to include participants who are skeptical of climate change.

Materials and Methods
As Study 1 indicated that most MTurk participants were, on average, supportive of climate change, we conducted a pre-screening survey via CloudResearch [43] to identify participants who both support and oppose the existence of climate change. We classified a participant as a supporter if they indicated that they believed that climate change is happening now, were concerned about climate change, and believed that climate change is caused by human activity (measured with three binary items). Conversely, an opponent was someone who disagreed with these statements. These items are often used to capture one's stance on the issue [14]. We then invited all climate change opponents (N = 355) and a similarly sized random sample of climate change supporters to complete the main study;

Discussion
Study 1 offered some initial support for the proposed effects (H1, H2). First, we found that participants were more likely to engage with content (meme and comments) that aligned with their personal position on climate change. Yet, there was one important nuance: the propensity to "like" a meme that aligned with one's stance on climate change was attenuated when the comments misaligned (rather than aligned) with one's position on climate change. In other words, opposing comments decreased participants' readiness to "like" a meme that was aligned with their views on climate change. Further, the pattern of results was similar, although attenuated, for the propensity to "share" the meme. Yet, the predominantly pro-climate change sample in Study 1 makes it unclear whether the effects will hold in a more balanced sample that includes a higher number of opponents of climate change. We address this limitation in the next study by making a targeted effort to include participants who are skeptical of climate change.

Study 2 4.1. Materials and Methods
As Study 1 indicated that most MTurk participants were, on average, supportive of climate change, we conducted a pre-screening survey via CloudResearch [43] to identify participants who both support and oppose the existence of climate change. We classified a participant as a supporter if they indicated that they believed that climate change is happening now, were concerned about climate change, and believed that climate change is caused by human activity (measured with three binary items). Conversely, an opponent was someone who disagreed with these statements. These items are often used to capture one's stance on the issue [14]. We then invited all climate change opponents (N = 355) and a similarly sized random sample of climate change supporters to complete the main study; once we reached a pre-determined quota of about 300 participants per group (N supporters = 359, N opponenets = 298), we closed the survey.
The stimuli and measures were identical to those used in Study 1. Again, we recoded the meme and comments conditions in terms of alignment with each participant's view regarding climate change (cell sizes ranged from 139 to 153 participants).
Intentions to "like" the comments. The expected main effect of comments condition was significant (F(1, 577) = 243.55, p < 0.001), similar to Study 1, such that participants were more likely to "like" the comments when they aligned (M = 3.87) with their position on climate change than when they misaligned (M = 1.54). The effect of meme condition (F(1, 577) = 2.93, p = 0.09) was marginal, suggesting that when the meme aligned with participants' views, they were more likely to "like" the comments (M = 2.89), than when the meme misaligned (M = 2.54); yet, the interaction effect was not significant (F < 1, see Figure 6).

Discussion
Study 2 replicated the results from Study 1; we found that participants were more likely to "like" a meme when the meme and its comments aligned with the participant's position on climate change. Yet, the position expressed in the comments moderated participants' readiness to "like" the meme: misaligned (vs. aligned) comments attenuated intentions to "like" a meme that aligned with participant's stance on climate change. Importantly, this result was replicated with a balanced sample that included both opponents and supporters of climate change, suggesting that climate change organizations might be more successful in (1) spreading pro-climate change information when pro-climate change posts are followed by pro-climate change comments, and in (2) curbing climate change skeptical information when climate change skeptical posts are followed by pro-climate change comments. We return to this point in the General Discussion. (Note that the meme by comments interaction on propensity to "like" the meme is observed when the data from participants supporting and opposing climate change is analyzed separately, indicating that the conclusions apply to both groups). Intentions to "like" the comments. The expected main effect of comments condition was significant (F(1, 577) = 243.55, p < 0.001), similar to Study 1, such that participants were more likely to "like" the comments when they aligned (M = 3.87) with their position on climate change than when they misaligned (M = 1.54). The effect of meme condition (F(1, 577) = 2.93, p = 0.09) was marginal, suggesting that when the meme aligned with participants' views, they were more likely to "like" the comments (M = 2.89), than when the meme misaligned (M = 2.54); yet, the interaction effect was not significant (F < 1, see Figure  6).

Discussion
Study 2 replicated the results from Study 1; we found that participants were more likely to "like" a meme when the meme and its comments aligned with the participant's position on climate change. Yet, the position expressed in the comments moderated par-  Intentions to "like" the comments. The expected main effect of comments condition was significant (F(1, 577) = 243.55, p < 0.001), similar to Study 1, such that participants were more likely to "like" the comments when they aligned (M = 3.87) with their position on climate change than when they misaligned (M = 1.54). The effect of meme condition (F(1, 577) = 2.93, p = 0.09) was marginal, suggesting that when the meme aligned with participants' views, they were more likely to "like" the comments (M = 2.89), than when the meme misaligned (M = 2.54); yet, the interaction effect was not significant (F < 1, see Figure  6).

Discussion
Study 2 replicated the results from Study 1; we found that participants were more likely to "like" a meme when the meme and its comments aligned with the participant's position on climate change. Yet, the position expressed in the comments moderated participants' readiness to "like" the meme: misaligned (vs. aligned) comments attenuated intentions to "like" a meme that aligned with participant's stance on climate change. Importantly, this result was replicated with a balanced sample that included both opponents and supporters of climate change, suggesting that climate change organizations might be Interestingly, both studies 1 and 2 demonstrated similar patterns of results for propensity to "share" the meme, but the interaction did not reach significance. We reason that there may be some inherent differences in participants' desire to "share" content that may be influencing this result. To account for such heterogeneity, in the next study we capture participants' general propensity to "like" and "share" content on social media and control for it in the analysis. Further, both studies 1 and 2 indicated that participants' intentions to "like" the comments are largely dependent on the comments' stance regarding the issue, which is somewhat less surprising. We thus focus our attention in the next study on the measures related to the meme. Importantly, Study 3 was designed to offer an initial investigation into the actions that an organization can take to encourage engagement with information supporting climate change on social media. Our results so far suggest that the position of the comments following the post plays an important role in encouraging engagement with the meme. Then, how should organizations moderate comments that appear on their social media posts? Should they delete all opposing comments? Hide them? Respond to them? Or is it better to disable comments altogether (an option that is available on some social media platforms such as YouTube and Instagram)? And are there any individual factors that might influence participants' attitudes towards some of these moderation techniques? We explore these questions in the next study by (1) including a "comments disabled" condition to examine whether it results in similar or lower engagement intentions than when the comments support or oppose climate change, and by (2) capturing participants' approval of and attitudes toward entities that delete, hide, or respond to comments opposing climate change.
To the best of our knowledge, prior research has not examined strategies for responding to comments that follow social media posts about climate change. However, research on other controversial topics has found that policy organizations that hide or delete comments are less trusted than those that do not (assuming that users are aware that comments have been hidden or deleted; [44]). In contrast, responding to comments garners more positive attitudes toward the organization (assuming that the responses are appropriately person-centered; [45]). Extending this work to climate change, we predict that participants will be most supportive of organizations that respond to (rather than delete, flag, or hide) opposing comments.
H3: Social media users will have a more favorable attitude toward an organization that responds to (vs. deletes, hides, or flags) opposing comments.

Materials and Methods
Given this study's focus, we invited a random sample of the pro-climate change participants who completed the pre-screener in Study 2 but did not participate in Study 2 to complete Study 3. Of the 222 participants who completed Study 3, we excluded 14 who did not pass the manipulation checks, resulting in a sample of 208 (M age = 44.72, SD = 13.13, range 21-80; 61.1% female). Similar to prior studies, the sample was diverse and reasonably representative of the general US population: the majority of the participants indicated White/Caucasian ethnicity (74.5%), followed by Asian/Asian-American (13.0%), Black/African-American (5.8%), Hispanic (4.8%), and other (1.9%); all US geographic regions were represented (Midwest: 20.7%; Northeast: 24.0%; South: 32.2%; West: 23.1%); 59.1% indicated liberal political orientation.
Participants responded to the same materials as in Study 1, with a few notable differences. First, in all conditions, the meme aligned with participant's stance (see Appendix B for the modified meme). Second, in addition to the two comment conditions from Study 1 (comments supporting and comments opposing climate change), we included a comments disabled condition, where participants were informed that "Comments have been disabled for this post." Thus, each participant was assigned randomly to one of the three comment conditions. After seeing the meme, all participants indicated their intentions to "like" and "share" the meme, using the items from Study 1, followed by the manipulation checks.
In the second part of the study, participants considered a pro-climate change entity (e.g., an organization, a community group, or a person) that often posts memes regarding climate change on social media, where social media users frequently comment on these posts, opposing climate change. Participants were told that the entity has decided to moderate these comments and asked to indicate their approval of five possible moderation techniques, displayed in random order: (1) Delete all comments opposing climate change-These comments will not be accessible once deleted, (2) Hide all comments opposing climate change-You can manually unhide each comment to read it, (3) Use an algorithm that will automatically hide as irrelevant comments opposing climate change-You can manually unhide each comment to read it, (4) Flag all comments opposing climate change as misinformation-You will still see these comments as usual, but there will be a red flag indicating misinformation, (5) Respond to all comments opposing climate change with scientific information about climate change-You will see all comments as usual.
Participants indicated an answer ranging from 1 = Strongly disapprove to 7 = Strongly approve. Next, we captured one's attitude towards the entity, if it decided to use each of the five techniques (1 = Extremely negative to 7 = Extremely positive).
We also captured a few exploratory items to gain a deeper understanding of participants' attitudes towards removing opposing comments about climate change, as this practice is often adopted online [46]. Thus, we asked participants to indicate the extent to which they think a pro-climate change entity has the right to remove comments opposing climate change, has a responsibility to remove such comments, and limits freedom of speech in an important way by removing these comments (all three measures captured on a scale ranging from 1 = Not at all to 7 = Very much so). We reasoned that the more respondents believed an entity has a responsibility to remove opposing comments (i.e., to prevent the spread of misinformation), the more accepting they would be of this practice. Further, we expected that perceived infringement of freedom of speech will be negatively associated with a perceived right to remove opposing comments. Finally, we captured how frequently participants "like" posts on social media and "share" posts they have seen on social media-to serve as covariates in the focal analysis-as well as how frequently they read the comments on posts on social media, to ensure that users pay attention not only to the main post but also to the comments.

Results
Attention to comments in social media posts. First, we examined whether participants pay attention to the comments following social media posts in general. Participants indicated they frequently read the comments, with 61% of the sample selecting an answer above the scale midpoint (M = 4.84, SD = 1.56; significantly above the scale midpoint, t(207) = 7.78, p < 0.001). This result reaffirms the relevance of the current research.
Intentions to "like" the meme. We conducted an ANCOVA on propensity to "like" the meme with comment condition as the between-subjects factor and propensity to like content on social media as the covariate. The results indicated the expected main effect of condition (F(2, 204) = 4.98, p = 0.008; see Figure 7), adjusting for general propensity to like content (F(2, 204) = 73.63, p = < 0.001). Planned comparisons showed that participants were more likely to "like" the meme when the comments supported climate change (and thus their beliefs) compared to when the comments opposed climate change (p = 0.002), replicating the results from the previous two studies. Importantly, participants' intentions to "like" the meme were marginally higher in the comments disabled condition than in the opposing comments condition (p = 0.10), but not significantly different from the supporting comments condition (p = 0.13). Not surprisingly, the higher participants' propensity to like content in general, the more likely they were to indicate readiness to "like" the meme (b = 0.60, t = 8.58, p < 0.001), keeping all else constant.
Intentions to "share" the meme. Mirroring the results on "liking" the meme, participants' intentions to "share" the meme differed across conditions (F(2, 204) = 3.25 p = 0.04; see Figure 7), adjusting for general propensity to share content (F(2, 204) = 73.63, p < 0.001). Participants indicated lower intentions to share the meme when the comments opposed climate change than when they supported it (p = 0.012), in support of H1; the comments disabled condition was not statistically different from the comments supporting (p = 0.27) or the comments opposing (p = 0.15) conditions. Again, higher participants' propensity to share content in general was positively associated with intentions to "share" the meme (b = 0.76, t = 9.30, p < 0.001). see Figure 7), adjusting for general propensity to share content (F(2, 204) = 73.63, p < 0.001). Participants indicated lower intentions to share the meme when the comments opposed climate change than when they supported it (p = 0.012), in support of H1; the comments disabled condition was not statistically different from the comments supporting (p = 0.27) or the comments opposing (p = 0.15) conditions. Again, higher participants' propensity to share content in general was positively associated with intentions to "share" the meme (b = 0.76, t = 9.30, p < 0.001).

Figure 7.
Intentions to "like" and "share" the meme claiming climate change is real by comments condition in Study 3.
Attitudes towards comment moderation techniques. We next examined participants' attitudes toward five comment moderation techniques, using repeated measures ANOVA with Bonferroni adjustment. As expected, condition did not interact with any of the measures reported next, so we do not discuss it further. The results indicated differences in level of approval of the five techniques (F(4, 828) = 111.01, p < 0.001); responding to opposing comments with scientific information about climate change had by far the highest approval rating, followed by flagging comments as misinformation and hiding opposing comments (see Figure 8 for means; all means except for hiding comments and using an algorithm to hide comments were significantly different from each other, p's < 0.05). Note that these means are also significantly different from the scale midpoint (p < 0.001), except for the flagging comments technique (p = 0.62).
Participants' attitude towards the entity demonstrated a similar pattern of results (F(4, 828) = 177.37, p < 0.001), with one notable difference: all techniques, even the two involving hiding comments, were significantly different from each other (p's < 0.05; see Figure 8). The option of responding to the comments resulted in the most positive attitude towards the entity. These results support H3. Note that, again, all means are significantly different from the scale midpoint (p < 0.001), except for flagging comments (p = 0.37). Attitudes towards comment moderation techniques. We next examined participants' attitudes toward five comment moderation techniques, using repeated measures ANOVA with Bonferroni adjustment. As expected, condition did not interact with any of the measures reported next, so we do not discuss it further. The results indicated differences in level of approval of the five techniques (F(4, 828) = 111.01, p < 0.001); responding to opposing comments with scientific information about climate change had by far the highest approval rating, followed by flagging comments as misinformation and hiding opposing comments (see Figure 8 for means; all means except for hiding comments and using an algorithm to hide comments were significantly different from each other, p's < 0.05). Note that these means are also significantly different from the scale midpoint (p < 0.001), except for the flagging comments technique (p = 0.62). Finally, we looked at participants' beliefs about the entity's right to moderate comments on social media. Using linear regression, we examined the extent to which participants believed an entity has the right to remove opposing comments as a function of responsibility, infringement of freedom of speech, political orientation, age, and gender. The results indicated that perceived responsibility to remove comments and age were positively associated with approval of removing opposing comments; conversely, freedom of speech infringement and conservative political orientation were negatively associated with an entity's right to remove comments (see results in Table 1).  Participants' attitude towards the entity demonstrated a similar pattern of results (F(4, 828) = 177.37, p < 0.001), with one notable difference: all techniques, even the two involving hiding comments, were significantly different from each other (p's < 0.05; see Figure 8). The option of responding to the comments resulted in the most positive attitude towards the entity. These results support H3. Note that, again, all means are significantly different from the scale midpoint (p < 0.001), except for flagging comments (p = 0.37).
Finally, we looked at participants' beliefs about the entity's right to moderate comments on social media. Using linear regression, we examined the extent to which participants believed an entity has the right to remove opposing comments as a function of responsibility, infringement of freedom of speech, political orientation, age, and gender. The results indicated that perceived responsibility to remove comments and age were positively associated with approval of removing opposing comments; conversely, freedom of speech infringement and conservative political orientation were negatively associated with an entity's right to remove comments (see results in Table 1).

Discussion
Study 3 replicated and extended the focal result that opposing comments decrease engagement intentions for pro-climate change content. Specifically, we showed that, when an organization does not want to engage in comments moderation, disabling comments may be a successful tactic of encouraging engagement with climate change memes-the results showed that participants were as likely to "like" a meme when comments were disabled as when comments supported the meme's message; participants' "share" intentions mirrored this pattern, although the comments disabled condition was not significantly different from the opposing comments condition. Importantly, this study offered several interesting insights about comments moderation. First, we found that responding to opposing comments was viewed more positively than hiding, flagging, or deleting comments. Second, participants were more likely to accept that an entity has the right to remove opposing comments when they believed that the entity has a responsibility to do so and it does not limit freedom of speech, and when participants were more liberal and older.

Conclusions
In three studies, we found that social media users' intentions to "like" a meme related to climate change were influenced by the stance taken in the accompanying comments. Specifically, when the meme aligned with users' beliefs about climate change, comments that opposed the meme reduced users' willingness to "like" the meme; yet, the stance of the comments did not seem to matter when the meme misaligned with users' position on the issue. Note that the pattern of results was similar, although attenuated, for propensity to "share" the meme, offering further support that the message expressed in a meme and the comments that follow it interact to influence online engagement with climate change content. We also examined how entities that are faced with such opposing comments may respond. Specifically, Study 3 showed that responding to each opposing comment-rather than deleting it, hiding it, or flagging it-was the preferred response strategy.
This work offers important theoretical implications. First, it is the first to demonstrate that the response to social media content, particularly when relating to a controversial topic such as climate change, depends not only on the main post's message but also on the stance of the accompanying comments. While prior literature has examined engagement with posts and comments in isolation, our results highlight the importance of simultaneously considering both elements. This is an important contribution as commenting on posts is a hallmark feature of social media platforms. Second, while the studies reported here focus on climate change, we believe that the observed novel results might apply to other controversial topics such as abortion or gun control, expanding the usefulness of the insights. Future research could examine the generalizability of the results to other topics. Nevertheless, these findings are of critical importance for climate change advocates seeking to garner at-large support for climate change action. Third, the findings suggest that what matters may be not only how social media users engage with a post but also when they engage with it. If early comments are misaligned with the main message, users' likelihood to "like" the post is attenuated, which may consequently limit the number of those who will ultimately see the post.
The results also provide several important practical implications. First, we find that comments that contradict the information communicated by the main post make supporters of the issue less likely to "like" the post and potentially less likely to share it. Given that "likes" and "shares" are key inputs in the algorithms that determine which posts are displayed with priority, fewer "likes" and "shares" pose a problem for organizations looking to spread content on social media about the existence and importance of climate change. Our findings suggest that, in such cases, organizations may be better off disabling comments, even if this may run counter to the "social" aspect of the platform. Alternatively, as usually only a few comments are displayed on posts, organizations may consider encouraging their members to comment supporting a post that has received many opposing comments or to "like" the comments that support the post, thereby decreasing the visibility of opposing comments.
On the flip side, however, the results of Study 2 suggest that opposing comments may also be used strategically against entities that promote misinformation about climate change. Generating comments supporting climate change in response to a meme opposing climate change may weaken opponents' intentions to engage with the content, potentially decreasing the spread of misinformation. Future research can examine what proportion of the comments need to be positive to observe this effect (i.e., would it take 10%, 50%, or 80% of all visible comments to be pro-climate change?).
Second, the results show a less consistent pattern for "sharing" a meme. While we find the effect in Study 3, controlling for general sharing proclivities, the effect does not reach significance in Studies 1 and 2. In addition, in all studies participants reported lower intentions to "share" than to "like" the meme (Study 1: M share = 2.39, M like = 2.96; F(2, 502) = 17.20, p < 0.001; Study 2: M share = 2.23, M like = 2.72; F(2, 1154) = 39.92, p < 0.001; Study 3: M share = 2.90, M like = 4.10; F(1, 205) = 125.04, p < 0.001). These findings suggest that "sharing" behaviors may be more complex, and organizations looking to promote the sharing of climate change content may need to carefully evaluate users' motivations to share. Future work can examine meme-sharing behaviors in further depth.
Interestingly, the results of Study 1 indicated that social media users' decision about whether to "like" the comments following the meme were not affected by the message of the meme itself. Yet, in Study 2, we found a marginal main effect of meme; in both studies, though, the interaction effect of meme by comments condition was not significant. This suggests that users' behaviors regarding comments are based primarily on the content of the comment itself, rather than its juxtaposition with the meme. Yet, user reactions to comments may serve as input for the algorithms used to determine content relevance, necessitating further research into what leads users to react to others' comments.
Several limitations of the current research deserve further discussion. First, we did not examine the underlying process behind the observed results. Thus, it is unclear whether the opposing comments decrease the credibility of the meme or exert normative influence on the target user. We believe that the former explanation is more likely, as divergent opinions are a defining feature of controversial topics such as climate change.
Second, all comments in the stimuli we used were in consensus-either in support or in opposition of the existence of climate change. This was essential to cleanly test our hypotheses. Yet, social media posts often involve a mix of supporting and opposing-and even perhaps ambiguous-comments. Further research is needed to understand how these mixed comments affect social media users' likelihood to "like" or "share" posts. Similarly, future research can examine these effects in samples that include social media users who are undecided or easily swayed in their opinions about climate change. The latter group may demonstrate greater sensitivity to opposing comments, raising the importance of moderating comments.
Third, the comments used in the stimuli were relatively short, civil, and mild in their stance on climate change, which is not always the case on social media. In fact, research shows that stronger, insulting comments polarize readers' attitudes toward the topic under discussion [47]. Yet, research has not examined how these types of comments affect social media users' willingness to "like" or "share" the main post, making it unclear whether social media managers should prioritize responding to such comments.
Finally, future work should examine other online behaviors such as posting a comment to the main post or responding to a comment following the meme. In addition, here we measured social media users' intentions to engage with content. Future research should capture actual "likes" and "shares" on social media and attempt to better understand the motivations underlying consumers' desire to "like" and "share" social media content, especially regarding controversial topics. Such insights can help public policy managers design interventions to help promote the spread of factual information that has the potential to improve societal welfare.
In conclusion, we find that the comments on memes about climate change have a significant effect on social media users' engagement with the meme itself, such that comments that oppose the post reduce people's willingness to engage with meme even when it expresses ideas they believe in. This novel result shows the power of comments and the importance of comment moderation, especially when looking to engage audiences on a topic as controversial and important as climate change.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A. Study 1 Materials
[Introduction Screen] Next, you will be shown a meme that recently appeared on a social media platform. Memes refer to images, videos, and/or text that are shared on social media and tend to spread rapidly. Social media users tend to share and comment on memes they find interesting.
Please spend adequate time reviewing this information carefully, as we will then ask you to answer several questions about it.
You will have at least 30 s to view this meme and associated comments before you can move on to the next page. After 120 s, the page will auto-advance.