Next Article in Journal
Rethinking the Connections between Ecosystem Services, Pollinators, Pollution, and Health: Focus on Air Pollution and Its Impacts
Next Article in Special Issue
Understanding Aboriginal Models of Selfhood: The National Empowerment Project’s Cultural, Social, and Emotional Wellbeing Program in Western Australia
Previous Article in Journal
Assessing Walking Programs in Fibromyalgia: A Concordance Study between Measures
Previous Article in Special Issue
Sound Decision Making in Uncertain Times: Can Systems Modelling Be Useful for Informing Policy and Planning for Suicide Prevention?
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

The Tool for Evaluating Media Portrayals of Suicide (TEMPOS): Development and Application of a Novel Rating Scale to Reduce Suicide Contagion

Depression Clinical & Research Program, Massachusetts General Hospital, Boston, MA 02114, USA
Suicide Prevention Program, County of Santa Clara Behavioral Health Services Department, San Jose, CA 95128, USA
Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
Center for Care Innovations, Oakland, CA 94612, USA
Department of Psychology, Palo Alto University, Palo Alto, CA 94304, USA
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(5), 2994;
Submission received: 29 January 2022 / Revised: 1 March 2022 / Accepted: 2 March 2022 / Published: 4 March 2022
(This article belongs to the Special Issue Novel Approaches to Suicide Prevention)


Research suggests that media adherence to suicide reporting recommendations in the aftermath of a highly publicized suicide event can help reduce the risk of imitative behavior, yet there exists no standardized tool for assessing adherence to these standards. The Tool for Evaluating Media Portrayals of Suicide (TEMPOS) allows media professionals, researchers, and suicide prevention experts to assess adherence to the recommendations with a user-friendly, standardized rating scale. An interdisciplinary team of raters constructed operational definitions for three levels of adherence to each of the reporting recommendations and piloted the scale on a sample of articles to assess reliability and clarify scale definitions. TEMPOS was then used to evaluate 220 news articles published during a high-risk period following the suicide deaths of two public figures. Post-hoc analyses of the results demonstrated how data produced by TEMPOS can be used to inform research and public health efforts, and inter-rater reliability analyses revealed substantial agreement across raters and criteria. A novel, wide-reaching, and practical approach to suicide prevention, TEMPOS allows researchers, suicide prevention professionals, and media professionals to study how adherence varies across contexts and can be used to guide future efforts to decrease the risk of media-induced suicide contagion.

1. Introduction

Suicide is one of the leading causes of death worldwide, with over 800,000 people dying by suicide annually, more than war, malaria, or breast cancer [1]. Media representations of suicide can influence the contagion of suicidal behavior, particularly in vulnerable populations. Rates of self-harm and suicide attempts have been increasing in recent years [2,3], and a growing body of research has established a link between self-harm and increased media use [4,5,6]. Suicide mortality tends to increase following highly publicized suicide events, a phenomenon known as the Werther Effect [7]. The association between media reporting and suicide contagion is well established as a globally reaching public health concern, documented in over 150 empirical studies and systematic reviews from around the world [8,9,10,11,12]. Newspaper coverage of suicide has been found to be significantly associated with the initiation of suicide clusters [13], and a substantial number of suicide attempt survivors report being affected by a media story about suicide [14,15]. Increases in suicide rates following a highly publicized suicide event tend to be proportionate to the volume, duration, and prominence of the coverage [16], and are greater when the majority of the coverage is sensationalistic or includes details about the suicide method [17,18]. For example, in the three months following the highly publicized suicide of American comedian Robin Williams in 2014, there were 16% more suicides than expected; moreover, the greatest increases were seen in deaths by asphyxiation (the method used by Williams) and in males over 30, suggesting an imitative effect [19].
Despite these findings, media can also play an important role in suicide prevention [18,20]. When media outlets minimize the inclusion of certain kinds of harmful information (e.g., information about how the suicide was completed), the risk of imitative behavior decreases [18,20]. For example, a campaign by suicide prevention experts in Austria to implement guidelines for reporting on railway suicides in Vienna in the 1980’s led to a reduction in the volume of coverage and, in turn, an 84% reduction in suicides [21]. More recent research demonstrates that media reports that portray suicide as a preventable outcome and disseminate resources and information about suicide prevention can help decrease suicide rates, a phenomenon known as the Papageno Effect [20,22].
In order to help media professionals decrease the risk of imitative suicide and instead promote a Papageno Effect, the World Health Organization published a set of recommendations for safely reporting on suicide [23]. These recommendations—which draw upon decades of research on suicide contagion and were developed with input from leading experts in the fields of suicide prevention, journalism, and public health—are continually expanded and adapted to reflect current empirical understanding. In addition to the WHO recommendations, suicide prevention professionals around the world have created and disseminated similar sets of recommendations, such as the Recommendations for Reporting on Suicide, developed in the United States [24], the Mindset guidelines from Canada [25], and the Mindframe guidelines from Australia [26]. Despite the existence of many different sets of recommendations, their content is consistent: Media professionals are advised to avoid sensationalizing or glamorizing the person who died, to avoid including explicit details about the death or suicide method, and to include prevention resources for those who may be struggling and at risk of suicide. Adherence to such guidelines is associated with a reduction in suicide rates [27], decreased use of highly lethal suicide methods [28], and increased utilization of support resources [20].
Adherence to these guidelines is especially vital during surges of suicide-related coverage, such as when a high-profile figure dies by suicide. Famed American fashion designer Kate Spade died by suicide on 5 June 2018, leading to a spike in suicide-related news coverage in the United States. Celebrity chef and TV personality Anthony Bourdain took his own life just three days later, on 8 June 2018. Celebrity suicides that occur in such close succession are exceedingly rare but provide a valuable opportunity to study how media outlets cover suicide. However, there currently exists no standardized method for measuring adherence levels, making it difficult to compare results across studies or understand how adherence varies across contexts.
Many studies examining adherence use a binary rating metric, noting the presence or absence of each recommended reporting practice [29,30,31]. Although simple to use, binary rating systems fail to account for the fact that degrees of adherence to a particular reporting recommendation may have differential impacts on the audience. For example, a newspaper article that provides graphic details about a suicide method may be significantly more harmful than an article that mentions the method in passing; however, under a binary rating system, these two articles would both be coded the same way. In order to capture more nuance in reporting, some researchers have utilized traditional content analysis methods [13,17,32,33], while others have developed their own rating methods tailored specifically to their research aims [34,35]. Although these more complex approaches succeed in capturing more nuance, they tend to be difficult and time-consuming to execute. Consequently, they are unlikely to be widely adopted by suicide prevention programs or media professionals in need of evaluation and monitoring tools.
The lack of a standardized, user-friendly rating system also poses a challenge to suicide prevention programs aiming to monitor and evaluate their progress in working with the media to increase adherence to the reporting recommendations. For example, in response to the clusters of youth suicides in 2009 and 2014, the Suicide Prevention Program of Santa Clara County in Northern California has been working with local media to improve suicide-related reporting since 2011. A 2016 study conducted by the Centers for Disease Control (CDC) in response to these clusters found that among the 246 media reports analyzed, only 17% included any sort of suicide prevention resource. On average, each media report contained 4.3 potentially harmful characteristics, compared to an average of only 0.5 protective characteristics [36]. After the study ended, however, program staff had no standardized method to assess the progress of their work with the media over time, to identify targeted areas for further improvement, or to provide quantified feedback to media partners about their adherence. Program leadership identified the need for the development of a new standardized assessment tool that would allow suicide prevention programs to track changes in media adherence over time and identify targeted areas for improvement.
To address these gaps, the County of Santa Clara’s Suicide Prevention Program and the Stanford Department of Psychiatry and Behavioral Sciences collaborated to develop the Tool for Evaluating Media Portrayals of Suicide (TEMPOS). The primary aim of this work is to develop a novel, user-friendly, and non-binary rating tool that can be used by members of the media (i.e., journalists, editors), suicide prevention professionals, and researchers. The second aim is to illustrate how TEMPOS may be used to monitor and evaluate media coverage. The present study describes the development of TEMPOS and its subsequent application to a dataset of 220 suicide-related news articles collected during a surge of suicide-related coverage. Through the process of applying the scale, tool characteristics and reliability were further explored.

2. Materials and Methods

2.1. Scale Development

TEMPOS was developed by an interdisciplinary team of researchers with backgrounds in psychology, psychiatry, public health, media, and community mental health. TEMPOS consists of ten criteria (Table 1), which were derived from the American version of the suicide reporting recommendations, the Recommendations for Reporting on Suicide [19]. Rather than utilizing a binary scoring system (adherence/non-adherence), TEMPOS utilizes a three-point rating scale to capture more complexity without being onerous. When using TEMPOS, a rating of 2 indicates full adherence to the guideline, a rating of 1 indicates partial adherence, and a rating of 0 indicates non-adherence. In order to delineate what qualifies as full, partial, or non-adherence, operational definitions were constructed for each rating level (three for each of the 10 criteria, and 30 definitions in total). Wherever possible, definitions were constructed using language drawn directly from (accessed on 28 January 2022), in order to maximize alignment with the recommendations.
The interdisciplinary TEMPOS team followed a four-step scale development process (Figure 1). First, following initial construction of the 30 rating choices, the team discussed each operational definition and revised any wording that was deemed unclear or ambiguous. Second, the 3-point rating system was pilot-tested with a subsample of 5 suicide-related articles drawn from a larger dataset of 220. Four raters independently rated these articles in order to assess inter-rater reliability and identify any scale definitions that were too vague or difficult to apply. Third, the team reviewed the ratings and worked together to refine the scale definitions in response to common points of confusion and disagreement that arose during the pilot coding process.
The fourth step in scale development intended to strengthen the validity of the scale through an external review process with experts on suicide contagion and media-influenced harm. We invited five external reviewers, each an expert in suicide contagion and media-influenced harm, to provide feedback and critique of TEMPOS. Each external reviewer was sent a draft of the scale and asked to provide comments on the structure of the scale, as well as the wording and validity of the constructs. Based on the feedback from these external reviewers, the team revised the scale and completed one final round of test coding on a subsample of 10 articles in order to assess inter-rater reliability prior to applying the scale to the full dataset of 220 articles.

2.2. Suicide News Media Dataset

The suicide deaths of Kate Spade and Anthony Bourdain in early June of 2018 triggered a surge in suicide-related coverage, providing a natural opportunity to study how regional and national news outlets cover suicide. Over the course of a month, the County of Santa Clara’s Suicide Prevention Program compiled a dataset of suicide-related news articles published in the United States. Articles were obtained using Google Alerts and manual searches of the keywords “suicide”, “suicide prevention”, “mental health”, “mental illness”, and “self-harm”. Letters to the editor, articles from publications focused on gossip (e.g., TMZ), non-English articles, obituaries, and articles covering murder-suicides were excluded.

2.3. Application of Scale

To illustrate how TEMPOS may be used to monitor and evaluate media coverage, the first author applied all ten TEMPOS criteria to each of the 220 articles in the dataset. In order to assess the inter-rater reliability of the scale, each article was also independently rated by one of five secondary raters.
Following the completion of the coding process, raters met to discuss and resolve discrepancies between the two sets of ratings in order to examine the inter-rater reliability of TEMPOS when applied to a large sample of media and produce a final set of ratings. In addition to rating each article for adherence to each of the ten criteria, researchers also calculated an overall TEMPOS score by dividing the total number of points scored by the total number of points possible. If any criteria were rated as “not applicable”, the total number of points possible was adjusted (20 total minus 2 points for each criterion that was rated “not applicable”). For ease of interpretation, scores were converted to percentages, with 0% indicating total non-adherence to the reporting recommendations, and 100% indicating full adherence.

3. Results

3.1. Characteristics of the Dataset

In total, 226 articles were collected from several media outlets covering national and local Bay Area news, including broadcast networks, online magazines and newspapers, and blogs that had readership of at least 1000 people. By the time the scale was fully developed, six articles from the dataset were no longer available, leaving a total of 220 articles. As expected, there was a surge in suicide-related coverage immediately following the death of Kate Spade, and coverage peaked three days later following the death of Anthony Bourdain (Figure 2). Article characteristics are presented in Table 2.

3.2. Inter-Rater Reliability

Inter-rater reliability was calculated by identifying the number of agreements between the two sets of ratings for each article and calculating the overall percentage of agreement. Across all raters and criteria, pure inter-rater agreement was 81.31%. To adjust for chance agreements, we calculated Cohen’s Kappa (κ) for each criterion [37]. Since the κ statistic depends on marginal values to calculate chance agreement, low prevalence of a variable can produce lower κ values. Accordingly, reporting characteristics that displayed low variability–for example, very few articles in our dataset contained content that glamorized suicide–displayed significantly lower κ values. Therefore, in addition to Cohen’s κ, percentage agreement is also presented for each criterion (Table 3). The average κ value across all criteria was 0.62; a κ value between 0.6 and 0.8 is generally understood to indicate substantial agreement among raters [37].

3.3. Analysis of TEMPOS Scores

We performed a series of exploratory analyses to understand overall levels of adherence, as well as how adherence levels varied between publications, across criteria, and over time. Overall TEMPOS percentage scores ranged from 5% to 100% (M = 74.7%, MDN = 75.0%, and SD = 18.2%). The distribution of overall scores was negatively skewed, as illustrated in Figure 3.
Adherence levels varied significantly by criterion (Figure 4). The criterion that displayed the lowest mean levels of adherence was “suicide prevention and mental health resources” (M = 0.96, SE = 0.05), which aligns with past findings that media reports of suicide often fail to provide information and resources that could help those who may be struggling [29,30,38]. The criterion that displayed the highest mean levels of adherence was “glamorization of suicide” (M = 1.79, SE = 0.03), suggesting that very few media outlets portray suicide in a positive manner.
We then examined how TEMPOS scores varied across and within publications (Figure 5). The average TEMPOS scores of each publication ranged from 4% to 96.9% (M = 76.6%, SD = 14.6%).
Lastly, we examined whether overall adherence to the guidelines changed over the 1-month study period (Figure 6). An independent-samples t-test was conducted to compare TEMPOS scores on the day of Kate Spade’s death and the day of Anthony Bourdain’s death. Reporting on Bourdain’s death (M = 79.7%, SD = 13.9%) was significantly more adherent than reporting on Spade’s death (M = 62.3%, SD = 26.3%); t(23) = −2.82, p = 0.01.

4. Discussion

This paper describes the development and application of a novel, user-friendly, non-binary rating system to assess media adherence to suicide reporting recommendations. While many studies have examined the relationship between adherence to reporting recommendations and suicide rates, disparate measurement approaches make it difficult to draw meaningful comparisons across studies. To address these gaps, the current study explains the scale development process for the Tool for Evaluating Media Portrayals of Suicide (TEMPOS), as well as its application to a dataset of 220 media reports collected during a surge of suicide-related coverage. Results demonstrate the scale’s reliability, validity, and its utility as a tool for researchers, journalists, and public health professionals engaged in suicide prevention.
The application of TEMPOS yielded high inter-rater agreement among coders. Consistent with prior research [32,34], rater agreement levels were directly related to the degree of subjective interpretation required by raters. More concrete criteria (i.e., details about the suicide method) displayed higher levels of inter-rater reliability than more abstract criteria (i.e., glamorization of suicide), which may require more subjective judgment from the raters.
The application of TEMPOS to a dataset of suicide-related media articles from June 2018 demonstrated that adherence increased slightly following the death of Kate Spade, which aligns with previous findings that media reporting on Bourdain’s death was more guideline-adherent [38]. At the time of these highly publicized deaths, many media outlets received public criticism for the inappropriate ways that they reported on Spade’s death [38], which may have alerted media outlets to the importance of adhering to the reporting recommendations. TEMPOS provides a method for monitoring trends in reporting adherence over time, which may help shed light on the links among public suicides, media adherence levels, and suicide rates. This is a vital step in developing prevention strategies for combatting suicide contagion and protecting public health.
A critical contribution from this study is an illustration of how TEMPOS can be used by a range of constituents involved in the prevention of suicide. Professionals engaged in regional or broad-reaching prevention programs can use TEMPOS to better understand how media adherence to suicide reporting recommendations varies among criteria, across and within publications, and over time. TEMPOS can also be used to identify which reporting recommendations are commonly violated and which practices have already been widely adopted. For example, our analyses revealed that very few articles in the dataset featured suicide prevention and mental health resources, which aligns with previous research [29,30,38]. Systematic ratings from TEMPOS can yield powerful datapoints that allow suicide prevention professionals to develop more targeted and efficient interventions focused on improving adherence to the most commonly ignored recommendations, or media outlets in particular need of further training. TEMPOS can also be directly used by editors and publication leadership to determine if there are specific sections or reporters within their organizations that are in need of training. Importantly, TEMPOS also makes it possible to examine the impact of such trainings by providing a standardized way of measuring adherence before and after the training is administered. Taken together, the development, testing, and application of this novel rating scale for assessing adherence to suicide reporting recommendations provides a promising springboard for a diverse set of constituents to make strides in evidence-based suicide prevention efforts.

Limitations & Future Directions

Several limitations should be noted. First, the dataset may not be fully representative of all news articles published in June 2018 because the article collection process relied on publicly accessible media. In addition, social media and articles shared via social media were not included in the dataset; however, social media is among the most common sources of news media for young people ages 18–29, and 48% of adults report receiving news from social media [39]. In addition to sharing and consuming news media, many people use social media to express personal experiences with mental health and suicide, which can further contribute to the spread of suicidal behavior [40]. Consequently, adapting TEMPOS to be suitable for assessing social media content is an especially important direction for future work. One promising example is the #Chatsafe Project in Australia, which introduced a set of evidence-based guidelines aimed at helping young people communicate about suicide safely online [41]. Future work can draw upon the #Chatsafe project and other research on the relationship between social media and suicide contagion to adapt TEMPOS for use with social media content.
Second, applying TEMPOS to every type of news article about suicide proved difficult. While the scale was easily applied to reports on individual suicide deaths, assessing articles that addressed suicide more broadly (e.g., articles discussing suicide trends, the general topic of suicide, or other related issues) was more challenging. Although efforts were made to expand the functionality of the tool (e.g., adding a “not applicable” option for some criteria), further work is needed to optimize the scale for application across a wide range of media types.
Third, under the current scoring system, all criteria are considered equal in significance when calculating a total score. While all ten criteria are important aspects of responsible reporting, failure to adhere to certain guidelines may result in higher risk for imitative behavior than others. For example, sharing specific details of the suicide method and location may be more likely to elicit copycat suicide attempts than the use of stigmatizing language [18]. Further development of the rating scale might consider assigning weights to criteria based on specific elements of reporting that have been identified as being most harmful. Furthermore, elements such as article size, tone, and ratio of positive to negative content are potentially important factors underlying the risk of imitative behavior [20,32,34]. Future iterations of TEMPOS may choose to leverage text analysis software programs such as Linguistic Inquiry & Word Count (LIWC) [42] to automatically calculate such variables and factor them into the overall TEMPOS score.
Finally, although TEMPOS offers a relatively efficient approach to measuring adherence, newsrooms often operate on extremely tight timelines. A primary aim of this work was to create a rating system that was complex enough to capture more nuance than a binary rating system, yet still accessible enough to be used by media professionals who are unfamiliar with the topic of suicide contagion. Ideally, media professionals would be able to use TEMPOS as a ‘self-check’ tool to assess the adherence levels of their content prior to publishing, and thereby reduce the risk of releasing harmful content. However, applying the scale prior to publication may not be realistic or feasible in the context of breaking news stories (e.g., the suicide of a high-profile figure). One way to further increase the practical utility of TEMPOS would be the use of artificial intelligence to automate evaluation of adherence to the reporting recommendations. Recent advances in machine learning and natural language processing may present an opportunity for TEMPOS to become partially or fully automated, which would allow for a wider range of applications. TEMPOS could also be used to intervene “upstream” by increasing reporters’ awareness of the reporting recommendations prior to scenarios in which they would need to apply them. For example, schools of journalism and publication companies could incorporate TEMPOS into their curricula, standards of practice, and professional training.

5. Conclusions

The application of the Tool for Evaluating Media Portrayals of Suicide (TEMPOS) has the potential to dramatically change how suicide is discussed and ultimately perceived. TEMPOS is a novel, user-friendly, and reliable tool for assessing adherence to suicide reporting recommendations that can be used by researchers, suicide prevention professionals, and media professionals alike. A key strength of TEMPOS is that it acknowledges the nuances in communication around suicide and the complexity of suicidal behaviors. In a departure from other rating scales, which typically employ binary measures of adherence, TEMPOS’s three-point scale allows raters to capture more nuance in adherence to reporting recommendations. As illustrated in this study, TEMPOS makes it possible to examine how media adherence to suicide reporting recommendations varies among criteria, across and within publications, and over time. Suicide prevention professionals seeking to work with media outlets to increase adherence can use TEMPOS to develop more targeted programming, both preemptively through education and training and during more urgent periods, such as during heightened-risk periods following surges of suicide-related coverage. Internally, media organizations can employ TEMPOS as an ongoing tool for self-assessment and monitoring. Ultimately, TEMPOS provides a platform for promoting widespread awareness of how reporting on suicide impacts individuals and communities, potentially leading to reduced stigma and improved visibility of a looming public health issue that is not commonly discussed.

Supplementary Materials

The following are available online at The Tool for Evaluating Media Portrayals of Suicide, full rating scale.

Author Contributions

Conceptualization, C.C.S., M.L., V.H., J.J.D., J.S.K., N.T.T., S.V.J. and S.G.P.; Formal analysis, C.C.S.; Funding acquisition, M.L.; Investigation, C.C.S., M.L., V.H., J.J.D., J.S.K., N.T.T. and S.G.P.; Methodology, C.C.S., M.L., V.H., J.J.D., J.S.K., N.T.T., S.V.J. and S.G.P.; Project administration, M.L., V.H. and S.H.K.; Resources, M.L. and V.H.; Supervision, M.L., V.H. and S.G.P.; Validation, S.V.J.; Visualization, C.C.S.; Writing—original draft, C.C.S. and M.L.; Writing—review & editing, C.C.S., M.L., V.H., J.J.D., J.S.K., S.H.K., N.T.T., S.V.J. and S.G.P. All authors have read and agreed to the published version of the manuscript.


This research was funded by Proposition 63, the California Mental Health Services Act.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The dataset of articles and TEMPOS ratings can be provided upon request.


Thank you to our panel of expert reviewers: Madelyn Gould, Sansea Jacobson, Thomas Niederkrotenthaler, Jane Pirkis, and Dan Reidenberg. Thank you also to Charlotte Pohl and Jasmine Lopez for assisting with the article rating process, and to Jeanne Tsai and Maren Nyer for providing mentorship.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.


  1. World Health Organization. Suicide in the World: Global Health Estimates; World Health Organization: Geneva, Switzerland, 2019.
  2. Ostertag, L.; Golay, P.; Dorogi, Y.; Brovelli, S.; Bertran, M.; Cromec, I.; Vaeren, B.V.D.; Khan, R.; Costanza, A.; Wyss, K.; et al. The implementation and first insights of the french-speaking Swiss programme for monitoring self-harm. Swiss Med. Wkly. 2019, 149, w20016. [Google Scholar] [CrossRef] [Green Version]
  3. Griffin, E.; McMahon, E.; McNicholas, F.; Corcoran, P.; Perry, I.J.; Arensman, E. Increasing rates of self-harm among children, adolescents and young adults: A 10-year national registry study 2007–2016. Soc. Psychiatry Psychiatr. Epidemiol. 2018, 53, 663–671. [Google Scholar] [CrossRef] [PubMed]
  4. Twenge, J.M. Increases in depression, self-harm, and suicide among U.S. adolescents after 2012 and links to technology use: Possible mechanisms. Psychiatr. Res. Clin. Pract. 2020, 2, 19–25. [Google Scholar] [CrossRef] [Green Version]
  5. Biddle, L.; Derges, J.; Goldsmith, C.; Donovan, J.L.; Gunnell, D. Using the internet for suicide-related purposes: Contrasting findings from young people in the community and self-harm patients admitted to hospital. PLoS ONE 2018, 13, e0197712. [Google Scholar] [CrossRef] [PubMed]
  6. Arendt, F.; Scherr, S.; Romer, D. Effects of exposure to self-harm on social media: Evidence from a two-wave panel study among young adults. New Media Soc. 2019, 21, 2422–2442. [Google Scholar] [CrossRef] [Green Version]
  7. Phillips, D.P. The influence of suggestion on suicide: Substantive and theoretical implications of the werther effect. Am. Sociol. Rev. 1974, 39, 340–354. [Google Scholar] [CrossRef] [PubMed]
  8. Fu, K.-W.; Yip, P.S.F. Estimating the risk for suicide following the suicide deaths of 3 asian entertainment celebrities: A meta-analytic approach. J. Clin. Psychiatry 2009, 70, 869–878. [Google Scholar] [CrossRef] [PubMed]
  9. Niederkrotenthaler, T.; Fu, K.; Yip, P.S.F.; Fong, D.Y.T.; Stack, S.; Cheng, Q.; Pirkis, J. Changes in suicide rates following media reports on celebrity suicide: A meta-analysis. J. Epidemiol. Community Health 2012, 66, 1037–1042. [Google Scholar] [CrossRef] [PubMed]
  10. Pirkis, J.; Blood, R.W. Suicide and the media: Part I. reportage in nonfictional media. Crisis J. Crisis Interv. Suicide Prev. 2001, 22, 146–154. [Google Scholar] [CrossRef] [PubMed]
  11. Sisask, M.; Varnik, A. Media roles in suicide prevention: A systematic review. Int. J. Environ. Res. Public Health 2012, 9, 123–138. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Stack, S. Media impacts on suicide: A quantitative review of 293 findings. Soc. Sci. Q. 2000, 81, 957–971. [Google Scholar]
  13. Gould, M.S.; Kleinman, M.H.; Lake, A.M.; Forman, J.; Midle, J.B. Newspaper coverage of suicide and initiation of suicide clusters in teenagers in the USA, 1988–1996: A retrospective, population-based, case-control study. Lancet Psychiatry 2014, 1, 34–43. [Google Scholar] [CrossRef]
  14. Cheng, A.T.A.; Hawton, K.; Chen, T.H.H.; Yen, A.M.F.; Chang, J.-C.; Chong, M.-Y.; Liu, C.-Y.; Lee, Y.; Teng, P.-R.; Chen, L.-C. The influence of media reporting of a celebrity suicide on suicidal behavior in patients with a history of depressive disorder. J. Affect. Disord. 2007, 103, 69–75. [Google Scholar] [CrossRef] [PubMed]
  15. Zahl, D.L.; Hawton, K. Media influences on suicidal behaviour: An interview study of young people. Behav. Cogn. Psychother. 2004, 32, 189–198. [Google Scholar] [CrossRef] [Green Version]
  16. Gould, M.; Jamieson, P.; Romer, D. Media contagion and suicide among the young. Am. Behav. Sci. 2003, 46, 1269–1284. [Google Scholar] [CrossRef] [Green Version]
  17. Lee, J.; Lee, W.-Y.; Hwang, J.-S.; Stack, S.J. To what extent does the reporting behavior of the media regarding a celebrity suicide influence subsequent suicides in South Korea? Suicide Life-Threat. Behav. 2014, 44, 457–472. [Google Scholar] [CrossRef] [PubMed]
  18. Sinyor, M.; Schaffer, A.; Nishikawa, Y.; Redelmeier, D.A.; Niederkrotenthaler, T.; Sareen, J.; Levitt, A.J.; Kiss, A.; Pirkis, J. The association between suicide deaths and putatively harmful and protective factors in media reports. CMAJ 2018, 190, E900–E907. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Whitley, R.; Fink, D.S.; Santaella-Tenorio, J.; Keyes, K.M. Suicide mortality in Canada after the death of robin williams, in the context of high-fidelity to suicide reporting guidelines in the Canadian media. Can. J. Psychiatry 2019, 64, 805–812. [Google Scholar] [CrossRef] [PubMed]
  20. Niederkrotenthaler, T.; Voracek, M.; Herberth, A.; Till, B.; Strauss, M.; Etzersdorfer, E.; Eisenwort, B.; Sonneck, G. Role of media reports in completed and prevented suicide: Werther v. papageno effects. Br. J. Psychiatry 2010, 197, 234–243. [Google Scholar] [CrossRef] [Green Version]
  21. Etzersdorfer, E.; Sonneck, G. Preventing suicide by influencing mass-media reporting. the viennese experience 1980–1996. Arch. Suicide Res. 1998, 4, 67–74. [Google Scholar] [CrossRef]
  22. Niederkrotenthaler, T. A Suicide-protective papageno effect of media portrayals of coping with suicidality. Inj. Prev. 2016, 22, A8–A9. [Google Scholar] [CrossRef] [Green Version]
  23. World Health Organization; International Association for Suicide Prevention. Preventing Suicide: A Resource for Media Professionals, 2017 Update; World Health Organization: Geneva, Switzerland, 2017.
  24. SAVE Reporting on Suicide. Available online: (accessed on 4 January 2022).
  25. Canadian Journalism Forum on Violence and Trauma Mindset: Reporting on Mental Health. Available online: (accessed on 25 January 2022).
  26. Hunter Institute of Mental Health. Reporting Suicide and Mental Illness: A Mindframe Resource for Media Professionals; Hunter Institute of Mental Health: Newcastle, Australia, 2014. [Google Scholar]
  27. Niederkrotenthaler, T.; Sonneck, G. Assessing the impact of media guidelines for reporting on suicides in Austria: Interrupted time series analysis. Aust. N. Z. J. Psychiatry 2007, 41, 419–428. [Google Scholar] [CrossRef] [PubMed]
  28. Barker, E.; Kolves, K.; Leo, D.D. Rail-suicide prevention: Systematic literature review of evidence-based activities. Asia Pac. Psychiatry 2017, 9, e12246. [Google Scholar] [CrossRef] [PubMed]
  29. Creed, M.; Whitley, R. Assessing fidelity to suicide reporting guidelines in Canadian news media: The death of robin williams. Can. J. Psychiatry 2017, 62, 313–317. [Google Scholar] [CrossRef]
  30. McTernan, N.; Spillane, A.; Cully, G.; Cusack, E.; O’Reilly, T.; Arensman, E. Media reporting of suicide and adherence to media guidelines. Int. J. Soc. Psychiatry 2018, 64, 536–544. [Google Scholar] [CrossRef]
  31. Pirkis, J.; Dare, A.; Blood, R.W.; Rankin, B.; Williamson, M.; Burgess, P.; Jolley, D. Changes in media reporting of suicide in Australia between 2000/01 and 2006/07. Crisis 2009, 30, 25–33. [Google Scholar] [CrossRef]
  32. Gould, M.; Midle, J.B.; Insel, B.; Kleinman, M. Suicide reporting content analysis: Abstract development and reliability. Crisis 2007, 28, 165–174. [Google Scholar] [CrossRef] [PubMed]
  33. Pitman, A.; Stevenson, F. Suicide reporting within British newspapers’ arts coverage: Content analysis of adherence to media guidelines. Crisis J. Crisis Interv. Suicide Prev. 2015, 36, 13–20. [Google Scholar] [CrossRef] [PubMed]
  34. Nutt, R.; Kidd, B.; Matthews, K. Assessing the adherence to guidelines of media reporting of suicide using a novel instrument--the “Risk of imitative suicide scale” (RISc). Suicide Life-Threat. Behav. 2015, 45, 360–375. [Google Scholar] [CrossRef] [PubMed]
  35. Sumner, S.A.; Burke, M.; Kooti, F. Adherence to suicide reporting guidelines by news shared on a social networking platform. Proc. Natl. Acad. Sci. USA 2020, 117, 16267–16272. [Google Scholar] [CrossRef]
  36. Garcia-Williams, A.; O’Donnell, J.; Spies, E.; Zhang, X.; Young, R.; Azofeifa, A.; Vagi, K. Epi-Aid 2016–018: Undetermined Risk Factors for Suicide Among Youth, Ages 10–24—Santa Clara County, CA, 2016; Centers for Disease Control and Prevention: Atlanta, GA, USA, 2017. [Google Scholar]
  37. McHugh, M.L. Interrater reliability: The kappa statistic. Biochem. Med. 2012, 22, 276–282. [Google Scholar] [CrossRef]
  38. Sheftall, A.H.; Tissue, J.L.; Schlagbaum, P.; Singer, J.B.; Young, N.; Stevens, J.H.; Ackerman, J.P. Newspaper adherence to media reporting guidelines for the suicide deaths of kate spade and anthony bourdain. JAMA Netw. Open 2019, 2, e1914517. [Google Scholar] [CrossRef] [PubMed]
  39. Shearer, E.; Mitchell, A. News use across social media platforms in 2020. Pew Res. Cent. J. Proj. 2021. Available online: (accessed on 28 January 2022).
  40. Luxton, D.D.; June, J.D.; Fairall, J.M. Social media and suicide: A public health perspective. Am. J. Public Health 2012, 102, S195–S200. [Google Scholar] [CrossRef] [PubMed]
  41. Robinson, J.; Hill, N.T.M.; Thorn, P.; Battersby, R.; Teh, Z.; Reavley, N.J.; Pirkis, J.; Lamblin, M.; Rice, S.; Skehan, J. The #chatsafe project. developing guidelines to help young people communicate safely about suicide on social media: A delphi study. PLoS ONE 2018, 13, e0206584. [Google Scholar] [CrossRef] [Green Version]
  42. Pennebaker, J.W.; Francis, M.E.; Booth, R.J. Linguistic inquiry and word count: LIWC 2001. Mahway Lawrence Erlbaum Assoc. 2001, 71, 2001. [Google Scholar]
Figure 1. TEMPOS Development Process.
Figure 1. TEMPOS Development Process.
Ijerph 19 02994 g001
Figure 2. Surge in Articles Following Two High Profile Deaths by Suicide.
Figure 2. Surge in Articles Following Two High Profile Deaths by Suicide.
Ijerph 19 02994 g002
Figure 3. Distribution of TEMPOS scores.
Figure 3. Distribution of TEMPOS scores.
Ijerph 19 02994 g003
Figure 4. Average Scores for Evaluation Criteria.
Figure 4. Average Scores for Evaluation Criteria.
Ijerph 19 02994 g004
Figure 5. TEMPOS scores, sorted by publication. Each light blue point represents an article; dark blue diamonds indicate publication averages. Error bars represent standard error of the mean. Publications with less than 3 articles in the dataset were dropped from the plot.
Figure 5. TEMPOS scores, sorted by publication. Each light blue point represents an article; dark blue diamonds indicate publication averages. Error bars represent standard error of the mean. Publications with less than 3 articles in the dataset were dropped from the plot.
Ijerph 19 02994 g005
Figure 6. TEMPOS scores over time. Red and green dashed lines represent the deaths of Kate Spade and Anthony Bourdain, respectively.
Figure 6. TEMPOS scores over time. Red and green dashed lines represent the deaths of Kate Spade and Anthony Bourdain, respectively.
Ijerph 19 02994 g006
Table 1. Overview of TEMPOS Criteria.
Table 1. Overview of TEMPOS Criteria.
#Criterion Name
1Framing of Suicide
2Factual & Non-Speculative Information
3Appropriate/Non-Stigmatizing Language
4Details About Suicide Method/Scene
5Details About Suicide Note
6Visual Content
7Reasons & Risk Factors for Suicide
8Sensational Language
9Glamorization of Suicide
10Suicide Prevention/Mental Health Resources
Note: The full TEMPOS rating scale has been included as a Supplementary Material.
Table 2. Article Characteristics.
Table 2. Article Characteristics.
Characteristicn (%)
News89 (40.5%)
Blog46 (20.9%)
Health28 (12.7%)
Entertainment24 (10.9%)
Other24 (10.9%)
Opinion5 (2.27%)
Sports4 (1.82%)
National (USA)178 (80.9%)
Local (SF Bay Area)42 (19.1%)
Publication Type
Broadcast News Company115 (52.3%)
Magazine60 (27.3%)
Newspaper51 (23.2%)
Table 3. Pure Agreement and Weighted Kappa Values by Criterion.
Table 3. Pure Agreement and Weighted Kappa Values by Criterion.
Criterion% Pure Agreementκ (Linear Weights)
Framing of Suicide 73.64%0.513
Factual & Non-Speculative Information68.18%0.452
Appropriate/Non-Stigmatizing Language83.10%0.587
Details About Suicide Method/Scene91.82%0.941
Details About Suicide Note97.27%0.975
Visual Content 92.27%0.832
Reasons & Risk Factors for Suicide76.82%0.569
Sensational Language73.18%0.378
Glamorization of Suicide75.90%0.173
Suicide Prevention/Mental Health Resources80.90%0.743
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sorensen, C.C.; Lien, M.; Harrison, V.; Donoghue, J.J.; Kapur, J.S.; Kim, S.H.; Tran, N.T.; Joshi, S.V.; Patel, S.G. The Tool for Evaluating Media Portrayals of Suicide (TEMPOS): Development and Application of a Novel Rating Scale to Reduce Suicide Contagion. Int. J. Environ. Res. Public Health 2022, 19, 2994.

AMA Style

Sorensen CC, Lien M, Harrison V, Donoghue JJ, Kapur JS, Kim SH, Tran NT, Joshi SV, Patel SG. The Tool for Evaluating Media Portrayals of Suicide (TEMPOS): Development and Application of a Novel Rating Scale to Reduce Suicide Contagion. International Journal of Environmental Research and Public Health. 2022; 19(5):2994.

Chicago/Turabian Style

Sorensen, Chloe Chang, Mego Lien, Vicki Harrison, John J. Donoghue, Jeevanjot Singh Kapur, Song Hi Kim, Nhi Thi Tran, Shashank V. Joshi, and Sita G. Patel. 2022. "The Tool for Evaluating Media Portrayals of Suicide (TEMPOS): Development and Application of a Novel Rating Scale to Reduce Suicide Contagion" International Journal of Environmental Research and Public Health 19, no. 5: 2994.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop