1. Introduction
The information flow among researchers and scholars experienced a major change in recent decades. In the “Social Web”, scholars can still cite a paper but also upload it to their website, tweet it, bookmark it, blog it, download it, share it, or post an online review. The general public can also react to a scientific finding in an easy and fast way, thanks to the wide diffusion of social media platforms and their easy accessibility. The online activity surrounding a published item can be tracked and can define an article’s impact in a different, broader, and updated way [
1]. When the impact of a research output on social media is assessed, it should be considered that the general public could have limited knowledge on some scientific topics.
Altmetric (Altmetric LLP, London, UK) is an altmetrics aggregator that tracks the online activity about a published item on social media (Facebook, Twitter, blogs); in mainstream media (
The Guardian,
The New York Times); in science-specific media (
Scientific American,
New Scientist); and in online reference managers (Mendeley, CiteULike). A specific algorithm calculates an Altmetric Attention Score (AAS) for an article, based on its mentions in those sources [
2]. Moreover, Almetric generates a graphic summary of the article’s impact in which the different colors represent a different altmetric resource (
Figure 1), enabling the reader to quickly assess which resources contributed most to the global score. Altmetric has already been implemented by many publishers in the scientific field. Altmetric is a subscription service that provides to customers all of the application’s services and tools but also offers a free widget that allows retrieving the AAS and other basic Altmetric data [
3].
Relying on the AAS, a reader can filter the published items and detect which articles are catalyzing more interest [
4]. According to Elmore in 2018, many young researchers are already accustomed to disseminating their publications using social media platforms [
5]. It should be mentioned that the majority of funding institutions (such as the US National Science Foundation) expect that the funded scientific projects are relevant from a scientific standpoint but also present a social impact. According to some authors, Altmetric does not only track the online dissemination of research on social and mainstream media but also plays a role in the definition of its societal impact [
6].
Due to the easy and fast access to social media platforms and the activity of spamming profit-driven companies trading on users’ votes, tweets, and profiles, data manipulation is one of the main concerns about Altmetric. To limit any unfair activity, Altmetric declared that it uses a specific algorithm to highlight artificial patterns of attention or detect social media automation tools [
7].
The objective of the current study is to assess the online attention to research in the field of legal medicine and to explore the actual correlation between the classic citation count in Web of Science (WOS), Scopus, and Dimensions and the AAS.
2. Materials and Methods
On 22 September 2020, a search, not limited in time nor in language, was performed through the free Dimensions app (
https://app.dimensions.ai/discover/publication) accessed on 22 September 2020. Dimensions is provided by Digital Science & Research Solutions Inc., London, UK. The search was limited to the sixteen journals which were listed in the Legal Medicine category in the 2019 edition of the Journal Citation Reports (JCR) 2019. The journals were: “
Legal Medicine”, “
Regulatory Toxicology and Pharmacology”, “
Journal of Law and the Biosciences”, “
International Journal of Legal Medicine”, “
Forensic Science International”, “
Science & Justice”, “
Forensic Science”, “
Medicine and Pathology”, “
Medical Law Review”, “
Journal of Forensic Sciences”, “
Journal of Forensic and Legal Medicine”, “
Australian Journal of Forensic Sciences”, “
American Journal of Forensic Medicine & Pathology”, “
Rechtsmedizin”, “
Romanian Journal of Legal Medicine”, “
The Journal of Law, Medicine & Ethics”, and “
Medicine Science and the Law”.
The search retrieved a total of 66,308 items that were then ordered by Altmetric Attention Score in descending order. The first 200 items were then exported to an Excel datasheet (Office for Mac 2011 package format, Microsoft, Albuquerque, New Mexico, USA). The citation counts were extracted from the corresponding databases (WOS, Scopus, and Dimensions). Although a few studies about classical citation advocate the use of the h-index as a reference for sample size determination, there is not a strong consensus on how to determine the sample size [
8]. In the current literature, many bibliometric studies included the top-100 articles in terms of AAS [
9,
10], but it is common to retrieve articles by analyzing only 50 or 20 articles [
11,
12]. Considering that the h-index for the items published by the 16 analyzed journals in the last 3 decades was 160 at the time of the search, a sample size of 200 articles can be considered representative.
Two researchers ( ) screened the retrieved published items and extracted, by consensus, information regarding: (1) article title; (2) journal title; (3) DOI; (4) time interval since publication; (5) number of citations in the Dimensions database; (6) Altmetric Attention Score; (7) number of authors and affiliations; (8) type of the affiliation of the authors, i.e., university or other; (9) origin of the article (as defined by the corresponding author), i.e., USA, Canada, Italy, China; (10) article subject, i.e., forensic anthropology, forensic pathology, toxicology and pharmacology, or other; (11) full-text availability, i.e., free full text or subscription required; (12) funding, i.e., whether the study was funded or not. All reviewed articles were classified according to the above-mentioned subcategories for each area of interest. Additionally, citation counts were extracted from the expanded science citation index of WOS (property of Clarivariate), from Scopus (registered trademark of Elsevier BV), and Dimensions. It was decided to gather citations from three different databases because WOS and Scopus, despite the differences in journal indexing, with Scopus having larger numbers of publications, can be considered a gold standard, while Dimensions as a novel database has not been extensively validated.
The different Altmetric data resources as well as the Twitter demographic data were harvested through the Dimensions search engine.
The websites of the journals were accessed to verify which journals were offering the altmetrics data of the published articles. When offering these data, the altmetrics provider was recorded.
The stratification by study subjects was based on the categories proposed by other bibliometric studies published in the field of forensic medicine [
13]. If an article did not fit in one of the proposed categories, a new one was created based on the article’s main topic [
14,
15].
Bibliometric data of the selected articles were further analyzed using the VOSviewer 1.6.6 software that constructs and visualizes bibliometric networks (
http://www.vosviewer.com/, Leiden University Centre for Science and Technology Studies, accessed on 20 February 2020). The science mapping of the top-200 articles was performed on the basis of keywords’ co-occurrence and co-authorship network analysis.
Descriptive statistics using counts and proportions were used to describe the articles and journals included in the study. The Pearson’s correlation coefficient (r) was employed to assess the association between citation counts for individual articles, the article AAS, and the total citation count in the analyzed databases.
3. Results
The top-200 articles were published by fourteen journals out of the sixteen included in the analysis and are listed in
Supplementary Table S1.
As reported in
Table 1, 75% of the articles were published by five journals: “
Journal of Forensic Sciences”, “
Forensic Science International”, “
Regulatory Toxicology and Pharmacology”
, “
Journal of Law and the Biosciences”, and “
The Journal of Law, Medicine & Ethics”. According to the data in
Table 1, a variable but lower online attention was displayed by the other journals included in the sample.
The “
Journal of Forensic Sciences” was the most prevalent journal with 41 items (21%), followed by “
Forensic Science International” with 38 items, equal to 19% of the retrieved items (
Table 1).
“Regulatory Toxicology and Pharmacology” displayed the highest number of citations per article. The “Journal of Forensic and Legal Medicine” published 12 of the 200 most discussed articles but displayed the highest mean AAS per article, especially thanks to the article by Yasemin et al. (2007) about simultaneous sudden infant death syndrome and the one by Won et al. (2020) about COVID-19 in forensic medicine unit personnel. These articles were the second and third most discussed articles ever in legal medicine.
A breakdown of the Altmetric data that contributed most to the global attention score of the different journals is presented in
Table 2. In all journals, the number of
Mendeley users was the most prevalent data, but the algorithm does not account for these data when the global AAS of an article is calculated. The reason behind is that since the users’ profiles are anonymized, these data are not fully auditable.
Tweets were the most popular data resource in every journal. According to Twitter demographic data, the research stakeholders were involved in 21.8% of the tweets, while the vast majority were authored by the general public.
News outlets were the second most common resource for the rest of the journals except for the “Journal of Forensic and Legal Medicine”, that displayed a significant activity on Facebook. Blogs and Facebook were in the third position for most of the journals. A high number of patents (26) were related to the items published by “Regulatory Toxicology and Pharmacology”, while 9 were related to “Forensic Science International” and 1 to the “Journal of Forensic Science”. “Regulatory Toxicology and Pharmacology” was also the journal with the highest number of policy sources, followed by the “Journal of Law, Medicine & Ethics”.
The oldest articles in the list were published in 1988, but 73.5% of the articles were published between 2013 and 2020 and another 19.5% between 2005 and 2012 (
Figure 2).
The top-200 articles involved 740 authors and 444 institutions from 38 countries all around the world, 64.33% of them were universities. Published items had a mean of 2.94 authors. Byard, RW (University of Adelaide), and Ross, AH (University of North Carolina) with four published items were the ones who authored the most articles. Ross, AH was the first author of one of them while Byard first authored three of them. The authors belonged mainly to North American (42.9%) and European institutions (42.5%), with these regions being almost equally represented, followed by Oceania (6%) and Asia (4.8%).
Free full text was available (Gold Open Access) for 58 articles, while Green Open Access was available for 25 and a subscription was required for 117 articles. Funding was reported in 78 articles, with the research projects being funded by 88 agencies. Following governmental and nonprofit funding bodies, it was possible to highlight the presence of large enterprises such as Fontem Venture (Amsterdam, Netherlands) a leader in vaping technology, Monsanto (San Luis, Missouri, USA), who developed and patented the use of glyphosate under the Roundup brand name, or British American Tobacco (London, UK) among other private stakeholders.
The most frequent topics were the ones related to toxicology and pharmacology, followed by criminalistics and law and bioethics; the topic forensic pathology was the one that received the highest AAS per item (
Table 3). Science mapping via keyword occurrence displayed a high prevalence of keywords related to toxicology and forensic anthropology as well as ethics and law (
Figure 3).
The science mapping via author co-occurrence displayed how out of 740 authors only 5 clusters of 37, 30, 5, 4, and 3 authors were interconnected. Greely, HT (Faculty of Law, Stanford University, Stanford, CA, USA), Bush, MA (Buffalo School of Dental Medicine, Buffalo, NY, USA), and Caulfield, T (Faculty of Law, University of Alberta, Canada) were the authors with the strongest collaboration links (
Figure 4).
No correlation could be assessed between the AAS and the number of citations in WOS (r = −0.006,
p > 0.05), Scopus (r = −0.009,
p > 0.05), and Dimensions (r = 0.009,
p > 0.05). On the other hand, the number of citations in WOS and the total number of citations in Dimensions (r = 0.979,
p < 0.01) and Scopus (r = 0.982,
p < 0.01) displayed a strong positive correlation (
Table 4).
4. Discussion
To the best knowledge of the authors, this is the first article exploring the online attention to research in legal medicine. The search was performed on the JCR journals listed in the category “Legal Medicine”, theoretically allowing us to conduct a search involving all the topics related to this specific discipline and thus avoiding the narrowing to specific subtopics that a keyword-driven search would have introduced.
Reviewing the aims and scopes of the involved journals, it could be appreciated how a very broad range of topics was included in such a search. The frame of the included disciplines embraces the legal medicine field in the broader sense, including: forensic medicine training; forensic medicine national systems; forensic pathology and histochemistry; human identification; profiling; mass disasters/mass graves; imaging; wound assessment; sexual assault; child abuse and neglect; chemistry; biochemistry; toxicology; toxicological and pharmacological regulations in society; biology; DNA analysis; entomology, botany; serology; odontology; psychiatry; anthropology; archaeology; criminalistics; digital forensics; the physical sciences; firearms; questioned documents; and the important and wide area where the biosciences interact with the law and ethics.
The altmetrics analysis offers a very complex scenario, that is, the natural reflection of an area that sits at the intersection of medicine, science, law, policing, and policy. As reported in
Table 1, journals that lack an altmetrics provider or use a different altmetrics provider from Altmetric such as PlumX can still achieve a significant online engagement, suggesting that having an altmetrics provider is not a required condition to attract online interest. The same consideration applies to the open access (OA) format that theoretically facilitates access to a wider and nonspecialized public than subscription-based formats. Since the vast majority of the articles had some sort of access restriction, the correlation between online engagement and Gold Open Access could not be clearly supported. This finding is in disagreement with what was reported by other authors in the lumbar spine literature and in sport sciences journals where OA status appeared to significantly impact the AAS [
16,
17].
The high number of news outlets is a distinctive feature of the most-discussed articles in legal medicine, underlining the high interest of media in the topic. The tweet to news outlet ratio was 6.6 in our sample and 33.3 in gynecology [
18] and 15.5 in spine surgery [
19]. The media attention on the research outputs probably stems, on one hand, from the impact that this discipline has on the everyday life of almost every individual and, on the other, from the human attraction towards mystery and thrills. Thousands of convictions have been based in recent decades upon forensic analyses, evidence, and testimony. The development of more sophisticated means has made it possible to reopen and successfully close a large number of cold cases. The article with the highest AAS ever in the field is about the forensic assessment of a shawl linked to the “Jack the Ripper” murders, probably the most famous unidentified serial killer (
Table S1). From the 1976 series
Quincy, featuring a Los Angeles county chief medical examiner working on suspicious deaths, to the 2000 box office hit series
CSI: Crime Scene Investigation, where the bureau officers work with the most sophisticated scientific techniques to solve crimes and convict criminals, forensic science has been a trending subject for the entertainment industry.
CSI lasted 15 years on screen with more than 300 episodes and had such a deep influence on laypeople that the so-called CSI effect was described. The effect can alter a trial because of the possible alterations of jurors’ decisions derived from the viewing of forensic or criminal-based fiction [
20]. With such a sensitized public, the opportunistic use of the research outputs should be of special concern. As underlined by DesPortes in 2018, forensic science is regarded by the media as part of law enforcement, being considered an exclusive tool of the prosecution whose final target is to convict people [
21]. It is not surprising then that articles about the reliability of forensic procedures and methods can attract considerable online attention. This is the case for
Could secondary DNA transfer falsely place someone at the scene of a crime? by Cale et al. that had a strong influence on social and traditional media, whose headlines were
How DNA evidence imprisons the innocent (
Table S1). Once again, forensic procedures are regarded as partial, while the authors were just using scientific methods to draw balanced and reliable conclusions, based on evidence and objective analysis.
The second most-discussed article was about a case of simultaneous sudden infant death. Twins had received the second dose of oral polio and the first dose of hepatitis B vaccine. This article, similar to the one by D’Errico et al., 2008 (
Table S1), was highly disseminated on social media by anti-vaxxer groups that are known to be especially active on these platforms. The online activity about these two articles was mainly linked to tweets and Facebook posts, being the articles with the second and third highest number of Facebook posts. The finding is significant in the light of the new report by the Center for Countering Digital Hate (CCDH) that highlights how anti-vaxxer groups have more than 31 million followers on Facebook. The authors reported that the anti-vaxxer group-related accounts on social media have increased their following by at least 7 to 8 million people since 2019 [
22]. In 2019, several social media firms committed to acting against the misinformation promoted by the anti-vaxxer movement. Twitter, for example, announced the launch of a new search tool that would direct users to the National Health Service or Department of Health and Human Services accounts when searching for vaccine-related topics. On the opposite side, two articles by Caulfield et al., 2020 and Silverman et al., 2017 (
Table S1) explore the caveats of the anti-vaccine approach and the ways to implement successful vaccination policies. The online activity about these two articles was linked not only to tweets and Facebook posts but also to news outlets that have a high relative weight in the AAS calculation.
In our sample, the vast majority of the articles belonged to the last decade, contrary to what happened in the bibliometric studies by Jones in 2006 and Lei et al. in 2019 where the most cited articles were at least a decade old. When analyzing this finding, it should considered that no time limits were applied to the search while Altmetric was introduced around 2010, as well as the most of the tools for altmetrics tracking. Online engagement tracking could be biased due to the irregular coverage of academic sources. It should be underlined that most of the top Altmetric articles did not introduce a time limit in the search [
23,
24].
Due to the immediacy of the altmetrics, the trending articles can render an updated snapshot of the current time displaying what laypeople consider valuable from their standpoint. In such an up to date scenario, the COVID-19 pandemic is clearly present, although its prevalence among the most-discussed articles is maybe not as high as expected. Just six out of two hundred articles were focused on the topic. It is interesting to remark how only two of these articles were related to the forensic pathology of the disease, while the others were examining aspects such as vaccination access, profit-driven models of the pharmaceuticals of the vaccine, and ethnic minorities’ disparities in terms of virus exposure, susceptibility, treatment, and death rates. These findings highlight how the concern of laypeople about the pandemic is multifaceted. As discussed earlier, vaccines are per se a trending topic in many aspects.
The science mapping approach by keyword co-occurrence highlighted how a relevant number of articles were related to toxicity and toxicology. A cluster of keywords involved cocaine, marijuana, and cannabis, another cluster connected the keyword overdose with morphine, drugs, and alcohol, while a third cluster connected glyphosate with toxicity. The attention is high on drug abuse almost all over the world, but especially in the US where the opioid crisis caused in 2019 a death toll of more than 130 people per day. According to the 2019 National Survey of Drug Use and Health (NSDUH), 57,203,000 people in the US used illicit drugs in 2019, 4,021,000 more than what was reported in 2018. Taking into account the dimension of drug abuse, the high attention on the topic is somewhat expected. The high attention on glyphosate toxicity reflects the controversies related to the glyphosate-based weed killer Roundup. Since a report of the International Agency for Research on Cancer (IARC) concluded the herbicide’s main component, glyphosate, was probably carcinogenic in humans [
25], the controversy is still ongoing, with US regulators in an opposed position [
26]. Despite the European Union extending the commercial license in 2017, many member states decided unilaterally at different times to ban the use of the compound, keeping the attention on the topic high on both social and traditional media.
De Kinder et al., in a recent article, called for enhancing international collaboration among legal medicine stakeholders to allow the transfer of evidence and expert opinions among countries, to allow further forensic analysis or to meet the needs of the prosecution. Moreover, forensic laboratories need to increase their connections on the operational side [
27]. The results of our altmetrics data and the science mapping approach to co-authorships display that the level of collaboration, at least on the research side, should be enhanced as suggested by the low number of authors per published item and the scarcity of collaboration links. This result is in disagreement with what is usually highlighted in bibliometric studies on most-cited articles that is related to them having a higher number of authors compared to the less-cited ones. Most cited articles are usually systematic reviews and randomized clinical trials (RCTs), and both study designs usually depend on significant teamwork due to their complexity [
28]. The low percentage of university institutions linked to the published items, when compared to other branches of health science [
15,
29], is a peculiar finding of this area and it is probably because forensic science is usually linked to governmental or private laboratories far apart from the academic environment. According to Jones in 2005, in forensic science labs, the managing of daily tasks and administrative duties are considered more worthwhile for staff promotion than research and publications [
30].
The high number of patents in the studied period, especially related to the field of toxicology and pharmacology, highlights a strong collaboration between the research and the pharmaceutical industry. The collaboration between research and industry, in contrast, appears weaker in the field of forensic medicine and pathology.
In our sample, no correlation could be highlighted between the number of citations in WOS, Scopus, and Dimensions and the AAS. This result is consistent with what was reported by other authors, who found no correlation between the citation count in Scopus or WOS and the AAS [
15,
31]; nevertheless, in cardiovascular research, a moderate correlation was found between these indicators [
32]. Wang et al., after assessing 2406 articles published in six PLOS journals, detected that papers with a higher attention score have a higher chance of being highly cited [
3]. Despite their possible correlation, AAS and citations indeed describe two diverse features of a published item. The AAS describes the online attention, while citations indicate how, at an academic level, previous research outputs are used to design, support, or discuss a new research project. To be discussed on social media is, for a newly published item, much faster than getting new citations. To assess an article’s impact through its citations requires a long time. Conversely, Altmetric data are quickly available in most cases. Further research is needed to clarify if and in which aspect the online engagement is related to the academic interest or value of a published item, but it is clear that altmetrics combined with classic citations allow a better assessment of the research outputs, one does not substitute the other.