Next Article in Journal
Research Support Services in Spanish Academic Libraries: An Analysis of Their Strategic Plans and of an Opinion Survey Administered to Their Directors
Previous Article in Journal
Grades of Openness: Open and Closed Articles in Norway
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations

Departamento de Física y Matemáticas, Universidad de Alcalá, Alcalá de Henares, 28871 Madrid, Spain
Publications 2018, 6(4), 47; https://doi.org/10.3390/publications6040047
Received: 6 September 2018 / Revised: 24 October 2018 / Accepted: 20 November 2018 / Published: 26 November 2018

Abstract

:
Journal self-citations may be increased artificially to inflate a journal’s scientometric indicators. The aim of this study was to identify possible mechanisms of change in a cohort of journals that rose from the fourth (Q4) to the first quartile (Q1) over six years or less in Journal Citation Reports (JCR), and the role of journal self-citations in these changes. A total of 51 different journals sampled from all JCR Science Citation Index (SCI) subject categories improved their rank position from Q4 in 2009 to Q1 in any year from 2010 to 2015. I identified changes in the numerator or denominator of the Journal Impact Factor (JIF) that were involved in each year-to-year transition. The main mechanism of change was the increase in the number of citations used to compute the JIF. The effect of journal self-citations in the increase of the JIF was studied. The main conclusion is that there was no evidence of widespread JIF manipulation through the overuse of journal self-citations.

1. Introduction and Objectives

Different criteria are used to evaluate and rank academic journals. Perhaps the most frequently used citation-based ranking is the Journal Impact Factor (JIF), introduced by Eugene Garfield and first published in 1975 by the Institute of Scientific Information (ISI), then by Thomson Reuters, and currently by Clarivate Analytics as part of Journal Citation Reports (JCR) [1,2]. The JIF of a given journal for year Y is calculated according to the following equation [3]:
JIF ( Y ) = Citations _ in _ Y _ to _ documents _ published _ in _ Y 1 _ and _ Y 2 Citable _ items _ published _ in _ Y 1 _ and _ Y 2 .
In Equation (1), Y1 and Y2 are the two years before Y. The “citable items” include only articles and reviews. However, other documents can be cited and they are often cited [4]. Journal rankings of specific research fields are often used for evaluation purposes, both of authors and institutions [5].
The use of reference sets based on Web of Science subject categories became an established practice in evaluative bibliometrics [6]. Many years ago, ISI created subject categories for JCR; these categories were assigned by ISI staff on the basis of a number of criteria including the journal’s title and its citation patterns [7,8]. Many journals appear in more than one category [9].
For a journal editor, increasing a journal’s impact factor may be an important objective [10]. Journal editors often publish editorials and letters in which they explain that one of their main goals is to maintain or improve their JIF or their journal’s rank position in JCR [11,12]. Actually, they can even publish editorials and commissioned opinion articles that cause these increases [13].
In some countries (for example, Spain), academics are awarded economic bonuses for publishing in prestigious journals, especially those ranked highly in their JCR categories [14,15]. Similarly, many Chinese universities pay monetary rewards to staff members who publish Science Citation Index (SCI) articles in journals with a high JIF [16]. Competition to publish in highly ranked journals can be fierce and, as a consequence, journals actively try to increase their rank position.
Pajić studied the stability of seven citation-based journal rankings [17], and discovered that many journals moved from one quartile to another. The plots he published illustrated instances of changes from Q4 to Q1 from one year to the next.
Percentiles are frequently used in journal ranking systems. Currently, many journals remain in the same JCR quartile for years, or move up only in small, single-quartile steps. This makes journals that experience relatively fast transitions from low to high rank positions a strategic research objective.

1.1. Journal Self-Citations and the JIF

According Bornmann and Haunschild, “citations are a target-oriented metric which measures impact on science” ([18], p. 230). Many comments and letters were published on the topic of author and journal self-citation. For example, authors complained that self-citations may be increased artificially to inflate a journal’s scientometric indicators [19,20,21,22,23,24,25,26]. However, as noted by Frandsen, little evidence exists that relates self-citations to the JIF [27].
Rousseau suggested that a high self-citation rate may be an expression of low journal visibility [28]. Peritz and Bar-Ilan studied the fields of bibliometrics and scientometrics, and discovered an increase in journal self-citation [29].
Mirsaeid, Motamedi, and Ghorbani selected 12 Iranian medical journals included in Web of Science and 26 Iranian medical journals included in ISC (Islamic World Science Citation Center), and studied the correlation between self-citation and JIF. These authors found that there was no significant difference between self-citation rates in these two databases. In addition, they found no significant differences between these two databases in the correlation of journal self-citation with impact factor [30].
Yang, Gao, and Zhang compared the journal self-citation rates for 99 Chinese scientific journals, and then compared the results with a similar set of 99 non-Chinese journals. They found that, in general, self-citation rates were higher in Chinese journals [31]. Humphrey, Kiseleva, and Schleicher studied the self-citation trends over the period of 2000–2012 for all business and management journals indexed in JCR. In some instances, they found strong increases in self-citation relative to external citations [32]. Lin studied the performance of Asian science and technology (S&T) journals in international citation indicators. She found that journal self-citations among the studied journals had no significant effect on the journals’ JIF values [33]. Hongling investigated the self-citation rates from 2007 to 2009 in journals from China, Japan, India, and Korea included in the SCI. She found that Korea had the highest self-citation rate, and Japan the lowest. She also discovered that the academic influence of journals with very low or very high self-citation rates was small [34].
Huang and Lin studied the influence of journal self-citations on the JIF and journal immediacy index (JII) [35]. They used data for 10 years of publications from 20 journals in environmental engineering, and found that the inclusion of journal self-citations only slightly modified the JIF and JII values.
Mimouni et al. investigated the self-citation rate of 117 pediatrics journals listed in the JCR. They discovered that there was a significant difference between JIF and corrected JIF (without self-citations) among all journals. They also found that self-citation was more prevalent in journals with a lower JIF and with a lower corrected JIF [36].
Torabian et al. studied the relationship between self-citation and JIF in open-access medical science journals indexed by ISI and Directory of Open-Access Journals (DOAJ) in 2007–2008. They found that, after deleting self-citations, 60% of the journals increased in rank, 27% decreased in rank, and 13% remained unchanged. They also found a significant relationship between self-citation and JIF [37].
Opthof studied the top 50 journals listed in the Web of Science category of cardiac and cardiovascular systems [38]. He studied inflation of the impact factor due to journal self-citation, and concluded that that journal self-citation among cardiovascular journals was substantial.
As noted above, a simple technique to increase the JIF consists of publishing editorial material with many journal self-citations [39]. However, in previous studies, we found that manipulation of the JIF by publishing large amounts of editorial material with many citations to the journal itself was not a widely used strategy [40].
In another study, we tried ascertaining the possible effect of journal self-citations on the increase in the JIF of journals for which this scientometric indicator rose at least fourfold in only a few years. Again, we found no evidence of widespread manipulation of the JIF through the massive use of journal self-citations [41]. In another article, I studied the factors (citations, self-citations, and number of articles) that led to large changes in the JIF in only one year in a sample of 360 journals. I discovered that about 54% of the increases in the JIF were associated with changes in journal self-citation patterns [42].
Yang et al. used data mining techniques to detect JIF manipulation [43]. These authors used eight algorithms to find suitable methods for detecting impact factor manipulation.
Note that, in general, the effect of added citations on the JIF will be more noticeable for journals ranked in lower positions. These journals have low JIFs and, thus, benefit more obviously from any additional citations they receive.

1.2. Coercive Citations

The concept of mandatory or coercive self-citation is not new. This refers to “requests that (i) give no indication that the manuscript was lacking in attribution; (ii) make no suggestion as to specific articles, authors, or a body of work requiring review; and (iii) only guide authors to add citations from the editor’s journal” ([44], p. 542). Resnik, Gutierrez-Ford, and Peddala provide another definition of coercive citations: “unnecessary references to his/her publication(s)” ([45], p. 307). Coercitive citations could also be used for personal editors’ benefit [46]. Esfe et al. discussed the types, reasons, benefits, and drawbacks of this type of self-citation [47]. Martin considers that coercive citations are in clear breach of the conventions of academic behavior [48]. Mahian and Wongwises considered that such editorial requests (or requirements) are not ethical and may consequently diminish the journal’s reputation [49].
Yu, Yu, and Wang used logistic regression to develop a classification scheme that identified journals as normal or abnormal in terms of the use of coercive citations [50]. Chorus and Waltman studied how the JIF of scholarly journals might be biased by self-citations [51]. These authors computed the ratio between the relative share of journal self-citations to papers published in the last two years, and the relative share of journal self-citations to papers published in preceding years. According these authors, a quotient higher than 1 suggests that the JIF may have been affected by self-citations. They discovered that there was a relationship between high ratios and coercive journal self-citation malpractices.

1.3. Objectives

In light of the research summarized above, the main aim of this work was to search for a new approach to the study of changes in academic journal ranking. I investigated the mechanisms of change in the JIF, and the effect of journal self-citations on rapid improvements (i.e., over a period of one to six years) in journal rank position from the bottom to the top quartile within its subject category. Percentiles are frequently used in journal ranking systems. Undoubtedly, journals that experience relatively fast transitions from low to high rank positions are a strategic research objective. Currently, many journals remain in the same JCR quartile for years, or move up only in small, single-quartile steps. Those that skyrocket are, thus, potentially important sources of data that can shed light on ways in which editorial policies may affect a given journal’s JIF and rank position.
Of course, other possibilities could be considered, such as to study improvements in seven years from Q4 to Q1 or changes in five years from Q3 to Q1 and so. However, in order to avoid a complex analysis or an apparent somewhat artificial effect of the role of journal self-citations, a more restrictive criterion is preferred.

2. Methodology

2.1. Data

Data on JIFs were obtained with the tool to consult JCR available for universities in Spain (https://www.recursoscientificos.fecyt.es/servicios/indices-de-impacto, accessed October 2017). This tool covers journals included in the SCI and Social Sciences Citation Index (SSCI) sections of JCR, and provides data for the JIFs and their corresponding numerator and denominator. These data can also be obtained from the usual Web of Science interface available from Clarivate Analytics, formerly managed by Thomson Reuters.
Data for the years 2009 to 2015 were used only for academic and research purposes. All SCI journals with a JIF greater than 0 and with no change in the short name field during the years sampled were selected. The “category code” field label was used to identify subject categories. In the dataset, each journal appears n times, one for each subject category in which it is included. Journals were listed in ascending order using the “category code” field, and then in descending order using the “impact factor” (JIF) field.
During the period studied here, 51 different journals in different JCR subject categories improved their rank position from Q4 to Q1. Some of these journals appeared in two (BIOFACTORS, IEEE J-STARS, J HEMATOL ONCOL, MOL ECOL RESOUR) or three (J MANUF SYST) different subject categories. Given that the ranks and, in some cases, the number of transitions (see below) may differ among journals in different subject categories, I report these cases here as different instances.

2.2. Procedure

Journals with the same JIF were assigned the same rank. Ranks were always strictly successive. Next, journals were assigned to quartiles with the algorithm provided on the Thomson Reuters website (http://ipscience-help.thomsonreuters.com/incitesLiveJCR/JCRGroup/jcrJournalProfile/jcrJournalProfileRank.html, accessed 19 June 2017). Finally, journals included in Q4 in 2009 and in Q1 in any year from 2010 to 2015 were selected.
The basic unit of analysis used here to study changes from Q4 to Q1 is the “transition”. Box 1 illustrates this approach, and explains the most important data underlying the change from Q4 to Q1 for the journal DES MONOMERS POLYM (subject category UY) by way of example. This change occurred across five transitions (2010–2009, 2011–2010, 2012–2011, 2013–2012, and 2014–2013). Each transition period was defined by an initial and a final year, and each transition was determined by changes in (1) the JIF; (2) the number of citations used to compute the JIF; (3) the number of citable items; (4) journal self-citations that contributed to the JIF; and (5) the total number of journal self-citations. These were the basic variables used here to study the role of journal self-citations in changes from Q4 to Q1.
In theory, the number of transitions could be zero. This would happen, for example, when a journal is ranked in Q1 in the same year it receives its first JIF. However, in this study, we always start from Q4.
Different mechanisms underlie the quartile transitions (see the work by Kiesslich, Weineck, and Koelblinger for a detailed discussion [52]). For example, some transitions in Box 1 involved increases both in the number of citations used to compute the JIF (JIF numerator) and the number of citable items (JIF denominator). Note also that some transitions from Q4 to Q1 can involve even a decrease in the number of the quartile. It is clear that, in general, not all transitions from Q4 to Q1 in all journals occurred in the same way. I, therefore, aimed to identify the mechanism of all transitions in the set of selected journals during their journey from Q4 to Q1.
Next, transitions involving an increase in citations used to compute the JIF were studied in more detail.
Box 1. Example of data used to calculate transitions. The journal DES MONOMERS POLYM was in the fourth quartile (Q4) in 2009 and entered the first quartile (Q1) in 2014. There are five transitions. Each transition is identified as the initial year (for example, 2010) and a final year (for example, 2011). C-Self-Citations-JIF: Self-citations-JIF(Y)/Self-citations-JIF(Y-1); C-Self-Citations: Self-citations(Y)/Self-citations(Y-1); C-JIF: JIF(Y)/JIF(Y-1); Q-Self-Citations: C-Self-citations-JIF/C-Self-citations; where JIF is the Journal Impact Factor, and Y is the year.
Box 1. Example of data used to calculate transitions. The journal DES MONOMERS POLYM was in the fourth quartile (Q4) in 2009 and entered the first quartile (Q1) in 2014. There are five transitions. Each transition is identified as the initial year (for example, 2010) and a final year (for example, 2011). C-Self-Citations-JIF: Self-citations-JIF(Y)/Self-citations-JIF(Y-1); C-Self-Citations: Self-citations(Y)/Self-citations(Y-1); C-JIF: JIF(Y)/JIF(Y-1); Q-Self-Citations: C-Self-citations-JIF/C-Self-citations; where JIF is the Journal Impact Factor, and Y is the year.
JournalDES MONOMERS POLYM
Subject CategoryUY
YearJIFQuartileCitations JIFCitable Items JIFSelf-Citations-JIFSelf-CitationsTransitionsC-Self-Citations-JIFC-Self-CitationsC-JIFQ-Self-Citations
20090.500Q44488317
20100.711Q364909182010–20093.0001.0591.4222.833
20111.444Q21309038732011–20104.2224.0562.0311.041
20120.875Q3778827762012–20110.7111.0410.6060.682
20132.210Q217981461162013–20121.7041.5262.5261.116
20142.780Q1278100831552013–20121.8041.3361.2581.350

2.3. Variables

For every transition in which there was an increase in citations used to compute the JIF, I calculated the following variables intended to measure changes (see also Box 1):
C-JIF: Quotient (change) in the JIF in one year (Y) divided by JIF in the previous year (Y-1),
C JIF = JIF ( Y ) JIF ( Y 1 ) .
C-Self-Citations-JIF: Quotient (change) in the number of journal self-citations used to compute the JIF in one year (Y) divided by this number in the previous year (Y-1),
C Self Citations JIF = Self Citations JIF ( Y ) Self Citations JIF ( Y 1 ) .
C-Self-Citations: Quotient (change) in the total number of journal self-citations in one year (Y) divided by this number in the previous year (Y-1),
C Self Citations = Self Citations ( Y ) Self Citations ( Y 1 ) .
The rationale for the use of the above variables is that they represent the yearly changes in journal self-citations used to compute the JIF (variable C-Self-Citations-JIF) and in journal self-citations (variable C-Self-Citations). These two variables were used to compute an overall quotient that allowed me to detect cases in which the change in the number of journal self-citations that contributed to the JIF was greater than the change in the number of journal self-citations. A ratio higher than 1 suggested that the JIF might have been affected or inflated by journal self-citations. Thus, for each transition involving increased citations used to compute the JIF, the following quotient was calculated:
Q Self Citations = C Self Citations JIF C Self Citations .
The rationale for the use of the above quotient is clear: in a given journal, it is not unusual for the number of self-citations used to compute the JIF to increase when the total number of self-citations also increases. When the Q-Self-Citations quotient was greater than 1, this means that the increase in the contribution of journal self-citations to all citations used to compute the JIF was greater than the increase in journal self-citations from one year to the next. This approach is similar to the method used by Chorus and Waltman [51]. As explained above, these authors used the ratio between the relative share of journal self-citations to papers published in the previous two years, and the relative share of journal self-citations to papers published in preceding years. Note, however, that I calculated this quotient only for data in which the number of journal self-citations increased from one year to the next.
For example, consider again the journal DES MONOMERS POLYM (Box 1) and the 2011–2010 transition. The number of journal self-citations that contributed to the JIF changed from 9 to 38 (C-Self-Citations-JIF = 4.222) while the number of total journal self-citations changed from 18 to 73 (C-Self-Citations = 4.056). The variable Q-Self-Citations for this pair of years was 1.041.
Instances of transitions in which there was an increase in the number of journal self-citations that contributed to JIF and the variable Q-Self-Citations was greater than 1 were selected for further scrutiny.
For all calculations, when the denominator of any quotient was zero, the quotient was set to the dummy value 1 × 1032 (very large) to simulate a zero-division in Excel.

2.4. Further Details

The method described above is a procedure of successive “filtering” or “zooming” (see Box 2). I started with a large set of journals. Next, I selected those that rose from Q4 to Q1 in six years or less. Next, I studied the transitions in these journals. Next, I focused on transitions that involved an increase in the number of citations, and then I focused on the role of journal self-citations in transitions with an increase in the number of citations.
The statistical methods used in this study are straightforward, and all calculations were carried out in Microsoft Excel. In all figures, the numbers represent the upper limit of the ranges.
Box 2. Procedure of successive “filtering” or “zooming".
  • Data on JIFs were obtained, 2009–2015.
  • Selection of all Science Citation Index (SCI) journals with a JIF greater than 0 and with no change in the short name field during the years sampled.
  • Selection of journals in different Journal Citation Reports (JCR) subject categories that improved their rank position from Q4 to Q1.
  • Selection of transitions and study of changes in the JIF, the number of citations used to compute the JIF, the number of citable items, journal self-citations that contributed to the JIF, and the total number of journal self-citations.
  • Selection of transitions involving an increase in citations used to compute the JIF.
  • Selection of transitions in which the change in the number of journal self-citations that contributed to the JIF was greater than the change in the number of journal self-citations.
The “additonal data” file contains the data relative to the instances studied.

3. Results and Discussion

The results are presented below in three sections. Firstly, I present the overall results. Next, I study the transitions. Finally, I focus in journals in which the mechanism of transition involved an increase in citations.

3.1. Overall Results

A total of 6654 different journals met the conditions cited above to be selected. I found a total of 57 instances of journals that encompassed a total of 220 transitions from Q4 to Q1. For example, the journal IEEE J-STARS moved from Q4 to Q1 in categories KV and SR. In the former category, it experienced five transitions, while, in the latter, it experienced only three. I report these instances as the short name of the journal followed by the category code (for example, IEEE J-STARS-KV and IEEE J-STARS-SR). According to this criterion, there were a total of 57 different instances. A total of 42 different subject categories contained journals that experienced changes from Q4 to Q1. Some subject categories appeared twice (CO, CQ, DM, EX, IA, IQ, KV, RZ) or as many as 8 times (PQ).
Table 1 shows the distributions of instances according to the number of transitions from Q4 to Q1.
Some moved rapidly from Q4 to Q1 in a single transition (three instances: HIST SCI-MQ, INFLUENZA OTHER RESP-NN, J BIOPHOTONICS-CO) or two transitions (seven instances: ADV ECOL RES-GU, AUTISM RES-CN, BIOFACTORS-CQ, BIOFACTORS-IA, LASER PHYS-UB, LOG J IGPL-PQ, SEMICONDUCT SEMIMET-IQ).

3.2. Mechanisms of Transition

In the whole set of instances, there were 220 transitions. The distribution of these transitions according to the mechanism of change and the variable C-JIF are shown in Table 2.
As expected, most transitions involved an increase in the JIF numerator (81.4%). In almost all of these transitions, the change in JIF was positive.
Figure 1 shows the distribution of instances according to the quotient of the JIF for the year when the journal reached Q1 (JIF(Q1) divided by the 2009 JIF (JIF(2009)). In 28 instances (about 50%), the quotient was 4 or less. However, in five instances (about 9%), this quotient was greater than 8. These instances represent considerable increases in the JIF.
As explained in the Section 2, all transitions that involved an increase in the number of citations were selected for further study. There were 179 such transitions (Table 3).
Many transitions listed in Table 3 involved a decrease in the number of citable items (JIF denominator). This decrease may help increase the JIF regardless of whether the number of citations used to compute the JIF (i.e., citations to items published in the two previous years) increased, remained unchanged, or decreased (provided that this decrease was smaller than the decrease in the number of citable items). As noted by Opthof, Coronel, and Janse, a good strategy to increase the JIF might be to publish fewer, but better papers. These authors simulated the effect of deleting some papers from the JIF formula, and also deleted the citations that these papers received. They found that if only manuscripts with a 100% priority score were published, the JIF would have increased [53].

3.3. Role of Journal Self-Citations in Transitions Involving Increased Citations

In 81 of the 179 transitions listed in Table 3 (54.8%), the number of journal self-citations that contributed to the JIF increased, and the variable Q-Self-Citations was greater than 1. For 15 transitions, the variable Q-Self-Citations could be not calculated, because the data for journal self-citations or journal self-citations that contributed to the JIF were labeled as “not available” in JCR.
Table 4 shows the data on the percentages of journal self-citations that contributed to the JIF in the final year in all transitions that involved an increase in the number of citations and in the number of journal self-citations that contributed to the JIF, and the values of Q-Self-Citations were greater than 1. These transitions were detected in the 34 different journals shown in Table 4.
Figure 2 shows the distribution of the transitions listed in Table 4 according to the percentage of citations used to compute the JIF that were journal self-citations.
How many journal self-citations are acceptable? Many years ago, an ISI report stated that 82% of the total 2002 JCR coverage had self-citations rates at or below 20% [54]. According to Yu, Yu, and Wang, “journals with a self-citation rate beyond 20% are considered suspicious journals that may be involved with coercive self-citation” ([50], p. 125). In this connection, it appears that, in the three most recent editions of JCR, journals suppressed for excessive self-citation had a rate of self-citations that contributed to their JIF of at least 50% [55]. According to an intermediate “ad hoc” criterion, in about 63% of the transitions listed in Table 4, the percentage of journal self-citations that contributed to the JIF was 30% or less. Only in five transitions (among 81), the percentage of journal self-citations was greater than 50%.
Some instances merit additional comment. For example, the journal PLANT MOL BIOL REP (subject category CO) experienced three transitions listed in Table 4. In the first one, the percentage of journal self-citations among all citations used to compute the JIF was only about 6%. However, in the two subsequent transitions these percentages increased to 57.3% and 71.2%.
The instances with the highest percentages of self-citation were LASER PHYS (subject category UB), with 60.9% in 2011, and PLANT MOL BIOL REP (subject category CO), with 71.2% in 2012. These two journals showed a rising trend in the percentage of journal self-citations that contributed to the JIF in the final year, in transitions that involved an increase in the number of total citations. In the former journal, the increase was from 35.8% to 60.9%, while, in the latter, the increase was from 6.3% to 57.3%, and subsequently to 71.2%. It is clear that, in these instances, the change from Q4 to Q1 involved a considerable journal self-citation component.
In contrast, the journal BRAIN IMAGING BEHAV (subject category RX) rose from Q4 to Q1 in five transitions, only two of which appear in Table 4. In these two transitions, the percentage of journal self-citations was very low, i.e., 4.1% and 6.3%. The journal GEOCHRONOMETRIA (subject category TE) moved from Q4 to Q1 in six transitions, only one of which appears in Table 4. In this case, the percentage of journal self-citations used to compute the JIF was also very low at 3.4%. Note that Table 4 shows only transitions that involved an increase in the number of total citations and journal self-citations that contributed to the JIF (JIF numerator), in journals with a Q-Self-Citation value greater than 1.

4. Conclusions

Although the JIF is criticized for many reasons, it remains one of the most widely used scientometric indicators. Manipulation of this indicator should be considered one of the sins a journal can commit. Krell and Romano (among others) warned against excessive numbers of journal self-citations [56,57], and there is now growing awareness that journals with unusually high numbers of self-citations might be banned from JCR. However, to punish intentional manipulations of the JIF, Thomson Reuters started banning journals with too many self-citations from the JCR for two years. The use of the JIF computed without journal self-citations could help avoid manipulations. However, “citation circles”, defined as “groups of journals that tend to cite themselves mutually in order to increase their JIFs” could be more difficult to detect [58].
In the present study, I used a sample of journals with low rank positions in their respective subject categories in 2009 as source data to study how these journals attained better rank positions. Their progress was linked mainly to increases in the number of citations, but transitions that involved increased citations (JIF numerator) were often accompanied by reductions in the number of citable items (JIF denominator). As explained above, I believe this is an effective and legitimate way of raising the JIF, i.e., by publishing fewer, but better articles.
In about 50% of the instances I found, the JIF quotient for the year when the journal entered Q1 divided by the 2009 JIF (when the journal was in Q4) was 4 or less. This figure does not appear to reflect a very large increase in the JIF among journals with large improvements in their rank positions.
Next, I focused on transitions in which the increase in journal self-citations that contributed to JIF from one year to the next was greater than the change in journal self-citations. The target of this analysis was the percentage of journal self-citations that contributed to the JIF. In about 63% of the transitions listed in Table 4, the percentage of journal self-citations that contributed to the JIF was 30% or less. Only in five transitions (among 81), the percentage of journal self-citations was greater than 50%.
Taken together, these results suggest that, for this particular sample of journals which experienced a sharp upturn in their rank position within their subject categories, there is no evidence of widespread manipulation of the JIF through the massive use of journal self-citations. The percentages of journal self-citations that contributed to the JIF were, in general, not very high.

Funding

This research received no external funding.

Acknowledgments

I thank K. Shashok for improving the use of English in the manuscript and four anonymous referees for their comments. Some of these comments were included in the text.

Conflicts of Interest

The author declares no conflicts of interest.

References

  1. Fiala, J.; Mareš, J.J.; Šesták, J. Reflections on how to evaluate the professional value of scientific papers and their corresponding citations. Scientometrics 2017, 112, 697–709. [Google Scholar] [CrossRef]
  2. Song, M.; Kim, S.; Lee, K. Ensemble analysis of topical journal ranking in bioinformatics. J. Assoc. Inf. Sci. Technol. 2017, 68, 1564–1583. [Google Scholar] [CrossRef]
  3. Bornmann, L. Confidence intervals for journal impact factors. Scientometrics 2017, 111, 1869–1871. [Google Scholar] [CrossRef]
  4. Liu, X.-L.; Wei, Y.H.; Gai, S.S. Citation characteristics of non-citable documents and contributions to journal impact factor. Curr. Sci. 2018, 114, 1423–1429. [Google Scholar]
  5. Ferrer-Sapena, A.; Díaz-Novillo, S.; Sánchez-Pérez, E.A. Measuring time-dynamics and time-stability of journal rankings in Mathematics and Physics by means of fractional p-variations. Publications 2017, 5, 21. [Google Scholar] [CrossRef]
  6. Leydesdorff, L.; Bornmann, L. The operationalization of “fields” as WoS subject categories (WCs) in evaluative bibliometrics: The cases of “library and information science” and “science & technology studies”. J. Assoc. Inf. Sci. Technol. 2016, 67, 707–714. [Google Scholar]
  7. Leydesdorff, L.; Rafols, I. A global map of science based on the ISI subject categories. J. Am. Soc. Inf. Sci. Technol. 2009, 60, 348–362. [Google Scholar] [CrossRef][Green Version]
  8. Pudovkin, A.I.; Garfield, E. Algorithmic procedure for finding semantically related journals. J. Am. Soc. Inf. Sci. Technol. 2002, 53, 1113–1119. [Google Scholar] [CrossRef]
  9. Waltman, L. A review of the literature on citation impact indicators. J. Inf. 2016, 10, 365–391. [Google Scholar] [CrossRef][Green Version]
  10. Groesser, S.N. Dynamics of journal impact factors. Syst. Res. Behav. Sci. 2012, 29, 624–644. [Google Scholar] [CrossRef]
  11. Lundberg, G.D. The omnipotent science citation index impact factor. Med. J. Aust. 2003, 178, 253–254. [Google Scholar] [PubMed]
  12. Shugan, S.M. The mission of Marketing Science. Mark. Sci. 2002, 21, 1–13. [Google Scholar] [CrossRef]
  13. Heneberg, P. Parallel worlds of citable documents and others: Inflated commissioned opinion articles enhance scientometric indicators. J. Assoc. Inf. Sci. Technol. 2014, 65, 635–643. [Google Scholar] [CrossRef]
  14. Jiménez-Contreras, E.; de Moya Anegón, F.; López-Cózar, E.D. The evolution of research activity in Spain: The impact of the National Commission for the Evaluation of Research Activity (CNEAI). Res. Policy 2003, 32, 123–142. [Google Scholar] [CrossRef]
  15. García, J.A.; Rodríguez-Sánchez, R.; Fernández-Valdivia, J. Scientific subject categories of Web of Knowledge ranked according to their multidimensional prestige of influential journals. J. Am. Soc. Inf. Sci. Technol. 2012, 63, 1017–1029. [Google Scholar] [CrossRef]
  16. Shao, J.F.; Shen, H.Y. Research assessment and monetary rewards: The overemphasized impact factor in China. Res. Eval. 2012, 21, 199–203. [Google Scholar] [CrossRef]
  17. Pajić, D. On the stability of citation-based journal rankings. J. Inf. 2015, 9, 990–1006. [Google Scholar] [CrossRef]
  18. Bornmann, L.; Haunschild, R. Measuring field-normalized impact of papers on specific societal groups: An altmetrics study based on Mendeley Data. Res. Eval. 2017, 26, 230–241. [Google Scholar] [CrossRef][Green Version]
  19. Heneberg, P. From excessive journal self-cites to citation stacking: Analysis of journal self-citation kinetics in search for journals, which boost their scientometric indicators. PLoS ONE 2016, 11, e0153730. [Google Scholar] [CrossRef] [PubMed]
  20. Foo, J.Y.A. A study on journal self-citations and intra-citing within the subject category of multidisciplinary sciences. Sci. Eng. Ethics 2009, 15, 491–501. [Google Scholar] [CrossRef] [PubMed]
  21. Falagas, M.E.; Alexiou, V.G. The top-ten in journal impact factor manipulation. Arch. Immunol. Ther. Exp. 2008, 56, 223–226. [Google Scholar] [CrossRef] [PubMed]
  22. Tighe, P.; Rice, K.J.; Gravenstein, N. Artifactual increase in journal self-citation. Anesth. Analg. 2011, 113, 378–382. [Google Scholar] [CrossRef] [PubMed]
  23. Epstein, D. Impact factor manipulation. J. Eur. Med. Writ. Assoc. 2007, 16, 133–134. [Google Scholar]
  24. Yu, G.; Wang, L. The self-cited rate of scientific journals and the manipulation of their impact factors. Scientometrics 2007, 73, 321–330. [Google Scholar] [CrossRef]
  25. Mannino, D.M. Impact factor, impact, and smoke and mirrors. Am. J. Respir. Crit. Care Med. 2005, 171, 417–418. [Google Scholar] [CrossRef] [PubMed]
  26. Neuberger, J.; Counsell, C. Impact factors: Uses and abuses. Eur. J. Gastroenterol. Hepatol. 2002, 14, 209–211. [Google Scholar] [CrossRef] [PubMed]
  27. Frandsen, T.F. Journal self-citations-Analysing the JIF mechanism. J. Inf. 2007, 1, 47–58. [Google Scholar] [CrossRef]
  28. Rousseau, R. Temporal differences in self-citation rates of scientific journals. Scientometrics 1999, 44, 521–531. [Google Scholar] [CrossRef]
  29. Peritz, B.; Bar-Ilan, J. The sources used by bibliometrics-scientometrics as reflected in references. Scientometrics 2002, 54, 269–284. [Google Scholar] [CrossRef]
  30. Mirsaeid, S.J.G.; Motamedi, N.; Ghorbani, N.R. Correlation between self-citation and impact factor in Iranian English medical journals in WoS and ISC: A Comparative Approach. Iran. J. Public Health 2015, 44, 1234–1243. [Google Scholar]
  31. Yang, Z.G.; Gao, F.; Zhang, C.T. Comparison of journal self-citation rates between some Chinese and non-Chinese international journals. PLoS ONE 2012, 7, e49001. [Google Scholar] [CrossRef] [PubMed]
  32. Humphrey, C.; Kiseleva, O.; Schleicher, T. A time-series analysis of the scale of coercive journal self-citation and its effect on impact factors and journal rankings. Eur. Account. Rev. 2018. [Google Scholar] [CrossRef]
  33. Lin, W.Y.C. The performance of Asian S&T journals in international citation indicators. Learn. Publ. 2017, 30, 193–204. [Google Scholar]
  34. Hongling, F. Self-citation rates of scientific and technical journals in SCI from China, Japan, India, and Korea. Learn. Publ. 2013, 26, 45–49. [Google Scholar] [CrossRef][Green Version]
  35. Huang, M.H.; Lin, W.Y.C. The influence of journal self-citations on journal impact factor and immediacy index. Online Inf. Rev. 2012, 36, 639–654. [Google Scholar] [CrossRef]
  36. Mimouni, M.; Ratmansky, M.; Sacher, Y.; Aharoni, S.; Mimouni-Bloch, A. Self-citation rate and impact factor in pediatrics. Scientometrics 2016, 108, 1455–1460. [Google Scholar] [CrossRef]
  37. Torabian, R.; Heidari, A.; Shahrifar, M.; Khodadi, E.; Vardanjani, S.A.E. The relation between self-citation and impact factor in medical science open access journals in ISI & DOAJ Databases. Life Sci. J. 2012, 9, 2206–2209. [Google Scholar]
  38. Opthof, T. Inflation of impact factors by journal self-citation in cardiovascular science. Neth. Heart J. 2013, 21, 163–165. [Google Scholar] [CrossRef] [PubMed][Green Version]
  39. Foo, J.Y.A. Impact of excessive journal self-citations: A case study on the Folia Phoniatrica et Logopaedica journal. Sci. Eng. Ethics 2011, 17, 65–73. [Google Scholar] [CrossRef] [PubMed]
  40. González, L.; Campanario, J.M. Structure of the impact factor of journals included in the social sciences citation index: Citations from documents labeled “editorial material”. J. Assoc. Inf. Sci. Technol. 2007, 58, 252–262. [Google Scholar] [CrossRef]
  41. Andrade, A.; González-Jonte, R.; Campanario, J.M. Journals that increase their impact factor at least fourfold in a few years: The role of journal self-citations. Scientometrics 2009, 80, 515–528. [Google Scholar] [CrossRef]
  42. Campanario, J.M. Large increases and decreases in journal impact factors in only one year: The effect of journal self-citations. J. Assoc. Inf. Sci. Technol. 2011, 62, 230–235. [Google Scholar] [CrossRef]
  43. Yang, D.H.; Li, X.; Sun, X.; Wan, J. Detecting impact factor manipulation with data mining techniques. Scientometrics 2016, 109, 1989–2005. [Google Scholar] [CrossRef]
  44. Wilhite, A.W.; Fong, E.A. Coercive citation in academic publishing. Science 2012, 335, 542–543. [Google Scholar] [CrossRef] [PubMed]
  45. Resnik, D.B.; Gutierrez-Ford, C.; Peddada, S. Perceptions of ethical problems with scientific journal peer review: An exploratory study. Sci. Eng. Ethics 2008, 14, 305–310. [Google Scholar] [CrossRef] [PubMed]
  46. Herteliu, C.; Ausloos, M.; Ileanu, B.V.; Rotundo, G.; Andrei, T. Quantitative and qualitative analysis of editor behavior through potentially coercive citations. Publications 2017, 5, 15. [Google Scholar] [CrossRef]
  47. Esfe, M.H.; Wongwises, S.; Asadi, A.; Karimipour, A.; Akbari, M. Mandatory and self-citation; types, reasons, their benefits and disadvantages. Sci. Eng. Ethics 2015, 21, 1581–1585. [Google Scholar] [CrossRef] [PubMed]
  48. Martin, B.R. Editors’ JIF-boosting stratagems–Which are appropriate and which not? Res. Policy 2016, 45, 1–7. [Google Scholar] [CrossRef]
  49. Mahian, O.; Wongwises, S. Is it ethical for journals to request self-citation? Sci. Eng. Ethics 2015, 21, 531–533. [Google Scholar] [CrossRef] [PubMed]
  50. Yu, T.; Yu, G.; Wang, M.Y. Classification method for detecting coercive self-citation in journals. J. Inf. 2014, 8, 123–135. [Google Scholar] [CrossRef]
  51. Chorus, C.; Waltman, L. A large-scale analysis of impact factor biased journal self-citations. PLoS ONE 2016, 11, e0161021. [Google Scholar] [CrossRef] [PubMed]
  52. Kiesslich, T.; Weineck, S.B.; Koelblinger, D. Reasons for journal impact factor changes: Influence of changing source items. PLoS ONE 2016, 11, e0154199. [Google Scholar] [CrossRef] [PubMed]
  53. Opthof, T.; Coronel, R.; Janse, M.J. Submissions, impact factor, reviewer’s recommendations and geographical bias within the peer review system (1997–2002). Cardiovasc. Res. 2002, 55, 215–219. [Google Scholar] [CrossRef]
  54. Institute for Scientific Information (no date) Journal Self-citation in the Journal Citation Reports—Science Edition. 2002. Available online: http://wokinfo.com/essays/ (accessed on 15 October 2018).
  55. Davis, P. Reverse Engineering JCR’s self-Citation and Citation Stacking Thresholds. The Scholarly Kitchen. 5 June 2017. Available online: https://scholarlykitchen.sspnet.org/2017/06/05/reverse-engineering-jcrs-self-citation-citation-stacking-thresholds/ (accessed on 15 October 2018).
  56. Krell, F.T. Losing the numbers game: Abundant self-citations put journals at risk for a life without an impact factor. Eur. Sci. Ed. 2014, 40, 36–38. [Google Scholar]
  57. Romano, N.C., Jr. Journal self-citation V: Coercive journal self-citation-manipulations to increase impact factors may do more harm than good in the long run. Commun. Assoc. Inf. Syst. 2009, 25, 41–56. [Google Scholar] [CrossRef]
  58. Teodorescu, D.; Andrei, T. An examination of “citation circles” for social sciences journals in Eastern European countries. Scientometrics 2014, 99, 209–231. [Google Scholar] [CrossRef]
Figure 1. Distribution of journals according the quotient of Journal Impact Factor (JIF) in the year when it entered the first quartile (Q1), JIF(Q1), divided by the 2009 JIF, JIF(2009).
Figure 1. Distribution of journals according the quotient of Journal Impact Factor (JIF) in the year when it entered the first quartile (Q1), JIF(Q1), divided by the 2009 JIF, JIF(2009).
Publications 06 00047 g001
Figure 2. Distribution of transitions listed in Table 4 according to the percentage of citations used to compute the JIF that were journal self-citations.
Figure 2. Distribution of transitions listed in Table 4 according to the percentage of citations used to compute the JIF that were journal self-citations.
Publications 06 00047 g002
Table 1. Number of instances according to the number of transitions journals experienced from Q4 to Q1.
Table 1. Number of instances according to the number of transitions journals experienced from Q4 to Q1.
N TransitionsN InstancesPercentage
135.3
2712.3
31526.3
4915.8
51628.1
6712.3
Total57100.0
Table 2. Distribution of the numbers (N) and percentages (%) of transitions according to the mechanism of change involved in the variable C-JIF. Symbols: - Decrease; + Increase; 0 No change. Empty cells represent instances that are not possible according to the JIF definition.
Table 2. Distribution of the numbers (N) and percentages (%) of transitions according to the mechanism of change involved in the variable C-JIF. Symbols: - Decrease; + Increase; 0 No change. Empty cells represent instances that are not possible according to the JIF definition.
C-JIF
NumeratorDenominator− (N)− (%)+ (N)+ (%)Total (N)Total (%)
156.8167.33114.1
010.5 10.5
+73.2 73.2
0 20.920.9
+ 6931.46931.4
+0 104.5104.5
++52.39543.210045.5
Total2812.719287.3220100.0
Table 3. Transitions in which the citations used to compute the JIF increased. +: the JIF denominator increased; 0: the JIF denominator did not change; −: the JIF denominator decreased. Gray cells represent years outside the range of study. Empty cells represent transitions that did not involve increased citations used to compute the JIF.
Table 3. Transitions in which the citations used to compute the JIF increased. +: the JIF denominator increased; 0: the JIF denominator did not change; −: the JIF denominator decreased. Gray cells represent years outside the range of study. Empty cells represent transitions that did not involve increased citations used to compute the JIF.
Journal+Category2010–20092011–20102012–20112013–20122014–20132015–2014
ACTA DIABETOL-IA++
ADV ECOL RES-GU+
ADV GEOPHYS-LE
AM J NURS-RZ
ASTERISQUE-PQ+
AUTISM RES-CN++
B MALAYS MATH SCI SO-PQ++++
BANACH J MATH ANAL-PQ+ +
BIOCHEM MEDICA-PW+++
BIOFACTORS-CQ
BIOFACTORS-IA
BRAIN IMAGING BEHAV-RX+++
CHINESE J AERONAUT-AI++++
CURR TOP MEMBR-DA
DES MONOMERS POLYM-UY+0 +
DIGEST ENDOSC-YA++++
DISS MATH-PQ + 0
ELECTRON T NUMER ANA-PN
FETAL DIAGN THER-SD+ 0
GENES NUTR-SA +
GEOCHRONOMETRIA-TE + +
HIST SCI-MQ+
HYSTRIX-ZM ++0
IEEE J-STARS-KV+++++
IEEE J-STARS-SR+++
INFLUENZA OTHER RESP-NN+
INT J DIGIT EARTH-KV+ ++
INT J GEN SYST-EX+ 0++
INT J GREEN ENERGY-DT+++
IRISH VET J-ZC+
J BIOPHOTONICS-CO+
J EXP CLIN CANC RES-DM++
J FOOD SCI TECH MYS-JY+
J GENET GENOMICS-KM
J HEMATOL ONCOL-DM++
J HEMATOL ONCOL-MA++
J MANUF SYST-IJ 0++
J MANUF SYST-IK 0++
J MANUF SYST-PE 0++
J NEUROPSYCHOL-VI+ ++
J NUCL SCI TECHNOL-RY +
LASER PHYS-UB++
LOG J IGPL-PQ+
MAR RESOUR ECON-JU
MAT SCI SEMICON PROC-IQ ++++
MATH MED BIOL-MC+
MINER METALL PROC-PZ +0+
MOL ECOL RESOUR-CQ+
MOL ECOL RESOUR-HT+
MULTIDIM SYST SIGN P-EX+ ++
NURS INQ-RZ0 +
PLANT MOL BIOL REP-CO+++
PUBL MAT-PQ++
REV SYMB LOGIC-PQ++++
RUSS MATH SURV+-PQ
SEMICONDUCT SEMIMET-IQ
Z MED PHYS-VY++
Table 4. Percentage of journal self-citations that contributed to the JIF in the final year in transitions that involved an increase in the number of total citations and journal self-citations that contributed to the JIF (JIF numerator), and with values of Q-Self-Citations greater than 1. Empty cells represent transitions that did not meet the above conditions. Grey cells represent years outside the study period.
Table 4. Percentage of journal self-citations that contributed to the JIF in the final year in transitions that involved an increase in the number of total citations and journal self-citations that contributed to the JIF (JIF numerator), and with values of Q-Self-Citations greater than 1. Empty cells represent transitions that did not meet the above conditions. Grey cells represent years outside the study period.
Journal+Category2010–20092011–20102012–20112013–20122014–20132015–2014
ACTA DIABETOL-IA17.727.6
ADV ECOL RES-GU13.845.8
B MALAYS MATH SCI SO-PQ30.835.0 44.8
BIOCHEM MEDICA-PW 51.145.9
BRAIN IMAGING BEHAV-RX 4.1 6.3
CHINESE J AERONAUT-AI22.6 34.5
DES MONOMERS POLYM-UY14.129.2 25.729.9
DIGEST ENDOSC-YA 14.415.016.8 11.8
ELECTRON T NUMER ANA-PN 12.1
FETAL DIAGN THER-SD 16.716.6
GEOCHRONOMETRIA-TE 3.4
HIST SCI-MQ17.2
HYSTRIX-ZM 28.6 7.7
IEEE J-STARS-KV 34.844.0
IEEE J-STARS-SR 34.8
INT J DIGIT EARTH-KV32.5 12.7
INT J GEN SYST-EX 38.2 9.4
INT J GREEN ENERGY-DT14.9
IRISH VET J-ZC7.7
J BIOPHOTONICS-CO5.0
J EXP CLIN CANC RES-DM5.6 17.6
J FOOD SCI TECH MYS-JY42.154.7
J GENET GENOMICS-KM 2.110.0
J HEMATOL ONCOL-DM14.8 13.6
J HEMATOL ONCOL-MA14.8 13.6
J MANUF SYST-IJ 39.147.839.1
J MANUF SYST-IK 39.147.839.1
J MANUF SYST-PE 39.147.839.1
LASER PHYS-UB35.860.9
LOG J IGPL-PQ 25.4
MAR RESOUR ECON-JU29.0 34.5 33.3
MAT SCI SEMICON PROC-IQ1.3 12.118.525.5
MINER METALL PROC-PZ 23.1 13.9 21.9
MULTIDIM SYST SIGN P-EX16.2 21.143.3
NURS INQ-RZ14.8 7.0
PLANT MOL BIOL REP-CO6.357.371.2
PUBL MAT-PQ6.9
Z MED PHYS-VY 19.0 10.4

Share and Cite

MDPI and ACS Style

Campanario, J.M. Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations. Publications 2018, 6, 47. https://doi.org/10.3390/publications6040047

AMA Style

Campanario JM. Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations. Publications. 2018; 6(4):47. https://doi.org/10.3390/publications6040047

Chicago/Turabian Style

Campanario, Juan Miguel. 2018. "Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations" Publications 6, no. 4: 47. https://doi.org/10.3390/publications6040047

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop