Reducing the Matthew Effect on Journal Citations through an Inclusive Indexing Logic: The Brazilian Spell (Scientific Periodicals Electronic Library) Experience

Abstract

The inclusion of scientific journals in prestigious indexers is often associated with higher citation rates; journals included in such indexers are significantly more acknowledged than those that are not included in them. This phenomenon refers to the Matthew effect on journal citations, according to which journals in exclusive rankings tend to be increasingly cited. This paper shows the opposite: that the inclusion of journals in local indexers ruled by inclusive logic reduces the Matthew effect on journal citations since it enables them to be equally exposed. Thus, we based our arguments on the comparison of 68 Brazilian journals before and after they were indexed in the Scientific Periodicals Electronic Library (Spell), which ranks journals in the Brazilian management field based on local citations. Citation impact indicators and iGini (a new individual inequality analysis measure) were used to show that the inclusion of journals in Spell has probably increased their impact factor and decreased their citation inequality rates. Using a difference-in-differences model with continuous treatment, the results indicated that the effect between ranking and inequality declined after journals were included in Spell. Additional robustness checks through event study models and interrupted time-series analysis for panel data point to a reduction in citation inequality but follow different trajectories for the 2- and 5-year impact. The results indicate that the indexer has reduced the Matthew effect on journal citations.

1. Introduction

It is known that scientific journals aim to differentiate from one another based on their quality. Thus, one of the most adopted strategies, if not the main one, lies in being inserted in databases and indexers that rank the impact of journals based on their citations [1,2,3,4]. Consequently, most journals focus their efforts on the most prestigious indexers, mainly Web of Science and Scopus [5], to increase their visibility and reputation in the global scientific debate [6,7]. It makes no sense since many researchers define the journals they want to publish based on their ranking position in such indexers [8].

Such a strategy of differentiating quality through prestigious indexes is also valid for journals from countries that are on the margins of global scientific production [9,10,11,12]. However, since some scientific fields have a more regional or local focus [13,14], not all journals would be eligible based on criteria adopted by such indexers [15], regardless of their quality. In fact, this fact ends up dividing journals into two classes, namely, indexed and non-indexed in international indicators. Since it is extremely hard to assess the quality of a given journal, researchers tend to trust any reliable status indication [16], which is the case with indexers such as Web of Science and Scopus. Thus, these differences in status attribution—which can be subtle when journals join a given indexer—can increase over time, generate cumulative advantages, and influence one’s perception of a given journal’s quality [16], as well as the volume of citations [7,8,17]. This process relates to the Matthew effect in Science [18,19], which also applies to journals since status differentiation signs lead to increasing gains in audience and citations.

If there is a journal stratification process inherent to their participation in prestigious indexers, what would be the alternatives for journals that do not participate in them in order to have their quality acknowledged? Being indexed in alternative databases [20,21,22], such as DOAJ (Directory of Open Access Journals), is likely the most common path, mainly for Open Access (OA) journals. The Ibero-American context presents some successful experiences [23,24,25,26], such as Redalyc (Red de Revistas Científicas de América Latina y El Caribe, Spain and Portugal) and Scielo (Scientific Electronic Library Online).

Although the influence of Open Access databases and indexers has been extensively explored at the researcher, journal, and country level, little is known about the effects of creating such indexers on the set of journals in these databases, mainly on the dynamics of citations and impact factors of indexed journals. Overall, studies focusing on investigating such dimensions have put greater emphasis on prestigious indexers [7,8,27], which often show higher Matthew effect intensity over time. However, the ranking logic adopted in such indexers is based on the exclusion principle, according to which the best journals are selected, whereas the others are excluded. Would the Matthew effect of journal citations also happen in databases and indexers that adopt an open and more inclusive logic?

In order to answer this question, this paper has analyzed a successful alternative indexing experiment carried out in Brazil, namely: the Scientific Periodicals Electronic Library (Spell), which is a scientific database that indexes Brazilian open access journals in the Business Administration, Accounting, and Tourism fields and that currently accounts for more than 55 thousand documents, 41 million accesses, and 14 million downloads. Spell was created by the local scientific community in the field in order to fill the gap observed in databases and indexers focused on Brazilian journals. Based on the inclusive logic, Spell was launched in mid-2012 to incorporate most journals in the field and to provide free access to all articles published on a single platform, and it enabled researchers to search for articles based on topics and terms. Since all journals—regardless of their ranking—had the same visibility in the database, they were expected to be evaluated based on the perceived quality of their articles, rather than on the previous ranking of journals.

In methodological terms, we selected 68 journals that had been present in Spell since its beginning of operation. Through fixed panel data models, we compared the impact indicators of citations before (2010–2012) and after (2013–2015) Spell was created, which allowed us to verify if the rise of Spell increased the impact of the journals. Additionally, we used a new measure of individual inequality analysis called iGini [28], which allowed us to analyze whether Spell reduced the Matheus effect of journal citations.

The analysis of whether the rise of Spell has simultaneously increased the mean citation of journals and reduced the inequality between them has contributed to the scientometric field. First, we explored an alternative database of local contents rather than focusing on large databases and indexers [3,4]. The impact of alternative databases on journal citations is hardly analyzed since many of them do not often provide such information, and when they do, there are often doubts about their validity and reliability [29,30]. Secondly, data about journal citations before and after Spell was created has enabled comparing the effect of the emergence of the investigated database in terms of impact and inequality, which is extremely rare in this type of research [8]. Third, the current study is a pioneer in introducing a new inequality assessment measure called iGini [28] in the scientometry field. This assessment measure enables analyzing individual cases in comparison to all others, which allows a direct measurement of the Matheus effect. Fourth, as it was not possible to compare the impact results of Spell with a control group, we employed a recent difference-in-differences model with continuous treatment [31] to assess the average treatment effect on the treated (ATET), assuming that all journals were treated continuously. Adopting this contemporary analytical approach utilized in studies evaluating treatment effects in the absence of control groups [32,33], we were able to examine the hypothesis that Spell’s implementation mitigated the cumulative impact of citations, thereby reducing the inequality among journals. Finally, we checked the robustness of the results by comparing the reduction in inequality between the years before and after the Spell intervention through an event study for panel data [34], as well as investigating the trajectory of change after the intervention through an intermittent time-series analysis [35].

The current article was structured into six parts. First, it contextualizes the journal evaluation system available in Brazil. Next, it addresses the herein-analyzed database and indexer called Spell. Then, it analyzes the incidence of the Matthew effect in journal evaluation processes. Subsequently, it provides detailed information about the analysis method and, finally, it presents the results, which are discussed in the final section.

1.1. Science, Indexers, and the Evaluation of Journals in Brazil

Before highlighting the relevance of Spell to the local academic community, it is necessary to contextualize some aspects of the way science is organized in Brazil since the emergence of this indexer is associated with peculiarities of Brazilian science, as well as with the production logic adopted in the Business Administration, Accounting and Tourism fields, the assessed indexer was consolidated in.

Unlike what happens in other countries, science in Brazil is strong, directed, and regulated by the government, both in terms of financing and in directing policies focused on contributing to scientific progress in the country [36,37]. However, the scientific environment in the country is featured by low interaction between universities and companies, great financial and structural dependence on the federal government [38], and internationalization barriers due to low proficiency in the English language [39]. This profile also leads to many journals and articles written in the local language in certain science fields such as applied human social sciences [40,41].

However, despite having a large number of studies published in local journals, Brazil is currently the 13th country in the world that mostly produces articles indexed in the Web of Science, 95% of which are produced by public universities [42]. According to the Science-Metrix report (2018), Brazil presents the highest rates of open access articles in the world (74%). Most of them are published in Brazilian journals that are not indexed in databases such as Scopus and the Web of Science [40]. Even when articles are published in indexed databases such as the Web of Science, 80.4% of journals are Brazilian [37].

The successful internationalization of scientific production in international journals, as well as its outspread in the country through open access systems, can be significantly attributed to the university evaluation system implemented by the Coordination for the Improvement of Higher Education Personnel (Capes), which is linked to the Brazilian Ministry of Education [43]. Capes is one of the main foundations accounting for the expansion of university post-graduate programs; it plays a key role in assessing and monitoring the performance of Brazilian universities in the scope of academic research since it guides the scientific production of public and private universities.

Nowadays, Brazil has more than 400 universities and colleges that have, in total, 4642 master’s and/or doctorate programs distributed among 49 assessment fields and organized into 9 major fields: Agrarian Sciences, Biological Sciences, Health Sciences, Human Sciences, Applied Social Sciences, Linguistics, Letters and Arts, Exact and Earth Sciences, and Engineering and Multidisciplinary fields1. Such information highlights the wide dimension of Brazilian science and its heterogeneous nature with respect to its evaluation fields. Therefore, specific commissions in each of these fields were created; they account for evaluating all Brazilian postgraduate programs every 4 years, in order to outline strategies focused on expanding, internationalizing, and improving Brazilian science.

Publications of post-graduate programs linked to Brazilian universities and colleges are evaluated by a scoring system called Qualis. Articles published by researchers associated with these programs are evaluated based on criteria set by Capes, which are used along with other indicators to evaluate the quality of post-graduate programs nationwide. Qualis Capes comprises 8 assessment levels, from A1 to C (A1, A2, B1, B2, B3, B4, B5, and C); each level corresponds to certain criteria and scores (from 100 to 0) established for each of the 49 assessment fields. For example, level A1 corresponds to high-impact journals; since high impacts of JCR and/or H-Scopus are overall established to define the journals to be part of this level [44]. This procedure shows certain alignment between Qualis and international assessments [41]. As previously mentioned, depending on the field, not all criteria are dimensioned based on international impact measurements (e.g., JCR and SCOPUS) Overall, they can only include indexing in specific databases and directories (e.g., Scielo, Redalyc, Latindex, DOAJ) or other mostly subjective criteria.

Given these dimensions, what could explain the internationalization and, at the same time, the endogenous nature of Brazilian science? One of the likely answers to this question lies in different journal assessment paths established by Capes for each science field. The so-called hard science fields have favored—since the beginning—international impact metrics used to classify the quality of journals at the time to assess their researchers and programs. On the other hand, the so-called soft sciences did not use impact metrics as a criterion to evaluate their publications for approximately 10 years [45]. Consequently, the process of classifying journals, over a period of time, was artisanal and poorly systematized [46,47]. Although journal evaluation processes have parameterized criteria based on normalization, regularity, circulation, visibility, and editorial management, in addition to the number, proportion, and perceived quality of published articles, the essence of evaluation in social sciences and humanities remained based on the opinions of experts [48].

The lack of criteria based on international metrics has also enabled the emergence and proliferation of national journals, which mainly contributed to scientific fields such as Business Administration, Accounting, and Tourism in Brazil to have approximately 304 open access journals [49]. This feature of the field has highlighted a quite different publication dynamic in comparison to the hard science field. Thus, there is a community featuring by high concentration of studies published in national journals—such as studies belonging to the Business Administration, Accounting, and Tourism fields—that are not indexed in the mainstream indexers.

Despite the current local endogeny in the Business Administration, Accounting, and Tourism field, from 2010 onwards, the improvement observed in evaluations carried out by Capes ended up putting pressure on the field itself to use metrics such as JCR and H-Scopus (Elsevier) at the time to evaluate journals published by researchers [45]. However, these metrics worked as a basis to qualify part of the international journals indexed in these databases. In the absence of such metrics, national journals ended up being evaluated again based on poorly consistent criteria; they were subjected to the scrutiny of specialists at the time to determine the most or least prestigious journals, since most of them did not appear in any indexer [46].

Qualis Capes has recently made radical changes to make evaluations more objective based on international metrics. However, there was a rupture in the institutional evaluation process because most Brazilian journals were not indexed in such databases. On the one hand, a small number of journals were already indexed, or in the process of being indexed, to adapt to institutional requirements. On the other hand, most national journals did not have indexing alternatives in such databases due to several factors like the large number of requirements for foreign journals to be indexed, a long evaluation time on the part of the mainstream databases (Web of Science, Scopus) and lack of managerial and technical resources to enable the internationalization process.

Part of this endogeny can also be explained by the valorization of Brazilian journals indexed in alternative national or Ibero-American open access databases [41], such as Scielo, Redalyc, and Spell itself, which will be analyzed in the present article. The Brazilian experience with national and Ibero-American databases—not only in the Business Administration, Accounting, and Tourism fields—has shown that databases such as Scielo and Redalyc have contributed to improving the quality level of national journals, as well as their visibility and outspread at national and international level [24,26]. According to Mesquita et al. [25], the success of such initiatives goes beyond the open access and free publication in Brazilian journals and databases, since they are also inducers of the knowledge democratization principle called Open Access (AO). They aim at increasing the access to, and speed of, information; knowledge sharing; data visibility; as well as transparency and collaboration of published research by using the San Francisco Declaration on Research Assessment (DORA) as a reference.

The research carried out by Carvalho Neto et al. [44] has emphasized the important role played by Open Access databases and directories in the Brazilian context. The authors have investigated a sample comprising 350 journals classified by Qualis Capes evaluation and found a strong association between open access journals presenting high Qualis classification level: the OA rate at the highest evaluation levels (from A1 to B2) ranged from 90% to 80%, whereas the most poorly evaluated journals recorded much lower rates (36% for B4 with and 28% for C). Rosa and Romani-Dias [49] observed similar results for the Business Administration, Accounting, and Tourism fields: journals presenting higher Qualis levels presented high incidence in databases and indexers such as Google Scholar, Latindex, Spell, Redalyc, Scielo, among others. In addition, evidence has also shown that these journals have higher mean indexing than journals presenting the worst Qualis evaluation.

It is worth emphasizing that Brazilian publishing and indexing initiatives keep journals away from predatory and commercial models of scientific publication, which is not often seen in the international OA context. According to Xia [30], who investigated 27 alternative and open access indexers, many of them are associated with predatory journals, which, in their turn, do not show evidence of quality at the time to be indexed. This is not the case with the Brazilian experience [50], since publications in predatory journals that charge fees are discouraged and morally repressed; furthermore, these journals are excluded from official evaluation mechanisms such as Qualis.

In addition to the local scope, the relevance of Open Access goes beyond the borders of peripheral countries like Brazil and the interest in it has gained room in several countries worldwide. According to Piwowar et al. [51], at least 28% of the world literature comprises OA. If one analyzes the difference in impact between OA and non-OA articles, OA articles receive 18% more citations than non-OA articles, on average. Thus, it is worth analyzing how open access databases and indexers based on inclusive logic—such as the Scientific Periodicals Electronic Library (Spell)—contribute to the rise in the number of citations and the impact of scientific articles.

1.2. The Brazilian Experience with the Scientific Periodicals Electronic Library (Spell)

The Brazilian experience with open access databases has been successful in terms of their institutional assessment quality, increased visibility, and understanding of their scientific impacts [24,25,26,44]. One of the reasons for such success lies in the widespread use of the Open Journal System (OJS) of the Public Knowledge Project developed at the University of British Columbia (Canada), which was translated and customized in Brazil by the Brazilian Institute of Information in Science and Technology (IBICT—Instituto Brasileiro de Informação em Ciência e Tecnologia) and made available through the Electronic Journal Editing System (SEER—Sistema Eletrônico de Editoração de Revistas). According to IBICT, this initiative enabled more than a thousand Brazilian scientific journals to be created or to migrate to online versions in the first decade of the 2000s.

However, access to such journals took place in a fragmented way, since researchers rarely use databases and search engines to access articles produced in Brazil. Despite international search initiatives in free access journals such as DOAJ (Directory of Open Access Journals), they were unable to draw enough attention from Brazilian journals and researchers because they aim at the general public. Thus, Scielo has conquered a prominent position at the national level in the attempt to prepare, store, spread, and evaluate national scientific production. Its project was originally supported by official Brazilian development agencies such as FAPESP and CNPq. However, Scielo was not capable of presenting a fast response to the demand for indexing national journals—so far, it has only incorporated 5.92% of journals in the Business Administration, Accounting, and Tourism fields [52]. It mainly happened because technologies adopted for Scielo’s indexing process were not accessible to most national journals, which also ended up being excluded from the indexing process due to several editorial adequacy requirements.

Although Scielo has emerged as a platform aimed at fostering open access to national journals, it was not capable of meeting the need of different research fields in Brazil to be included in databases and indexers. Thus, instead of being a project focusing on including non-indexed national journals in international databases, it was structured as a journal selection mechanism that excluded most national initiatives. Given the low number of journals indexed by Scielo, in association with Capes’ interest in increasingly valuing impact indicators in the Brazilian journal evaluation system, the Business Administration, Accounting and Tourism fields started discussing the likelihood of building their own database and indexing system. The initial inspiration to create a national portal for journals emerged in 2006; however, it was only in 2010 that a real development proposal was articulated with the National Association of Postgraduation and Research in Administration (ANPAD). Given ANPAD’s prominence in the field—since it incorporates the main scientific meetings and the most prominent journals in the field, as well as represents postgraduate programs before official research and development agencies—the design of a database and indexing system started to take shape.

The database and indexer had no name yet, but their idealization was based on four principles, namely: inclusion, free access, visibility, and information. With respect to inclusion, despite the increasing number of publications in the field, as well as the significant increase in the number of professors and postgraduate programs in Brazil, national and international databases, and indexers, such as Web of Science, Scopus, and Scielo, used exclusion criteria, since they adopted deliberately strict access criteria that were inaccessible to most national journals in our field. According to Rosa and Romani-Dias [52], Brazilian journals in the Business Administration, Accounting, and Tourism fields present total indexation rates of 0.99%, 4.38%, and 5.92%, respectively. The new database aimed at breaking up with the exclusion logic and at including any national journal in the field that was classified in Qualis and met the criteria of good publishing practices, such as peer review.

If inclusive logic was a principle, it would make no sense to have a closed database with restricted access. This is the reason why Spell should necessarily be a free access portal for any user. The free access would enable increasing the visibility of national journals—regardless of their prestige and Qualis stratum—as well as organizing the entire content in a single database to enable integrated searches based on topic and subject. Such an organizational process was essential to increase the visibility of national production. At that time, it was much more practical to search for articles in English than in Portuguese, because there was no database capable of integrating a relatively large volume of national journals in the Business Administration, Accounting, and Tourism fields.

With respect to the higher quality information expected to be acquired through the database and indexer, Capes has implemented a process to review the journal evaluation approach in the Qualis System, which started to take into consideration the impact factor and the indexing in databases with academic-scientific reputation. This change, which meets the worldwide trend to use the impact factor of journals as an instrument to evaluate scientific publications, would lead to changes in the stratification of national journals. Hence, the idea of producing impact indicators based on citations of national journals by the journals themselves would highlight the relevance of what is produced for Brazilians in terms of use. In strategic terms, the concern in the field was to have an alternative local impact in order to avoid the massive downgrade of national journals in Qualis, since most of them are not part of international indexers.

Bearing these four principles in mind, the Scientific Periodicals Electronic Library (Spell) was officially launched in June 2012, after two years of discussions and development; it is freely accessible at www.spell.org.br (accessed on 3 August 2023). Seventy (70) journals composed the database in its first operation year—8784 articles, in total (Figure 1). At that time, the database was limited to articles published from 2000 onwards and it only presented mechanisms focused on searching and sharing articles and journals. From 2013 to 2015, the procedure adopted to include all collection articles, which comprised documents published since 1961, was implemented. Thus, in 2015, the database presented 33,253 documents deriving from 100 journals. Spell impact indicators—which included citations per document, immediacy, and self-citation rate, among others2—started to be generated in that very same year. By the end of 2020, Spell had 53,149 documents distributed in 130 journals; most of them had their impact assessed yearly.

Although Spell database/indexer is restricted to the Business Administration, Accounting, and Tourism fields in Brazil, it has some advantages over alternative indexers. The first advantage refers to the reliability of its sources. Most of the 21 alternative indexers evaluated by Gutierrez et al. [29] incorporate predatory journals with questionable quality, not to mention that their journal impact measurement method is not properly described. In addition, according to Xia [30], such indexers are not very judicious since they accept many journals that do not require peer review. In addition to not incorporating any predatory journals, Spell also exposes its data sources.

The second advantage of Spell lies in its scope in indexing journals researchers in the Administration, Accounting, and Tourism fields publish in. As shown in Figure 2, Spell indexes 11% of journals that have received publications from researchers in the field; however, these journals account for 49% of the entire volume of articles produced in the field. Although Scopus and the JCR of Web of Science present higher rates of journals (35% and 16%, respectively), their volume of published articles is much lower (17% and 7%, respectively). Google Scholar, which indexes 58% of all publications, only exceeds Spell’s publication volume by two percentage points. Although Google Scholar has greater coverage than Spell, its use as an impact indicator is discouraged in many cases due to limitations observed in its indicators [4].

Given these advantages, CAPES’ evaluation department accounting for defining Qualis classification criteria to rank the journals has incorporated Spell as an indexer for intermediate extracts, alongside h-index and CiteScore of Scopus and the JCR of Web of Science, which were used as ranking criteria for superior extracts. However, what we want to highlight with Spell’s experience is how the incorporation of both a database and indexer based on inclusive logic helped reduce asymmetries in, and partly revert the Matthew effect on journal citations.

1.3. Citation Inequalities and the Matthew Effect on Journal Evaluations

As previously mentioned, the system used to evaluate journals published by Brazilian researchers is based on CAPES’ Qualis system. One of the advantages of such a system lies in the fact that it allows journals that are not listed in impact indicators to have their quality acknowledged. This evaluation was based on the assessment of experts, who ranked the journals based on subjective criteria used to assess the quality of the publications. However, although the ranking by experts is useful and often carried out, it tends to perpetuate asymmetries and biases of judgment, as well as to favor the most prestigious journals [7,16] and the ones presenting the best status [8]—either due to their history or to the status of the publisher or institution of origin [53]—regardless of effective changes in the quality of the least prestigious journals [27]. Consequently, journals that were best evaluated in the past always tend to be better evaluated [17]. Likewise, journals that were poorly evaluated in the past are likely to be poorly evaluated in the present. Such an unequal merit, and quality acknowledgment allocation phenomenon, is called the Matthew effect in science [18,19].

The Matthew effect in Science, in the sense treated by Merton [19], refers to social processes inherent to academic activity, which lead some social actors in Science, such as researchers, institutions, and—in the herein investigated case—journals, to accumulate material or symbolic rewards in a disproportional way, regardless of merit. Such an accumulation of rewards takes place through initial comparative advantages in status [8], prestige [54], or access to resources [55], which produce incremental benefits overtime for journals presenting privileged positions or evaluation [19]. Such a cumulative advantage generation mechanism can be identified in several social stratification systems [56]; it is not only observed in science. In fact, it always produces the same result, namely: the rich get richer, whereas the poor become relatively poorer [18].

The Matthew effect was named after the Bible verse of Matthew 25:29: “for to everyone who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away”. More than metaphorical, the consolidation of this effect took place through the concept of cumulative advantage, which is often used in the literature about social mobility and poverty [56]. However, there is a striking cumulative advantage typification in bibliometrics. According to Price [57], success tends to generate success, mainly in the case of journals—those that were mostly consulted in the past are likely to be most cited in the future.

Price’s [57] finding about cumulative advantages generated due to unequal attention and merit attribution to international journals is not an isolated case [58]. There is evidence that the initial status differentiation can generate long-lasting asymmetries in assessment processes carried out in Brazil, mainly in expert-based evaluation systems such as Qualis. Rossoni and Guarido Filho [46] have investigated how the prestige of the editorial boards of different journals affected both the Qualis score (subjective assessment) and the impact of the assessed journals (objective assessment). Despite the ceremonial nature surrounding the activity performed by editorial boards, boards comprising both more prestigious and international members tended to have higher Qualis scores, but not greater impact factors. It only happened due to the effect such boards had on experts’ evaluation, which did not show practical association with greater article reading and citations in the journals. However, in addition to enabling journals to have higher scores in Qualis, this effect has also increased the number of times they were cited in subsequent periods [46], which has self-fulfilled experts’ prophecy about the accuracy of their evaluation. Thus, the academic distinction element was given by the symbolic appeal of the evaluated journals rather than by the effective contribution of their publications.

Very little can be conducted about the accumulation of advantages by the most visible journals in expert-based evaluation systems. According to Machado-da-Silva et al. [48], the citation of Brazilian journals in the Business Administration, Accounting, and Tourism fields have increased by 22% from 2005 to 2007 (9 out of 10 journals), whereas the citation of the three most cited journals has increased by 44%—i.e., twice as much the increase observed in the previous ones, within a 3-year interval. In addition to the lack of search engines to enable all journals in the field to have the same chances of being identified through searches based on topic, the inequality in the acknowledgment of these journals would only increase.

According to Starbuck [27], the emphasis placed on prestigious journals results much more from administrative choices than from the natural adaptive result of scientific practice in Social Sciences; this factor leads to perception biases towards the value of published articles and hinders knowledge improvement processes. The emphasis on indicators and rankings further increases the biases of such a perception. Bornmann et al. [17] have used agent-based simulations to show that the introduction of new indicators, such as Hirsch’s hα-index, reinforced the Matthew effect at the researcher level and led to the disproportionate acknowledgment of journals that had initially presented high h-index. Lariviere and Gingras [7] have found similar effects between journals. They performed a natural experiment to compare 4532 identical studies published in different journals and found that the journals’ impact factors strongly influenced citation rates. Based on the comparison of each pair of duplicated articles, those published in the highest-impact journals had, on average, twice as many citations as the version published in the lowest-impact journals. Drivas and Kremmydas [8] have also found similar results. They assessed the ranking effect on the Academic Journal Guide published by the Association of Business Schools and found causal evidence that increasing journals’ ranking on the list would increase their number of citations.

The Scientific Periodicals Electronic Library (Spell) was generated due to quality assessment biases that led to Matthew effect on journal citations. It was carried out in order to increase the visibility of Brazilian academic productions in the Business Administration, Accounting, and Tourism fields, as well as to reduce asymmetries and inequalities in the journal quality acknowledgment system so that any journal would have the same chance to be identified in the database, regardless of ranking. The next section presents, in detail, the analysis method adopted in the current study to find evidence that Spell has increased the number of citations in articles available in the database, reduced the inequality between national journals in the field, and mitigated the Mathew effect on journal citations.

2. Materials and Methods

2.1. Data

Impact indicators of journals composing Spell were collected on its homepage (http://www.spell.org.br/impacto, accessed on 3 August 2023). Spell makes available impact indicators of indexed journals—whose first publication dates back to 2010—on a yearly basis. Until the end of the current research, 9 listings had been published between 2010 and 2018. The citations received by the analyzed articles come exclusively from the Spell database. Thus, although Spell had 130 journals registered in 2020, there was variation in the number of journals presenting impact indicators between 2010 and 2018. It happened because not all journals were in the listing, since the service started operating in late 2012, as well as because other journals had been terminated. In addition, journals must be up to date in order to be part of the list of impact indicators released every year. Consequently, the number of indexed journals has ranged from a minimum of 78 journals in 2010 to a maximum of 114 journals in 2018.

However, the aim of the current study was to investigate the impact of Spell implementation in 2012 on the journal’s impact and inequality. We herein compared the same interval before and after its launching, namely: 2010–2012 and 2013–2015. Only journals listed in all editions released within that time interval were taken into consideration in the analysis. This procedure was adopted to avoid the effect of including or excluding journals listed in different periods, as well as a significant increase in the volume of citations due to incoming journals. Thus, our database was formed by a balanced data panel comprising 68 journals observed for 6 consecutive years providing 408 cases, in total.

2.2. Variables

Spell freely makes available several impact indicators for journals organized by year, in which the citations for these impacts are exclusively sourced from Spell’s database; it is also possible to export them to electronic spreadsheets. Our job was to compile these indicators in a single matrix, whose definition—given by the database—can be seen at http://www.spell.org.br/impacto/ajuda (accessed on 3 August 2023). Of the available indicators, we stick to two:

Two-year impact without self-citations—remarkably similar to impact indicators used in databases such as Web of Science and SCOPUS, which take into consideration the number of times a given journal is cited—except for self-citations—in a given year (for example, 2015), in articles published in the two immediately preceding years (2014 and 2013), divided by the number of articles published in the same period. Thus, the impact of the 68 analyzed journals was annually observed 6 times between 2010 and 2015.

Five-year impact without self-citations—it was computed in the same way as the previous index; however, it took into consideration a window of citations to articles published in the previous five years and divided by the total number of articles published in those same five years. For example, the impact of each journal in the sample was assessed by taking into consideration the number of citations recorded in 2015 for articles published between 2010 and 2014, divided by the total number of articles produced in that same period. The same process was applied to the other years, until 2010.

It should be taken into consideration that, in addition to disregarding self-citations that can inflate impact indicators, the indicators described above have the advantage of being a relative measurement, since they take into consideration the number of received citations in comparison to the total number of published articles. This factor helps avoid the effect of journal longevity and size on impact indicator composition, as observed in absolute measurements, such as the H Index, which end up reinforcing the Matthew effect [17,59].

In addition to indicators generated by Spell, it was necessary to create two other variables: the first one was created to show that the emergence of Spell in 2012 has increased the impact of journals and reduced the inequality of citations. The second variable was created to capture the relative impact of a given journal in comparison to the impact of all other journals—this variable enabled exposing the inequality in citations:

Period. When one variable with two categories was created: 2010–2012—aggregates all the observations recorded in the years immediately before Spell creation; 2013–2015—also aggregates observations in our sample in the three years following the database emergence. In terms of effect capturing, this variable was treated as a dummy—value 1 corresponds to the period between 2013 and 2015.

Our intention was to meet the criteria of event studies focused on evaluating the effect of external events on journals [60]. In order to do so, it was necessary to specify an equal time window before and after Spell creation in 2012. As Spell citation data refer to the year 2010, the time window was limited to six years: three years before and three years after its creation. Although this cut has reduced the sample to 204 observations for each period, it did not compromise the power of the herein-performed tests, since the sample size remained large.

iGini (individual Gini) is an individual inequality measurement tool developed by Liao [28]. It derives from one of the aggregate inequality indicators mostly used in studies about wealth distribution [61], namely: the Gini index. Ceriane and Verme [62] have highlighted that the Gini index captures inequality based on the aggregation of individual contributions. According to Liao [28], the aim of iGini is to analyze inequality at the individual level to help improve the understanding of the ways inequality takes place. Thus, iGini can be seen as the difference in scale between a given observation and all other observations in the same sample [28]. In the current case, iGini captures the difference between the impact of one journal and that of all other journals in the same year. Thus, based on the formalization by Liao [28] (p. 7, Equation (2)), for each journal I, the iGini or $gI$_i can be defined as:

$$g_i=\frac{\sum _{j=1}^n\left|x_i-x_j\right|}{{2n}^2\stackrel{̿}{x}}$$

(1)

wherein x_i represents the Spell impact of journal i and x_j refers to the Spell impact of all other journals j on the sample with size n. The denominator is used to normalize the indicator between 0 and 1. The higher the value, the greater the distance in terms of citations from one journal i to the other journals j.

It is worth emphasizing that the use of Liao’s iGini [28] as a measurement tool enabled analyzing the effect of Spell on reducing the inequality of citations since any other aggregate inequality assessment tool would significantly reduce the degrees of freedom of the tests.

2.3. Analysis Models

Since annual observations referring to three years before and three years after Spell creation were recorded for all 68 journals, we used panel data models. Panel data deals with omitted variable bias due to heterogeneity in the data [63]. It does this by controlling for variables that we cannot observe, are not available, and/or cannot be measured but are correlated with the predictors [64]. It is worth emphasizing that these models are mainly useful when heterogeneity is not observed in balanced panels In our study, these cases refer to journals, whose other effects in the period would not be easily identified.

Given the various alternatives for adjusting panel data models, we conducted some verification tests. First, we did the Hausman test, which showed that individual characteristics correlate with the regressors (chi² = 8.8, p = 0.0123), requiring us to use fixed rather than random models. Second, we checked whether it was necessary to control the effect of the year, where the joint F-test indicated that the combined effect of the years was significant (F = 48.54, p < 0.001). Third, we checked whether the panel suffered from contemporary correlation problems using Pesaran’s test [65], in which no cross-sectional dependence was identified (−1.051, p = 0.2935). Fourth, the modified Wald test for groupwise heteroskedasticity in fixed effect regression models indicated that the error distribution was not homoskedastic (chi² = 24,059.2, p < 0.001). Fifth, the Wooldridge test for autocorrelation in panel data (Lagrange Multiplier test) pointed to serial correlation (F = 16.013, p = 0.0002). Faced with these last two problems, we used the robust standard error clustered at the journal level [65]. We ran the same models as a robustness check using random-effect models. There was no significant change in coefficients, regardless of the adopted models and error types.

Thus, 6 models were generated in the current study: the first 4 were used to evaluate the effect of Spell implementation on the 2- and 5-year impacts and on inequality in the 2- and 5-year impacts based on the dummy variable “Period”, which captured the effect of the 2013–2015 period on dependent variables. As for the last two models, the variable “Impact” was added to Spell as a moderator in inequality models to capture whether Spell implementation affected journals with different impact levels in different ways.

In addition to the fixed and random models of panel data, we used the Analysis of Variance between Subjects (ANOVA) to calculate the size of the Spell effect; the explained variance rate was calculated by dividing the value of the sum of Type III squares by the total corrected variance, as suggested by Snijders and Bosker [66]. Thus, 2- and 5-year impacts, as well as 2- and 5-year iGini, were incorporated into these models as variables to be tested. The effect of including journals in Spell was also tested—based on the variable “Period”—in order to check whether the means recorded for impact and inequality differed from each other between 2013–2015 and 2010–2012. Similar to fixed panel data models, we controlled part of the non-observed effects by incorporating the factor “Journal” to increase the concurrent validity degree and the Year effect. With this, we hope that fluctuations that occurred at the journal level caused by variables or events not included in the model, will not severely contaminate the analysis of the Spell effect. ANOVA models were included in Appendix A.

We seek to attest to the causality of Spell’s effect on reducing inequality between journals by employing the difference-in-differences (DID) model with continuous treatment [31] to assess the average treatment effect on the treated (ATET). To this end, we assume the impact measures are treated continuously, so that, after the entry of Spell, the effect of the impact on inequality between journals is smaller. So, we first conducted the treatment effect test through the interaction term between the period before and after the adoption of Spell with the impact of 2 and 5 years [33,67]. Thus, we considered that there was a significant reduction in inequality if the interaction terms were significant (p < 0.05) and if the Wald test of comparison between the coefficients with and without interaction pointed to a significant value. Additionally, we conducted the DID test using STATA’s xtdidregress command to verify whether the average treatment effect on the treated (ATET) was significant, providing greater robustness to the results.

Finally, we sought to analyze the robustness of the results by investigating the temporal effect of Spell in reducing citation inequality. To this end, we used two models suitable for panel data, especially when they present serial correlation problems [65]. First, we sought to evaluate the variation between the years before (leads) and after the Spell intervention through the event studies model operationalized through STATA’s eventdd command.

In addition to being suitable for panel data, it allows the “full set of event leads allows for the inspection of parallel trends in the pretreatment period [and] the policy lags allow for inspection of the temporal nature of treatment effects, noting any dynamics in the appearance of effects, for example, increasing or decreasing effects over time, and whether effects are transient or permanent (p. 858)” [34]. The model implementation is straightforward, in which we follow the standard definitions of the package, including panel-fixed effects with standard errors clustered in units (Journal).

In the second model, we seek to analyze the trajectory of change after the intervention through an intermittent time-series analysis (ITSA model), which cannot be carried out with the event studies model). We use STATA’s xtitisa command [68], including panel-fixed effects with standard errors clustered in units (Journal). The ITSA model, in addition to evaluating the significance of the slope before the intervention (T), analyzes its effect immediately posterior (X), as well as the slope after the intervention (X × T). Finally, it demonstrates the acceleration of the effect by checking the linear trend after the intervention [35].

3. Results

3.1. Spell-Indexed Journal Citations

The citation rate recorded for articles belonging to the Business Administration, Accounting, and Tourism fields, which were published in journals listed in the Spell database has been constantly increasing in Brazil, as shown in Figure 3. A little more than 37% of citations observed in 2010 referred to articles in journals, whereas the total citation rate recorded in 2015 has increased to 52%. This outcome has indicated a trend to increasingly cite peer-reviewed documents to the detriment of books or other document types.

Regardless of the reasons why this increase has been taking place—from the larger number of databases providing access to articles to the constant valuing of journal citations, mainly of the international ones—it was pertinent to compare the increased number of citations of Spell-indexed journals (all Brazilian journals) to the total number of journal citations. Figure 3 shows this proportion, according to which the rate of article citations in Spell has increased from 12.8% in 2012 (when Spell was launched) to 15.2% in 2015.

Therefore, despite the emphasis given to the fact that Brazilian research in the Business Administration, Accounting and Tourism fields have been increasingly using foreign literature, primarily in English, the data points out exactly the opposite: even though it is not the majority of citations, there is a constant increase in citations of Brazilian papers indexed in Spell. This outcome meets the aim of the database, which was created to increase the access to, and visibility of, Brazilian scientific production.

3.2. Impact of, and Inequality in, Spell Citations

The percentile growth of citations in Spell must also be taken into consideration based on the impact index, which compares the total number of citations recorded for a given journal to the total number of published articles. Figure 4 shows the mean impact of journals indexed in Spell, in a 5-year window—self-citations were always excluded. Up to Spell implementation in 2012, the impact of database journals ranged from 0.09 citations per document in 2010 to 0.17 in 2012. After Spell implementation, this rate increased to 0.20 in 2013 and reached its peak (0.28) in 2015.

Although the citation increase trend was observed before Spell was implemented in 2012, we understand that this trend was consolidated by the emergence of the aforementioned indexer, since it was widely adopted by the research community in the Business Administration, Accounting, and Tourism fields, which needed topic-based search engines comprising all database journals. This outcome has encouraged us to make more robust comparisons.

However, it is worth emphasizing that the increase in mean citations of Spell articles does not happen in a concentrated manner. On the contrary, since the platform enables equal access to articles, data shown in Figure 4 indicate that the distribution of citations is getting lesser unequal.

This outcome was evidenced through the Gini coefficient of Spell impact recorded for a 5-year window. Such a measurement tool is used to measure inequality in distributions, mainly to measure income inequality between countries. As previously mentioned, this index ranges from zero to one—higher values correspond to greater inequality. The Gini coefficient tended to remain stable until 2012. After this year, when Spell started its activities, the coefficient systematically declined, and it indicates that the inequality of citations among journals in the database has decreased over time.

For the purpose of analogy with income inequality between countries, it is as if we have left an inequality condition equal to that of Brazil to the inequality of countries such as Israel and Bulgaria. However, without analogy, the decreased inequality of Spell citation impact indicates that, despite the differences in resources and power asymmetries observed between journals listed in the database, Brazilian researchers in the Business Administration, Accounting, and Tourism fields started to acknowledge the usefulness of journals in a more egalitarian and lesser centralized way. As we have pointed out, Spell’s prominence in promoting fairer conditions for Brazilian journals has likely led to lesser inequality, since all journals have the same visibility space and can be mainly accessed based on content rather than on journal status.

Apparently, “the richest” journals in terms of citations are not proportionally getting “richer”, which is the reason why we must carefully analyze whether Spell has reduced the Mathew effect on journal citations. Since such assessments cannot be conducted with aggregate indices such as Gini, we used the measurement tool recently formulated by Liao [28], iGini, whose trend is also shown in Figure 4. It is worth emphasizing that Gini corresponds to the exact sum of iGini; however, the Gini excludes cases presenting value 0, which is not the case for iGini calculation. This is the reason why iGini recorded higher values than Gini between 2010 and 2012. These journals without citations were listed in the database; therefore, they could not be featured as omitted cases; thus, they were included in the analysis applied to the Spell effect on reducing the inequality of citations.

3.3. Comparing the Impact of and Inequality in Citations before and after Spell Implementation

Table 1. Impact and mean inequality of Journals in Spell.

	2-Year Impact			5-Year Impact
	2010–2012	2013–2015	Growth (%)	2010–2012	2013–2015	Growth (%)
Spell Impact Factor	0.097	0.181	87%	0. 123	0. 235	90%
25th percentile	0.017	0.068	300%	0.027	0.092	241%
50th percentile	0.063	0.131	108%	0.073	0.172	136%
75th percentile	0.132	0.235	78%	0.150	0.291	94%
iGini	0.0086	0.0067		0.0085	0.0065

Note: N = 408 (68 journals × 6 years). Spell Impact Factor without self-citations. iGini was calculated based on the impact of Spell. The higher the mean value recorded for iGini, the greater the impact of inequality.

presents the mean impact of Spell, without self-citations, recorded for 2- and 5-year windows in the periods right before (2010–2012) and after (2013–2015) Spell implementation. It also presents the mean iGini recorded for the same periods, whose inequality was calculated based on the impact of journals. As expected, the mean 2-year (0.097 to 0.181) and 5-year impacts (0.124 to 0.236) have increased between the two periods. Impact inequality recorded for the 2-year (0.086 to 0.0067) and 5-year (0.0085 to 0.0065) windows between the two periods has decreased.

Table 1 also shows that citation impact grew in all quartiles (25%, 50%, 75%). However, growth was more remarkable precisely in the lower quartiles. The lowest inequality of citations observed between the evaluated periods is highlighted in Figure 5, which depicts the distribution of the 2-year (violin chart on the left) and 5-year (on the right) impacts. A greater volume of journals recorded lower impact values in 2010–2012; however, this volume decreased in 2013–2015, due to a higher density of cases in intermediate portions of journals listed in the database. Figure 6 corroborates these results, as there is an increase in both the median of the 2-year impact (from 0.065 to 0.131) and the 5-year impact (from 0.073 to 0.172). Thus, Spell’s mean impact has shown an increase in the number of citations recorded for documents between the analyzed periods, whereas iGini has evidenced whether such an increase was better distributed.

Although the herein recorded means have indicated an association of Spell implementation with increased impact and reduced inequality, it was necessary to test whether the period-change effect was significant, or not. It was conducted based on the application of regression tests to fixed panel models; results are shown in

Table 2. Regression results recorded panel time fixed effect models.

	(1)	(2)	(3)	(4)	(5)	(6)
	2-Year Impact	5-Year Impact	iGini2	iGini5	iGini2	iGini5
2013–2015	0.143 ***	0.197 ***	−0.004 ***	−0.003 ***	−0.005 ***	−0.003 ***
	(0.015)	(0.019)	(0.001)	(0.001)	(0.001)	(0.001)
2-Year Impact					0.035 ***
					(0.005)
2013–2015 × 2-Year Impact					−0.017 *** (0.003)
5-Year Impact						0.020 ***
						(0.003)
2013–2015 × 5-Year Impact						−0.013 ***
						(0.002)
Constant	0.075 ***	0.088 ***	0.010 ***	0.009 ***	0.007 ***	0.007 ***
	(0.009)	(0.011)	(0.001)	(0.001)	(0.001)	(0.001)
R2	0.328	0.498	0.203	0.286	0.630	0.593
Adjusted R2	0.320	0.491	0.193	0.278	0.624	0.586
F	21.49 ***	24.99 ***	12.07 ***	10.92 ***	24.73 ***	27.9 ***

Note: N = 408. Robust standard errors are presented in parenthesis (adjusted for 68 clusters in the Journal). Dummy variables with year effect omitted. *** p < 0.01, ** p < 0.05, * p < 0.1.

. After the effects of journals were controlled in models 1 and 2, there was a significant increase in the impact of journals after Spell implementation (variable 2013–2015), both for the 2-(b = 0.143) and 5-year (b = 0.197) windows. In other words, based on the comparison between three years before Spell implementation and three years after it, there was a significant increase in mean impact recorded for both time windows, whose explanatory power ranged from 7.6% to 7.7% according to the ANOVA test (Appendix A).

Models 3 and 4 presented significant effects on reducing impact inequality between journals in the 2- (b = 0.004) and 5-years (b = 0.003) windows. Thus, based on the comparison between 2010–2012 and 2013–2015, there was a decreased difference in the impact of journals (explanatory power of 3.3%, according to the ANOVA test). This outcome enables saying that journals’ inclusion in Spell is associated with increased equitable distribution of citations per document, which, consequently, reduced the Matthew effect at the journal level.

Finally, the dummy variable representing journals’ inclusion in Spell (2013–2015) has interacted with the absolute impact of journals (2- and 5-Year Impacts) in models 5 and 6. It was carried out to investigate whether inequality reduction was associated with journals’ impact factor. Results have shown that the greater the impact of the journal, the less unequal it became in relation to the others, both for the 2- (b = −0.017) and 5-year impacts (b = −0.020).

Figure 7 depicts the association between impact and inequality before and after Spell implementation. Both impact measurements—2 and 5 years—have shown decreased association between impact and iGini in 2013–2015 in comparison to 2010–2012. To corroborate these results, we conducted the Wald test between the Impact coefficients before and after Spell. The difference in the effect of citations on inequality reduced significantly for both the 2-year (F = 144.62, p < 0.001) and 5-year impact (F = 115.39, p < 0.001). Furthermore, the difference-in-differences regression showed that the average treatment effect on the treated (ATET) was 0.0201 (p < 0.001), adjusted for the covariates, time, and panel effects. Such a test could not be conducted for the 5-year impact, as the model did not converge due to its limitations. In any case, all other findings indicate that the difference occurred for the 5-year impact, even if in a less intense way. Thus, the current data show that journals’ inclusion in Spell has reduced the Mathew effect and made them more egalitarian.

3.4. Robustness Check: Analyzing the Impact of Spell over Time

To corroborate the results, we analyze the effect of Spell reduction on the inequality of citations between journals over time. Referring to the Spell intervention year (2012), we included leads and lags in the panel data event study model that indicates the effect between years. The results in

Table 3. Fixed effects panel event study model.

	(1)	(2)
	iGini2	iGini5
lead3 (2010)	0.002 ***	0.001 **
	(0.001)	(0.001)
lead2 (2011)	−0.001	0.001
	(0.001)	(0.001)
lag0 (2013)	−0.001	−0.0001 ***
	(0.001)	(0.001)
lag1 (2014)	−0.001 ***	−0.002 ***
	(0.001)	(0.001)
lag2 (2015)	−0.002 ***	−0.002 ***
	(0.001)	(0.001)
Constant	0.008 ***	0.008 ***
	(0.001)	(0.001)
R-squared (within)	0.203	0.286
F(5, 67)	12.07 ***	10.92 ***
Joint test for leads (2010–2011)	6.057 ***	3.287 **
Joint test for lags (2013–2015)	9.274 ***	10.693 ***

Note: N = 408. Robust standard errors are presented in parenthesis (adjusted for 68 clusters in the Journal). Fixed effects. The baseline of eventdd model: 2012 year. *** p < 0.01, ** p < 0.05, * p < 0.1.

suggest no significant difference between the years 2012 and 2011 for both iGini2 and iGini5 (b_lead2 = 0.001). However, it cannot be ruled out that there was a trend in the period before the intervention, as there are differences in inequality between the years 2010 and 2012 for both iGini2 (b_lead3 = 0.002, p < 0.01) and iGini5 (b_lead3 = 0.001, p < 0.05). This result is corroborated by the joint test for leads, which proved significant.

When analyzing the trajectory after the intervention, the reduction in inequality was significant both one year later for iGini2 (b_lag1 = −0.001, p < 0.01) and iGini5 (b_lag1 = −0.002, p < 0.01) and two years later: b_lag2 = −0.001, p < 0.01 for iGini2; b_lag1 = −0.002, p < 0.01 for iGini5. The joint test for lags corroborates this. However, what is notable is that the reduction in inequality for iGini2, which is based on document citations from the last two years, took time (b_lag1 = −0.001, p = 0.414). For iGini5, which is based on citations of documents with up to five years of publications, the effect immediately after the arrival of Spell is significant (b_lag1 = −0.001, p < 0.01). Such trajectories can be visualized in Figure 8. While the iGini2 graph (left) points to a gradual drop in citation inequality, the iGini5 graph points to an immediate reduction at the time of intervention (time 0 = 2013).

The magnitude of the effects between lags and leads indicates a more significant reduction in inequality after the Spell intervention. However, there are doubts whether this reduction was accelerated or not, as well as what trajectory this change took. To answer these questions, in

Table 4. Interrupted time-series panel model.

	(1)	(2)
	iGini2	iGini5
T (2010–2012)	−0.0013 ***	−0.0009 ***
	(0.0002)	(0.0001)
X (2013)	0.0005	−0.0005 **
	(0.0004)	(0.0002)
X × T (2013–2015)	−0.0005 **	−0.0001
	(0.0002)	(0.0002)
2-Year Impact	0.0246 ***
	(0.0033)
5-Year Impact		0.0106 ***
		(0.0019)
Constant	0.0075 ***	0.0080 ***
	(0.0002)	(0.0006)
Postintervention Linear Trend (2013)	−0.0017 ***	−0.0009 ***
	(0.0003)	(0.0001)
Wald chi²	63.20 ***	48.36 ***

Note: N = 408. GEE population-average model (family: Gaussian; link: identity) using xtitsa Stata command. Robust standard errors are presented in parenthesis (adjusted for 68 clusters in the Journal). Legend: T: slope prior to intervention; X: change in the level immediately following intervention; X x T: difference between preintervention and postintervention slopes. *** p < 0.01, ** p < 0.05, * p < 0.1.

, we present the results of the intermittent time-series panel model. Firstly, the model points out that there was already a temporal downward trend in inequality before Spell, both for iGini2 (b_T = −0.0013, p < 0.01) and for iGini5 (b_T = −0.0009, p < 0.01). Corroborating the event study model, the reduction in inequality for iGini2 was not immediate (b_X = 0.0005, p = 0.223). However, it was for iGini5 (b_X = −0.0005, p < 0.05). The trajectory after the Spell intervention pointed to a significant change in slope for iGini2 (b_XxT = −0.0005, p < 0.05), which was not observed for iGini5 (b_XxT = −0.0001, p = 0.531).

Despite different trajectories for the two types of inequality indicators, where in iGini2 there was a gradual change, and for iGini5 the change was immediate, the linear trend postintervention test points to an acceleration in the reduction of citation inequality after the Spell intervention for both iGini2 (b = −0.0017, p < 0.01) and iGini5 (b = −0.0009, p < 0.01). Added to the results of previous models, there are elements to infer the effect of Spell in reducing citation inequality.

4. Discussion and Conclusion

This paper has shown how the inclusion of journals in open databases and indexers based on inclusive logic has the potential to increase the impact of citations and reduce inequality between journals. Based on the Brazilian experience in the Business Administration, Accounting, and Tourism research fields, which created a database and indexer to cover most national scientific productions in the field in 2012, the current study has: (1) compared the impact of journals before and after Spell implementation; (2) investigated whether these journals became lesser unequal in terms of impact after Spell implementation.

With respect to the first point, after the non-observed heterogeneity at the journal level was controlled, results have shown that the mean 2- and 5-year impact of journals recorded in the three years right after Spell implementation was significantly higher than that recorded in the three years before it. It is worth emphasizing that the comparison of citations before and after journals’ inclusion in Spell was only possible because the database and indexer had calculated the impact indicators for the three years prior to its launch.

As previously mentioned, the most likely interpretation for the fact that Spell has increased the mean number of citations lies in the fact that the platform made available articles openly published in the management research field on a single database that enabled equal search possibilities for all journals. Before Spell, there was no efficient search engine encompassing the scientific production of the management field in a single environment; consequently, researchers in the field had to search journals’ websites separately, a fact that favored the best-known ones. The visibility Spell has given to journals results from the prominence of its owner, the National Association of Postgraduation and Research in Administration (ANPAD), which is the association of graduate programs and researchers in the field that has outspread its use countrywide. Thus, without ANPAD’s prominence in the field, the database adoption process would have been much slower and would not allow the improvement in information and visibility to be used as an impact increase mechanism.

As for the second point, the inequality measurement newly created by Liao [28]—iGini—enabled showing that journals’ inclusion in Spell has made the impact of citations less unequal within the three years after such an inclusion than within the three years before it, both for the 2- and 5-year impact measurements. In addition, we have also shown that the influence of the ranking of journals in Spell on the inequality between journals has decreased after journals were indexed in it. This outcome enabled the conclusion that Spell has likely reduced the Matthew effect on journal citations since the highest-impact journals were not so far from the other journals.

Our understanding is that the reduced Matthew effect on Spell journals’ citations is linked to two mechanisms. The first mechanism refers to reduced information asymmetry. As most journals in the management field were not ranked on databases such as Scopus and Web of Science, the evaluation of their likely quality was based on Qualis CAPES, which is the basis used to classify Brazilian journals [47]. Until 2012 Qualis CAPES guided the evaluation of Brazilian journals almost exclusively based on subjective criteria [46,52]. Moreover, such an assessment was based on the status of the institution editing the journal [53]. Thus, after the creation of Spell, which provided the same possibilities for articles to be found through thematic searches, researchers had a better opportunity to assess the quality of articles, regardless of the journal. In fact, if articles with better-perceived quality tend to be more cited [54], it has likely happened at Spell.

The second mechanism refers to a decrease in the perception of the differentiated status attributed to some journals that are better classified in rankings such as Qualis or due to the prestige of institutions accounting for editing them. This status leads researchers to cite articles deriving from better-evaluated journals, rather than basing the citation on articles’ quality [7,8]. On the contrary, limitations in the process of investigating all the relevant literature often lead researchers to only search for studies in the most prestigious journals [17,54]. This phenomenon reinforces the Matthew effect since the most prestigious journals always have disproportionate acknowledgment gains. However, the rise of Spell—which has incorporated an equitable search engine to impact measurements based on local production—has decreased the incentives to only cite the most prestigious journals. Individual inequality measurements point towards decreased differences in citations between journals and show that such a decrease is inversely proportional to the impact of journals.

Our findings have important implications for studies about the impact of journals, as well as about scientific research acknowledgment mechanisms. First, based on the analytical perspective, results have indicated that the unwanted trend of researchers to keep on citing scientific articles from better-ranked journals mainly takes place because indexers (e.g., Scopus and Web of Science) and listings of journals with differentiated status (e.g., Academic Journal Guide of the Association of Business Schools) operate based on a segregation logic and exclude most relevant journals. This process is evident in a significant part of the literature, which shows a strong association between journal status and citations [7,17,27,54,59]. Without this segregation mechanism, the cumulative advantages of such rankings would most likely be mitigated. According to Merton [18,56], Price [57], and DiPrete and Eirich [56], the Matthew effect only happens in systems based on different statuses that are not always associated with quality. The effects of citing the best-evaluated journals would be mitigated in systems based on inclusive logic, such as Spell and Redalyc [26].

Secondly, in methodological terms, the analysis of inequality between actors in a given social system has always been a methodological challenge [62], mainly if one investigates the individual contribution of each actor. Thus, only recently, Liao [28] has developed an individual indicator—iGini—which has several advantages such as aggregated or segmented measurements between and within groups. Thus, we advocate that incorporating iGini as a measurement to assess differences in impact and citations can open room for relevant research to be developed by scholars focused on investigating the Matthew effect and science acknowledgment, mainly because iGini can capture a given actor’s distance from everyone else, as well as enables analyzing different forms of inequality.

Thirdly, it is necessary to highlight the practical implications of this study, mainly to research assessment policies. By incorporating an open, inclusive, free database and indexer without the intermediation of commercial publishers, as in the case of Spell, such a system directly attacks one of the biggest issues observed in the use of rankings, listings, and impact factors: acknowledgment based on citations that do not necessarily refer to the quality of articles. In other words, since such initiatives reduce the Matthew effect, they bring the practice of citations closer to an idealistic ethos of scientific acknowledgment based on merit [69], which is only possible through inclusive, non-exclusive databases and indexers. At the individual level, if there is clear evidence that authors cite articles based on journals’ status and impact factor [7,8], the academic community must explicitly acknowledge these citations for the quality of the documents, regardless of the journals’ prestige. One of the recent alternatives lies in the likelihood of journals, indexers, institutions, and research funds being signatories to the San Francisco Declaration on Research Evaluation (DORA), which makes recommendations about the separation of journal metrics from the research evaluation itself. What DORA signatories want—and there are indications that they are achieving it [70]—is that science assessment systems that currently attribute impact through citations do not distort the system to the point of making such metrics useless. In addition, it is worth emphasizing that Spell is the initiative of a large, although peripheral, academy that faces great issues in legitimizing itself at local and international levels, namely: the Brazilian academy. Since the Matthew effect on citations also happens at the country level [58], initiatives focused on promoting the dissemination of local science, in an open and freeway, help fight the hegemonic practices of commercial publishers that limit knowledge dissemination and distort the acknowledgment of academic ventures.

Finally, it is necessary to highlight that this research has some limitations that should be overcome in future research. The main limitation lies in the fact that we conducted an event study in which the exogenous shock—i.e., the creation of Spell—affected all journals at the same time. Although there is no suspicion of another relevant event for the journals in the sample, our research strategy may have contemporary correlation issues. In addition, as data about citations of Brazilian journals in the management field that are not included in Spell are not available in virtually any other index, it was not possible to perform a quasi-experimental treatment to compare the sample of journals included in Spell to those that were not included in it. Future studies could investigate mechanisms used to estimate citations of journals that are not included in indexers, such as Spell, in order to check the causality of including a given journal in them, both in terms of impact and inequality. Since we were not interested in explaining the individual impact of journals, but rather the change caused by Spell implementation, our study has ignored effects at the author and article level, as well as some proxies at the journal level. Future studies focused on better understanding the inequality of citations could decompose the effect at these levels. Finally, since iGini is a measurement of individual inequality, we strongly recommend that this indicator should be used by other researchers to investigate the effect of other databases on the stratification of science, mainly by comparing the effect of international indexers.