Evaluation of a Company’s Media Reputation Based on the Articles Published on News Portals

Abstract

A company’s reputation is an important, intangible asset, which is heavily influenced by media reputation. We developed a method to measure a company’s reputation based on sentiments detected in online articles. The sentiment of each sentence was evaluated and categorized into one of three polarities: positive, negative, or neutral. Then, we developed another method to assess a company’s media reputation using all available online articles about the company. The company’s media reputation is presented as a tuple consisting of their media reputation on a scale from 0 to 100, the number of articles related to the company, and the margin of error. Experiments were conducted using articles written in Lithuanian published on major news portals. We used two different tools to assess the sentiments of the articles: Stanford CoreNLP v.4.5.10, combined with Google API, and the pre-trained transformer model XLM-RoBERTa. Google API was used for translation into English, as Stanford CoreNLP does not support the Lithuanian language. The results obtained were compared with those of existing methods, based on the coefficients of media endorsement and media favorableness, showing that the results of the proposed method are less moderate than the coefficient of media favorableness and less extreme than the coefficient of media endorsement.

1. Introduction

Reputation is as important to companies as it is to individuals. A company’s reputation is considered a vital intangible asset that includes brand equity, product and service quality, customer relationships, supplier relationships, and community standing [1]. Reputation is not inherent; to build it, a company must invest in marketing and advertising. To reflect the importance of media coverage in building this asset, Deephouse [2] coined the term “media reputation” in 2000, defining it as “the overall evaluation of a firm presented in the media” (p. 1097).

In a general sense, a company’s reputation is referred to as its corporate image. Vito [3] distinguished four different methods commonly used to assess corporate reputation in the literature: corporate reputation ranking, structured questionnaires, third-party indices, and media coverage. Structured questionnaires are the most common method, used in 33% of cases. In contrast, media coverage analysis remains relatively understudied, accounting for only 10% of cases. However, media reporting on companies is widespread. Since the internal workings of a company are often a “black box” to the public, media coverage is the main tool reducing information asymmetry regarding a company’s actions [4]. Opinions are often expressed using sentiment-laden words [5], which serve as a primary mechanism by which the media impacts readers’ perceptions. These words subtly influence readers to form judgments about a subject. For example, the positive or negative framing of events provides a visible public expression of approval or disapproval of companies. Thus, in addition to sharing information, business journalists also disseminate their own perceptions.

The tone of a company’s media coverage—the aggregate sentiment of articles—reflects a difficult-to-quantify assessment of a company’s fundamentals [6]. Positive media coverage is likely to increase credibility and trust among consumers. Over time, positive media sentiment makes a company’s offerings more attractive to existing and potential customers, thereby driving sales growth. Therefore, media reputation is a strategic resource and driving force behind a company’s overall reputation.

Modern media can be divided into two broad groups: articles published on news portals; social media. To date, research has primarily focused on sentiment analysis in social media [5,7,8]. Very few studies have assessed the media reputation of companies based on articles published on news portals. Therefore, our study focuses on this underexplored area.

The contributions of this paper are as follows:

• Providing a systematic literature review of methods that assess the media reputation of companies based on online news articles;
• Proposing a novel method to measure an article’s reputation index;
• Creating and implementing a method to measure the media reputation of companies, which can be easily adapted to any language;
• Collecting news articles from major Lithuanian portals, conducting experiments to measure the media reputation of companies, and comparing the results with those of other existing methods.

The remainder of this paper is organized as follows: Section 2 reviews existing methods for assessing media reputation. Section 3 presents the proposed methods. Section 4 explains the experimental results and compares them with those of different methods. Finally, Section 5 presents the conclusions drawn from this study.

2. Review of Related Work

Content and sentiment analysis methodologies are commonly used to assess media content [2]. Content analysis is one of the most frequently employed methods by media content raters to systematically analyze and categorize information to determine the emotional tone, which can be negative, neutral, or positive. Janis I. L. and Fadner R. H. [9] developed a coefficient of imbalance in 1943 to assess media content during wartime. Content was divided into two groups: relevant (r) and non-relevant (N). Relevant content was divided into three subgroups: favorable (f), unfavorable (u), and neutral (n). Content was favorable if it expressed a positive direction and unfavorable if it expressed a negative direction; if neither, it was categorized as neutral. The formula to calculate the coefficient of imbalance is as follows:

$$C=\left\{\begin{array}{c}{\displaystyle \frac{f^2-f\ast u}{(f+u+n)(f+u+n+N)}},f>u\\ 0,f=u\\ {\displaystyle \frac{{f\ast u-u}^2}{(f+u+n)(f+u+n+N)}},f<u\end{array}\right.$$

(1)

This research is extremely valuable. The coefficient was defined based on ten criteria, the values of which ranged in the interval [−1, 1]. This coefficient ensured robust standardization across different sample sizes and domains. The coefficient limited outliers, overcoming the influence of anomalous sentiment expressions. However, in current practice, including non-relevant content in the calculation of the coefficient is seen as a flaw in the formula. It is now understood that irrelevant content is innumerable.

One of the first researchers to notice the value of the Janis–Fadner coefficient was Deephouse [10]. He did not criticize the coefficient but derived his own coefficient from theirs called the coefficient of media endorsement (CME), calculated as follows:

$$CME=\left\{\begin{array}{c}{\displaystyle \frac{e^2-e\ast c}{{(e+c)}^2}},e>c\\ 0,f=u\\ {\displaystyle \frac{{e\ast c-c}^2}{{(e+c)}^2}},e<c\end{array}\right.$$

(2)

Deephouse [10] used the terms “endorsing” and “challenging”, which correspond to the terms “favorable” and “unfavorable”, respectively. It should be emphasized that neither neutral nor non-relevant units were included. Then, Deephouse [2] presented a new version of the Janis–Fadner coefficient called the coefficient of media favorableness, which is calculated as follows:

$$CMF=\left\{\begin{array}{c}{\displaystyle \frac{f^2-f\ast u}{(f+u+n)(f+u+n)}},f>u\\ 0,f=u\\ {\displaystyle \frac{{f\ast u-u}^2}{(f+u+n)(f+u+n)}},f<u\end{array}\right.$$

(3)

The coefficient of media favorableness (CMF) reflects the true purpose of the Janis–Fadner coefficient and is appropriate for current research. Moreover, in Formula (3), Deephouse [2] used the same symbols as they were presented in the original research by Janis and Fadner [9]. The adaptation of the Janis–Fadner coefficient went almost unnoticed by the research community; although cited in several works [11,12,13], the version presented tended to be that of Deephouse [2]. For example, Li et al. [14] cited the Janis–Fadner coefficient but used an earlier version provided by Deephouse [10] in 1996. Thus, although the initial work by Janis and Fadner [9] is credited, the modifications proposed by Deephouse [2,10] are often overlooked.

Zhang [15] is one of the few researchers who noticed the contributions of Deephouse. Zhang considered not only the nonlinear coefficient presented by Janis and Fadner [9] and its adaptations provided by Deephouse [2,10] but also the linear function models presented by other scholars to measure media reputation. In total, seven measures were used to investigate a correlation between the existing measures of media reputation and their correlation with corporate reputation. The findings of Zhang are as follows: (1) all pairs of measures were positively correlated, (2) media reputation and corporate financial performance were significantly correlated, and (3) a more accurate measurement of media reputation would enable “a company to more accurately compare its media reputation with those of its competitors” (899p.).

Zhang [16] developed a new measure of media reputation, called the media reputation index, which is a composite measure of media favorability, media visibility, and recency. The formula is expressed as follows:

$$MRI=w\times MF,w=MV+R,MV=\sum _{i=1}^n\left(logL\times P\right)$$

(4)

where MF—media favorability (the Janis–Fadner coefficient and so on); R—recency; n—number articles; L—length of the article (number of words); and P—prominence to measure the position, where the name of the company appeared (headline, the first paragraph, other places). Formula (4) possesses no qualities inherent to the Janis–Fadner coefficient. While an experiment was conducted, many details were omitted, making it impossible to replicate or interpret the results. While paper [16] has several citations, they are all generic. Moreover, Zhang himself did not cite paper [16] in a related publication [17] even though he examined all known measures of media reputation. As such, the value of Formula (4) is uncertain.

Zhang and Ha [17] compared five measures of media reputation at the attribute level, but they did not include the Janis–Fadner coefficient since it does not a relevant item that could be measured in a news sample at the attribute level. Zhang and Ha [17] argued that the differences in the formulas of these measures and the lack of discussion about these reasons call for an empirical analysis to compare the measures. The authors concluded that the measures identify different values of media reputation; they cannot be substituted for one another. Therefore, to assess media reputation, a single consensus measure needs to be developed.

A second method in the domain of media content analysis is sentiment analysis, which assumes that investors rely on the news media to obtain information about the reputation of companies they are interested in. This method uses natural language processing techniques to automatically determine the emotional tone of news media coverage. Many surveys [5,18,19] on sentiment analysis have been conducted in recent years. However, none of these surveys reviewed articles on media sentiment analysis to assess the reputation of companies. Nevertheless, Misra et al. [20] presented a method that is close to this field: the sentiments of the news article were measured using the event sentiment score (ESS), which was obtained from the RavenPack News Analytics database [21]. The value of the ESS variable is based on an expert coding system. Despite the ESS, this method has several points that can be built upon: (1) the assessment of media sentiment is computed by averaging the ESS values of all news items published during the period considered, (2) the ESS values range from 0 to 100, and (3) news volume is measured as a separate variable. In addition, the average media sentiment is confirmed by other works [22,23]. Also, including the news volume in the analysis is significant, as it is necessary to distinguish one negative news item from multiple negative news articles [24,25].

This review of related work can be summarized as follows: The Janis–Fadner coefficient and its derivatives were used to define media reputation in content analysis. Zhang and Ha [17] concluded that the Janis–Fadner coefficient and its derivatives identify different values of media reputation, and they are not interchangeable. A single consensus measure is needed to assess media reputation. The event sentiment score (ESS) from the RavenPack News Analytics database has been used in sentiment analysis. The relevance of the ESS has been verified in various studies [20,25,26,27]; however, its application has been limited to companies listed in the RavenPack News Analytics database, and an independent measure of sentiment is needed to evaluate the media reputation of companies. Thus, our goal, based on the rich research base of our content analysis, is to develop a measure that can analyze sentiment.

3. Method Used to Measure Media Reputation

Firstly, we introduce a method to measure an article’s reputation index. This method is novel and part of the process used to measure the media reputation of companies.

3.1. Method Used to Measure an Article’s Reputation Index

The steps implemented in the well-known and largely applied Janis–Fadner coefficient for content analysis can be used to develop an article reputation index in sentiment analysis. Zhang [16] attempted to develop a new measure of media reputation. However, his attempt was unsuccessful, since it lost all features inherent to the Janis–Fadner coefficient. Nevertheless, the idea of using the length of an article to calculate the reputation index of an article can be borrowed, as it is important to know the volume of the context in which positive or negative sentiments are expressed. The volume of context is included in the Janis–Fadner coefficient; however, its value is masked by nonlinear calculation. We take Formula (3) as the basis for a new formula that calculates the media reputation index of an article. The goal of our derivation is two-fold: (1) to preserve the properties of the Janis–Fadner coefficient and (2) to explicitly consider the length of the article. We performed several experimental derivations of the formulas before reaching a successful outcome. We divided Formula (3) by the ratio f/(f + u + n), if f > n, which represents the number of favorable sentences compared to the total number of sentences. In the same manner, if f < n, we can divide Formula (3) by the ratio u/(f + u + n), which represents the number of unfavorable sentences compared to the total number of sentences. Thus, the article reputation index is presented as Formula (5):

$$ARI=\left\{\begin{array}{c}{\displaystyle \frac{f^2-f\ast u}{(f+u+n)(f+u+n)}}÷{\displaystyle \frac{f}{f+u+n}},f>u\\ 0,f=u\\ {\displaystyle \frac{{f\ast u-u}^2}{(f+u+n)(f+u+n)}}÷{\displaystyle \frac{u}{f+u+n}},f<u\end{array}\right.$$

(5)

After manipulating and simplifying Formula (5), we obtain Formula (6):

$$ARI=\left\{\begin{array}{c}{\displaystyle \frac{f^2-f\ast u}{\left(f+u+n\right)\ast f}},f>u\\ 0,f=u\\ {\displaystyle \frac{{f\ast u-u}^2}{\left(f+u+n\right)\ast u}},f<u\end{array}\right.$$

(6)

Next, we prove that Formula (6) retains the features of the Janis–Fadner coefficient with a value in the interval of [−1, 1]. In an extreme case, let us say that all sentences are positive. In this case, the numerator is f², and the denominator is f ∗ f. Thus, the result is one. A similar situation occurs if all sentences are negative. In this case, the numerator is −u² and the denominator is u ∗ u. Thus, the final result is −1. If an article contains sentences of different polarities and the number of positive sentences is greater than the number of negative sentences, the value of the numerator will decrease, since f² − f ∗ u; meanwhile, the value of the denominator will increase, since (f + u + n) ∗ f. The obtained value will be less than one. The same phenomenon occurs if the number of negative sentences is greater than the number of positive sentences. In this case, the obtained value will be greater than −1.

The values of the ESS variable, designed to measure the sentiment score of an event, range on a scale from 0 to 100. Such scaling has been confirmed in various studies [20,25,26,27] and was followed in this study. Therefore, we convert the obtained ARI value to a scale of 0 to 100 according to Formula (7):

$$ARI\%=ARI\ast 50+50$$

(7)

Next, we compare the values of the CMF, CME, and ARI to confirm the suitability of the proposed ARI for assessing an article’s reputation. The comparison was carried out using synthetic values. The scaled values used are shown in

Table 1. Comparison of CMF%, CME%, and ARI%.

No	Number of Positive Sentences	Number of Negative Sentences	Number of Neutral Sentences	CMF%	CME%	ARI%
1	1	0	0	100	100	100
2	2	1	0	61.11	61.11	66.67
3	2	1	1	56.25	61.11	62.5
4	3	1	0	68.75	68.75	75
5	3	1	1	62	68.75	70
6	1	1	0	50	50	50
7	0	1	0	0	0	0
8	1	2	0	38.89	38.89	33.33
9	1	2	1	43.75	38.89	37.5
10	1	3	0	31.25	31.25	25
11	1	3	1	38	31.25	30

.

Firstly, we must note that the article’s reputation index CME% does not take into account the number of neutral sentences. The article in the second row contains two positive sentences and one negative sentence, meaning that the values of the article’s reputation indices differ. Thus, the value of the article’s reputation index ARI% is 66.67. This proportion means that two-thirds of the article is positive. Meanwhile, the values of the other two indices are 61.11, which is lower than 66.67. The article in the tenth row contains one positive and three negative sentences, meaning that three-quarters of the article is negative. Only ARI% had a value of 25, which is consistent with the tone of the article. Similar reasonings can be applied to each row in Table 1.

Next,

Table 2. Assessment of real articles’ reputation.

No	Number of Positive Sentences	Number of Negative Sentences	Number of Neutral Sentences	CMF%	CME%	ARI%
1	10	2	19	54	78	63
2	11	6	10	54	60	59
3	30	10	11	62	69	70
4	21	16	26	51	54	54
5	16	33	31	46	38	39
6	27	17	39	52	57	56
7	16	2	10	64	85	75
8	6	3	2	57	61	64
9	9	3	8	57	69	64
10	13	2	10	61	82	72
11	5	1	3	62	78	72
12	7	2	10	55	72	63
13	14	1	8	67	90	78
14	8	1	4	67	85	77
15	16	1	10	66	92	78
16	10	2	19	54	78	63
Average	13.93	6.67	13.40	58.33	71.33	65.60

presents the reputation assessment of real articles dedicated to a real, large and well-known Lithuanian company. The articles were collected from different open access Lithuanian portals. We present the assessment of just a small proportion of these collected articles.

We can observe that the reputation indices of various articles differ as with synthetic data. For instance, the values given in the third row for positive, negative, and neutral sentences are ten times larger than those in the fifth row of Table 1. The values for the reputation indices are the same as in Table 2. In addition, we calculated the average values presented in Table 2 for each column separately. The average reputation index of articles should reflect the reputation value of the company being reported. However, the average value of our proposed reputation index ARI% is less moderate than the coefficient of media favorableness and less extreme than the coefficient of media endorsement. The moderate nature of the values in the CMF% index is explained by its nonlinear dependence on the length of the article. The extremeness of the values derived for the CME% index is due to the lack of consideration of neutral sentiments.

3.2. Description of the Method Used to Measure Media Reputation

During the review of related work, we noticed that several authors [20,25,26,27] used the ESS variable from RavenPack to measure event-related sentiments. When assessing sentiment across multiple events, the average of the ESS variables was calculated to obtain a single overall measure for the company. Since the ESS variable is only available for companies listed in the RavenPack dataset, we propose ARI%, introduced in Section 3.1, as an independent measure of article sentiment. We suggest using an averaging method, such as the application of ESS variables, to assess company reputation based on available articles. The averaging of media sentiment is supported by other studies [22,23]; however, this only yields a summary measurement. For instance, it is also important to know how many elements are included in the average, as it is necessary to distinguish a single negative news article from a high volume of negative news articles [24,25]. The second limitation of the average is that it completely obscures opposite assessments of the sentiment polarity. This effect can be evaluated by calculating the margin of error. Therefore, the media reputation of a company should be based on a combination of variables, including the average of the article’s reputation indices, the total number of articles, and the margin of error.

We approach the problem of assessing a company’s media reputation under the assumption that sentence-level sentiments within an article can be calculated. Our field of application focuses on articles published in open access online news media. The language used in these articles is regulated and less noisy compared to that in social networks, as it is supervised by editors-in-chief and language editors [28,29,30]. Therefore, the methods required to analyze sentiment in news articles on the sentence level are less computationally demanding compared to social media. Existing tools can be used to determine the polarity of sentences, such as Stanford CoreNLP [31], which has been employed by several authors [32,33]. However, the articles in our study are written in Lithuanian, which is not compatible with Stanford CoreNLP. To solve this problem, we utilized Google Translate API. Many researchers [34,35,36,37] expressed the concern that the translation process could alter sentiment polarity. However, experiments revealed that the performance rates of sentiment analysis on the original and translated datasets are comparable. The results obtained showed that translating the input text from a specific language to English and using compatible tools may be better than developing language-specific methods.

To diversify the assessment of sentiment polarity, we used an additional tool, the pre-trained transformer model XLM-RoBERTa, which is the latest development in the field of NLP [38]. XLM-RoBERTa is a large multi-language model trained on 100 languages, including Lithuanian [39], and has outperformed other transformer models on various NLP tasks under low-resource language settings [40].

We combined all components of the proposed method in Figure 1, which demonstrates the workflow of the proposed method for determining the media reputation of a specific company. The main steps include article collection, text preprocessing, sentence segmentation, sentence-level sentiment analysis using either CoreNLP with a translated text or a multilingual RoBERTa model, aggregation of sentiments (article level), computation of emotional indices (including ARI), and aggregation of company-level media reputation statistics.

Next, we will discuss the initial steps of the workflow in more detail, as they have not received sufficient attention. During text preprocessing, the first step is the removal of HTML tags. The next step involves segmenting the text into sentences, as all tools assign sentiments to sentences. Sentence segmentation was performed using a deterministic rule-based tokenizer, which temporarily masks dots in short 1–2 letter abbreviations, and a predefined list of words such as “pvz.”, “t.t.”, and “t.y.”. The masked dots are restored after segmentation. Empty segments are removed, and copyright footer phrases are filtered out.

4. Experiment and Discussion

To assess the media reputation of selected companies, we followed the steps defined for the method proposed in Section 3. The first step involves collecting company-related articles published in online Lithuanian media. We collected articles from major open access Lithuanian news portals published over the past two years. The articles were then grouped by company. Larger companies received significantly more media attention than smaller ones, ranging from 36 to 64 and from 6 to 7 articles, respectively. A significant difference in volume was observed across the dataset. Subsequently, we carried out the remaining steps of the proposed method. Company data were anonymized, and the results for the larger and smaller companies are presented in

Table 3. Media reputation of larger companies.

No	Sentiment Analysis Tool	Number of Positive Articles	Number of Negative Articles	Number of Neutral Articles	Total Number of Articles	CMF%	CME%	ARI%
L01	CoreNLP	7	53	4	64	43 ± 2	34 ± 4	38 ± 3
	XLM-RoBERTa	3	54	7		47 ± 1	21 ± 5	40 ± 2
L02	CoreNLP	10	29	2	41	46 ± 2	38 ± 6	42 ± 4
	XLM-RoBERTa	14	20	7		48 ± 1	46 ± 11	46 ± 3
L03	CoreNLP	29	20	7	56	50 ± 2	52 ± 6	51 ± 4
	XLM-RoBERTa	18	28	10		49 ± 1	49 ± 9	47 ± 3
L04	CoreNLP	18	30	5	53	48 ± 2	45 ± 6	46 ± 4
	XLM-RoBERTa	14	34	5		48 ± 1	35 ± 8	44 ± 3
L05	CoreNLP	8	40	3	51	43 ± 3	37 ± 6	38 ± 4
	XLM-RoBERTa	4	41	6		44 ± 2	24 ± 7	37 ± 3
L06	CoreNLP	20	30	5	55	47 ± 3	44 ± 6	45 ± 4
	XLM-RoBERTa	19	28	8		48 ± 1	44 ± 9	46 ± 3
L07	CoreNLP	3	41	5	49	42 ± 3	29 ± 5	36 ± 3
	XLM-RoBERTa	11	27	11		47 ± 1	41 ± 9	44 ± 3
L08	CoreNLP	13	36	4	53	47 ± 2	43 ± 6	44 ± 3
	XLM-RoBERTa	10	30	13		48 ± 1	37 ± 8	46 ± 2
L09	CoreNLP	6	31	6	43	47 ± 3	40 ± 5	42 ± 4
	XLM-RoBERTa	0	34	9		43 ± 2	23 ± 6	37 ± 3
L10	CoreNLP	3	28	5	36	45 ± 2	37 ± 5	40 ± 3
	XLM-RoBERTa	1	32	3		43 ± 2	22 ± 7	36 ± 3
L11	CoreNLP	14	31	1	46	48 ± 2	44 ± 5	46 ± 4
	XLM-RoBERTa	21	23	2		50 ± 2	49 ± 9	50 ± 4
L12	CoreNLP	33	16	4	53	52 ± 1	58 ± 6	55 ± 3
	XLM-RoBERTa	32	12	9		51 ± 1	65 ± 9	54 ± 3
L13	CoreNLP	16	33	2	61	47 ± 2	44 ± 7	45 ± 4
	XLM-RoBERTa	19	23	9		50 ± 1	48 ± 10	49 ± 3
L14	CoreNLP	25	22	10	57	51 ± 1	52 ± 7	53 ± 3
	XLM-RoBERTa	29	8	20		51 ± 1	69 ± 9	54 ± 2
L15	CoreNLP	24	12	5	41	52 ± 3	58 ± 7	54 ± 4
	XLM-RoBERTa	27	10	4		52 ± 2	63 ± 11	56 ± 4
L16	CoreNLP	17	28	1	46	47 ± 2	42 ± 4	44 ± 4
	XLM-RoBERTa	19	24	3		48 ± 2	44 ± 7	47 ± 4
L17	CoreNLP	3	43	8	54	46 ± 2	29 ± 6	40 ± 3
	XLM-RoBERTa	6	37	11		47 ± 1	30 ± 8	42 ± 3
L18	CoreNLP	22	21	8	51	51 ± 2	51 ± 7	51 ± 4
	XLM-RoBERTa	20	22	9		49 ± 2	45 ± 11	48 ± 4
L19	CoreNLP	25	18	10	53	50 ± 6	55 ± 19	51 ± 11
	XLM-RoBERTa	26	14	13		51 ± 6	63 ± 19	52 ± 11
L20	CoreNLP	10	29	15	54	48 ± 2	46 ± 7	46 ± 4
	XLM-RoBERTa	16	33	5		47 ± 2	42 ± 9	44 ± 4
L21	CoreNLP	21	26	5	54	48 ± 2	49 ± 6	47 ± 4
	XLM-RoBERTa	17	27	8		48 ± 2	46 ± 9	46 ± 4
L22	CoreNLP	22	29	6	57	47 ± 3	47 ± 6	46 ± 4
	XLM-RoBERTa	24	28	5		48 ± 1	47 ± 9	46 ± 3
L23	CoreNLP	22	26	6	54	49 ± 2	49 ± 6	49 ± 3
	XLM-RoBERTa	17	29	8		49 ± 1	40 ± 10	47 ± 3
Aver.	CoreNLP	16	29	6	51	48 ± 2	44 ± 6	46 ± 4
	XLM-RoBERTa	16	27	8		48 ± 2	43 ± 9	46 ± 4

and

Table 4. Media reputation of smaller companies.

No	Sentiment Analysis Tool	Number of Positive Articles	Number of Negative Articles	Number of Neutral Articles	Total Number of Articles	CMF%	CME%	ARI%
S01	CoreNLP	2	5	0	7	42 ± 6	30 ± 15	36 ± 9
	XLM-RoBERTa	0	6	1		47 ± 2	14 ± 15	39 ± 5
S02	CoreNLP	2	4	1	7	48 ± 3	52 ± 20	46 ± 8
	XLM-RoBERTa	2	4	1		49 ± 2	50 ± 26	48 ± 6
S03	CoreNLP	2	5	0	7	48 ± 9	36 ± 24	44 ± 14
	XLM-RoBERTa	1	5	1		46 ± 4	29 ± 25	41 ± 7
S04	CoreNLP	1	6	0	7	46 ± 4	42 ± 9	43 ± 8
	XLM-RoBERTa	3	3	1		49 ± 3	44 ± 19	49 ± 8
S05	CoreNLP	1	5	1	7	45 ± 5	37 ± 11	40 ± 9
	XLM-RoBERTa	0	5	2		46 ± 3	24 ± 14	40 ± 6
S06	CoreNLP	3	4	0	7	46 ± 5	46 ± 11	44 ± 9
	XLM-RoBERTa	2	5	0		48 ± 3	32 ± 26	44 ± 8
S07	CoreNLP	0	7	0	7	39 ± 4	30 ± 5	31 ± 4
	XLM-RoBERTa	0	7	0		46 ± 3	18 ± 12	38 ± 4
S08	CoreNLP	3	4	0	7	48 ± 6	46 ± 20	46 ± 13
	XLM-RoBERTa	1	5	1		47 ± 2	36 ± 13	43 ± 6
S09	CoreNLP	2	4	1	7	51 ± 3	47 ± 7	51 ± 5
	XLM-RoBERTa	3	2	2		51 ± 1	60 ± 20	52 ± 5
S10	CoreNLP	2	4	1	7	49 ± 1	43 ± 6	47 ± 4
	XLM-RoBERTa	3	2	2		51 ± 2	51 ± 19	52 ± 6
S11	CoreNLP	1	3	3	7	45 ± 5	35 ± 15	41 ± 9
	XLM-RoBERTa	1	4	2		49 ± 5	37 ± 15	46 ± 9
S12	CoreNLP	4	2	1	7	54 ± 9	66 ± 23	57 ± 13
	XLM-RoBERTa	4	1	2		48 ± 7	64 ± 24	50 ± 11
S13	CoreNLP	2	3	2	7	47 ± 5	44 ± 15	45 ± 9
	XLM-RoBERTa	2	5	0		49 ± 1	27 ± 20	46 ± 4
S14	CoreNLP	2	2	2	6	50 ± 2	48 ± 19	49 ± 7
	XLM-RoBERTa	3	1	2		50 ± 1	67 ± 30	51 ± 6
S15	CoreNLP	1	6	0	7	45 ± 5	38 ± 12	40 ± 10
	XLM-RoBERTa	2	4	1		50 ± 10	41 ± 25	47 ± 16
S16	CoreNLP	3	2	2	7	48 ± 4	46 ± 16	48 ± 9
	XLM-RoBERTa	1	5	1		43 ± 7	21 ± 27	37 ± 12
S17	CoreNLP	1	6	0	7	43 ± 7	24 ± 18	37 ± 11
	XLM-RoBERTa	3	3	1		48 ± 6	49 ± 29	48 ± 10
S18	CoreNLP	5	1	1	7	55 ± 4	66 ± 13	60 ± 7
	XLM-RoBERTa	4	2	1		52 ± 2	67 ± 26	55 ± 6
S19	CoreNLP	3	4	0	7	50 ± 5	58 ± 21	49 ± 10
	XLM-RoBERTa	2	4	1		48 ± 2	35 ± 26	45 ± 6
S20	CoreNLP	0	6	1	7	44 ± 5	28 ± 12	37 ± 7
	XLM-RoBERTa	2	4	1		46 ± 5	38 ± 12	43 ± 8
S21	CoreNLP	3	3	1	7	50 ± 2	51 ± 9	51 ± 6
	XLM-RoBERTa	2	3	2		48 ± 2	48 ± 28	46 ± 6
Aver.	CoreNLP	2	4	1	7	47 ± 5	43 ± 14	45 ± 9
	XLM-RoBERTa	2	4	1		48 ± 3	41 ± 21	46 ± 7

, respectively.

To confirm the suitability of the proposed article reputation index (ARI), we compared its values with those of the media favorability coefficient (CMF) and the media endorsement coefficient (CME). The values of the CMF, CME, and ARI indices were converted to a scale of 0 to 100 using Formula (7); consequently, these values are denoted as CMF%, CME%, and ARI%. To account for variability in the results, a 95% confidence level [41,42] was chosen, and the corresponding confidence intervals were calculated. The sample size of larger companies (Table 3) was sufficient, i.e., n ≥ 30; therefore, the distribution was assumed to be normal [41]. The values of all indexes and margins of error were rounded to the closest integer value. The sample size of smaller companies (Table 4) was limited, so the distribution of values was checked and observed to be close to normal. We made a cautious assumption that the same formula could be used to calculate the confidence interval.

Let us now discuss the results presented in Table 3. Firstly, from the results summarized in the right-hand column of Table 3, it is surprising to note that the average values of indices CMF% and ARI% do not depend on the sentiment assessment tool. Instead, they are the same for both tools. A small difference is observed only in the CME% index. The average number of positive articles is the same for both tools. XLM-RoBERTa is more moderate than Stanford’s CoreNLP, as the average number of negative articles (two) neared the average number of neutral articles.

For the results of individual companies, there is no correlation between the results of all three indices for each company and the values only correlate individually with the indices. The values of the CMF% index are the same for the following companies: L04, L14, L15, L21, and L23; the values of the CME% index are the same for two companies: L06 and L22; and the values of the ARI% index are the same only for company L22.

The distribution of ARI% index values across all companies and both sentiment analysis tools is shown in Figure 2. The largest differences are observed for the following companies: L03, L07, L09, and L10. For company L03, the number of positive articles decreased, while the number of negative articles increased in the case of XLM-RoBERTa. The opposite effect is observed for company L07 compared to company L03. The number of positive articles for company L09 became 0 in the case of XLM-RoBERTa. Three of these articles were negative, and the other three were neutral. The number of positive articles and neutral articles for company L10 decreased in the case of XLM-RoBERTa. Evidently, in the case of XLM-RoBERTa, changing the sentiment polarity assessment resulted in different values.

To observe the general tendencies of the indices considered, we conclude that the CMF% index is the most moderate. The values of this index vary the least between the two sentiment analysis tools. In addition, the CMF% index has the smallest margin of error. The range of values for the CME% index is the largest one, with the highest margin of error as well. The increase in the range of values is particularly noticeable for the sentiment analysis tool XLM-RoBERTa. A significant change in sentiment polarity is observed for company L14 between the two sentiment analysis tools. However, this radical change has little effect on the other two indices. The value of our proposed ARI% index increased by one in the case of XLM-RoBERTa compared to CoreNLP, as the number of positive articles increased by four. The CMF% index did not react to this increase in the number of positive articles. We conclude that this index is sensitive to changes in sentiment polarity. The ARI% index responds better to changes in sentiment polarity than the CMF% index, as shown by the increase in positive articles to four.

The values of all three indices across all companies are visualized in Figure 3 and Figure 4, which show that the trends of the indices remain the same for both sentiment analysis tools.

The margins of error for the ARI% index lie between those of the CMF% and CME% indices. This conclusion is valid for both sentiment analysis tools. Therefore, a diagram combining media reputation values and margins of error is shown for a single sentiment analysis tool in Figure 5. All margins of error are moderate, except for company L19. We examined the articles related to this company and found that three were very negative and one was very positive, resulting in a large margin of error for this company. Moreover, we observed that all three indices have the same margin of error for both sentiment analysis tools. This led us to conclude that a strongly positive or strongly negative article is assessed in the same manner by all tools.

Let us now examine the average values presented in the right-hand column of Table 4. The average values of the number of articles with different polarities are the same for both sentiment analysis tools. However, the average values of different indices differ slightly. The average values of the CMF% and ARI% indices increase, while the average value of the CME% index decreases in the case of XLM-RoBERTa compared to CoreNLP.

For the results of individual companies, there is no complete correspondence between the three indices for each company, as in the case of larger companies. The only correspondence between values exists for separate indices. Two companies have the same CMF% index value: S09 and S14. Moreover, while no companies share the same CMF% index value, two other companies share the same ARI% index value: S05 and S06.

If we compare the average media reputation values for companies in Table 3 and Table 4, the values of the CMF% and ARI% indices are present in the case of XLM-RoBERTa. This suggests that even a small number of articles can indicate trends in the media reputation of companies.

However, we observed that the number of negative articles overwhelmed the number of positive articles. In Table 3, the number of positive articles is greater than the number of negative articles for only six out of twenty-three companies, and in Table 4, the number of positive articles is greater than the number of negative articles for three companies out of twenty-one companies, and equal for two companies. The dominance of negative news can result from the deep-rooted negativity that is naturally present in all humans due to the evolutionary processes [43]. Researchers theorize that people have developed a defense mechanism in response to negative information and actively seek out threats [44]. Therefore, if information indicates a potential threat to human wellbeing, it should be processed to avoid risks and potential threats. It can be concluded that media providing negative information helps people avoid threats and form a more comfortable lifestyle.

A significant difference is observed when comparing the media reputation of larger and smaller companies. The margin of error is greater for smaller companies than for larger companies. This is influenced by the significantly limited number of articles available for smaller companies.

Although this study was conducted using the Lithuanian language only, we used two completely different tools to assess sentiment polarity. The results obtained show uniform behavior for the proposed method across both sentiment analysis tools. Moreover, we employed two different datasets of companies, and this method achieved the same results and revealed similar trends for the same company across different databases using both sentiment analysis tools. Therefore, we conclude that the proposed method will have similar outcomes for different languages.

5. Conclusions

The media rankings of companies are less demanding and offer many advantages over other measures of reputation. Researchers can develop a sentiment classification model that fits the conceptual definition of reputation. Using fully automated natural language processing techniques, such a method is less time-consuming, more cost-effective, and more easily applicable than other methods for assessing reputation. Methodological advances enable text-based measurements of reputation that are reliable, valid, and representative of the company, provided that the data collection process involves a wide range of online resources.

We developed a method to measure a company’s reputation based on sentiments detected in online articles. The sentiment of each sentence was evaluated and categorized into one of three polarities: positive, negative, or neutral. Based on this approach, we also developed a fully automated method for assessing a company’s media reputation using all available online articles. Experiments were conducted with articles written in Lithuanian and sourced from major news portals.

To validate the proposed method in different ways, two sentiment analysis tools were employed: Stanford CoreNLP combined with Google API and the pre-trained transformer XLM-RoBERTa. The average results of the proposed ARI% index for larger companies were similar for both sentiment analysis tools. However, when comparing specific companies, such correspondence only existed in 1 out of 23 three companies. Larger differences were observed in 4 out of 23 companies. Thus, the use of different sentiment analysis tools had a moderate impact on assessing a company’s media reputation.

We also conducted two experiments focusing on larger and smaller companies, yielding similar results for both, aside from the margin of error. Smaller companies had a greater margin of error because the number of articles available was almost ten times less than that of larger companies. It was also observed that negative articles prevailed for many companies. This result can be explained by human interest in negative news, as humans are evolutionarily primed to be aware of threats to avoid. The obtained results were compared with those of existing methods, specifically the coefficients of media endorsement and media favorableness, the results of the proposed method are less moderate than those of the coefficient of media favorableness and less extreme than those of the coefficient of media endorsement.