The Impact of Shanghai Epidemic, China, 2022 on Public Psychology: A Sentiment Analysis of Microblog Users by Data Mining

Abstract

The Shanghai epidemic (COVID-19) is another large-scale epidemic in China’s central cities in the post-epidemic era from the end of March 2022. It is of great significance to study the public’s attitude towards the Shanghai epidemic to support healthy psychology and a positive attitude. This study used crawler technology to obtain the Weibo data related to the epidemic situation in Shanghai that was published by users. The crawled microblogs were preprocessed, and the BosonNLP sentiment dictionary for attitude classification was selected. The Chinese vocabulary ontology for 21 emotion classifications was also used. The results showed that the general attitude of the public in Shanghai was positive. This fluctuated greatly in the initial stage and gradually increased in the later stage. Through text mining, it is clear that goods and materials, nucleic acid virus testing, and other aspects of the epidemic in Shanghai are concerning for the public. The public attitude in areas that are close to the epidemic center is relatively more negative. The study can provide references for policymakers to fight COVID-19 by improving public attitude and solving urgent matters.

1. Introduction

Since the outbreak of COVID-19 in 2020, the public’s lifestyle, working status, and other aspects of life have been widely affected and altered. COVID-19 has caused widespread public mental health problems all over the world, and various countries have conducted research on mental health in the epidemic, such as the UK [1], Egypt [2], and Korea [3]. The mental health and psychological status of the public after the outbreak are often more negative than before, especially in Wuhan [4]. The psychological status of the staff who were directly involved in the epidemic prevention and control has been greatly affected. Their depression and anxiety are worrying, and these are affected by many factors [5]. Moreover, the risk of their simultaneous occurrence is higher than before the COVID-19 period [6]. This situation is common during the COVID-19 pandemic [7]. Nurses are more anxious than other medical staff [8]. In addition to the medical staff with a high work risk, the mental health status of the public should also be given attention. At present, the Chinese population’s public health and mental health still face some challenges [9]. Young people are a group prone to psychological problems [10]. Work and employment issues can also affect the mental health of the public [11]. At the same time, the regional closure caused by the epidemic is also one of the important factors leading to public anxiety and depression [12]. Children and adolescents often lack companions in close situations, which is not conducive to their physical and mental health [13]. College students who have family members or relatives infected with COVID-19 are relatively more anxious and depressed [14]. College students should improve their ability to cope with the pressure caused by the restrictions of the epidemic [15]. In the situation of the current normalization of epidemic prevention and control, more attention should be paid to the emotional and psychological conditions of the public.

In the Internet era, in which information is widely disseminated in real-time, public concern is high regarding COVID-19, and web media accounts for a significant part of its information sources. The network medium in the outbreak of COVID-19 not only has the function of transmitting important information, but it is also an expression platform for the public’s view. Weibo is characterized by its high openness of content, a large user base, and strong subjectivity of expression. Social media platforms such as Weibo provide a channel for spreading epidemic prevention information. Therefore, they affect public attitudes and behaviors [16]. The government should make good use of social media to improve public awareness of COVID-19. Since the outbreak of COVID-19, people’s negative emotions have increased, and their life satisfaction has decreased [17]. COVID-19 also has a dynamic effect on suicide probability [18]. There are many research methods for determining the public’s psychological state on Weibo, such as machine learning [19], qualitative content analysis [20], and so on. Compared with Twitter, another important social platform, Twitter users tend to pay more attention to the economic situation, while Weibo users pay more attention to public health [21]. Weibo is an important medium to determine the public’s emotional changes and subjective attitudes during the epidemic.

Around 13 April 2022, major media reported a large-scale epidemic in Shanghai. In the post-epidemic era, COVID-19 in China’s important economic center city broke out again on a large scale. Affected by many factors, such as the strong transmission ability of the Omicron variant [22], the number of infected people exceeded 10,000 per day at the initial stage. The epidemic situation in Shanghai has its particularity and important research value in public health. The public pays high attention to the epidemic situation in Shanghai. They often obtain relevant news about the epidemic situation in Shanghai through Internet media. At the same time, they discussed relevant topics on the Internet, and there were many topics about the Shanghai epidemic on Weibo.

Sentiment analysis belongs to the category of natural language processing. Many studies in various countries have discussed the public state during the COVID-19 epidemic through sentiment analysis, including Nepal [23], Spain [24], and others. Such research can help to eliminate public panic and deal with public crises. A large-scale benchmark Twitter data set for COVID-19 sentiment analysis was established for research use [25]. Dictionary methods and machine learning methods are often used for sentiment analysis. Among them, the dictionary method is generally based on the already built sentiment dictionary design algorithm, and the result that is obtained depends on the dictionary. Therefore, when dealing with special field problems, it is necessary to build a professional field dictionary. Sentiment dictionaries in different languages have been designed and applied, such as the AGUST [26] dictionary in German, Affected Br [27] in Spanish, SentiWordNet [28] in English, etc. The sentiment dictionary approach has already contributed to online education during COVID-19 [29]. Machine learning methods need to convert text into word vectors or other forms, and then classify it by supporting a vector machine or neural network model. The disadvantage is that a large amount of data must be manually labelled. The process of data labeling can be solved by semi-supervised learning, requiring only a small amount of data labeling, but the labeling algorithm needs to have a high accuracy rate. According to the epidemic situation of the Twitter platform, sentiment analysis provided a visual analysis and suggestions for channeling public sentiment [30]. At the same time, a support vector machine method based on fuzzy rules is proposed, which can provide help for filtering the tweet forwarding path to reduce the spread of false messages [30]. Moreover, with the advent of the era of deep learning big models, it is also a good choice to choose a preprocessing framework such as palm [31] to process text, but there are few applications for such models at present.

This study crawled the microblog on Weibo related to COVID-19 in Shanghai. Then, text mining and sentiment analysis was used to understand the overall attitude and change process of the public towards the Shanghai epidemic. The purpose of this study is to understand the public’s concerns about the epidemic situation in Shanghai and to analyze the attitudes of people with different feature labels towards the Shanghai epidemic. It is of great reference significance to discuss the public’s attitude and the psychological state toward large-scale epidemics in the post-epidemic era. The results and discussions obtained can help to positively guide public health psychology.

2. Materials and Methods

2.1. Research Method

The research methods for affective tendencies and psychological analysis mainly include the questionnaire method, case analysis method, expert interview method, and so on. In this paper, sentiment analysis methods based on big data and sentiment dictionaries are applied, compared with other research methods, which have the advantages of high work efficiency and a large data volume. It is also more intuitive for microblog text to be directly expressed as public emotions. The overall experimental process is shown in Figure 1. The whole process of data mining includes data filtering, data preprocessing, design modeling, classification, and feature analysis. Social media platforms have massive amounts of textual data that generally contain user attitudes towards various topics. The big data method can analyze the attitudes and concerns of user groups with different characteristics, which can make a variety of disordered data orderly. First, we crawled the text of users who were concerned about the epidemic in Shanghai and obtained relevant content by searching keywords. Then, these data were cleaned, which belongs to the data preprocessing in data mining. The cleaned data were matched with the appropriate dictionary so that the text content could be quantified. Finally, a sentiment analysis was carried out by combining the user’s feature tag and publishing time. Sentiment analysis was the main method of this data mining.

2.2. Materials Acquisition

The COVID-19 epidemic broke out in Shanghai in mid-March. The news broke in late March. By the end of April, the epidemic was stably controlled, and the heat of relevant news gradually disappeared. Therefore, we chose to crawl the relevant comments from 24 March 2022 to 26 April 2022. During this period, a large amount of effective data could be obtained; we could also understand the psychological changes of the public and determine the key points in the fight against the epidemic. Considering that many official accounts with large numbers of fans have the characteristics of high posting frequency and high repetition of content, most of them are objective narratives. Posts on accounts with more than 10,000 fans were not crawled. This study used crawler technology to crawl 152,774 microblogs with the keyword “Shanghai epidemic” through Weibo’s advanced search interface. These microblogs mainly included text content, user name, gender, age, IP address, and sending time.

2.3. Text Preprocessing

The work of text preprocessing mainly includes two parts: overall data processing and text data processing. The overall data processing includes removing missing data (text) and removing duplicate data; 130,757 valid data were obtained after processing. Text data processing is shown in

Table 1. Text type and processing reason.

Type	Examples	Reason for Removal
Forwarding user name	//@username:	This is not a valid text.
Special symbols	%……	This is not a valid text.
Expression	[Happiness]	Expressions often have polysemy in different situations, which constitutes interference.
English case	Aaaaaaaa	English interferes with Chinese dictionary matching
Number	123444444	Interfere with the dictionary
Topic content	#Epidemic situation in Shanghai#	Topic as the title should not be used as subjective expression content of users
Website	https//:www.baidu.com (accessed on 20 May 2022)	Interfere with the dictionary

.

3. Experimental Process

3.1. Attitude Classification

In this paper, the BosonNLP sentiment dictionary was selected for score calculation. BosonNLP sentiment dictionary is a labeled dictionary launched by Posen, a natural language processing company. It applies to the language processing of social media such as Weibo and Baidu Post Bar. Compared with other dictionaries such as SnowNLP [32] and HowNet [33], BosonNLP [34] has the advantage that the data of this dictionary are from social media, which is more convenient for us to process the data that are crawled from Weibo.

For the sentiment analysis, first, the processed text corpus is word segmented. The word segmentation tool selected here is Jieba word segmentation. After that, the stop words list, degree adverb list, and negative words list are constructed. The functions of the three lists are shown in

Table 2. The role of vocabulary in attitude classification.

Word Name	Examples	Function
Stop words	Not only ... But ...	Stop words do not participate in the calculation to improve efficiency
Degree Adverb	Very, A little	Degree adverbs participate in the calculation and multiply with the weight of sentiment words
Negative words	No, Don’t	The negative value is involved in the calculation, and the weight value of sentiment words is reversed

.

At the same time, the emotion score of special words in the sentiment dictionary is removed, such as “Shanghai” and “epidemic situation”. The calculation rule of emotion score is: find a sentiment word existing in the dictionary, judge whether it is a negative word or degree adverb before the word, if so, multiply the values, and add the values of each sentiment word calculated in a text as the emotion score of the text corpus (a microblog). After scoring the text corpus, the emotion expressed by the corpus is divided into three categories: positive, negative, and neutral: a score >1 is positive, a score <1 is negative, and a score in [−1, 1] is neutral.

3.2. Emotion Classification

Using the Chinese vocabulary ontology, the emotion of the text corpus is further divided. According to the construction of emotional vocabulary noumenon [35], emotions can be divided into seven categories and 21 sub-categories. These seven categories are happiness, goodness, anger, sadness, fear, evil, and surprise, of which each category has subordinate sub-categories, which provide a standard for our emotional classification, as shown in

Table 3. Emotion classification.

Emotion Category	Subclass
Happiness	Happiness (PA)
	Peace (PE)
Goodness	Respect (PD)
	Admire (PH)
	Trust (PG)
	Love (PB)
	Wish (PK)
Anger	Anger (NA)
Sadness	Sadness (NB)
	Disappointment (NJ)
	Yearn (PF)
	Guilt (NH)
Fear	Flustered (NI)
	Fear (NC)
	Shame (NG)
Evil	Boredom (NE)
	Abhor (ND)
	Disparage (NN)
	Jealousy (NK)
	Doubt (NL)
Surprise	Surprise (PC)

. Each sentiment word has the characteristics of polarity, intensity, part of speech, category, and so on. According to the intensity, it is divided into five intensities: 1, 3, 5, 7, and 9. The overall intensity of 21 emotions were calculated.

It is not accurate to classify each text as a unique emotion category. Based on a large amount of data in the experiment, the method of studying emotion categories as a whole was adopted.

4. Data and Result Analysis

4.1. Overall Evaluation

After scoring 131,467 data, the results showed that there were 59,401 positive data, accounting for 45.18%, and 42,142 negative data, accounting for 32.55%. There are 29,924 neutral attitude corpora, accounting for 22.76%, as shown in Figure 2. It can be seen that, although the proportion of positive corpus is the highest, the proportion of negative corpus is still very high, close to one third. This shows that it is highly necessary to determine the psychological state of the public. The total average emotion score is 2.11. The overall sentiment of the public from 24 March 2022 to 26 April 2022, is positive. Differences in the amount of text should be considered when evaluating data within a specific range. The average emotion score (total emotion score of all corpora/number of corpora) within the eigenvalue range is used as the emotion score label, and the following emotion scores refer to the average emotion score.

The number of data with gender labels is 130,924. Among them, the proportion of male users is relatively less, and the performance attitude is more positive, as shown in

Table 4. Gender emotion score.

Gender	Total Score	Number of Persons	Average Emotion Score
Female	74,130.44	76,418	0.97
Male	204,739.00	54,506	3.76
Total	278,869.44	130,924	2.13

. In China, women usually not only have to work, but also need to take on more household chores. The inconvenience of not being able to purchase family necessities because of closures caused by the epidemic may be the reason for women’s more negative performance.

The independent sample t-test was used to study whether there were significant differences in gender attitudes towards the epidemic situation in Shanghai. The results are shown in

Table 5. Post-test of scores for different genders.

Variable Name	Variable Value		SD	T	P	MVD	Cohen’s D Value
score	Male	18,298	11.582	17.809	0.000 ***	1.963	0.180
	Female	21,155	10.305
	Total	39,453	10.959

Note: *** represents the significance level of 1% respectively; MD stands for median; SD stands for standard deviation; MVD represents the median difference.

. The mean values of males and females on the score are 4.119/2.156, respectively. The p-value of the F test is 0.000 **** < 0.05, so the statistical results are significant, indicating that there are significant differences in the emotional scores between different groups. Cohen’s D value of the difference amplitude is 0.18, and the difference amplitude is very small (0.20, 0.50 and 0.80, respectively, corresponding to the small, medium and large critical points). There are seven emotion categories in the overall data: good 48%, evil 27%, happiness 9%, sadness 8%, fear 6%, surprise 1%, and anger 1%. Among the positive emotions, admiration and happiness account for more, accounting for 31% and 8%, respectively, while negative emotions account for 9% of devaluation, 8% of disappointment, and 6% of sadness, as shown in Figure 3.

4.2. Statistical Analysis of Text

By calculating the word frequency of the result after word segmentation, we can obtain the top 20 words that appear frequently. The word cloud is generated according to these words. The word cloud can intuitively display the contents that the public pays more attention to in the epidemic situation in Shanghai, as shown in Figure 4. The word frequency table is shown in

Table 6. Word frequency table (top 20).

Word	Word Frequency	Word	Word Frequency
Nucleic acid	24,576	Personnel	11,855
Prevention and control	23,364	Novel coronavirus	10,947
Good	17,093	No	10,844
Quarantine	15,877	Asymptomatic	10,636
Residential quarters	14,603	Anti-epidemic	10,618
Testing	14,536	Illness	10,446
Hope	14,490	Life	10,020
Material	13,286	City	9878
Work	13,104	Think	9791
Infected individual	11,943	China	8143

. Through the word frequency table and word cloud diagram display, we can see that nucleic acid-based virus detection is the most popular concern in the epidemic, and material problems are also more concerning. There are problems in material coordination in the early stages of the epidemic, and the government should learn from this experience.

Select the words with the top 20 word frequencies to construct the keyword co-occurrence matrix. The co-occurrence rule is that the distance between keywords is no more than 1. The more co-occurrence times, the closer the relationship between keywords. When showing the total number of co-occurrence times, the visualization effect is poor. Only show the co-occurrence relationship to build the co-occurrence semantic network, as shown in Figure 5.

Use the TF-IDF algorithm to calculate keyword contributions, based on the Jieba analysis. The tags () method is implemented, as shown in Figure 6. As can be seen from Figure 6, the theme of the document revolves around the Shanghai epidemic, and words such as COVID-19, community, and nucleic acid also have a high degree of contribution.

It can be seen that citizens pay more attention to materials, nucleic acid virus testing, etc., in the anti-epidemic period, as well as personal life and work, and actively share helpful information through social platforms.

4.3. Emotional Trends

Considering the different numbers of microblogs per day, the daily average emotion score is used as the trend chart. The trend of emotion score can be divided into three periods: fluctuation period (FP), stabilization period (SP), and rising period (RP), as shown in Figure 7. It can be seen that the emotion scores fluctuated greatly from 23 March to 29 March, indicating that the public mood easily and greatly fluctuated in the initial stage of the epidemic. On 25 March, the only score below 0 appeared. From 30 March to 8 April, the emotion score fluctuated slightly compared with the previous stage, indicating that public emotion may be gradually stabilized due to the influence of the government and all sectors of society in the fight against the epidemic. From 10 April to 26 April, the overall emotion score showed an upward trend, indicating that the public attitude towards the fight against the epidemic was more positive.

Two independent samples of the Mann–Whitney U test were adopted for the emotion score groups over three different periods. The results are shown in

Table 7. Post-test of different attitudes in three periods.

Two Independent Samples		MD			MVD	Cohen’s D
Group A	Group B	Group A	Group B
Score_FP	Score_SP	1.922	1.705	1.625	0.218	0.145
Score_FP	Score_RP	1.922	2.625	0.540	0.703	0.7
Score_SP	Score_RP	1.705	2.625	0.025 **	0.92	1.01

Note: ** represents the significance level of 5% respectively; MD stands for median; MVD represents the median difference.

. The median scores of SP (stabilization period) and RP (rise period) were 1.705/2.625, respectively; The p-value of the test result is 0.025 ** < 0.05, so the statistical results are significant, indicating that there is a significant difference between SP and RP in scores. The difference range of Cohen’s F value is 1.01, and the difference is a very large range. This may be related to the time division between the stabilization period (SP) and the fluctuation period (FP) because the stabilization period still has large numerical fluctuations.

Take 25 March, the day with the lowest emotion score, as an example, and divide the 24 h of that day into four stages; each time stage is 3 h, and the emotion score trend of that day is shown in Figure 8. Based on the crawled text content and the Weibo hot spots of the day, the public raised great doubts about the grid management that was implemented at that time. Similarly, the incident of nurses in Shanghai Oriental Hospital, which appears many times in the text, is also one of the reasons for the low score of public emotion. From 9:00 to 12:00, Shanghai usually holds a press conference on the epidemic situation on that day. At the same time, the relevant topics are hot. The reason may be that the anti-epidemic work at the initial stage of the epidemic did not meet the public’s expectations.

4.4. Province Emotion Analysis

Due to the low mobility of citizens during the epidemic, the emotional score was calculated by province, according to the user’s IP address. The four provinces with smaller sample sizes (less than 200) were excluded. These four provinces were Macau, Tibet, Qinghai, and Taiwan. The scores are shown in Figure 9. The lowest score was 0.9 in Shanghai, the center of the epidemic. Jilin’s highest emotion score was 3.4. It can be seen that the provinces with scores in [2.0, 3.4] are far away from Shanghai, except for Zhejiang. As shown in Figure 10, the closer the geographical location to Shanghai, the more negative the overall attitude.

4.5. Significance of Results

The changing trend of public sentiment with time in the epidemic shows the characteristics of violent fluctuations in the early stage, stability in the middle stage, and rise in the late stage. The results are mutually verified with some existing research conclusions [36,37]. In the early stages of an outbreak, public sentiment tends to be volatile for two reasons: first, various problems are exposed in the early stage of the epidemic, although all regions have anti-epidemic experience. On the one hand, the normal living conditions of the public have been altered, and the inconvenience caused by the restrictions have led the public to behave more negatively. The authorities should pay attention to the deficiencies in the initial work of the epidemic, give feedback to the public promptly, use the media to publicize and guide the public, and pay attention to combating false information to increase the public’s trust.

5. Conclusions

The outbreak in Shanghai took place in about one month. The emotion score changed and could be divided into three stages: the fluctuation period, the stabilization period, and the rise period. During the fluctuation period, the emotion score fluctuated greatly, and the emotion score on 25 March was the lowest, showing a negative attitude. The stabilization period continued to fluctuate, and the fluctuation range gradually decreased. During the rising period, public emotion as a whole showed an upward trend. The authorities should pay attention to the stability maintenance work in the early stage of the epidemic to avoid public resistance. Citizens pay more attention to the working conditions in the fight against epidemic diseases, including nucleic acid virus testing, materials, isolation, etc., as well as personal life. At the same time, they actively use Weibo for help. The government should focus on the content of public online help, recognize the common problems, and solve them, such as the redundancy crisis caused by the restrictions. Male Weibo users are relatively fewer and have a more positive attitude. The epidemic ushered in an economic decline, which has generally affected vulnerable groups of women at work. Authorities should consider how to protect women’s rights at work. Among the provinces, Shanghai, which is at the center of the epidemic, has the lowest emotion score of 0.9, which is still in the positive range. The provinces closer to Shanghai were more negatively affected by the epidemic than those farther away. This shows that the areas in the epidemic center should strengthen the anti-epidemic publicity and guidance of the public, and the areas close to the epidemic center should strengthen the work to change the public’s aversion the epidemic center.