Mining Textual and Imagery Instagram Data during the COVID-19 Pandemic
Abstract
:1. Introduction
1.1. Instagram
1.2. COVID-19 and Social Media
1.3. Similar Studies
1.4. Aims of the Study
- Identify the key stakeholders in COVID-19 vaccine research and investigate the content of their Instagram posting, as well as how this is perceived by users;
- Detect any similarities/differences between the respective companies posting on both textual and visual features;
- Detect any similarities/differences between the respective users’ perception, by means of hashtags;
- Perform user posts’ intent classification, to explore a potential predictive modelling application for detecting what users desire to post;
- Perform user posts’ sentiment analysis, to quantify their feelings and opinions.
2. Materials and Methods
- Tozinameran from US-German cooperation Pfizer–BioNTech;
- BBIBP-CorV by Chinese Sinopharm;
- CoronaVac by Chinese Sinovac;
- Ad5-nCoV by Chinese CanSino Biologics;
- mRNA-1273 by US Moderna and its partner Johnson and Johnson;
- Gam-COVID-Vac by Russian Gamaleya Research Institute.
- DateTime, the date and time of post in UTC standard;
- PostText, the text body of the post;
- PostChars, the number of post characters;
- PostWords, the number of post words;
- HashTags, contained in post;
- Likes, the number of likes scored;
- Comments, the number of comments made;
- Images, the number of uploaded images;
- Videos, the number of uploaded videos, if any;
- VGG16, image classification output from pretrained VGG16 model, as a list in case of more images;
- InceptionV3, image classification output from pretrained InceptionV3 model, as a list in case of more images;
- ResNet50, image classification output from pretrained ResNet50 model, as a list in case of more images.
- Pretrained models used directly as classifiers in an application to classify new images;
- Pretrained models used as feature extractors, with features subsequently be used as input to another model;
- Pretrained models used for better weight initialization of the new integrated model.
- VGG16 [53] from Oxford Visual Geometry Group, where 16 refers to the number of layers, with VGG19 also available. Innovative for introducing consistent and repeating structural blocks
- InceptionV3 [54] where inception modules, blocks of parallel convolutional layers with different sized filters are introduced
- ResNet50 [55] where residual modules are introduced. These employ unweighted, shortcut connections that memorize, e.g., input to later layers in the network architecture
- Convert the color image to a Numpy array
- Extract the three (or four in case of .png) color channels and reshape as a single one-dimensional array
- Depending on model, scale pixel RGB intensities into either [0,1] (torch framework mode), [−1,+1] (tensorflow framework mode) or zero-center BGR intensities unscaled (caffe framework mode). These are internal details of the preprocess_input function, implemented differently for each model
- Use the model to make predictions (probabilities) for all classes
- Choose the highest probability as the most likely predicted result
3. Results
- ‘Acknowledge’ (ACK), for generic statements, reporting facts and sharing experience
- ‘Advise’ (ADV), for suggestions, recommendations, giving guidelines or offering help
- ‘Seek’ (SEK), for seeking help, advice, comments, or answers
- ‘Express’ (EXP), for any kind of expression, feeling, or thought, positive or negative (hybrid intent-sentiment)
- Substitute any other, possibly remaining, words containing language characters, accents, etc., with their closest ASCII equivalent, as user posts can be very noisy (e.g., changing the Greek word ‘ελληνικά’ to ‘ellenika’);
- Remove URLs in posts, as they are also frequently used, using regular expressions;
- Tokenize text and remove punctuation, using NLTK regular expressions tokenizer;
- Convert all tokens to lower case, using python’s string method;
- Normalize text, using NLTK lemmatization for verbs, nouns and adjectives;
- Remove tokens than contain non-alphabetic characters, e.g., numbers, using python’s string method;
- Remove English ‘stop words’, words of less importance that appear quite frequently in natural speech, using NLTK;
- Remove any remaining non-English, or English un-normalized words (e.g., ‘amigo’, ‘yeaah’, ‘lol’) that may have survived in post, using NTLK corpus.
- Embedding layer where input_dim = 100,000, output_dim = 32, input_length = 1000;
- Conv1D layer with filters = 32, kernel_size = 3, size = ‘same’, activation = ‘relu’;
- MaxPooling1D layer with pool_size = 2, strides = 2;
- Flatten layer where the previous layer’s 2d output is flattened to a 1d vector;
- Dense layer with 500 fully connected units, activation = ‘relu’;
- Dense layer with a single output neuron, activation = ‘sigmoid’.
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Hays, S.; Page, S.J.; Buhalis, D. Social media as a destination marketing tool: Its use by national tourism organisations. Curr. Issues Tour. 2013, 16, 211–239. [Google Scholar] [CrossRef]
- Dwivedi, Y.K.; Ismagilova, E.; Hughes, D.L.; Carlson, J.; Filieri, R.; Jacobson, J.; Jain, V.; Karjaluoto, H.; Kefi, H.; Krishen, A.S.; et al. Setting the future of digital and social media marketing research: Perspectives and research propositions. Int. J. Inf. Manag. 2020, 102168. [Google Scholar] [CrossRef]
- Hanika, I.M.; Miranti, A. Social Media and Fake News in 2017 Jakarta Governor Election. In Proceedings of the 5th International Conference on Education & Social Sciences (ICESS), “The Asia Network: Bringing Time, Space and Social Life Together”, Semarang, Indonesia, 26–27 July 2017. [Google Scholar]
- Srivastava, R.; Chawla, S.; Popat, V. Social Media a New Platform for Mass Marketing. Adv. Innov. Res. 2020, 7, 238. [Google Scholar]
- Saunders, J.F.; Eaton, A.A. Snaps, selfies, and shares: How three popular social media platforms contribute to the sociocultural model of disordered eating among young women. Cyberpsychol. Behav. Soc. Netw. 2018, 21, 343–354. [Google Scholar] [CrossRef] [Green Version]
- Amanatidis, D.; Mylona, I.; Mamalis, S.; Kamenidou, I.E. Social media for cultural communication: A critical investigation of museums’ Instagram practices. J. Tour. Herit. Serv. Mark. JTHSM 2020, 6, 38–44. [Google Scholar]
- Most Popular Social Networks Worldwide as of October 2020, Ranked by Number of Active Users. Available online: https://www.statista.com/statistics/272014/ (accessed on 29 January 2021).
- Hanan, H.; Putit, N. Express marketing of tourism destinations using Instagram in social media networking. Hosp. Tour. 2013, 471. [Google Scholar] [CrossRef]
- Liebhart, K.; Bernhardt, P. Political storytelling on Instagram: Key aspects of Alexander Van der Bellen’s successful 2016 presidential election campaign. Media Commun. 2017, 5, 15–25. [Google Scholar] [CrossRef] [Green Version]
- Zappavigna, M. Social media photography: Construing subjectivity in Instagram images. Vis. Commun. 2016, 15, 271–292. [Google Scholar] [CrossRef]
- Trulline, P.; El Karimah, K. Selfie Women’s Photo on Instagram (Virtual Etnography Study Post Photos Selfie on Instagram). In Proceedings of the 5th International Conference on Education & Social Sciences (ICESS), “The Asia Network: Bringing Time, Space and Social Life Together”, Semarang, Indonesia, 26–27 July 2017. [Google Scholar]
- Weilenmann, A.; Hillman, T.; Jungselius, B. Instagram at the museum: Communicating the museum experience through social photo sharing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France, 27 April–2 May 2013; pp. 1843–1852. [Google Scholar]
- Tyer, S. Instagram: What Makes You Post? Pepperdine J. Commun. Res. 2016, 4, 14. [Google Scholar]
- Anagnostopoulos, C.; Parganas, P.; Chadwick, S.; Fenton, A. Branding in pictures: Using Instagram as a brand management tool in professional team sport organisations. Eur. Sport Manag. Q. 2018, 18, 413–438. [Google Scholar] [CrossRef]
- Hunt, D.S.; Lin, C.A.; Atkin, D.J. Communicating social relationships via the use of photo-messaging. J. Broadcasting Electron. Media 2014, 58, 234–252. [Google Scholar] [CrossRef]
- Marengo, D.; Longobardi, C.; Fabris, M.A.; Settanni, M. Highly-visual social media and internalizing symptoms in adolescence: The mediating role of body image concerns. Comput. Hum. Behav. 2018, 82, 63–69. [Google Scholar] [CrossRef]
- Terttunen, A. The Influence of Instagram on Consumers’ Travel Planning and Destination Choice. Bachelor’s Thesis, Haaga-Helia University, Helsinki, Finland, 2017. [Google Scholar]
- Kusuma, K. Activities of the Cyber Public Relations of O Chanel TV in Promoting their Company on the Instagram Social Media. Am. J. Humanit. Soc. Sci. Res. AJHSSR 2018, 2, 50–56. [Google Scholar]
- Hu, Y.; Manikonda, L.; Kambhampati, S. What we instagram: A first analysis of instagram photo content and user types. In Proceedings of the International AAAI Conference on Web and Social Media, Ann Arbor, MI, USA, 1–4 June 2014; Volume 8. [Google Scholar]
- Singh, M. Instagram Marketing—The Ultimate Marketing Strategy. Adv. Innov. Res. 2020, 7, 379. [Google Scholar]
- Latiff, Z.A.; Safiee, N.A.S. New business set up for branding strategies on social media–Instagram. Procedia Comput. Sci. 2015, 72, 13–23. [Google Scholar] [CrossRef] [Green Version]
- Manovich, L. The aesthetic society: Instagram as a life form. In Data Publics: Public Plurality in an Era of Data Determinacy; Mörtenböck, P., Mooshammer, H., Eds.; Routledge: London, UK, 2020. [Google Scholar]
- Gaber, H.R.; Wright, L.T.; Kooli, K. Consumer attitudes towards Instagram advertisements in Egypt: The role of the perceived advertising value and personalization. Cogent Bus. Manag. 2019, 6, 1618431. [Google Scholar] [CrossRef]
- Fouda, A.; Mahmoudi, N.; Moy, N.; Paolucci, F. The COVID-19 pandemic in Greece, Iceland, New Zealand, and Singapore: Health policies and lessons learned. Health Policy Technol. 2020, 9, 510–524. [Google Scholar] [CrossRef] [PubMed]
- Morens, D.M.; Fauci, A.S. Emerging pandemic diseases: How we got to COVID-19. Cell 2020, 182, 1077–1092. [Google Scholar] [CrossRef]
- Matarese, A.; Gambardella, J.; Sardu, C.; Santulli, G. miR-98 regulates TMPRSS2 expression in human endothelial cells: Key implications for COVID-19. Biomedicines 2020, 8, 462. [Google Scholar] [CrossRef]
- COVID-19 Map. Johns Hopkins Coronavirus Resource Center. Available online: https://coronavirus.jhu.edu/map.html (accessed on 29 January 2021).
- Kamenidou, I.E.; Stavrianea, A.; Liava, C. Achieving a Covid-19 free country: Citizens preventive measures and communication pathways. Int. J. Environ. Res. Public Health 2020, 17, 4633. [Google Scholar] [CrossRef] [PubMed]
- Koirala, A.; Joo, Y.J.; Khatami, A.; Chiu, C.; Britton, P.N. Vaccines for COVID-19: The current state of play. Paediatr. Respir. Rev. 2020, 35, 43–49. [Google Scholar] [CrossRef] [PubMed]
- Lerner, A.M.; Folkers, G.K.; Fauci, A.S. Preventing the spread of SARS-CoV-2 with masks and other “low-tech” interventions. JAMA 2020, 324, 1935–1936. [Google Scholar] [CrossRef] [PubMed]
- CoVid19.gov.gr. Τα μέτρα της Κυβέρνησης για την αντιμετώπιση του κορονοϊού. Available online: https://covid19.gov.gr/ (accessed on 29 January 2021).
- O’Callaghan, K.P.; Blatz, A.M.; Offit, P.A. Developing a SARS-CoV-2 vaccine at warp speed. JAMA 2020, 324, 437–438. [Google Scholar] [CrossRef]
- Dai, L.; Zheng, T.; Xu, K.; Han, Y.; Xu, L.; Huang, E.; An, Y.; Cheng, Y.; Li, S.; Liu, M.; et al. A universal design of betacoronavirus vaccines against COVID-19, MERS, and SARS. Cell 2020, 182, 722–733. [Google Scholar] [CrossRef]
- Corey, L.; Mascola, J.R.; Fauci, A.S.; Collins, F.S. A strategic approach to COVID-19 vaccine R&D. Science 2020, 368, 948–950. [Google Scholar]
- Chen, Q.; Min, C.; Zhang, W.; Wang, G.; Ma, X.; Evans, R. Unpacking the black box: How to promote citizen engagement through government social media during the COVID-19 crisis. Comput. Hum. Behav. 2020, 110, 106380. [Google Scholar] [CrossRef]
- Saire, J.C.; Panford-Quainoo, K. Twitter Interaction to Analyze Covid-19 Impact in Ghana, Africa from March to July. arXiv 2020, arXiv:2008.12277. [Google Scholar]
- González-Padilla, D.A.; Tortolero-Blanco, L. Social media influence in the COVID-19 pandemic. Int. Braz. J. Urol. 2020, 46, 120–124. [Google Scholar] [CrossRef] [PubMed]
- Adly, A.S.; Adly, A.S.; Adly, M.S. Approaches based on artificial intelligence and the internet of intelligent things to prevent the spread of COVID-19: Scoping review. J. Med. Internet Res. 2020, 22, e19104. [Google Scholar] [CrossRef]
- Kushner, J. The Role of Social Media during a Pandemic. Available online: https://bit.ly/34mOPcK (accessed on 8 May 2021).
- Malecki, K.; Keating, J.A.; Safdar, N. Crisis communication and public perception of COVID-19 risk in the era of social media. Clin. Infect. Dis. 2020, 72, 697–702. [Google Scholar] [CrossRef] [PubMed]
- Kim, Y.; Kim, J.H. Using photos for public health communication: A computational analysis of the Centers for Disease Control and Prevention Instagram photos and public responses. Health Inform. J. 2020, 26, 2159–2180. [Google Scholar] [CrossRef] [PubMed]
- Basch, C.H.; MacLean, S.A. A content analysis of HPV related posts on instagram. Hum. Vaccines Immunother. 2019, 15, 1476–1478. [Google Scholar] [CrossRef] [PubMed]
- La Gatta, V.; Moscato, V.; Postiglione, M.; Sperli, G. An Epidemiological Neural network exploiting Dynamic Graph Structured Data applied to the COVID-19 outbreak. In IEEE Transactions on Big Data; IEEE: Piscataway, NJ, USA, 2020. [Google Scholar]
- AbdelFattah, M.; Galal, D.; Hassan, N.; Elzanfaly, D.; Tallent, G. A Sentiment Analysis Tool for Determining the Promotional Success of Fashion Images on Instagram. Int. J. Interact. Mob. Technol. 2017, 11, 66–73. [Google Scholar] [CrossRef] [Green Version]
- Moscato, V.; Picariello, A.; Sperli, G. An emotional recommender system for music. In IEEE Intelligent Systems; IEEE: Piscataway, NJ, USA, 2020. [Google Scholar]
- Information about the Pfizer-BioNTech COVID-19 Vaccine. Available online: https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/Pfizer-BioNTech.html (accessed on 29 January 2021).
- AstraZeneca-Research-Based Bio-Pharmaceutical Company. Available online: https://www.astrazeneca.com/ (accessed on 29 January 2021).
- Arashkia, A.; Jalilvand, S.; Mohajel, N.; Afchangi, A.; Azadmanesh, K.; Salehi-Vaziri, M.; Fazlalipour, M.; Pouriayevali, M.H.; Jalali, T.; Nasab, S.D.M.; et al. Severe acute respiratory syndrome-coronavirus-2 spike (S) protein based vaccine candidates: State of the art and future prospects. Rev. Med. Virol. 2020, e2183. [Google Scholar] [CrossRef] [PubMed]
- About Johnson & Johnson. Available online: https://www.jnj.com/about-jnj (accessed on 29 January 2021).
- How the Johnson & Johnson Vaccine Works. Available online: https://www.nytimes.com/interactive/2020/health/johnson-johnson-covid-19-vaccine.html (accessed on 29 January 2021).
- COVID-19 Vaccine. Available online: https://en.wikipedia.org/wiki/COVID-19_vaccine (accessed on 29 January 2021).
- Instaloader. Available online: https://instaloader.github.io/ (accessed on 29 January 2021).
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016. [Google Scholar]
- Adekotujo, A.S.; Lee, J.; Enikuomehin, A.O.; Mazzara, M.; Aribisala, S.B. Bi-lingual Intent Classification of Twitter Posts: A Roadmap. In International Conference in Software Engineering for Defense Applications; Springer: Cham, Germany, 2018. [Google Scholar]
- Purohit, H.; Dong, G.; Shalin, V.; Thirunarayan, K.; Sheth, A. Intent classification of short-text on social media. In Proceedings of the 2015 IEEE International Conference on Smart City/Socialcom/Sustaincom (Smartcity), Chengdu, China, 19–21 December 2015. [Google Scholar]
- Saha, T.; Saha, S.; Bhattacharyya, P. Tweet act classification: A deep learning based classifier for recognizing speech acts in twitter. In Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019. [Google Scholar]
- Inception. Available online: https://inception-project.github.io/ (accessed on 29 January 2021).
- Ekman, P. Are there basic emotions? Psychol. Rev. 1992, 99, 550–553. [Google Scholar] [CrossRef] [PubMed]
- Ekman, P. Emotions inside out. 130 years after Darwin’s ‘The Expression of the Emotions in Man and Animal. Ann. N. Y. Acad. Sci. 2003, 1000, 1–6. [Google Scholar] [CrossRef]
- Plutchik, R. The circumplex as a general model of the structure of emotions and personality. In Circumplex Models of Personality and Emotions; Plutchik, R., Conte, H.R., Eds.; American Psychological Association: Washington, DC, USA, 1997. [Google Scholar]
- Izard, C.E.; Libero, D.Z.; Putnam, P.; Haynes, O.M. Stability of emotion experiences and their relations to traits of personality. J. Personal. Soc. Psychol. 1993, 64, 847. [Google Scholar] [CrossRef]
- Cowen, A.S.; Keltner, D. Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proc. Natl. Acad. Sci. USA 2017, 114, E7900–E7909. [Google Scholar] [CrossRef] [Green Version]
- Souravlas, S.; Anastasiadou, S. Pipelined Dynamic Scheduling of Big Data Streams. Appl. Sci. 2020, 10, 4796. [Google Scholar] [CrossRef]
- Souravlas, S.; Anastasiadou, S.; Katsavounis, S. More on Pipelined Dynamic Scheduling of Big Data Streams. Appl. Sci. 2021, 11, 61. [Google Scholar] [CrossRef]
- Anastasiadou, S.; Kofou, I. Incorporating Web 2.0 Tools into Greek Schools. Int. J. Technol. Learn. 2013, 20, 11–23. [Google Scholar]
- Kofou, I.; Anastasiadou, S. Language and Communication Needs Analysis in Intercultural Education. Int. J. Divers. Educ. 2013, 12, 15–64. [Google Scholar] [CrossRef]
Facebook—2740 | YouTube—2291 | WhatsApp—2000 | FB Messenger—1300 |
Instagram—1221 | WeChat—1213 | TikTok—689 | QQ—617 |
Douyin—600 | Sina Weibo—511 | Telegram—500 | Snapchat—498 |
Kuaishou—481 | Pinterest—442 | Reddit—430 | Twitter—353 |
Pfizerinc (21 April 2016) | Astrazeneca (21 May 2013) | Jnj (17 August 2017) | |
---|---|---|---|
posts | 390 (rate: 6.8 posts/month) | 1309 (rate: 14.2 posts/month) | 33 (rate: 0.8 posts/month) |
followers | 67,800 | 23,600 | 32,600 |
following | 273 | 551 | 338 |
Characters | Words | Hashtags | Likes | Comments | Images | Videos | |
---|---|---|---|---|---|---|---|
Pfizer (390 posts—since 4/2016) | 272.6 (152.7) | 41.9 (24.0) | 3.6 (3.9) | 203.7 (364.9) | 17.5 (57.7) | 1.4 (1.3) | 0.4 (0.5) |
Astrazeneca (1309 posts—since 5/2013) | 177.2 (99.8) | 26.8 (15.4) | 2.4 (2.6) | 43.2 (73.5) | 1.8 (7.0) | 1.0 (0.4) | 0.1 (0.2) |
Johnson & Johnson (33 posts—since 8/2017) | 1168.8 (736.1) | 198.0 (127.9) | 3.9 (1.8) | 301.0 (287.5) | 139.1 (586.7) | 1.9 (1.5) | 0.3 (0.4) |
#pfizer | #astrazeneca | #jnj | Totals | |
---|---|---|---|---|
ACK | 155 | 120 | 7 | 282 |
ADV | 81 | 67 | 9 | 157 |
SEK | 6 | 4 | 1 | 11 |
EXP | 97 | 100 | 28 | 225 |
Totals | 339 | 291 | 45 | 675 |
LOG | NB | DT | KNN | SVM | MLP | RF | BN | LMT | |
---|---|---|---|---|---|---|---|---|---|
Accuracy (%) | 60.4 | 52.9 | 62.1 | 44.0 | 68.6 | 51.7 | 68.0 | 61.5 | 61.5 |
MCC | 0.409 | 0.394 | 0.422 | 0.269 | 0.523 | 0.349 | 0.523 | 0.412 | 0.414 |
ROC area | 0.777 | 0.763 | 0.704 | 0.618 | 0.790 | 0.702 | 0.854 | 0.766 | 0.772 |
F-measure | 0.609 | 0.572 | 0.614 | 0.373 | 0.677 | 0.549 | 0.652 | 0.576 | 0.609 |
Kappa | 0.407 | 0.344 | 0.419 | 0.163 | 0.516 | 0.310 | 0.498 | 0.394 | 0.412 |
Overall | #pfizer | #astrazeneca | #jnj | |
---|---|---|---|---|
mean | 0.38 | 0.42 | 0.34 | 0.39 |
standard deviation | 0.45 | 0.46 | 0.44 | 0.43 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Amanatidis, D.; Mylona, I.; Kamenidou, I.; Mamalis, S.; Stavrianea, A. Mining Textual and Imagery Instagram Data during the COVID-19 Pandemic. Appl. Sci. 2021, 11, 4281. https://doi.org/10.3390/app11094281
Amanatidis D, Mylona I, Kamenidou I, Mamalis S, Stavrianea A. Mining Textual and Imagery Instagram Data during the COVID-19 Pandemic. Applied Sciences. 2021; 11(9):4281. https://doi.org/10.3390/app11094281
Chicago/Turabian StyleAmanatidis, Dimitrios, Ifigeneia Mylona, Irene (Eirini) Kamenidou, Spyridon Mamalis, and Aikaterini Stavrianea. 2021. "Mining Textual and Imagery Instagram Data during the COVID-19 Pandemic" Applied Sciences 11, no. 9: 4281. https://doi.org/10.3390/app11094281
APA StyleAmanatidis, D., Mylona, I., Kamenidou, I., Mamalis, S., & Stavrianea, A. (2021). Mining Textual and Imagery Instagram Data during the COVID-19 Pandemic. Applied Sciences, 11(9), 4281. https://doi.org/10.3390/app11094281