Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication
Abstract
:1. Introduction
2. Literature Review
3. Materials and Methods
3.1. Study Site
3.2. Datasets and Processing
3.2.1. Tweets of 2013 Colorado floods
3.2.2. GIS Data
3.2.3. National Oceanic and Atmospheric Administration (NOAA) Warning/Alert Messages
3.2.4. Reference Documents
3.3. Analytics and Techniques
4. Results and Discussion
4.1. Evaluation of Text Content
4.2. Evaluation of Image Content
4.2.1. Manual Approach
4.2.2. AI Approach
4.3. Extracting Added Tweets Using Verified Keywords
5. Discussion, Implications for Risk Communication, and Future Research
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Hannah, R.; Max, R. Natural Disasters. Available online: https://ourworldindata.org/natural-disasters (accessed on 5 February 2020).
- Newell, B.R.; Rakow, T.; Yechiam, E.; Sambur, M. Rare disaster information can increase risk-taking. Nat. Clim. Chang. 2016, 6, 158–161. [Google Scholar] [CrossRef]
- Bradley, D.T.; McFarland, M.; Clarke, M. The effectiveness of disaster risk communication: A systematic review of intervention studies. In Effective Communication during Disasters; Apple Academic Press: Waretown, NJ, USA, 2016; pp. 81–120. [Google Scholar]
- Elisa, S.; Grieco, E. Americans Are Wary of the Role Social Media Sites Play in Delivering the News; Pew Research Center: Washington, DC, USA, 2019. [Google Scholar]
- Richards, D. Mainstream Media vs. Social Media: What Does the Future Hold? Available online: https://tuckerhall.com/mainstream-media-vs-social-media-future-hold/2017 (accessed on 5 September 2020).
- Amber, S. The use of social media in crisis communication. In Risk Communication and Community Resilience; Routledge: London, UK, 2019; pp. 267–282. [Google Scholar]
- Moon, G. Why People Share: The Psychology of Social Sharing. Available online: https://coschedule.com/blog/why-people-share/ (accessed on 1 May 2019).
- Oh, S.; Syn, S.Y. Motivations for sharing information and social support in social media: A comparative analysis of Facebook, Twitter, Delicious, YouTube, and Flickr. J. Assoc. Inf. Sci. Technol. 2015, 66, 2045–2060. [Google Scholar] [CrossRef]
- Starbird, K. Examining the alternative media ecosystem through the production of alternative narratives of mass shooting events on Twitter. In Proceedings of the Eleventh International AAAI Conference on Web and Social Media 2017, Montreal, QC, Canada, 15–18 May 2017. [Google Scholar]
- Liu, X.; Kar, B.; Zhang, C.; Cochran, D.M. Assessing relevance of tweets for risk communication. Int. J. Digit. Earth 2018, 12, 781–801. [Google Scholar] [CrossRef]
- Boulton, C.A.; Shotton, H.; Williams, H.T. Using social media to detect and locate wildfires. In Proceedings of the Tenth International AAAI Conference on Web and Social Media 2016, Cologne, Germany, 17–20 May 2016. [Google Scholar]
- Brengarth, L.B.; Mujkic, E. WEB 2.0: How social media applications leverage nonprofit responses during a wildfire crisis. Comput. Hum. Behav. 2016, 54, 589–596. [Google Scholar] [CrossRef]
- Sachdeva, S.; McCaffrey, S. Using social media to predict air pollution during California wildfires. In Proceedings of the 9th International Conference on Social Media and Society 2018, Copenhagen, Denmark, 18–20 July 2018. [Google Scholar]
- Bonney, R. ‘Citizen Science: A Lab Tradition’ [in] Living Bird: For the Study and Conservation of Birds. Living Bird Study Conserv. Birds 1996, 15, 7–15. [Google Scholar]
- Greengard, S. Following the crowd. Commun. ACM 2011, 54, 20–22. [Google Scholar] [CrossRef]
- Hetmank, L. Components and Functions of Crowdsourcing Systems-A Systematic Literature Review. Wirtschaftsinformatik 2013, 4, 2013. [Google Scholar]
- Onsrud, H.; Camara, G.; Campbell, J.; Chakravarthy, N.S. Public commons of geographic data: Research and development challenges. In Proceedings of the International Conference on Geographic Information Science 2004, Adelphi, MD, USA, 20–23 October 2004. [Google Scholar]
- Onsrud, H.; Campbell, J. Big opportunities in access to” Small Science” data. Data Sci. J. 2007, 6, OD58–OD66. [Google Scholar] [CrossRef]
- Wiggins, A.; Crowston, K. From conservation to crowdsourcing: A typology of citizen science. In Proceedings of the 2011 44th Hawaii International Conference on System Sciences 2011, Kauai, HI, USA, 4–7 January 2011. [Google Scholar]
- Sutton, J.N.; Palen, L.; Shklovski, I. Backchannels on the Front Lines: Emergency Uses of Social Media in the 2007 Southern California Wildfires. In Proceedings of the 5th International ISCRAM Conference, Washington, DC, USA, 4–7 May 2008. [Google Scholar]
- Meek, S.; Jackson, M.J.; Leibovici, D.G. A Flexible Framework for Assessing the Quality of Crowdsourced Data. In Proceedings of the AGILE’2014, Castellón, Spain, 3–6 June 2014. [Google Scholar]
- Castillo, C.; Mendoza, M.; Poblete, B. Information credibility on twitter. In Proceedings of the 20th International Conference on World Wide Web 2011, Hyderabad, India, 28 March 2011. [Google Scholar]
- Liu, J.; Li, J.; Li, W.; Wu, J. Rethinking big data: A review on the data quality and usage issues. ISPRS J. Photogramm. Remote Sens. 2016, 115, 134–142. [Google Scholar] [CrossRef]
- Lee, K.; Caverlee, J.; Pu, C. Social spam, campaigns, misinformation and crowdturfing. In Proceedings of the 23rd International Conference on World Wide Web 2014, Seoul, Korea, 8 April 2014. [Google Scholar]
- Starbird, K.; Maddock, J.; Orand, M.; Achterman, P.; Mason, R.M. Rumors, false flags, and digital vigilantes: Misinformation on twitter after the 2013 boston marathon bombing. IConference 2014 Proc. 2014. [Google Scholar] [CrossRef] [Green Version]
- Dale, S. Heuristics and biases: The science of decision-making. Bus. Inf. Rev. 2015, 32, 93–99. [Google Scholar] [CrossRef]
- Covello, V.T. Risk communication: An emerging area of health communication research. Ann. Int. Commun. Assoc. 1992, 15, 359–373. [Google Scholar] [CrossRef]
- Kar, B. Citizen science in risk communication in the era of ICT. Concurr. Comput. Pract. Exp. 2016, 28, 2005–2013. [Google Scholar] [CrossRef]
- Greenhalgh, A. Social Media Flooded with Rescue Requests during Hurricane Harvey. Professional Projects from the College of Journalism and Mass Communications: 2018. Available online: https://digitalcommons.unl.edu/journalismprojects/19/ (accessed on 5 September 2020).
- Kishore, N.; Marqués, D.; Mahmud, A.; Kiang, M.V.; Rodriguez, I.; Fuller, A.; Ebner, P.; Racy, F.; Sorensen, C.; Maas, L.; et al. Mortality in puerto rico after hurricane maria. N. Engl. J. Med. 2018, 379, 162–170. [Google Scholar]
- Jahanbin, K.; Rahmanian, V. Using Twitter and web news mining to predict COVID-19 outbreak. Asian Pac. J. Trop. Med. 2020, 13. [Google Scholar] [CrossRef]
- Rosenberg, H.; Syed, S.; Rezaie, S. The Twitter pandemic: The critical role of Twitter in the dissemination of medical information and misinformation during the COVID-19 pandemic. Can. J. Emerg. Med. 2020, 1–7. [Google Scholar] [CrossRef] [Green Version]
- Government Accountability Office. Applied Research and Methods: Assessing the Reliability of Computer-Processed Data (GAO-09-680G). Available online: https://www.gao.gov/assets/80/77213.pdf (accessed on 10 March 2020).
- Resch, B.; Usländer, F.; Havas, C. Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment. Cartogr. Geogr. Inf. Sci. 2018, 45, 362–376. [Google Scholar] [CrossRef] [Green Version]
- Viviani, M.; Pasi, G. Credibility in social media: Opinions, news, and health information—A survey. WIREs Data Min. Knowl. Discov. 2017, 7, e1209. [Google Scholar]
- Xiang, Z.; Du, Q.; Ma, Y.; Fan, W. Assessing Reliability of Social Media Data: Lessons from Mining TripAdvisor Hotel Reviews. In Proceedings of the Information and Communication Technologies in Tourism 2017, Rome, Italy, 24–26 January 2017. [Google Scholar]
- Alonso, O.; Marshall, C.C.; Najork, M.A. A human-centered framework for ensuring reliability on crowdsourced labeling tasks. In Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing 2013, Palm Springs, CA, USA, 7–9 November 2013. [Google Scholar]
- Oh, O.; Agrawal, M.; Rao, H.R. Community intelligence and social media services: A rumor theoretic analysis of tweets during social crises. Mis Q. 2013, 407–426. [Google Scholar] [CrossRef]
- Guberman, J.; Schmitz, C.; Hemphill, L. Quantifying toxicity and verbal violence on Twitter. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion 2016, San Francisco, CA, USA, 27 February 2016. [Google Scholar]
- McCormick, T.H.; Lee, H.; Cesare, N.; Shojaie, A.; Spiro, E.S. Using Twitter for demographic and social science research: Tools for data collection and processing. Sociol. Methods Res. 2017, 46, 390–421. [Google Scholar] [CrossRef] [PubMed]
- Williams, J.; Dagli, C. Twitter language identification of similar languages and dialects without ground truth. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial) 2017, Valencia, Spain, 3 April 2017; (accessed on 5 September 2020). [Google Scholar]
- Liu, W.; Ruths, D. What’s in a name? using first names as features for gender inference in twitter. In Proceedings of the 2013 AAAI Spring Symposium Series 2013, Palo Alto, CA, USA, 25–27 March 2013. [Google Scholar]
- Burger, J.D.; Henderson, J.; Kim, G.; Zarrella, G. Discriminating gender on Twitter. In Proceedings of the Conference on Empirical Methods in Natural Language Processing 2011, Scotland, UK, 27–31 July 2011. [Google Scholar]
- Denis, L.A.S.; Palen, L.; Anderson, K.M. Mastering Social Media: An Analysis of Jefferson County’s Communications during the 2013 Colorado Floods. In Proceedings of the 11th International ISCRAM Conference, University Park, PA, USA, 18–21 May 2014. [Google Scholar]
- Caragea, C.; McNeese, N.; Jaiswal, A.; Traylor, G.; Kim, H.W.; Mitra, P.; Wu, D.; Tapia, A.H.; Giles, L.; Yen, J.; et al. Classifying Text Messages for the Haiti Earthquake. In Proceedings of the 8th International ISCRAM Conference, Lisbon, Portugal, 8–11 May 2011. [Google Scholar]
- Chen, S.-H.; Chen, Y.-H. A content-based image retrieval method based on the google cloud vision api and wordnet. In Proceedings of the Asian Conference on Intelligent Information and Database Systems, Kanazawa, Japan, 3–5 April 2017. [Google Scholar]
- d’Andrea, C.; Mintz, A. Studying the Live Cross-Platform Circulation of Images With Computer Vision API: An Experiment Based on a Sports Media Event. Int. J. Commun. 2019, 13, 21. [Google Scholar]
- Colorado Climate Center. The 2013 Colorado Floods Accumulated Precipitation. Available online: https://www.ncdc.noaa.gov/news/visualizing-september-2013-colorado-flood (accessed on 5 July 2016).
- Liu, X. Evaluating Relevance and Reliability of Twitter Data for Risk Communication; ProQuest Dissertations Publishing, 2017. Available online: https://aquila.usm.edu/dissertations/1415/ (accessed on 5 September 2020).
- City of Boulder. The 2013 Boulder Flood Extent Map. Available online: https://bouldercolorado.gov/open-data/city-of-boulder-september-2013-flood-extents (accessed on 10 July 2016).
- City of Boulder. Main Roads and Streets Map. Available online: https://bouldercolorado.gov/maps (accessed on 15 July 2016).
- NOAA. NOAA Warning/Alert. Available online: https://alerts.weather.gov/cap/co.php?x=1 (accessed on 18 July 2016).
- FEMA. Reducing Losses through Higher Regulatory Standards. Available online: https://www.fema.gov/media-library-data/1429759760513-f96124536d2c3ccc07b3db4a4f8c35b5/FEMA_CO_RegulatoryLAS.pdf (accessed on 24 July 2016).
- Severe Flooding on the Colorado Front Range—September 2013. Available online: https://wwa.colorado.edu/resources/front-range-floods/assessment.pdf (accessed on 3 August 2016).
- NOAA. The Record Front Range and Eastern Colorado Floods of 11–17 September; NOAA: Silver Spring, MD, USA, 2014.
- Kingkade, T. University of Colorado Students Post Scenes from Flooded Boulder Campus; Huffpost: New York, NY, USA, 2013. [Google Scholar]
- Department of Higher Education. 2013 Colorado Flood: Campus Impacts; Department of Higher Education: Little Rock, AR, USA, 2013. Available online: https://highered.colorado.gov/campusflooding.pdf (accessed on 25 July 2016).
- De Choudhury, M.; Diakopoulos, N.; Naaman, M. Unfolding the event landscape on twitter: Classification and exploration of user categories. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work 2012, Bellevue, WA, USA, 11–15 February 2012. [Google Scholar]
- Google. Google Vision API. Available online: https://cloud.google.com/vision (accessed on 5 April 2020).
- Vis, F.; Faulkner, S.; Parry, K.; Manyukhina, Y.; Evans, L. Twitpic-ing the Riots: Analysing Images Shared on Twitter during the 2011 UK Riots. In Twitter and Society; Weller, K., Bruns, A., Burgess, J., Mahrt, M., Puschmann, C., Eds.; Peter Lang: New York, NY, USA, 2013; pp. 385–398. [Google Scholar]
- Kelly, R. Twitter study reveals interesting results about usage. Available online: https://www.blogomator.com/blog/tweets-are-pointless-twitter-study-reveals-interesting-results-about-usage-pear-analytics/ (accessed on 1 August 2016).
- Gupta, A.; Lamba, H.; Kumaraguru, P.; Joshi, A. Faking sandy: Characterizing and identifying fake images on twitter during hurricane sandy. In Proceedings of the 22nd international conference on World Wide Web 2013, Rio de Janeiro, Brazil, 13–17 May 2013. [Google Scholar]
- Benevenuto, F.; Magno, G.; Rodrigues, T.; Almeida, V. Detecting spammers on twitter. In Proceedings of the Collaboration, Electronic Messaging, Anti-Abuse and Spam Conference (CEAS) 2010, Washington, DC, USA, 13–14 July 2010. [Google Scholar]
- Horita, F.E.A.; Degrossi, L.C.; de Assis, L.F.G.; Zipf, A.; de Albuquerque, J.P. The Use of Volunteered Geographic Information (VGI) and Crowdsourcing in Disaster Management: A Systematic Literature Review. In Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, IL, USA, 15–17 August 2013. [Google Scholar]
- Klonner, C.; Marx, S.; Usón, T.; de Porto Albuquerque, J.; Höfle, B. Volunteered Geographic Information in Natural Hazard Analysis: A Systematic Literature Review of Current Approaches with a Focus on Preparedness and Mitigation. ISPRS Int. J. Geo-Inf. 2016, 5, 103. [Google Scholar] [CrossRef] [Green Version]
ID | Roads/streets | Posted Time | Associated Risk Information |
---|---|---|---|
1 | West of Broadway | 09/12 03:02 | Boulder Creek is about to spill its bank. |
2 | Broadway & Arapahoe Avenue | 09/12 05:30 | Water at Boulder Creek has come up 2.5 feet in 10 min. |
3 | 8th Street & Marine Street | 09/12 05:52 | Gregory canyon drainage overtopping the underground culvert, flowing onto 8th St. near Marine. |
4 | 28th Street & Colorado Avenue | 09/12 06:09 | Knee deep water at 28th St & Colorado Ave. |
5 | 15th Street | 09/12 08:39 | River taking back Boulder neighborhood street. |
6 | Highway 36 underpass | 09/12 22:23 | It’s raining! It’s pouring! |
7 | 8th Street between University of Colorado and Marine | 09/13 03:22 | …basically, a raging torrent. |
8 | 30th Street & Foothills | 09/13 00:49 | Colorado Avenue is closed between 30th and Foothill. |
9 | 30th Street | 09/13 01:08 | Water is coming up through drains on 30th and Colorado Ave…this could get ugly. |
10 | Highway 36 | 09/13 01:30 | Barely make it out of Boulder. Couldn’t get to hwy 36. |
11 | Highway 36 | 09/13 02:33 | Highway 36 is flooded, not way out. |
12 | Highway 36 & Foothills | 09/13 05:32 | Over 3 feet of water flooding. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, X.; Kar, B.; Montiel Ishino, F.A.; Zhang, C.; Williams, F. Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication. ISPRS Int. J. Geo-Inf. 2020, 9, 532. https://doi.org/10.3390/ijgi9090532
Liu X, Kar B, Montiel Ishino FA, Zhang C, Williams F. Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication. ISPRS International Journal of Geo-Information. 2020; 9(9):532. https://doi.org/10.3390/ijgi9090532
Chicago/Turabian StyleLiu, Xiaohui, Bandana Kar, Francisco Alejandro Montiel Ishino, Chaoyang Zhang, and Faustine Williams. 2020. "Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication" ISPRS International Journal of Geo-Information 9, no. 9: 532. https://doi.org/10.3390/ijgi9090532