Next Article in Journal
Integration of Biometrics and Steganography: A Comprehensive Review
Next Article in Special Issue
CogBeacon: A Multi-Modal Dataset and Data-Collection Platform for Modeling Cognitive Fatigue
Previous Article in Journal
Nematic Liquid Crystal Composite Materials for DC and RF Switching
Open AccessCommunication

A Pipeline for Rapid Post-Crisis Twitter Data Acquisition, Filtering and Visualization

Information Sciences Institute, University of Southern California, Los Angeles, CA 90292, USA
*
Author to whom correspondence should be addressed.
Technologies 2019, 7(2), 33; https://doi.org/10.3390/technologies7020033
Received: 18 January 2019 / Revised: 19 March 2019 / Accepted: 30 March 2019 / Published: 2 April 2019
(This article belongs to the Special Issue Multimedia and Cross-modal Retrieval)
  |  
PDF [805 KB, uploaded 2 April 2019]
  |  

Abstract

Due to instant availability of data on social media platforms like Twitter, and advances in machine learning and data management technology, real-time crisis informatics has emerged as a prolific research area in the last decade. Although several benchmarks are now available, especially on portals like CrisisLex, an important, practical problem that has not been addressed thus far is the rapid acquisition, benchmarking and visual exploration of data from free, publicly available streams like the Twitter API in the immediate aftermath of a crisis. In this paper, we present such a pipeline for facilitating immediate post-crisis data collection, curation and relevance filtering from the Twitter API. The pipeline is minimally supervised, alleviating the need for feature engineering by including a judicious mix of data preprocessing and fast text embeddings, along with an active learning framework. We illustrate the utility of the pipeline by describing a recent case study wherein it was used to collect and analyze millions of tweets in the immediate aftermath of the Las Vegas shootings in 2017. View Full-Text
Keywords: data acquisition; social web; twitter; crisis informatics; case study; Las Vegas shootings; fastText; active learning; data preprocessing; visualization; embeddings data acquisition; social web; twitter; crisis informatics; case study; Las Vegas shootings; fastText; active learning; data preprocessing; visualization; embeddings
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Kejriwal, M.; Gu, Y. A Pipeline for Rapid Post-Crisis Twitter Data Acquisition, Filtering and Visualization. Technologies 2019, 7, 33.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Technologies EISSN 2227-7080 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top