Health Use Cases of AI Chatbots: Identification and Analysis of ChatGPT Prompts in Social Media Discourses

Abstract

The popularity of ChatGPT has raised questions surrounding AI’s potential for health use cases. Since the release of ChatGPT in 2022, social media users have shared their prompts and ChatGPT responses on different topics such as health. Despite editorials and opinion articles discussing the potential uses of ChatGPT, there is a lack of a systematic approach to identify and analyze the use cases of ChatGPT in health. This study establishes a framework for gathering and identifying tweets (i.e., posts on social media site “X”, formerly known as Twitter) that discuss health use cases of ChatGPT, integrating topic modeling with constructivist grounded theory (CGT) to organize these topics into common categories. Using this framework, nine topics were identified, which were further grouped into four categories: (1) Clinical Workflow, (2) Wellness, (3), Diseases, and (4) Gender Identity. The Clinical Workflow category was the most popular category, and included four topics: (1) Seeking Advice, (2) Clinical Documentation, (3) Medical Diagnosis, and (4) Medical Treatment. Among the identified topics, “Diet and Workout Plans” was the most popular topic. This research highlights the potential of social media to identify the health use cases and potential health applications of an AI-based chatbot such as ChatGPT. The identified topics and categories can be beneficial for researchers, professionals, companies, and policymakers working on health use cases of AI chatbots.

1. Introduction

Artificial intelligence (AI) has reshaped our daily interactions by enabling the development and assessment of sophisticated applications and devices, known as intelligent agents, capable of executing a variety of functions, such as medical diagnoses [1]. Chatbots, also known as artificial conversation entities, interactive agents, smart bots, or digital assistants, are human-computer programs that simulate interactions with humans in different health applications [2]. The history of chatbots stretches back to 1966, when EIZA, a chatbot simulating a psychotherapist’s operation, was developed [3]. The first AI chatbot was developed in 1988. Since then, the emergence of more advanced chatbots like Apple’s Siri, Google Assistant, and Amazon Alexa has risen in recent years [2].

Chatbots adopts computational linguistics models to facilitate communication with humans in various domains, such as education [4]. Chatbots can improve productivity and offer efficient, engaging assistants capable of addressing questions directly [5]. Sometimes, users approach chatbots with emotional requests and perceive them more as friendly companions than mere assistants [6]. Figure 1 illustrates a rising trend in the research interest surrounding chatbots starting from 2002, with a significant spike in interest after 2022.

Chatbots can also be used to address users’ health questions [7,8]. Relevant studies have focused on earlier versions of health chatbots and on single health issues such as oncology [9]. Traditional health chatbots are often linked to poor patient adherence, which is attributed to a perceived lack of consistency and transparency compared to the face-to-face interactions provided by in-person meetings [10]. Moreover, clinicians view chatbots as being better suited for administrative tasks like scheduling appointments, locating hospitals, and sending prescription reminders, while noting significant risks in using chatbots, such as the risk of receiving inaccurate medical information. Consequently, clinicians remain skeptical of chatbots replacing complex decision-making processes that require expert medical advice [10].

Chat Generative Pretrained Transformer (ChatGPT), the latest version of GPT-3 (Generative Pretrained Transformer 3), is an innovative generative AI chatbot based on language modeling. ChatGPT predicts text via encoding through translating input into a vector and decoding through converting the model’s output to generate iterative human-like responses [11,12]. ChatGPT addresses the semantic limitations of previous models and can “answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests” [11]. ChatGPT’s user-friendly chat interface makes it a readily adoptable tool for individuals with basic digital literacy skills [13].

Before the implementation of ChatGPT, chatbots were developed for a specific task, such as checking for health symptoms [10]. Healthcare chatbots, with efficiencies comparable to clinicians, offer benefits such as aiding in medical decisions, enhancing physical activity, supporting cognitive behavioral therapy, and treating somatic disorders [14]. The emergence of ChatGPT has also boosted the development and market share of health chatbots [10] p. 10, thus providing businesses, startups, and the healthcare industry with a unique opportunity to develop new AI-based applications, improve workflows, and drive positive outcomes.

Social media has significantly transformed the way information is accessed, shared, and consumed. Today, social media not only serves as a venue for individuals to develop and sustain personal relationships but also functions as a dynamic platform. Users can seek and gather information, create original content, share updates with their personal networks, and disseminate information to the broader public. These platforms facilitate a rich exchange of ideas and interaction, making them central to modern communication strategies [15]. With over 5 billion users, social media sites [16] have been considered a key platform for sharing public opinion and experiences on different issues such as health [17] and politics [18]. Social media data can be analyzed to understand behaviors and identify emerging societal trends from diverse societies for decision-making [19].

It was found that 70% of the population searches for health information online, and 37% of those individuals specifically seek health information on social media [20]. Social media has profoundly influenced healthcare by revolutionizing patient education, health communication, and professional networking. Through social media platforms, healthcare organizations can disseminate educational content, promote health awareness campaigns, and mobilize support for public health initiatives [21]. Online patient support communities provide invaluable emotional support and connection for individuals facing similar health challenges [22]. Moreover, healthcare professionals utilize social media for networking, sharing research, and collaborating on projects, fostering knowledge exchange and professional development [23]. Real-time health surveillance through social media data analysis enables the tracking of disease outbreaks and identification of emerging health trends [24].

After the release of ChatGPT in 2022, the number of ChatGPT users reached 100 million users within two months [25], and this number has been increasing exponentiall, with over 1.5 billion visits in 2023 [26]. ChatGPT has been adopted and discussed by a wide range of users to gain first-hand AI experience through simple communications with the chatbot [27]. Interacting with an AI chatbot involves sending a message known as a prompt, which guides the AI in generating textual responses [28]. Health applications of ChatGPT can be identified through the content of prompts developed by health professionals and the general public [29]. These prompt data, however, are not available to researchers. Nonetheless, ChatGPT users have started to share on social media platforms their prompts and responses that can be used to assess the use cases of ChatGPT for health [30].

Although using chatbots is not a new concept, the popularity of ChatGPT has raised questions of AI’s potential for health use cases. Several editorials, perspectives, and commentaries have been published discussing the uses, challenges, and opportunities of GPTs for health [29,31], as well as the associated ethical issues; however, most of these articles focus on ChatGPT applications for health professionals, researchers, and regulators. These studies consider topics such as analyzing the performance of ChatGPT in medical exams such as parasitology exams [32] and the USMLE [33]; evaluating the responses of ChatGPT for specialized questions on different issues such as cancer [34], retinal diseases [35], genetics/genomics [36], gastroenterology [37], and pathology [38]; and using ChatGPT to write patient letters for different clinical scenarios [39]. Others have compared the quality and clarity of health messages generated by AI to those presented in tweets about folic acid [40] and websites about lower back pain, meniscal injury, and gonarthrosis [41]; investigated challenges related to mental health [42] and healthy lifestyles [43]; identified AI chatbot applications in medical, dental, pharmacy, and public health education [44]; developed customized chatbots [45]; explored user acceptance [46]; and measured the efficiency of AI chatbots created before the release of ChatGPT for promoting healthy lifestyles, aiding smoking cessation, and improving treatment or medication adherence [47]. In brief, the current literature primarily provides insights on potential applications and challenges, presents use case examples in specific medical domains, evaluates ChatGPT’s responses, develops tailored methods, and explores user perceptions.

Additionally, some studies have looked into ChatGPT discussions on social media [30,48,49]. An examination revealed three primary themes: general subjects (e.g., news), functional areas (e.g., creative writing), and potential consequences (e.g., human impact). One study categorizing themes on Twitter (now known as “X”) found that users predominantly used ChatGPT for writing and asking questions [30]. A similar study examined posts on the Chinese Weibo platform, revealing discussions about ChatGPT that focused on its technical support capabilities, the effectiveness of AI-related features, its influence on human labor, and its impact on the fields of education and technology [48]. Another study examined remarks concerning ChatGPT and marketing, uncovering 10 marketing themes, including the notion of revolutionizing B2B marketing [49]. While these studies provide value insights, health topics were not detected in the results due to the lack of focus on health issues. Thus, generalizability analyses and systematic studies are lacking regarding the use cases of ChatGPT and their categories related to health issues from the perspective of the public. In summary, although analyzing social media data offers a valuable approach for identifying and researching the health-related use cases of ChatGPT, relevant studies are currently scarce. To overcome this constraint, this study aims to identify and analyze the health use cases of ChatGPT, an AI chatbot. To accomplish this objective, we utilize both topic modeling, extracting themes from large text datasets, and grounded theory, providing a contextual understanding as our foundational framework. Our research question revolves around determining the primary categories of health-related use cases for AI chatbots. The findings underscore the importance of identifying and understanding health use cases offered by ChatGPT through social media conversations. This study provides a distinctive contribution to the burgeoning research surrounding the health applications of AI chatbots, utilizing social media data.

2. Materials and Methods

This research integrates grounded theory with topic modeling to systematically analyze data and reduce human biases. There are three schools of thought in grounded theory: positivism, interpretivism, and constructivism, proposed by Glaser [50], Strauss [51], and Charmaz [52], respectively. This study uses constructivist grounded theory (CGT) due to its ability to combine the strengths of positivism and interpretivism [53]. This research embeds topic modeling into CGT in three steps, including initial coding, focused coding, and theoretical coding following data collection and preprocessing, as illustrated in Figure 2 [52,53]. CGT is grounded in empirical evidence to develop theory from data, emphasizing personal reality, subjective reality, meaning-making, or the construction of reality [54] and offers a set of concepts and their categories [55]. Combining topic modeling with qualitative coding can effectively eliminate irrelevant and meaningless content and clarify the interpretation of topics. Our methodology represents a form of computational grounded theory, wherein topics and their categories are derived through topic modeling and continuously refined through iterative qualitative coding [56].

2.1. Data Collection and Processing

This study focused on ChatGPT prompts posted on Twitter (now renamed to/known as X). In this paper, we will use the terms “Twitter” and “tweet” instead of “X” and “post”, as the former are more widely known and recognizable. We chose the Twitter Academic API for several reasons compared to other social media platforms like Facebook and Reddit. First, Twitter data can be either public or private, but, unlike Facebook, the default setting is public [57]. Second, Twitter is a unique and valuable data source because it is easily accessible, provides real-time content, and is rich in detail [58]. Additionally, Twitter allows for access to large datasets compared to other social media platforms such as Reddit [59]. TikTok and ChatGPT have both experienced explosive growth, quickly ascending to prominence among global audiences. When ChatGPT was launched, it quickly became the fastest-growing chatbot and application, surpassing even TikTok in popularity. Within just a few months, it attracted millions of users, and it garnered billions of visits within its first year [25,26]. In comparison to competitors like Google Bard (now Gemini), which receives about 330 million visits each month [60], ChatGPT significantly outpaces this with approximately 1.6 billion visits monthly [61]. We conducted a t-test comparing the Google trend data for ChatGPT, Microsoft Copilot, Google Bard, and Facebook Llama in 2023 [62]. The results showed that searches for ChatGPT were significantly higher (p-value < 0.05) compared to the other chatbots, indicating ChatGPT’s superior popularity. Figure 3 illustrates the Google search trend data normalized to a scale of 0 to 100 to represent the relative popularity of search terms over a specific time period. This means that a value of 100 represents the peak popularity of that search term within the selected time frame. A value of 50 means that the term was half as popular as at its peak.

Among social media platforms, Twitter is a popular platform for research because of the following reasons. First, Twitter has over 300 million monthly active users, generating more than 500 million tweets per month [63,64] to share health information and advice [65]. Second, Twitter has provided Application Programming Interfaces (APIs) to facilitate data collection on a large scale [66] and has earned the focus of several studies to address a wide range of research questions in different applications such as health and politics [67]. Third, Twitter is by far the most extensive and widely accessible public source of online data for studying human behavior and social interactions [68]. The surge in user-generated content on Twitter has significantly impacted the field of communication. Twitter data have been used as a popular cost-effective source of data for studying different phenomena, such as common health issues [69], disaster communication [70], and politics [71].

Twitter content is valuable for analyzing how the public shares and disseminates information about health. In previous studies, Twitter data have been analyzed to understand the conversations surrounding different issues, such as cancer [72], COVID-19 [73], and HIV [74]. These studies began by harvesting data from Twitter using a specified search query (e.g., #HIV) over a defined period (e.g., November 2018–January 2019). To ensure data quality, several preprocessing steps were implemented. Initially, duplicate tweets, retweets, brief messages, and tweets not in English were filtered out. Moreover, an additional preprocessing layer, such as the removal of commonly occurring but irrelevant words (stop words), was employed to refine the dataset. This entire process was likely facilitated by Python packages such as Tweepy for data collection and Pandas for data manipulation, as well as R packages like twitteR and dplyr.

To ensure the quality and reliability of our data, we collected tweets posted between November 2022 and March 2023 via Twitter’s Academic API in R using the academictwitteR package. The chosen timeframe ensured the reliability of our dataset by covering a continuous and recent period, capturing trends in discussions about ChatGPT technologies. This extended collection period also allowed us to accumulate a substantial volume of data, essential for statistical robustness and mitigating the impact of outliers in the analysis. We employed a carefully constructed query designed to target specific discussions relevant to our study on ChatGPT technologies. The query used was: “((GPT AND chat) OR (chat AND GPT) OR GPTchat OR #GPTchat OR chatGPT OR #chatGPT OR GPT3 OR #GPT3 OR GPT3Chat OR #GPT3Chat OR GPT) AND (asked)”, focusing on capturing conversations explicitly asking about or referencing GPT-related topics. This approach ensured that the data were highly relevant to our research questions. To enhance data quality, we applied stringent filters to exclude non-English tweets using lang:en, retweets using -is:retweet, comments posted by verified users using -is:verified, and duplicates using unique() function in R. The exclusion of non-English tweets allowed for a uniform dataset in terms of language, facilitating more consistency and accurate text analysis and interpretations. Analyzing documents in multiple languages would require complex language processing tools and translation services, which could introduce variability and potential inaccuracies in the results. Removing retweets and duplicates was crucial to ensure that our analysis was based on original content, providing a clearer picture of genuine user engagement without artificial inflation from repeated posts. Removing verified users from Twitter data is crucial to minimize bias and better represent the opinions of the general public, as these accounts often belong to influential figures whose views may not reflect those of ordinary users. The resultant dataset comprised 79,234 tweets posted by 61,217 unique users, representing 1.29 tweets per user (Figure 3), which were rigorously collected and filtered to meet high standards of data integrity. This large volume of data provided a robust foundation for our analysis, ensuring that our findings would be based on a comprehensive and reliable sample of the discourse surrounding GPT technologies during the specified period. These measures collectively bolster the reliability and validity of this study, affirming that the conclusions drawn are well supported by high-quality empirical evidence.

To identify health-related tweets, we used the LIWC (Linguistic Inquiry and Word Count) 2015 software, which has a health detection category to recognize tweets containing health-related terms. LIWC is a lexicon-based tool developed to measure the psychological, emotional, and linguistic components contained within written texts. It categorizes words into predefined groups that reflect different dimensions such as emotions and health. LIWC operates by counting words in the categories and computing percentages, thus offering researchers and professionals a rapid, reliable, and data-driven method to analyze language in various contexts [75]. Figure 4 illustrates the process used to collect and identify tweets containing health-related terms, as well as the monthly tweet frequency. This figure reveals an upward trend in the volume of tweets over the studied four-month period.

After excluding short tweets with fewer than five words, we identified 4537 tweets containing health-related terms, posted by 4355 users, averaging 1.09 tweets per user. On average, these tweets included 38.66 words, 0.79 hashtags initiated by ‘#,’ and 0.62 mentions beginning with ‘@’. Figure 5 shows the distribution of the number of words in the tweets. For example, 174 tweets contained 44 words.

Figure 6 depicts a Zipf’s law distribution for the word frequencies, where the y-axis measures the frequency of each word and the x-axis represents their rank. Zipf’s law is an empirical rule that suggests that the frequency of words has an inverse relationship with their frequency rank [76]. The steep decline from left to right illustrates that the most frequent words are used significantly more often than less frequent ones, showing the inverse relationship between word frequency and rank. A red vertical line marks the threshold for the 50 most commonly used words, emphasizing their higher frequency relative to others in the corpus.

2.2. Initial Coding

In CGT, the initial coding step is the first exploratory stage of coding qualitative data to categorize data. This step helps the researcher to identify patterns within the data, which can be further refined and explored in subsequent stages of the grounded theory process. This study used topic modeling for the initial coding process of CGT to identify main codes (topics) within the data [52]. Topic modeling techniques utilize statistical machine learning algorithms to identify abstract themes present within a collection of documents [77]. Topic modeling can mine large corpora and inductively generate code among the corpora and identify patterns that might be missed with manual coding to render knowledge [78]. Prior research has shown that topic modeling can be used as an initial coding step in the coding of large corpora [56] and produce similar results to traditional grounded theory [79].

This research adopted the Latent Dirichlet Allocation (LDA) technique [80], which is a method commonly used for topic modeling [81] to learn latent structural elements (topics) of a corpus. LDA is an unsupervised method used to discover hidden topics within a large collection of documents without any prior annotations. Various methodologies for identifying topics within a corpus have emerged, leveraging both neural network architectures and statistical distributions. Over the past decade, neural networks, including deep neural networks (DNNs), have garnered attention for enhancing data analysis techniques [82]. However, neural network-based models are often resource-intensive and time-consuming [83], necessitating substantial amounts of training data, and are also susceptible to overfitting issues [84]. Furthermore, deep learning approaches have not consistently outperformed alternative methods when applied to small or medium-sized corpora [83]. In contrast, LDA has emerged as a widely accepted and utilized model for topic discovery within a corpus. The literature underscores LDA’s validity [85] and widespread adoption [86].

LDA operates under the assumption that words and documents interact within a corpus, represented through a bag-of-words model that reduces the complexity of text data by viewing documents as mere sets of words, ignoring both their sequence and contextual relationships. In this model, corpora are quantified into vectors where each element denotes the occurrence or count of words. LDA iterates thousands of times over the documents, analyzing the topic distribution of each word token. Words that commonly appear together are likely to be grouped under the same topic, enabling the model to automatically identify semantically related words. This grouping offers a summary view of the corpus’s contents and serves as a reference for identifying documents that predominantly feature specific topics. The corpora undergo iterative sampling, with the model adjusting topic–word associations to better reflect their underlying latent distributions. These collections of words, or “topics”, are subsequently interpreted by humans as significant “themes” based on intuitive understanding [80] in the focused coding step. For instance, if words like “gene”, “dna”, and “genetic” are grouped together in a topic, one might interpret and label this topic as “genetics” [87].

In our study, we treated tweets as documents, henceforth using the terms “tweet” and “document” interchangeably. LDA estimates the probability of each set of words belonging to each topic and the probability of each topic being associated with each document. In essence, LDA quantifies the relationship between topics and documents, denoted as P(Topic|Document), as well as between words and topics, represented as P(Word|Topic).

The topics were represented by the top words for each topic, ranked in descending order of their probability, P(Word|Topic). We developed some preprocessing steps including removing punctuation, stop words, and numbers. To assess various models and their learning processes in LDA, we examine different sets of topics and iterations. As an unsupervised method, topic modeling groups similar documents into a predetermined number of topics. Therefore, identifying the optimal number of topics for the corpus is a crucial step. We used a density-based method to determine the optimal number of topics, resulting in 53 topics [88]. This method indicates that the LDA model achieves optimal performance when the average cosine distance between topics is at its minimum. The density-based method helped us to compare different models to identify the best set of topics. We used Mallet, a Java-based tool, with $\alpha ={\displaystyle \frac{5}{numberoftopics}}=0.09$ controlling the per-document topic distributions, $\beta =0.01$ influencing the topic–word distributions, and 4000 iterations. To ensure model stability, we performed three independent LDA runs. No statistically significant differences (p-value < 0.05) were observed between the runs, indicating the robustness of the LDA learning process model, as shown in Figure 7. This figure also shows that the log likelihood started to plateau after 3000 iterations, meaning that there were minimal changes between iterations; this suggests that the model reached convergence.

2.3. Focused Coding

The goal of focused coding is to synthesize and condense initial codes into broader categories [52]. In the initial coding phase, LDA categorized large volumes of data into topics representing categories of tweets. The subsequent step after initial coding was focused coding, which was employed to identify and interpret meaningful and relevant topics. LDA offers two main outputs: the first output is the probability of each word per topic, or $P\left(W_i\right|T_k)$, while the second output is the probability of each topic per document, or $P\left(T_k\right|D_j)$ [80]. Each document represents a tweet in this study. Following [89], we investigated the top 10 words per topic based on the descending order of $P\left(W_i\right|T_k)$. The top 10 words helped us to efficiently grasp the essence of the topics without being overwhelmed by too much information. It also ensured that the most relevant and frequently occurring words were highlighted, providing a clear and concise representation of each topic’s thematic content.

The focused coding procedure involved two critical steps. First, coders closely examined the top 10 words in each topic to address two specific questions. The first question aimed to evaluate the coherence and meaningfulness of the topic, ensuring that it formed a logical and understandable theme. The second question scrutinized whether the topic was pertinent to health issues, aligning with the research’s thematic focus. This thorough examination involved assessing how well each topic mirrored actual health-related use cases, confirming their direct applicability and relevance to the field. Coders also evaluated the authenticity of the topics, ensuring that they accurately represented genuine health discussions. Furthermore, the consistency of the topics was scrutinized across different data segments to confirm that they reliably conveyed the same thematic elements throughout the dataset. This multi-faceted approach ensured a comprehensive validation of each topic’s meaningfulness, relevance, and reliability within the health context. This dual-question approach helped us to meticulously filter out any topics that were either irrelevant or did not contribute meaningful insights. Through this detailed and selective process, the coders effectively narrowed down the topics to a final set of nine, each of which was both substantively meaningful and closely related to health topics.

In the second step of the focused coding process, we engaged in a detailed and systematic iterative examination of each topic. This involved alternating between the top 10 words of each topic and the top 20 most relevant documents (tweets), like in [90], which were ranked based on the descending order of $P\left(T_k\right|D_j)$. While other studies [91,92] have chosen to analyze the top 5 or top 10 documents to interpret topics, we opted for the top 20 to gain a more comprehensive understanding of each topic. This meticulous process allowed us researchers to delve deeply into the semantics and contextual usage of words within the tweets in order to fully grasp the underlying concepts of the nine topics and assign them clear and descriptive labels. Focusing on the top 20 most relevant documents per topic balances depth, relevance, and efficiency. This approach ensures that the analysis is both thorough and manageable to deeply explore the most significant content without being overwhelmed by less relevant material.

Our aim was not just to identify the predominant themes but also to understand how these themes were represented in the data. This back-and-forth process, as part of a reflective approach, facilitated a nuanced understanding, helping us to identify the meaning of the topics within the tweets and to assign a label to each topic. For instance, when the top 10 words of a topic included “advice”, “give”, “therapist”, “specific”, “job”, “personal”, “therapy”, “prompt”, “ways”, and “professional”, the coders labeled this topic as “Seeking Advice”. This label not only captures the essence of the topic but also reflects its practical significance in the context of the tweets. This method ensured that each topic was not only identified but also labeled in a manner that was useful for subsequent analysis and interpretation. To calculate the weight (popularity) of each topic, we measured the normalized weight of each topic with the following formula for n documents and k topics, where $T_k$ represents topic k and $D_j$ denotes document j:

$$NW(T_k)={\displaystyle \frac{\sum _{j=1}^{n}P\left(T_k\right|D_j)}{\sum _{k=1}^t\sum _{j=1}^{n}P\left(T_k\right|D_j)}}$$

2.4. Theoretical Coding

Theoretical coding is an advanced stage in the grounded theory methodology, where researchers conceptualize the relationships between codes or themes derived from the data. This stage extends beyond initial coding and focused coding to integrate these codes into a cohesive theoretical framework that is grounded in the empirical data collected [52]. The theoretical coding step in this study involved identifying relations between the labels of the topics by performing thorough zoom-ins and zoom-outs on each topic [53] and merging topics into dimensions or constructs [52]. This abstraction process included revisiting the top 10 words and the top 20 tweets along with searching for similarities and differences to recognize a meaningful link between the labels of the nine topics. Coders systematically evaluated each topic for its empirical grounding and relevance to the broader constructs, ensuring that each category was not only reliably consistent across different datasets but also comprehensive in covering the thematic scope of the study. This included an iterative review process where topics were continually reassessed and refined to ensure they accurately reflected the underlying data while maintaining logical coherence within the emerging theoretical framework. This meticulous approach guaranteed that the final constructs were both robust and representative of the diverse elements present in the health discussions captured in our data. This process led to the merging of the 9 labels into 4 constructs. For instance, coders categorized the topics labeled “Cancer” and “COVID-19” under a broader construct named “Diseases”. The weight of each construct was calculated by summing the weights of all topics that belong to construct $C_d$ using the following formula:

$$\mathit{W}({\mathit{C}}_{\mathit{d}})=\sum _{\mathit{k}\in {\mathit{C}}_{\mathit{d}}}\mathit{N}\mathit{W}\left({\mathit{T}}_{\mathit{k}}\right)$$

3. Results

The topic modeling analysis of tweets in

Table 1. Three steps of constructivist grounded theory.

Initial Coding	Focused Coding	Theoretical Coding
advice, give, therapist, specific, job, personal, therapy, prompt, ways, professional	Seeking Advice	Clinical Workflow
interesting, work, find, writing, clinical, change, thinking, made, essay, info	Clinical Documentation
doctor, medical, patients, diagnose, found, provide, information, accurate, medicine, symptoms	Medical Diagnosis
bad, high, heart, pain, low, hard, years, disease, treatment, chronic	Medical Treatment
plan, workout, diet, create, meal, week, make, day, fitness, exercise, based, give	Diet and Workout Plans	Wellness
health, mental, people, data, based, generate, public, care, physical, improve	General Health
cancer, cure, ill, similar, current, alcohol, important, risk, science, thinks	Cancer	Diseases
COVID, response, vaccine, immunity, disease, virus, symptoms, infection, prevent, pandemic	COVID-19
physical, human, men, woman, body, women, gender, abortion, pregnant, man	Anatomical Differences	Gender Identity

shows the nine health-related topics within the corpus, each characterized by its top 10 words. Below, we briefly explore each topic and its category, providing insights into the diverse range of questions Twitter users have posed to ChatGPT.

3.1. Clinical Workflow

We begin the qualitative analysis with a discussion of the biggest group of topics. This analysis grouped four topics as processes/steps of the Clinical Workflow category. The first topic, named Seeking Advice, involves a range of inquiries from users on various issues, prominently featuring health-related concerns such as allergies (“give us an advice on how to help someone who is having an allergic reaction”). The nature of these prompts indicates a perception among users of generative AI as a source of authoritative guidance, akin to consulting with a clinician. Users in this group expressed their experience in using AI for personalized recommendations and insights into managing their health conditions, indicating a level of trust in the AI’s capability to provide accurate and helpful information. Additionally, this category includes questions about health-related professions, with users seeking guidance on career paths and professional development within the healthcare sector. These users asked generative AI to act as a mentor, offering strategies to navigate the complexities of healthcare careers. This trend underscores the expanding role of AI as a versatile tool for both personal health management and professional advancement in the medical field.

The second topic in this category is Clinical Documentation. This topic encompasses inquiries related to managing clinical documents, focusing on tasks like drafting reports for clinical trials and organizing data from multiple clinical studies. Examples of such inquiries included tasks like “writing a phase 3 clinical trial report” or “compiling data from clinical trials”. These tasks are essential in clinical research, where detailed documentation and rigorous analysis are required to ensure the accuracy and reliability of trial outcomes. Meanwhile, compiling clinical trials entails the systematic gathering and synthesis of data across various studies to aid in comprehensive reviews and meta-analyses, essential for shaping evidence-based medical practices and policies.

The next topic is Medical Diagnosis, involving inquiries directed towards obtaining diagnoses based on medical information provided to AI. Users input symptoms, medical histories, or specific health concerns with the intention of receiving an analysis or identification of potential health conditions. These users are not limited to patients and include medical providers too. For example, one user stated “I’m an ER doctor… I asked ChatGPT to diagnose my patients”.

The final topic in the Clinical Workflow category is Medical Treatment. This topic revolves around inquiries concerning the treatment of various diseases, with a particular focus on conditions like Alzheimer’s disease and other chronic diseases. Questions in this realm often explore the efficacy of different treatment modalities, including conventional medical interventions, lifestyle changes, and alternative therapies. For instance, a growing interest was found in understanding how dietary modifications can play a role in managing or potentially alleviating symptoms of chronic conditions. Discussions in this topic might delve into specific dietary approaches, such as anti-inflammatory or plant-based diets, and their impact on disease progression, symptom management, and overall health outcomes.

3.2. Wellness

Two topics were assigned to this category. The first one is Diet and Workout Plans. This topic includes users requesting the generative AI to devise both general and tailored diet and exercise regimes, ranging from specific dietary programs like a “7-day high protein vegan meal plan” to detailed fitness schedules such as an “18-week marathon training plan for an intermediate runner”. These inquiries highlight the user perception of generative AI as a virtual coach capable of supporting their health and fitness aspirations. By soliciting personalized advice, users demonstrate their trust in the AI’s ability to understand their unique dietary preferences, nutritional requirements, and fitness levels, and to translate these into actionable, effective plans. This reliance on AI for customized health guidance reflects a broader trend of utilizing technology to achieve personal wellness objectives, showcasing AI’s potential to provide informed, adaptable, and accessible coaching solutions for diverse health and fitness goals.

General Health is the second topic in the Wellness category. This topic encompasses a broad spectrum of inquiries related to both physical and mental health, offering insights into how lifestyle changes and personalized wellness practices can profoundly impact one’s well-being. For instance, one user asked ChatGPT for “examples of people who improved their health by going plant-based”. Such conversations illuminate the potential benefits of dietary choices on physical health, including weight management, improved heart health, and a reduced risk of chronic diseases. Similarly, the Wellness category delves into tailored approaches to mental wellness, such as “Customized meditation or relaxation scripts for stress reduction focus or sleep assistance”. These topics underscore the importance of personalized health strategies that cater to the unique circumstances and goals of individuals, demonstrating the interconnectedness of physical and mental health in achieving overall well-being.

3.3. Diseases

The Diseases category includes two topics/diseases: COVID-19 and Cancer. The COVID-19 topic encompasses a broad range of prompts related to the COVID-19 pandemic, delving into critical issues such as the effectiveness of face masks in preventing virus transmission, the efficacy and safety of vaccines in building immunity, the exploration of various treatment options for those infected, and the concept of herd immunity as a potential strategy to curb the spread of the virus. It also covers discussions on identifying and understanding the symptoms of COVID-19, thereby enabling early detection and treatment. Within this context, questions may seek to explore scientific evidence and research findings that support or challenge the use of specific preventive measures, therapeutic drugs, or vaccination campaigns, as well as the broader implications of these strategies on public health and safety.

The second topic within this category delves into inquiries related to cancer. This topic encompasses a wide array of discussions, from the impact of dietary factors, like “the effects of dietary iron in cancer risk”, to the exploration of innovative treatments such as “What frequency kills cancer”. Questions within the Cancer topic seek to understand the complex interplay between lifestyle choices, environmental factors, and cutting-edge medical interventions in the prevention, development, and treatment of cancer.

3.4. Gender Identity

Our qualitative analysis of the questions posted on social media identified one topic pertaining to differences in sex organ anatomy across genders, which was named Anatomical Differences and grouped as a Gender Identity category. This topic encompasses inquiries from users about the distinctions between the sexes, particularly focusing on biological and physiological aspects that differentiate females and males. Questions within this category are diverse and range from foundational inquiries such as “What is a Woman?” to more complex and contemporary topics such as “Can a man be pregnant?”. These questions often delve into the scientific/political/social understanding of sex and gender, exploring the chromosomal and hormonal differences that traditionally define males and females.

The most discussed topic was Diet and Workout Plans, while the least discussed one was COVID-19 (Figure 8). Clinical Workflow received the highest weight, followed by Wellness, Diseases, and Gender Identity (Figure 9). The weight value represents the popularity of that category or its topics.

4. Discussion

Social media serves as a communication channel for disseminating health information and can also be used to monitor technology use cases. The study presented here investigates how health use cases of ChatGPT are represented, exploring their various categories and levels of popularity.

4.1. Theoretical Contributions

To the best of our knowledge, this research is the first to identify and code social media discussions on health prompts from an AI chatbot (i.e., ChatGPT). To achieve this objective and to fill the gap in the literature, we developed a data collection and processing framework and embedded topic modeling into CGT to categorize Twitter discourse on health use cases of ChatGPT. The theoretical contributions of this paper are three-fold. First, this study shows the utility of social media data for investigating the health use cases of a chatbot. While most of the existing literature primarily lacks systematic research on the health use cases offered by chatbots developed by generative AI [29], we grounded our study in social media data to directly obtain public prompts representing health use cases. This paper demonstrates the strengths of social media data for mining public experiences with a chatbot.

Second, this research offers a new approach using topic modeling and CGT to understand how users interact with a chatbot using health-related prompts. Using computational methods and qualitative coding provides a holistic identification and interpretation of social media comments on health-related prompts given to chatbots. Our analysis on Twitter conversations surrounding interactions with an AI-based chatbot disclosed topics through the initial coding step. The focused coding processes narrowed down the topics and identified nine topics relating to health issues. The theoretical coding step enabled the research team to categorize these nine topics into four categories: (1) Clinical Workflow, (2) Diseases, (3) Wellness, and (4) Gender Identity.

Third, this work surpasses the limitations of existing studies on the potential uses of AI-based chatbots for health applications and highlights the importance of understanding the possible use cases expressed in social media comments. The results show that prompts could be on a wide range of issues from Gender Identity to Clinical Documentation, which has not been detected in previous social media studies [30,48,93]. The diversity of topics and categories in this study reflects health information needs and health information-seeking behaviors among the public, which could suggest a theoretical framework for understanding how AI platforms are used in health information seeking compared to traditional sources. In line with non-social-media studies, this study demonstrates that AI chatbots are versatile tools utilized in wellness-related behaviors, in the management of various diseases, and in clinical workflows, including specifics such as mental health [42], cancer [34], healthy lifestyles [43], medical treatments [47], and clinical documents [31]. While non-social-media research primarily concentrates on perspectives from healthcare providers (e.g., [31]), this study reveals that the four identified categories of topics are prevalent on social media and are largely discussed by ordinary users. This shift illustrates how everyday users actively leverage AI chatbots for multiple health-related purposes: they seek advice, request medical diagnoses and treatments, explore options for mental and physical well-being, and pose detailed questions about human anatomy. This broader engagement highlights a significant expansion in the use of AI chatbots beyond professional medical contexts, making health information more accessible to the general public.

The diversity of questions posted on social media shows the willingness of users to ask a chatbot about serious health issues (e.g., the Cancer and Medical Treatment topics) and sensitive concerns (e.g., mental health). This indicates that users have developed the required level of trust in AI to accept it as a credible source of health information. This has implications for research related to trust in digital health technologies and the perceived credibility of AI-generated prompts. This study’s ranking of topics and their categories can help researchers to develop studies that work towards understanding the information-seeking behavior of users and the socio-technical factors that influence the prioritization of certain topics over others and how these preferences shift over time.

4.2. Implications for Practice

The results of this research can provide practical implications for the health sector to inform the development of more sophisticated, responsive, and ethically aware AI systems. First, our results open a new direction for the health industry and startups to develop AI applications for health needs. This study reveals the potential of AI chatbots for disease analysis and preventive/diagnostic/treatment procedures. This means that AI chatbots can be used for different health applications and decision-making processes, such as analyzing historical data (e.g., clinical documents), providing health consultant (e.g., diet plans), developing precision medicine (e.g., treatments), and interpreting diagnoses (e.g., assessing pain). Our findings suggest a demand for personalized health information. This could drive the development of AI systems capable of providing more tailored health advice based on users’ prompts and, potentially, their clinical conditions.

Second, the diversity of categories and topics detected in this study suggests a need for AI chatbots to provide accurate responsible responses. Topics such as COVID-19 and Cancer are susceptible to misinformation. Monitoring questions on these topics can help identify prevalent myths or misinformation, allowing health authorities to address these directly through public information campaigns and AI chatbots. The diversity also highlights the need for clear policies and regulations regarding the use of AI in providing health information. Understanding popular topics can guide innovation in medical technology and the development of new tools that align with public interest or needs.

Third, our findings show the potential of chatbot prompts and relevant social media data for two purposes. First, data from chatbot prompts and social media can aid in monitoring digital epidemiology, emerging health concerns, misinformation trends, and public information needs. Second, social media discussions can help chatbot developers to identify possible problematic responses given by chatbots.

While the focus of this study is on health use cases of AI chatbots, there are some practical ethical challenges. The first challenge is the legal issues posed by AI chatbots, such as the privacy risk involved in scraping large amounts of data that may contain personal data. The second challenge is the humanistic concerns, such as the loss of human touch and the lack of empathy in AI chatbots. Another challenge is presented by algorithmic issues, such as algorithmic biases (e.g., producing discriminatory outcomes). The last challenge involves information concerns, such as concerns about the validity of AI-produced information, referring to its accuracy; the reliability of AI chatbots regarding the possibility of them being manipulated to produce misinformation; and the appropriateness of the information provided by AI chatbots [94,95]. These challenges highlight the need for developing comprehensive ethical guidelines to support all health stakeholders in the effective use of AI chatbots.

4.3. Limitations and Future Work

This study has a few limitations. First, our data collection might have excluded relevant tweets that did not contain our query. Second, we excluded non-English tweets that might have added more dimensions to our analysis, and this could have imposed an unknown sampling bias. Third, focusing on ChatGPT prompts on Twitter during the COVID-19 pandemic introduces some potential selection biases that could affect this study’s generalizability. First, as we concentrated solely on Twitter, this research may not capture broader user interactions that could be prevalent on other platforms, given Twitter’s unique demographic and user behavior. The majority of Twitter users fall within the 18 to 49 age range. Individuals within this demographic tend to be healthier than older adults, have lower rates of geriatric medical conditions, possess higher levels of education, and earn higher incomes [96]. Second, while ChatGPT is the most popular generative AI chatbot, with 1.54 billion visits per month and over 180.5 million monthly users in the first half of 2024 [97], our analysis being limited to ChatGPT might mean that it does not represent interactions with other AI chatbots that may be configured differently or engage users in diverse ways. There has been an update for ChatGPT (ChatGPT-4), alongside the emergence of other AI chatbots like Microsoft Copilot, Facebook Llama, and Google Bard (now Gemini), all proposed in 2023 [98]. Additionally, conducting this study while the COVID-19 pandemic was ongoing could have skewed the data, as changes in the patterns of social media discussions during this time would not necessarily be indicative of standard behavior. Future work can address the limitations of this study, including expanding the query, including non-English tweets, inferring the demographic information of users, and studying other social media platforms (e.g., Reddit) and AI chatbots (e.g., Google Gemni). In addition, exploring each of the identified topics and categories could empower the theoretical contributions and practical implications of this study’s findings.

5. Conclusions

This study establishes a comprehensive framework designed to gather and categorize tweets related to health-related inquiries directed at the ChatGPT chatbot. We utilized a dual-method approach, incorporating both topic modeling and grounded theory techniques, to thoroughly analyze the content of these tweets. This approach enabled us to delve deeply into how the public leverages this chatbot to address their health questions and requirements. Our analytical process revealed four distinct categories of prompts: Clinical Workflow, Diseases, Wellness, and Gender Identity, ranked in order of decreasing popularity. The results of this research underscore the significant potential of social media platforms as tools for identifying and exploring health use cases and the practical applications of AI-based chatbots in the health sector. These insights are particularly valuable for a diverse array of stakeholders, including researchers, healthcare professionals, technology developers, and policymakers, who are involved in evaluating and leveraging the health applications of chatbots. This study not only aids in understanding current interactions between users and health chatbots but also contributes to strategic planning and development within health technology fields, potentially guiding future enhancements to chatbot functionalities in order to better meet users’ needs.