Application of Machine Learning and Natural Language Processing Techniques for the Analysis of Surveys with Open-Ended Questions: A Scoping Review

Abstract

The use of open-ended survey questions for data collection has increased significantly across various areas, as has the application of machine learning (ML) and natural language processing (NLP) techniques to analyze respondents’ opinions. In this study, we conducted a scoping review of 79 studies that analyze open-ended answers given in surveys. We structured our review around six main criteria: application of supervised learning, unsupervised learning, Supervised Descriptive Rule Discovery (SDRD), open-ended questions, NLP, and opinion comparison. This approach allowed us to identify the most used tasks, algorithms, and technologies in ML and NLP, revealing areas of opportunity and the main future challenges. We based our review on the methodological framework of Arksey and O’Malley and adapted PRISMA for reporting systematic reviews. Our findings suggest that most studies addressing surveys with open-ended questions were published in 2020 and 2022, predominantly focusing on research and health domains.

1. Introduction

Artificial intelligence (AI) is defined as the analysis of agents that perceive data from the environment and carry out actions; in turn, each agent has a specific function. AI focuses on developing algorithms and models that allow machines to learn, solve problems, and even reason to make decisions, all autonomously. There are six main areas within AI: (1) NLP, (2) Computer vision, (3) Robotics, (4) Automated reasoning, (5) Knowledge representation, and (6) ML [1]. For this review, we will focus on ML and NLP.

ML is the ability of computers to mimic human behavior; in other words, it is the field of study that allows computers to learn without the need for rigorous programming [2]. Within ML, there are different types of learning, including supervised learning, which learns from input data to predict an output, i.e., it works with previously labeled data; unsupervised learning, on the other hand, does not have labeled data but can learn from it to predict the next output [1], and semi-supervised learning, which uses supervised and unsupervised learning techniques and is usually employed when there is a large amount of input data and little labeled output data [3].

On the other hand, supervised descriptive rule discovery (SDRD) is defined as a set of techniques based on rules for obtaining descriptive knowledge [4]. SDRD encompasses both types of learning: supervised and unsupervised. SDRD techniques are designed to uncover subtler and more complex patterns and relationships in data. The goal is to explore information that is hidden or difficult for experts to obtain about the value of a predefined category. This allows for greater accuracy in identifying influential factors and trends in survey responses [5].

Currently, in the field of research, there are various methods for collecting data. When it comes to gathering opinions, the most efficient method or tool for collecting data is a survey. Surveys have been used for several years and are a social research approach that involves gathering information from respondents through questions designed based on the purpose of the research or the purpose for which it was created. A survey can contain closed and open questions [6].

Open-ended responses represent a valuable and effective alternative for analysis for various reasons, including their breadth, as they allow respondents to express their opinions and comments in their own words, i.e., without restrictions imposed by predefined options, such as the Likert scale, to mention one example. This ensures that a wide variety of perspectives are captured in the responses. They also provide greater context and depth, i.e., they often explain why a respondent chose one or more specific options and provide additional information about their preferences and/or experiences. Open-ended responses often reveal unexpected situations, allowing researchers to discover new topics, trends, or issues that may otherwise go unnoticed in survey design. Another very important point is the focus on the respondent’s “opinion.” By allowing respondents to express themselves freely, open-ended responses place the entire focus on respondents’ opinions. This is especially valuable in fields such as customer service and product development, where understanding customer needs and desires is critical.

There are various studies related to the use of surveys, including studies aimed at promoting the use of surveys with open-ended questions, such as [7], where respondents had the freedom to express their opinions through open-ended questions, as the authors claimed they have multiple benefits in the data collection process, since this way the ones polled are not forced to agree with predefined confirmations. Likewise, Kosmajac et al. [8] presented a new method of topic modeling using graphs to represent text, overcoming analysis limitations and improving understanding of the opinions and feelings expressed by respondents. In addition, Baburajan et al. [9] focused on demonstrating the value of open-ended questions in surveys for obtaining more information by applying topic modeling and factor analysis. Similar works have focused on using surveys with open-ended questions and analyzing the opinions of those polled, applying NLP and ML techniques [10,11,12,13].

Talking about open-ended questions gives us an idea of how difficult it can be to process data, or rather, responses of this type. Currently, countless tools and methods make this process easier. An example of this is the work of Espinoza et al. [13], who proposed an application for extracting themes from responses to open-ended questions, thereby increasing the reliability of the data obtained and helping analysts carry out this process more quickly. Since manual coding often presents various difficulties, requires a lot of effort and human interaction, which is reflected in time and money costs, Gweon and Schonlau [14] conducted empirical research to evaluate the pre-trained language model Bidirectional Encoder Representations from Transformers (BERT) for automatic coding. This is where ML takes center stage, performing these activities more efficiently and with less effort.

Given this context, this study reviewed 79 papers on opinion comparisons considering NLP and ML, and the application of SDRD. These works spanned the period from 2014 to 2026. At the end of the analysis, it was found that, among the studies that used SDRD, several applied only some of the following tasks: Emerging Pattern Mining (EPM), Contrast Sets (CS), or Subgroup Discovery (SD). It was found that none of the studies used all three techniques together. Likewise, the development of a system that integrates these techniques with NLP for the analysis of surveys with open-ended questions was identified as an area of future opportunity, as it is an active field of research.

In the search for prior related work for this scoping review, it was identified that Kastrati et al. [15] conducted a scoping review on NLP, ML, and Deep Learning solutions for sentiment analysis in the educational domain. This study reviewed 612 papers and identified only 92 as relevant from 2015 to 2020. They also based their research on surveys with open-ended questions for student feedback on learning platforms, highlighting that the area is expanding, especially in distance learning. In [16], a review of trends and challenges in the use of AI, NLP, and open-ended questions was carried out. Although the authors did not specify how many papers were analyzed, the period covered by these papers was from 2017 to 2022. This study focused on reviewing the methodologies, challenges, trends, and adaptability presented by NLP. In [17], a cross-sectional online survey was conducted targeting physicians, educators, researchers, knowledge users, community organization representatives, graduate students, and policy stakeholders who had experience or interest in conducting outreach studies. This study was based on the application of 54 surveys, although it does not mention how many works were analyzed. The period of this study spanned from 2012 to 2016. The objective was to understand experiences and perspectives regarding the conduct and reporting of such studies.

In a previous study [18], we reviewed works that utilize SDRD in their investigations. This study includes a systematic review that identified 39 studies published from 2018 to 2022 incorporating CSM and SD techniques, concluding that the use of SDRD techniques enabled the identification of key features within a dataset of interest, to drive the adoption of rules that facilitate the understanding and interpretation of the obtained data. Also, after analyzing some review articles on SDRD [4,19] and natural language processing [20], we did not find any approach that integrates SD, EPM, and CSM for analyzing surveys with open-ended questions.

These related studies, compared to this comprehensive review, covered the period from 2012 to 2022, reviewing the existing literature. Although some reviews address certain criteria, such as NLP, AI, and open-ended questions, none address the review of previous studies that integrate both NLP and SDRD techniques, especially to compare results from surveys with open-ended questions. Also, none of the related review papers address the six search criteria taken into account for the research (use of supervised learning, unsupervised learning, SDRD, open-ended questions, NLP, and comparison of opinions). This scoping review aims to describe and understand the current state of ML and NLP techniques for analyzing open-ended survey responses, covering the period from 2014 to 2026. By combining SDRD and NLP techniques, it is possible to compare the results of surveys with open-ended questions for more efficient decision-making, reducing costs in terms of time and human resources.

The remainder of this document is organized as follows: Section 2 presents the materials and methods used to conduct the review. In Section 3, we present our results, while in Section 4, we discuss these findings. Finally, our conclusions are summarized in Section 5.

2. Materials and Methods

Our evaluation is based on the methodological approach proposed by Arksey and O’Malley [21] and adopts the PRISMA model presented by Levac et al. [22]. The scope of the research comprises five phases of development: (1) identifying the research questions, (2) identifying relevant studies, (3) selecting relevant studies, (4) graphically representing the data, and (5) summarizing and reporting the findings.

2.1. Research Questions

We developed six research questions that structured our scoping review, supporting us in achieving our research objectives and guiding us throughout the review process.

• RQ1. What types of machine learning are most used for data analysis in surveys with open-ended questions?
• RQ2. What are the most common ML and NLP tasks for analyzing surveys with open-ended questions?
• RQ3. What are the main ML algorithms applied for analyzing surveys with open-ended questions?
• RQ4. What are the most popular NLP models and architectures for analyzing surveys with open-ended questions?
• RQ5. What technologies are frequently employed for analyzing surveys with open-ended questions?
• RQ6. In which areas of research are surveys with open-ended questions predominant?

2.2. Inclusion and Exclusion Criteria

In the initial stage of the research search process, we defined the digital libraries we searched for related works, including IEEE Xplore, ACM Digital Library, Springer Nature Link, ScienceDirect (Elsevier), and Wiley Online Library. We also used Google Scholar to expand our search. In the second phase of the search strategy, a set of keywords was identified and used as search criteria to retrieve studies published between 2014 and 2026. It is worth noting that the search queries were consistent across all digital libraries; however, the syntax was adapted to the specific requirements of each database (for example, using specific field tags, identifying characters, or Boolean operator limits required by IEEE Xplore, ACM Digital Library, SpringerLink, ScienceDirect, Wiley and Hindawi, and Google Scholar).

Table 1. Keywords and related concepts.

Area	Keywords	Related Concepts
Surveys with open-ended questions	Analysis	Data mining, Data analysis, Artificial intelligence
	Machine learning
	Survey
	Open-ended question
	Open-ended
	NLP Natural language processing
	SDRD
	Supervised descriptive rule discovery
	Free text questions
	Open-ended answers
	Open-ended survey

details these keywords, which were used both individually and in combination with the connectors “AND” and “OR” to broaden our results.

The following queries were performed to search for primary studies in each selected repository.

• ‘Analysis’ AND ‘Machine learning’ AND ‘Survey’ AND (‘open-ended question’ OR ‘open-ended’) AND ‘Natural language processing’. Analysis of the preliminary results of this query revealed relevant search terms related to different applications of machine learning in survey analysis. Query 2 includes these search terms to broaden the identified relationship.
• (‘Machine learning’ OR ‘Supervised descriptive rule discovery’ OR ‘SDRD’ OR ‘Natural language processing’ OR ‘NLP’) AND (open-ended question’ OR ‘open-ended’ OR ‘free text questions’ OR ‘open-ended answers’ OR ‘open-ended survey’).

Once all relevant studies had been gathered, they were included: (1) if they addressed NLP or ML techniques and utilized surveys with open-ended questions, as this allows us to obtain highly complex unstructured data; (2) if they were published in English between 2014 and 2026; the time frame was defined to capture the evolution of the field, thereby ensuring that the state of the art remains current; the language was chosen because English is the universal language, thus guaranteeing access to globally validated methodologies and technologies; and (3) if they provided empirical results, allowing us to ensure that the results or conclusions of each study were not merely theoretical. Regarding the exclusion criteria: we excluded (1) studies not written in English, as mentioned above, in order to have access to globally proven technologies or methodologies and thereby ensure that all studies meet the same international review standards; we excluded (2) studies not relevant to the research questions to avoid diverting the research; and (3) we also excluded theses at any academic level to ensure that all literature has undergone a rigorous prior review process. This process was carried out and verified independently by more than one author. In the final phase, we used the PRISMA model as a guide to organize and present the results obtained.

2.3. Selection and Eligibility of Studies

Overall, 79 related articles were identified as relevant to this research after eliminating duplicates and analyzing each to obtain a summary and the most important points, such as the technologies used, year of publication, and NLP and ML tasks performed. The 79 selected articles were used to conduct the scoping review (Figure 1).

The methodology of this study was implemented following the methodological framework of Arksey and O’Malley, integrated with the PRISMA 2020 flowchart:

• Relevant studies were identified through an exhaustive search of five digital databases (IEEE, ACM, Springer, ScienceDirect, Wiley) and Google Scholar, yielding 29,288 initial records.
• Following the PRISMA flowchart, the selection was conducted in three phases. First, a manual review of titles from 16,808 records; next, a review of abstracts and keywords from 5111 records; followed by an evaluation of the full text of 96 reports to ensure eligibility.
• Studies that did not use at least one search criterion were not directly related to the application of surveys, or were theses of any level were excluded, ultimately yielding a total of 79 relevant studies.
• Finally, the results were compiled, summarized, and presented in graphical and tabular formats.

After identifying the final 79 articles, data were extracted using a standardized form that included: (1) bibliographic data, (2) the study’s contribution, (3) technologies implemented, (4) type of learning used, (5) machine learning and NLP tasks employed, (6) problem solved, (7) results obtained, (8) use of NLP, (9) use of open-ended questions, (10) comparison of opinions, (11) type of publication, and finally, (12) country.

This process was carried out by two reviewers (AOV, LRM), with the first reviewing and analyzing each study, while the second reviewed the results; any discrepancies were resolved through technical discussion, ensuring the integrity of the synthesized information.

After this step, we found that most works were published in journal articles, while a minority were published in conference proceedings, as shown in Figure 2. Regarding the trend in the years of publication of the articles, we found that most were concentrated in the years 2020 and 2022 (e.g., [8,23,24,25,26,27,28,29]). As shown in Figure 2, in these years, there were more works published related to survey analysis.

Figure 3 shows the geographical distribution of the studies analyzed, where we can see that most belong to the United States, followed by the United Kingdom, Switzerland, and China (for instance, [27,28,29,30,31]).

Figure 4 shows the distribution of the quality of the selected evidence; articles published in high-impact journals (Q1 and Q2) according to the Journal Citation Reports (Web of Science) predominate, ensuring a high level of methodological and scientific rigor in the analyzed evidence. Furthermore, the conferences included are mostly CORE (Computer Research and Education) A*, A, and B, ensuring the technological relevance of the examined ML and NLP proposals and supporting the technical validity of this review’s findings.

2.4. Data Collection and Analysis

Once we defined the main studies to be used for the review, their bibliographic and content data were retrieved, and an analysis was performed, taking into account authors, main contributions, issues addressed, technologies and algorithms used, main results, and whether they utilized NLP and/or open-ended questions.

3. Results

The results of this review are divided into two phases: state-of-the-art analysis and technology analysis.

Results of the State-of-the-Art Analysis

Table 2 below list each article and its compliance with the following criteria: (1) Use of supervised learning (SL); (2) Application of unsupervised learning (UL); (3) Focus on SDRD, here not only are those that employ this technique, but it is also specified which tasks were performed: Emerging Pattern Mining (EPM), Subgroup Discovery (SD) or Contrast Set Mining (CSM); (4) Consideration of open-ended questions (OQ); (5) Utilization of NLP, (6) Take into account opinion comparison (OC), and (7) Number of Surveys Conducted (SC) where NS means Not Specified.

The state-of-the-art analysis revealed the following: As shown in Table 2, Duan et al. [32] meet all the comparison criteria; however, they implemented only EPM. None work performed all three SDRD tasks (EPM, CS, and SD). Also, most related studies show a greater inclination to use supervised and unsupervised learning together to analyze open-ended survey questions. In addition, most approaches (45 works) analyzed more than 1000 survey participant answers.

Although EPM, CSM, and SD [4,18,19,27,32,33,34] algorithms have been widely applied to discover trends in data and solve real-world problems in domains like education, health, and commerce, Table 2 shows that only five works [18,27,32,33,34] applied them to analyze surveys with open-ended questions. Despite the significant advantages offered by each technique, none of the studies shown in Table 2 employ them in combination. One practical challenge that may limit their adoption is the overwhelming number of rules generated by SDRD algorithms when applied to large datasets. Nevertheless, several strategies for reducing the number of rules obtained have been developed [4,19,35].

Table 2. Comparative table of related works.

Article	SL	UL	SDRD	OQ	NLP	OC	SC
Singer and Couper [7]	NO	NO	NO	YES	YES	YES	NS
Kosmajac et al. [8]	NO	YES	NO	YES	YES	YES	50,792
Baburajan et al. [9]	YES	YES	NO	YES	YES	YES	364
Bardutz and Bigazzi [10]	NO	YES	NO	YES	YES	YES	366
Moreo, Esuli, and Sebastiani [11]	YES	NO	NO	YES	YES	YES	201–10,788
Schonlau and Couper [12]	NO	NO	NO	YES	YES	YES	1212 and 1758
Espinoza et al. [13]	NO	YES	NO	YES	YES	YES	9800–20,000
Gweon and Schonlau [14]	YES	NO	NO	YES	YES	YES	585–1212
Olmos-Vallejo et al. [18]	YES	YES	CS and SD	YES	NO	YES	NS
Jojoa et al. [23]	YES	NO	NO	YES	YES	YES	365
Zucco et al. [24]	YES	NO	NO	YES	YES	YES	169
Jayaratne and Jayatilleke [25]	YES	YES	NO	YES	YES	YES	46,888
He and Schonlau [26]	YES	YES	NO	YES	YES	NO	1096–1756
Ríos-Méndez et al. [27]	YES	YES	EPM	YES	NO	YES	7856
Supraja et al. [28]	YES	YES	NO	YES	YES	YES	39
Ferrario and Stantcheva [29]	NO	YES	NO	YES	YES	YES	5144
Spasić et al. [30]	YES	NO	NO	YES	YES	YES	55
Roberts et al. [31]	YES	YES	NO	YES	YES	NO	2323
Duan et al. [32]	YES	YES	EPM	YES	YES	YES	359–1091
Machorro-Cano et al. [33]	YES	YES	EPM	YES	NO	YES	7856
Olmos-Vallejo et al. [34]	YES	YES	SD	YES	NO	YES	289 and 47,093
Koufakou [36]	YES	YES	NO	YES	YES	NO	10,610
Haensch et al. [37]	YES	NO	NO	YES	YES	YES	5000
Jacennik et al. [38]	YES	NO	NO	YES	YES	YES	104
van Buchem et al. [39]	YES	YES	NO	YES	YES	YES	534
Nanda et al. [40]	NO	YES	NO	YES	YES	YES	130,500–158,000
Rubio Delgado et al. [41]	YES	YES	NO	YES	NO	NO	86
Schonlau et al. [42]	YES	NO	NO	YES	YES	YES	1006–2350
Smoll et al. [43]	NO	YES	NO	YES	YES	YES	723
Wang et al. [44]	YES	YES	NO	YES	YES	YES	402
Yawson et al. [45]	NO	NO	NO	NO	YES	YES	119
Pestian et al. [46]	NO	NO	NO	YES	YES	YES	30
Kjell et al. [47]	YES	YES	NO	YES	YES	YES	92–854
Robinson et al. [48]	YES	NO	NO	YES	YES	YES	9862
Tvinnereim and Fløttum [49]	YES	YES	NO	YES	YES	YES	4634
Maramba et al. [50]	YES	NO	NO	YES	YES	NO	3426
Guetterman et al. [51]	NO	YES	NO	YES	YES	NO	58 and 68
Hahn et al. [52]	NO	YES	NO	YES	YES	YES	3183
McGillivray et al. [53]	NO	YES	NO	YES	YES	NO	599
Nawaz et al. [54]	YES	YES	NO	YES	YES	YES	4400
Popping [55]	NO	NO	NO	YES	YES	NO	NS
Liu et al. [56]	YES	YES	NO	YES	NO	YES	1448
Jaeger and Rasmussen [57]	YES	NO	NO	YES	YES	YES	4341
Küfner et al. [58]	YES	NO	NO	YES	NO	YES	674,138
Mourtgos and Adams [59]	NO	YES	NO	YES	YES	YES	396
Yamano et al. [60]	NO	YES	NO	YES	YES	YES	107
Lasri et al. [61]	YES	YES	NO	YES	NO	YES	4388
Linton et al. [62]	NO	NO	NO	YES	YES	YES	5634 and 59,768
González Canché [63]	YES	YES	NO	YES	YES	YES	10,684
Marengo et al. [64]	YES	YES	NO	YES	NO	YES	5.048
Buenano-Fernández et al. [65]	NO	YES	NO	YES	YES	YES	900
Pietsch and Lessmann [66]	YES	YES	NO	YES	YES	YES	5001
Baumer et al. [67]	NO	YES	NO	YES	YES	NO	1095
Shiwaku et al. [68]	NO	NO	NO	YES	YES	YES	1.978
Koufakou et al. [69]	YES	YES	NO	YES	YES	YES	204
Etz et al. [70]	NO	YES	NO	YES	YES	NO	320,500
Costales et al. [71]	NO	YES	NO	YES	YES	YES	65
Zhang et al. [72]	NO	YES	NO	YES	YES	YES	168
Suadaa et al. [73]	YES	NO	NO	YES	YES	NO	19,944
Grönberg et al. [74]	NO	YES	NO	YES	YES	YES	742 and 6087
Singha and Mohapatra [75]	YES	YES	NO	YES	NO	YES	170
Mahendher et al. [76]	YES	NO	NO	YES	NO	YES	153
Kazi et al. [77]	YES	YES	NO	YES	YES	YES	124
Alshaikh et al. [78]	YES	NO	NO	YES	YES	YES	1.506
Wang [79]	YES	NO	NO	YES	NO	YES	1019
Pestian et al. [80]	YES	NO	NO	YES	YES	YES	371
Clark et al. [81]	YES	YES	NO	YES	YES	YES	4.068
Li et al. [82]	YES	YES	NO	YES	YES	YES	3336
Torrao et al. [83]	YES	YES	NO	YES	YES	YES	41
Kobra et al. [84]	YES	NO	NO	YES	YES	YES	400
Inoue et al. [85]	YES	YES	NO	YES	YES	YES	201
Nikulchev et al. [86]	NO	YES	NO	YES	YES	YES	20,443
Cook et al. [87]	YES	NO	NO	YES	YES	YES	1453
Onan [88]	YES	YES	NO	YES	YES	YES	154,000
Wijngaards et al. [89]	YES	NO	NO	YES	YES	YES	1122
Robin et al. [90]	YES	YES	NO	YES	YES	YES	13,705
Tvinnereim and Fløttum [91]	YES	YES	NO	YES	YES	YES	2115
Gish et al. [92]	NO	YES	NO	YES	YES	YES	510
Cheese et al. [93]	YES	YES	NO	YES	YES	YES	98

Table 2. Comparative table of related works.

Article	SL	UL	SDRD	OQ	NLP	OC	SC
Singer and Couper [7]	NO	NO	NO	YES	YES	YES	NS
Kosmajac et al. [8]	NO	YES	NO	YES	YES	YES	50,792
Baburajan et al. [9]	YES	YES	NO	YES	YES	YES	364
Bardutz and Bigazzi [10]	NO	YES	NO	YES	YES	YES	366
Moreo, Esuli, and Sebastiani [11]	YES	NO	NO	YES	YES	YES	201–10,788
Schonlau and Couper [12]	NO	NO	NO	YES	YES	YES	1212 and 1758
Espinoza et al. [13]	NO	YES	NO	YES	YES	YES	9800–20,000
Gweon and Schonlau [14]	YES	NO	NO	YES	YES	YES	585–1212
Olmos-Vallejo et al. [18]	YES	YES	CS and SD	YES	NO	YES	NS
Jojoa et al. [23]	YES	NO	NO	YES	YES	YES	365
Zucco et al. [24]	YES	NO	NO	YES	YES	YES	169
Jayaratne and Jayatilleke [25]	YES	YES	NO	YES	YES	YES	46,888
He and Schonlau [26]	YES	YES	NO	YES	YES	NO	1096–1756
Ríos-Méndez et al. [27]	YES	YES	EPM	YES	NO	YES	7856
Supraja et al. [28]	YES	YES	NO	YES	YES	YES	39
Ferrario and Stantcheva [29]	NO	YES	NO	YES	YES	YES	5144
Spasić et al. [30]	YES	NO	NO	YES	YES	YES	55
Roberts et al. [31]	YES	YES	NO	YES	YES	NO	2323
Duan et al. [32]	YES	YES	EPM	YES	YES	YES	359–1091
Machorro-Cano et al. [33]	YES	YES	EPM	YES	NO	YES	7856
Olmos-Vallejo et al. [34]	YES	YES	SD	YES	NO	YES	289 and 47,093
Koufakou [36]	YES	YES	NO	YES	YES	NO	10,610
Haensch et al. [37]	YES	NO	NO	YES	YES	YES	5000
Jacennik et al. [38]	YES	NO	NO	YES	YES	YES	104
van Buchem et al. [39]	YES	YES	NO	YES	YES	YES	534
Nanda et al. [40]	NO	YES	NO	YES	YES	YES	130,500–158,000
Rubio Delgado et al. [41]	YES	YES	NO	YES	NO	NO	86
Schonlau et al. [42]	YES	NO	NO	YES	YES	YES	1006–2350
Smoll et al. [43]	NO	YES	NO	YES	YES	YES	723
Wang et al. [44]	YES	YES	NO	YES	YES	YES	402
Yawson et al. [45]	NO	NO	NO	NO	YES	YES	119
Pestian et al. [46]	NO	NO	NO	YES	YES	YES	30
Kjell et al. [47]	YES	YES	NO	YES	YES	YES	92–854
Robinson et al. [48]	YES	NO	NO	YES	YES	YES	9862
Tvinnereim and Fløttum [49]	YES	YES	NO	YES	YES	YES	4634
Maramba et al. [50]	YES	NO	NO	YES	YES	NO	3426
Guetterman et al. [51]	NO	YES	NO	YES	YES	NO	58 and 68
Hahn et al. [52]	NO	YES	NO	YES	YES	YES	3183
McGillivray et al. [53]	NO	YES	NO	YES	YES	NO	599
Nawaz et al. [54]	YES	YES	NO	YES	YES	YES	4400
Popping [55]	NO	NO	NO	YES	YES	NO	NS
Liu et al. [56]	YES	YES	NO	YES	NO	YES	1448
Jaeger and Rasmussen [57]	YES	NO	NO	YES	YES	YES	4341
Küfner et al. [58]	YES	NO	NO	YES	NO	YES	674,138
Mourtgos and Adams [59]	NO	YES	NO	YES	YES	YES	396
Yamano et al. [60]	NO	YES	NO	YES	YES	YES	107
Lasri et al. [61]	YES	YES	NO	YES	NO	YES	4388
Linton et al. [62]	NO	NO	NO	YES	YES	YES	5634 and 59,768
González Canché [63]	YES	YES	NO	YES	YES	YES	10,684
Marengo et al. [64]	YES	YES	NO	YES	NO	YES	5.048
Buenano-Fernández et al. [65]	NO	YES	NO	YES	YES	YES	900
Pietsch and Lessmann [66]	YES	YES	NO	YES	YES	YES	5001
Baumer et al. [67]	NO	YES	NO	YES	YES	NO	1095
Shiwaku et al. [68]	NO	NO	NO	YES	YES	YES	1.978
Koufakou et al. [69]	YES	YES	NO	YES	YES	YES	204
Etz et al. [70]	NO	YES	NO	YES	YES	NO	320,500
Costales et al. [71]	NO	YES	NO	YES	YES	YES	65
Zhang et al. [72]	NO	YES	NO	YES	YES	YES	168
Suadaa et al. [73]	YES	NO	NO	YES	YES	NO	19,944
Grönberg et al. [74]	NO	YES	NO	YES	YES	YES	742 and 6087
Singha and Mohapatra [75]	YES	YES	NO	YES	NO	YES	170
Mahendher et al. [76]	YES	NO	NO	YES	NO	YES	153
Kazi et al. [77]	YES	YES	NO	YES	YES	YES	124
Alshaikh et al. [78]	YES	NO	NO	YES	YES	YES	1.506
Wang [79]	YES	NO	NO	YES	NO	YES	1019
Pestian et al. [80]	YES	NO	NO	YES	YES	YES	371
Clark et al. [81]	YES	YES	NO	YES	YES	YES	4.068
Li et al. [82]	YES	YES	NO	YES	YES	YES	3336
Torrao et al. [83]	YES	YES	NO	YES	YES	YES	41
Kobra et al. [84]	YES	NO	NO	YES	YES	YES	400
Inoue et al. [85]	YES	YES	NO	YES	YES	YES	201
Nikulchev et al. [86]	NO	YES	NO	YES	YES	YES	20,443
Cook et al. [87]	YES	NO	NO	YES	YES	YES	1453
Onan [88]	YES	YES	NO	YES	YES	YES	154,000
Wijngaards et al. [89]	YES	NO	NO	YES	YES	YES	1122
Robin et al. [90]	YES	YES	NO	YES	YES	YES	13,705
Tvinnereim and Fløttum [91]	YES	YES	NO	YES	YES	YES	2115
Gish et al. [92]	NO	YES	NO	YES	YES	YES	510
Cheese et al. [93]	YES	YES	NO	YES	YES	YES	98

Therefore, the lack of a unified framework that integrates EPM, CS, and SD with NLP suggests a missed opportunity for comprehensive knowledge discovery. Each task offers unique qualities that, when combined, provide robust analytical capability. For example, some studies [4,18,19,35] have reviewed the state of the art and highlighted the advantages of using SDRD. Also, according to Duan et al. [32], SDRD was essential for extracting relevant information from customer reviews to improve the online shopping experience. Beyond the focus on analyzing surveys with open-ended questions, we can mention two important studies with significant findings that give SDRD greater weight and prominence. Padilla et al. [94] presented the development of SDRDPy, a novel application for analyzing SDRD algorithm results. It stands out for its ability to process rules without the need for manual intervention. In addition, Carmona et al. [95] establish that the concept of SDRD is a category of data mining techniques that combines supervised learning for both predictive and descriptive purposes. This makes it possible to understand underlying phenomena in the data, regarding a variable of interest.

We therefore conclude that this gap represents an open research challenge in developing architectures, even new computational methods capable of transforming qualitative data into structured, descriptive evidence.

4. Discussion

4.1. Question 1: What Types of Machine Learning Are Most Used for Data Analysis in Surveys with Open-Ended Questions?

Regarding the results from the analysis of ML and NLP technologies,

Table 3. Types of machine learning.

Types of Learning	Total	Article
Supervised and Unsupervised	33	[9,18,25,26,27,28,31,32,33,34,36,39,41,44,47,49,54,56,61,63,64,66,69,75,77,81,82,83,85,88,90,91,93]
Supervised	20	[11,14,23,24,30,37,38,42,48,50,57,58,73,76,78,79,80,84,87,89]
Unsupervised	19	[8,10,13,29,40,43,51,52,53,59,60,65,67,70,71,72,74,86,92]
Semi-supervised	5	[12,29,46,62,68]
Not specified	2	[7,55]
Not applicable	1	[45]

shows that the combined use of supervised and unsupervised learning techniques stands out notably in implementation, appearing in a total of 33 articles. On the other hand, it is important to note that studies that opted to implement unsupervised or semi-supervised learning represent a considerably smaller proportion among the articles analyzed. An example of this is He and Schonlau [26], where supervised and unsupervised learning were applied simultaneously to examine the lack of reliability and validity in manual coding. Onan [88] also implemented both types of learning to analyze instructor evaluation reviews, demonstrating great adaptability across case studies.

These findings imply that combining supervised and unsupervised learning successfully extracts information from open-ended responses [9], describes data according to a property of interest [18,27], and predicts personality [25]. It also codes text answers [26], determines the factors impacting students’ creativity [28], and improves the ease and efficiency of the analysis of free text responses [31]. Additionally, it helps understand customer interests [32], student opinions [36], and patient experiences [39], and analyze medical beliefs [41], to mention a few.

It is worth noting that the total number of studies in this review increases by one because study [29] uses semi-supervised and unsupervised learning. It highlights how open-ended responses provide valuable and detailed information about people’s perceptions and priorities; therefore, it is counted twice.

4.2. Question 2: What Are the Most Common ML and Natural Language Processing Tasks for Analyzing Surveys with Open-Ended Questions?

NLP is widely used for survey analysis. For example, Schonlau et al. [42] implemented an n-gram and language-specific stemming approach in Stata to develop a method for automatically classifying open-ended questions by Support Vector Machine (SVM) with a linear kernel, while Pestian et al. [46] applied topic modeling and sentiment analysis to understand how students perceive the usefulness and ease of use of technology for a learning management system. Cook et al. [87] built an NLP-based model to analyze patterns and predictors of suicidal behavior.

ML methods have the potential to improve accuracy by modeling nonlinear relationships and hidden patterns in the data. Therefore, new ML-based methods can automate many stages of survey analysis, reducing manual workload. This leads to faster and more efficient analysis, freeing up time and resources. Figure 5 below shows the most used NLP and ML tasks across the 79 analyzed papers.

Table 4. ML and NLP Tasks.

Task	AI Area	Article
Classification	ML	[11,14,23,24,26,30,36,37,38,39,41,42,47,48,50,54,55,56,57,58,61,62,63,66,68,69,75,76,77,78,79,80,83,84,87,88,89,90]
Topic modeling	NLP	[8,9,10,25,28,31,39,40,43,44,49,52,54,56,59,63,65,66,67,71,74,82,85,86,91,93]
Clustering	ML	[8,13,28,32,43,44,46,51,61,65,70,72,86]
Regression	ML	[12,25,28,31,44,47,49,58,64,75,81,82,91]
Correlation	ML	[25,26,28,44,47,60,64,75,83,88,90]
Sentiment analysis	NLP	[23,24,28,36,39,69,77,78,83,89,93]
SDRD	ML	[18,27,32,33,34]
Association	ML	[41,69,81]
Key phrase extraction	NLP	[28]
Keyword-based topic analysis	NLP	[29]
Topic-based classification	NLP	[36]
Keyword extraction	NLP	[53]
Semantic Text Matching	NLP	[77]
Exact Answer Extraction (EAE)	NLP	[77]
Emotion classification	NLP	[83]
Term extraction	NLP	[90]
Not specified	NA	[7]
Not applicable	NA	[45]

illustrates the findings regarding ML and NLP tasks, showing that Classification (by ML) followed by Topic Modeling (by NLP) were the most used in the analyzed works, while Association and NLP tasks like Keyword Extraction and Semantic Text Matching were the least utilized.

Table 4 shows that SDRD was applied in only five studies. Nevertheless, Carmona and Elizondo [4] noted that SDRD not only allows the identification of patterns that conventional predictive models often overlook but also offers a clearer view of the relationships between variables. This contribution is vital to the field, as SDRD goes beyond simply “predicting” a response label to “explaining” which combinations of terms or conditions define a specific opinion group. Therefore, we conclude that low adoption of these tasks in conjunction limits survey analysis. It is missing the opportunity to achieve deeper knowledge discovery that reveals descriptive and meaningful rules in open-ended responses.

4.3. Question 3: What Are the Main ML Algorithms Applied for Analyzing Surveys with Open-Ended Questions?

In relation to the ML algorithms shown in Figure 6 and

Table 5. ML algorithms.

Algorithm	Article
SVM	[11,14,23,26,36,37,42,46,54,66,73,75,76,79,80,84,88]
RF	[14,25,26,30,37,54,58,64,73,75,76,79,84,88]
LR (Logistic regression)	[37,50,58,75,76,81,82,87,88]
Naive Bayes	[30,36,37,69,75,79,84,88]
K-Means	[28,32,44,46,61,86,93]
KNN (K-Nearest Neighbor)	[36,69,84,88]
Gradient boosting	[12,58,79]
Multilayer Perceptron	[23,41]
XGBoost	[14,58]
Lasso regression	[58,64]
Ridge regression	[58,64]
PA (Passive Aggressive)	[11]
Multinomial boosting	[12]
Best-first decision tree	[30]
SMO (Sequential Minimal Optimization)	[41]
Multilevel Bayesian Modeling	[52]
MNB (Multinomial Naïve Bayes)	[54]
Ordinal logistic regression	[56]
PLS-DA (Partial Least-Squares Discriminant Analysis)	[57]
BART (Bayesian additive regression trees)	[58]
C-Tree (Conditional inference tree)	[58]
CART (Classification and Regression Trees)	[58]
Stacking ensemble	[64]
SMOTE (Synthetic Minority Over-sampling Technique)	[76]

, Support Vector Machine (SVM) and Random Forest (RF) were found to be the most prevalent in the implementation throughout the analyzed works, as can be seen in various studies [11,14,23,36,37,54,66]. This trend is clearly reflected in the aforementioned table. However, it is important to note that, in addition to SVM and RF, multiple studies have explored other algorithms. These additional algorithms are used in only a single study, indicating a lower frequency of use compared to the previous ones.

The advantage of SVM and RF over other algorithms in survey analysis has profound implications. According to Moreo, Esuli, and Sebastiani [11], the popularity of SVM stems from its ability to classify text. This makes it ideal for analyzing free-text responses that can vary widely. RF, on the other hand, has demonstrated superior performance to other ML algorithms for tasks like regression [25] and classification [26]. For example, He and Schonlau [26] demonstrate that machine learning algorithms, specifically SVM and RF, perform similarly to humans in classifying text responses, and their accuracy increases as the training data set grows.

Despite the prevalence of SVM and RF implementations today, these methods still present considerable limitations. One of which is that it has been demonstrated that BERT has better classification performance than SVM and RF with the ngram model when the size of training data increases [14]. Also, BERT surpassed SVM performance for topic-based classification of course reviews [36]. Moreover, SVM has been overcome by RF in detecting grammatical ambiguity in software requirements using Bag-of-Words features [73]. RF again showed better performance than SVM to determine the satisfaction level of students on online learning during the COVID-19 pandemic [75].

4.4. Question 4: What Are the Most Popular NLP Models and Algorithms for Analyzing Surveys with Open-Ended Questions?

Figure 7 presents an analysis of the most widely used NLP models and algorithms. Latent Dirichlet Allocation (LDA), Structural Topic Modeling (STM), and BERT are the most prevalent approaches in the reviewed works. The predominance of these, according to our analysis, is due to their ability to address different levels of complexity in text analysis. For instance, LDA plus Term Frequency-Inverse Document Frequency (TF-IDF) demonstrated the best accuracy in terms of average correlation in textual content representation compared to Linguistic Inquiry and Word Count (LIWC), Doc2Vec, and Word2Vec language modeling approaches [25]. STM solved the disadvantages of LDA concerning the missing topic correlation consideration and the assumption that the same data generation process applies to each document, disregarding additional information from the analyst [31]. One advantage of BERT compared to CNN is its pre-training on big data [36]. Also, BERT overcame non-pretrained models like SVM and XGBoost in automatically coding answers [14].

In

Table 6. NLP models and algorithms.

Models/Algorithms	Article
LDA	[8,9,25,28,31,36,40,43,44,54,64,65,66,67,71,74,81,82,85,86,93]
STM	[10,31,36,49,52,56,59,74,91]
BERT	[14,28,36,39,52,73,78]
RoBERTa	[36,77,83,93]
BERTopic	[52,62,93]
BTM	[8,66]
NMF (Non-negative Matrix Factorization)	[39,93]
Gibbs Sampling	[63,66]
WNTM (Word Network Topic Model)	[8]
XLNet	[36]
Sentence-BERT (SBERT)	[44]
Based on lexicon and rules (Vader)	[71]
IndoBERT	[73]
SRoBERTa	[77]
AraBERT	[78]
MARBERT	[78]
QARiB	[78]
Local Context Focus-Aspect Term Extraction and Polarity Classification (LCF-ATEPC)	[78]
SentiStrength	[89]
OpenNLP Perceptron	[90]
GPT-4.1	[92]
GPT-4o	[92]
GPT-5	[92]
Gemini-2.0-Flash	[92]
Gemini-2.5-Flash	[92]
Twitter-roBERTa-base for Sentiment Analysis-Updated	[93]
Pegasus	[93]
GPT-2	[93]
Bidirectional and autoregressive transformers	[93]
T5	[93]
FLAN-T5-model	[93]

, the works that applied NLP algorithms are also specifically indicated. The analysis in Figure 7 shows that LDA remains the standard tool for analyzing open-ended survey responses. This demonstrates that researchers value LDA’s ability to uncover themes without the need for prior training. On the other hand, Roberts et al. [31] noted that the use of STM adds particular value by allowing the identified themes to be directly linked to respondent data, such as age or geographic location. This helps to understand not only what is being said but also who is saying it. In contrast, studies such as those by Gweon and Schonlau [14] and Alshaikh et al. [78] highlighted that, when a more precise opinion analysis (based on specific aspects) is required, BERT-based models are indispensable.

On the other hand, LDA, STM, and BERT have some disadvantages. LDA presents low topic modeling performance when the target documents are short or have many topics [8]. STM complicates the calculation of the adequate number of survey respondents for the analysis [31]. BERT is more difficult to program than non-pre-trained methods and demands high hardware requirements [14].

It is essential to account for the rapid evolution of large language models (LLMs) and transformer-based architectures, such as the GPT family [92,93], which have revolutionized survey analysis through prompting. While these approaches represent a significant shift in natural language processing, they appear underrepresented in this exploratory review, primarily due to our study’s conceptual framework. Most of the literature meeting our inclusion criteria predates the widespread accessibility and academic consolidation of these models. Consequently, although current research is shifting toward using LLMs, the primary focus of this review remains on established ML and NLP techniques that provided the methodological foundation for survey analysis during the period studied. Future research should explore integrating LLMs as feature extractors or reasoning engines to improve the interpretability of descriptive rules.

Table 7. Data representation models.

Model	Total	Article
n-grams	16	[12,14,29,37,42,47,48,50,57,62,64,69,80,84,85,87]
TF-IDF	12	[25,36,39,53,60,62,65,69,73,85,88,93]
BoW (Bag-of-Words)	10	[9,36,43,57,59,65,66,69,82]
Word2Vec	8	[8,23,25,36,39,66,69,88]
BERT vectors	7	[14,28,36,52,62,73,78]
Linguistic Inquiry and Word Count (LIWC) dictionary	4	[25,48,60,64,89]
Dense vector embeddings	2	[62,84]
GloVe (Global Vectors for Word Representation)	2	[66,88]
fastText	1	[72,88]
Swivel (Submatrix-wise Vector Embedding Learner)	1	[23]
Doc2Vec	1	[25]
Word/sentence embeddings	1	[37]
SBERT-based sentence embedding vectors	1	[44]
BoW vectorization model	1	[86]
WeSTClass embedding	1	[62]
Term-presence	1	[88]
Term-Frequency	1	[88]
LDA2Vec	1	[88]

classifies the analyzed studies by the feature extraction and data representation techniques employed. These tools are essential for transforming natural language into numerical formats that can be processed by machine-learning algorithms.

Most works represented free text answers using n-grams, for example, unigrams and bigrams [48,50,62,80] or 5-g [47]. Also, traditional models such as the BoW or TF-IDF predominate in the survey analysis approaches. Nevertheless, the advantages of vector representations (embeddings) in examining the positional arrangement and contextual environment of words were leveraged by several recent methodologies. The latter allows for more precise capture of the underlying semantics of documents, as observed in the specialized literature [36]. The most popular method in the reviewed studies is Word2Vec, which employs a feed-forward neural network to predict the words adjacent to a given word, thereby creating word representations. This resource provides an optimized implementation of the BoW and Skip-gram models to generate word vectors with high technical precision [96].

4.5. Question 5: What Are the Most Frequently Employed Technologies for Analyzing Surveys with Open-Ended Questions?

The most used technologies in related work were classified into libraries, programming languages, data mining tools, visualization tools and techniques, statistical tests, database management systems, and additional technologies in Table 7,

Table 8. Library Table.

Libraries	Article
stm package	[10,31,52,59,74]
NLTK	[60,82,86,93]
Gensim	[40,85,86,93]
spaCy	[8,24,38,60]
pyLDAvis	[9,40,74,85]
Keras	[36,61,84,88]
MALLET (MAchine Learning for LanguagE Toolkit)	[40,54,64,67]
pandas	[24,38,75]
quanteda	[37,52,56]
tidyverse	[57,81]
FastText	[8,88]
TensorFlow	[61,88]
HuggingFace	[36,77]
Caret	[57,58]
LiblineaR	[37,87]
VADER Library	[24,93]
Schedule	[24]
scikit-learn	[79]
Stanford Core NLP	[30]
Jieba package	[44]
SentiWordNet	[24]
MultiWordNet	[24]
PyTorch	[36]
Janome	[85]
dplyr packages	[57]
stringr	[57]
tidyr	[57]
R splitTools library	[64]
tableone de R	[57]
Hunspell packages	[37]
PrimeFaces	[18]
stm package	[10,31,52,59,74]
Broom	[57]
MLeval	[57]
BartMachine	[58]
Rpart	[58]
partykit	[58]
Gam	[58]
gamsel	[58]
R glmnet packages	[58]
Sentix	[24]
corpus	[57]
tyditext	[57]
Transformers Python library	[93]

,

Table 9. Table of programming languages used.

Programming Language	Article
Python	[9,14,24,32,36,38,39,40,44,53,66,75,76,83,84,85,86,93]
R	[10,42,43,49,56,57,58,59,64,66,74,76,81,82,89,91]
Java	[18,32,34,66]
Visual Basic	[70]
C++	[66]
JavaScript	[86]
Bash	[66]
Coffeescript	[86]

,

Table 10. Table of visualization tools and techniques.

Graphic	Article
Word cloud	[23,24,25,36,43,64,71,77,81,85,93]
Heatmap	[25,65]
Inter-topic distance plot	[74,85]
Violin plot	[24]
TagCrowd	[50]
Many Eyes	[50]
Topic-document	[74]
Bipartite network	[65]
Word map	[87]
Histogram charts	[23]

,

Table 11. Statistical test table.

Statistical Test	Article
Chi-square tests	[18,85]
ANOVA	[57,88]
F-test statistics	[24]
Granger causality hypothesis test model	[24]
Augmented Dickey–Fuller Test	[24]
Likelihood-ratio test	[24]
The Wilcoxon test	[28]
Harman’s one-factor testing	[44]
Wilcoxon rank-sum (Mann–Whitney) test	[87]

,

Table 12. Data mining tools table.

Programming Language	Article
WEKA	[18,30,34,41,64,88]
Stata	[12,42,50,58,87]
IBM SPSS	[38,44,45,53,83]
EPM Framework	[27,33]
VIKAMINE	[18]
Rapidminer	[69]
Orange3	[79]

and

Table 13. Database management systems.

Manager	Article
PostgreSQL	[18,34,41]
Neo4j	[8]
MySQL	[24]

. Figure 8 and Table 8 below break down all the libraries mentioned in the analyzed works. The stm package in R was the most widely implemented, followed by the Natural Language Toolkit (NLTK) and Gensim in Python.

The results shown in Figure 8 and Table 8 reflect significant technological fragmentation, which is a direct consequence of the multidisciplinary nature of the studies analyzed. For example, Nanda et al. [40] used pyLDAvis, Gensim, and MALLET to identify the most important characteristics of MOOCs to improve students’ learning experience. Baburajan et al. [9] also implemented pyLDAvis among the techniques they used. Inoue et al. [85] used Gensim among their technologies to explore the effects of the COVID-19 pandemic on nursing research and development in Japan. Other studies, such as [22,59,85,93], make use of these libraries for data analysis or visualization, to name a few.

As for the programming languages used in the analyzed works, most of them mainly use Python, R, and Java. This trend is clearly reflected in Figure 9 and Table 9, which provide a detailed breakdown of the programming languages used, as well as specifying which studies implemented each of them. This information allows us to appreciate the preferences and trends in the research community when choosing programming languages for data analysis.

An analysis of programming languages shows a balanced competition between Python and R. For example, Mahendher et al. [76] used both Python and R to predict changes in pharmaceutical sales driven by noncommunicable diseases and post-COVID-19 habits. Gweon and Schonlau [14] utilized Python to show the inefficiency and high cost of manually coding text from open-ended questions. Wijngaards et al. [89] used R to demonstrate how computer-aided text analysis can build measures of job satisfaction from open-ended survey responses.

Figure 10 and Table 10 present the graphic visualization technologies that have been most widely used in the reviewed works. Among the different tools and methods, word clouds stand out as one of the most widely used technologies in the analysis. This information is presented below in textual form with references, allowing for a better understanding of the graphic tools that have predominated in the research analyzed and in which studies they were implemented.

The statistical tests used in the studies analyzed are presented in Figure 11 and Table 11. This table shows that the Chi-square test is the technique that appears most frequently in the studies reviewed, followed by ANOVA (Analysis of Variance). This information highlights the preference for these methodologies in data analysis as presented in the examined literature.

Regarding data mining tools, WEKA, Stata, and IBM SPSS were found to be the most widely used in the analyzed studies, as shown in Figure 12 and Table 12. This table provides a summary of the frequency of use of these tools, highlighting their relevance in data analysis in the research reviewed.

As shown in Figure 12, Weka stands out as the most widely used data mining tool, which suggests a preference for rapid experimentation. Its visual interface lets researchers apply complex algorithms such as SVM or RF without programming from scratch, making data mining more accessible. As Schonlau and Couper [12] note, integrating NLP techniques into well-known statistical software helps maintain econometric rigor while automating text analysis.

Table 13 presents the database management systems identified in the review and specifies the studies in which each is used. Preference for PostgreSQL suggests a need for robustness and support for complex data types in survey analysis.

Figure 13 and

Table 14. Additional technologies.

Technology	Article
Text mining (Minería de texto)	[12,51,64,65,69,84,87]
Microsoft Excel	[45,70]
Canvas	[69,71]
Gavagai Explorer	[13]
LimeSurvey	[24]
MetaMap	[30]
Monte Carlo simulation	[31]
MPLus 8	[44]
Semantic Excel	[47]
Voyant Tools	[50]
Qualitative software MAXQDA 12	[51]
PsychArchives	[52]
SPSS Survey Analyzer 4.0.1	[53]
T-Lab	[55]
Text Component Analysis (TCA)	[55]
TextQuest	[55]
Wordscores	[55]
WordStat	[55]
Yoshikoder	[55]
Syuzhet	[74]
R/RStudio for statistical analysis	[76]
Pre-trained language models (CLLMs)	[77]
COVAREP (Cooperative Voice Analysis Repository for Speech Technologies)	[80]
NVIDIA Tesla T4 GPU	[84]
CUDA 12.0 toolkit	[84]
Plataforma Digital para la Investigación Psicológica Interdisciplinaria	[86]
Radboud Faces Database	[47]
cTakes (Clinical Text Analysis Knowledge Extract System)	[87]
Bayesian optimization with a Gaussian process	[88]
Minitab statistical software	[88]
Text mining	[12,51,64,65,69,84,87]
Computer-Aided Text Analysis	[89]
Saffron knowledge extraction tool	[90]
Methodological framework (ARC framework)	[90]

below show additional technologies, i.e., all those that do not fall into any of the above classifications, including the work in which they are mentioned. As can be seen, text mining, followed by Microsoft Excel and Canvas, was among the most widely implemented technologies.

As mentioned earlier, the diversity of technologies stems from the multidisciplinary nature of the data. These additional resources contribute to the field by showing that survey processing is not a linear task involving a single software program, but rather a combination of technologies for text preparation. This is just as critical to the rigor of the results as the choice of the final algorithm.

4.6. Question 6: In Which Areas of Research Are Surveys with Open-Ended Questions Predominant?

Extracting information from a large dataset has required a significant amount of effort on the part of researchers, which is why many have dedicated themselves to demonstrating that extracting information from free text, i.e., from surveys with open-ended questions, can be a low-cost approach [38]. Throughout the research, multiple studies have been found that propose new methodologies, methods, and/or tools that use ML to automate the categorization of responses to open-ended questions [27,28,29].

We noticed that the domains with the greatest inclination are research and health, i.e., most of the published papers work with diverse approaches to research or test methodologies or present tools for text processing and survey analysis, while the others focus on health issues, applying ML and survey analysis to obtain valuable information. For example, Rubio Delgado et al. [41] provided a tool for the pathology department of the Rio Blanco Regional Hospital in Veracruz that facilitate the analysis of surveys obtained on the decrease in autopsies at that hospital. For their part, Ríos Mendez et al. [27] used emerging pattern mining methods to identify the reasons that lead doctors to reject or accept the application of autopsies. Similarly, Olmos et al. [34] applied subgroup discovery to identify why Mexican hospitals perform too few autopsies and to determine whether the reasons vary due to cultural, geographical, or infrastructure factors. In contrast, Duan et al. [32] focused their work on the characteristics of online customer reviews to improve the shopping experience.

Table 15. Research areas.

Areas	Total	Article
Health	30	[18,24,27,30,33,34,38,39,41,43,45,46,47,50,52,61,62,64,70,76,80,81,83,84,85,86,87,90,91,93]
Research	24	[7,8,11,12,13,14,26,31,37,42,49,51,53,55,56,57,58,63,66,67,72,77,78,92]
Education	17	[23,25,28,29,36,40,48,54,60,65,68,69,71,74,75,82,88]
Service	8	[9,10,32,44,59,73,79,89]

classifies the reviewed works according to the domains of knowledge where they were focused.

The dominance of research and health suggests that the adoption of ML is not just a technical improvement, but a necessity in sectors with high volumes of unstructured data. In the field of health, the ability to process free text enables the extraction of deep and valuable insights at low cost, as shown in [18,27,33,41]. These studies show that specialized algorithms can identify critical patterns in sensitive data, such as the reasons for rejecting autopsies, with a level of precision that manual review cannot achieve. This trend contributes to the field by positioning NLP as a strategic ally in clinical and academic decision-making. We understand that the “Research” domain leads the frequency rankings in the studies because many of them focus on presenting tools, technology comparisons, and other topics. That is why we conclude that the health and education domains aim for more specific objectives.

5. Conclusions

Open-ended responses in surveys are a valuable option, as they reflect the richness and variety of respondents’ opinions, provide specific details, and facilitate the discovery of meaningful information for informed decision-making. In this research, we conducted a scoping review of 79 studies that analyzed open-ended survey answers. Upon discovering that there are no previous studies that integrate ML techniques, specifically SDRD with NLP, we concluded that the combination of these techniques is an open field of research and that the development of a software tool that integrates these techniques for the analysis of surveys with open-ended questions would be a valuable contribution to knowledge.

In summary, the use of SDRD techniques in survey analysis promises a significant improvement in the quality of survey data with open-ended questions, which supports the present review.

Future work will focus on developing new computational methods for analyzing surveys with open-ended questions, exploring new NLP techniques and integrating them with EPM, SD, and CSM—taking into account different scenarios to ensure the adaptability of these techniques. One study focused on the healthcare sector and another on the education sector by testing different algorithms for each technique, quality metrics, and objective and subjective evaluations conducted by experts in the field.

Despite the comprehensiveness of this review, it is important to note that approaches based on acoustic signals, multimodal data, or signal-level representations were excluded. Although the main focus of this study was NLP applied to open-question surveys, the literature also suggests integrating representations based on signal characteristics [97].