PromptTone: A Dataset for Evaluating Large Language Model Code Generation Under Varying Prompt Politeness Levels

Abstract

The increasing adoption of Large Language Models (LLMs) in software development has enabled automatic code generation from natural language, yet the influence of communicative factors such as prompt tone remains underexplored. This work introduces PromptTone, a controlled dataset designed to investigate how variations in prompt politeness affect LLM-based code generation in web development. The dataset is constructed through a structured experimental design combining three variables: programming paradigm (Vue.js Composition API vs. Options API), LLM provider (GPT, Claude, Gemini), and prompt tone (impolite, neutral, polite), resulting in 396 generated components across 22 implementations. Data were collected in an educational setting under a single-prompt constraint to capture first-shot model behavior, and are provided in both hierarchical and CSV formats, including prompts, generated code, and error annotations. Preliminary analysis reveals that prompt tone influences output characteristics such as verbosity, with model-specific patterns: for instance, some models exhibit increased output length with more polite prompts, while others remain stable. Differences also emerge across programming paradigms, suggesting an interaction between tone and code structure. These findings highlight that LLMs are sensitive not only to semantic content but also to pragmatic aspects of input. Overall, the dataset provides a novel benchmark for studying human–LLM interaction in code generation, supporting future research on prompt engineering, model evaluation, and socially-aware Artificial Intelligence (AI)-assisted development tools.

Dataset: https://doi.org/10.5281/zenodo.19224813.

Dataset License: Creative Commons Attribution 4.0 International

1. Summary

The rapid evolution of LLMs has significantly transformed the landscape of software development, particularly in the domain of code generation from natural language descriptions. Early systems such as Codex [1] demonstrated the feasibility of automatically synthesizing functional code from textual specifications, paving the way for modern AI-assisted development tools. More recent advances have further consolidated this paradigm, with contemporary LLMs achieving strong performance across a wide range of programming tasks, as highlighted in a recent survey on code generation [2]. These models are increasingly integrated into development workflows, enabling both expert and non-expert users to generate, modify, and reason about code through conversational interfaces.

Within this context, web development has emerged as a particularly relevant application domain. Modern web applications are characterized by dynamic, component-based architectures that require both structural correctness and semantic coherence. Prior work has explored how LLMs can support end-user development by translating natural language requirements into executable web artifacts [3], as well as how they can assist in testing and validation tasks, such as automated web-form test generation [4]. At the same time, research in web accessibility has highlighted both the opportunities and challenges of leveraging LLMs for improving code quality and inclusivity, emphasizing their ability to reason over complex structures while also exposing limitations such as hallucinations and inconsistencies [5,6].

A key yet underexplored dimension of human–LLM interaction concerns the role of language pragmatics, and in particular, the impact of communicative tone on model behavior. Politeness has long been studied in linguistics and communication theory as a factor influencing interaction outcomes [7,8]. In the context of Natural Language Processing (NLP), computational approaches to politeness have investigated how linguistic cues can be modeled and analyzed [9]. More recently, empirical studies have begun to examine how variations in prompt tone affect LLMs performance, revealing complex and sometimes counterintuitive relationships. For instance, Yin et al. [10] show that politeness can influence model effectiveness depending on linguistic and cultural context, while another study suggests that less polite prompts may, in some cases, yield higher accuracy [11]. These findings indicate that LLMs are sensitive not only to the semantic content of prompts but also to their stylistic and social dimensions.

Despite these advances, existing datasets do not provide adequate support for studying the interaction between prompt tone and code generation, particularly in the Italian language. As summarized in Table 1, current Italian datasets such as ITA-Bench [12], Evalita-LLM [13], ITALIC [14], and GITA [15] focus on general language understanding and evaluation tasks, without considering either politeness or code generation. On the other hand, datasets that explicitly model politeness, such as TyDiP [16] and XFORMAL [17], do not include code generation tasks, and in some cases either exclude Italian or rely on rewritten or translated data. Instead, HumanEval-XL [18] support multiple natural languages but do not incorporate pragmatic dimensions such as tone, and are often based on machine-translated prompts rather than native language inputs.

Table 1. Comparison of the PromptTone dataset with existing benchmarks for Italian LLM evaluation.

Dataset	Target Language(s)	Politeness	Code Gen	Origin
ITA-Bench [12]	Italian	No	No	Native Language & Translated
Evalita-LLM [13]	Italian	No	No	Native Language
ITALIC [14]	Italian	No	No	Native Language
GITA [15]	Italian	No	No	Native Language
TyDiP [16]	9 Languages (Excludes Italian)	Yes	No	Native Language
XFORMAL [17]	Brazilian-Portuguese, French, Italian	Yes	No	Human Rewrites
Table Accessibility [19]	Italian	No	Yes	Native Language
HumanEval-XL [18]	23 Natural Languages	No	Yes	Machine Translated
PromptTone	Italian	Yes	Yes	Native Language

This fragmentation highlights a gap: no existing dataset that supports (i) Italian language, (ii) controlled variations in politeness, and (iii) code generation tasks at the same time. This limitation prevents a systematic investigation of how pragmatic variations in prompts influence LLMs behavior in realistic development scenarios.

To address this limitation, we introduce the PromptTone dataset, a controlled and reproducible benchmark for analyzing the effect of prompt tone on LLM-based code generation in the context of Vue.js web forms. The dataset is grounded in a structured experimental design that combines three independent variables: programming paradigm (Composition API vs. Options API), LLM provider (GPT, Claude, and Gemini), and prompt tone (impolite, neutral, and polite). This design results in 18 distinct experimental conditions, systematically explored across 22 independent implementations, for a total of 396 generated Vue.js components.

The dataset is provided in two complementary formats. The first is a hierarchical file structure that preserves the original organization of each implementation, including the generated Vue single-file components and the corresponding input prompts, along with contextual materials describing the target forms. The second is a tabular CSV representation that consolidates all experimental conditions into a single analysis-ready format, enabling large-scale quantitative analysis. Each entry includes not only the prompt and generated output but also metadata capturing potential errors, facilitating the study of both successful and failure cases.

Compared to existing resources, the PromptTone dataset uniquely combines native Italian prompts, explicit control over politeness levels, and realistic code generation tasks within a single experimental framework. This enables researchers to isolate and analyze the impact of tone while controlling for other variables, a capability not supported by prior datasets. By combining realistic development tasks with a rigorous experimental design, the dataset supports a wide range of research directions, including prompt engineering, model comparison, code quality assessment, and human–AI interaction studies.

Overall, this work contributes to the growing body of research on LLMs by highlighting the importance of linguistic and social factors in shaping model outputs. It provides both a conceptual framework and a practical resource for investigating how communication styles influence AI-assisted programming, with implications for the design of more effective, robust, and user-aware development tools.

2. Data Description

The dataset comprises 22 independent implementations, each corresponding to a complete set of artifacts produced by a single student. Every implementation contains 18 Vue.js form components generated using large language models (LLMs), resulting in a total of 396 generated components. The dataset is distributed in two complementary formats: (i) a hierarchical file and folder structure that preserves the original organization of each implementation, and (ii) a CSV file designed to facilitate programmatic analysis. Both representations contain the same information, including the Vue API style, LLM provider, and prompt tone.

2.1. Hierarchical File Structure

The data is organized in a raw/ directory containing 22 numbered folders (e.g., 01, 02, …, 22), each representing an individual implementation. The internal structure of each implementation mirrors the experimental design in a nested and systematic way, allowing each implementation to be uniquely identified through its directory path alone.

The organization follows a consistent naming convention that directly encodes the experimental conditions. At the top level, folders distinguish between the two Vue programming paradigms, namely composition-api/ and options-api/. Within each paradigm, subfolders identify the LLM provider used to generate the code, namely chat-gpt/, claude/, and gemini/. Finally, each provider-specific folder contains three subfolders corresponding to prompt tone, labeled as 01-scortese/, 02-neutro/, and 03-cortese/, which map respectively to impolite, neutral, and polite categories in the CSV representation. This structure ensures that each implementation is uniquely identifiable without requiring additional metadata.

Each implementation folder contains exactly two files:

• Form.vue, a Vue single-file component (SFC) generated by the LLM, including template, script, and style sections;
• prompt.txt, containing the exact input prompt used for generation.

This pairing of generated artifact and corresponding prompt enables a direct analysis of the relationship between input prompt and output.

In addition to the generated components, each implementation includes a materials/ directory documenting the reference form that students try to replicate. This directory provides essential contextual information for interpreting and evaluating the generated outputs. It includes:

• form-link.txt: the URL of the original form from a real website;
• form-description.txt: a description of the form’s purpose and fields;
• form-screenshot.(png|jpg|jpeg): a visual screenshot of the target interface;
• notes.txt (optional): additional annotations provided by the student.

These materials ensure that the task context is fully documented and reproducible, allowing for qualitative and quantitative evaluation of the generated implementations.

2.2. CSV Dataset Format

For convenience in programmatic analysis, the dataset is also provided as a single CSV file derived from the hierarchical folder structure. Each row represents one implementation, and columns consolidate all experimental variations. The CSV includes the following columns:

• Metadata columns:

  –

  implementation_id: a unique identifier for each implementation (e.g., 1, 2, …, 22).

  –

  description: a description of the form’s purpose and fields.

  –

  link: the URL of the original form from a real website.

  –

  img_data_b64: a base64-encoded screenshot of the target form.

• Prompt and output columns: 54 columns following the naming pattern {api}_{model}_{tone}_{type}, where:

  –

  api∈ {composition-api, options-api}

  –

  model∈ {chat-gpt, claude, gemini}

  –

  tone∈ {impolite, neutral, polite}

  –

  type∈ {prompt, output, error}

For each experimental condition, the CSV stores three complementary pieces of information: the input prompt, the generated Vue.js component, and an additional field capturing any potential compilation or runtime errors encountered when processing the generated Form.vue component. When no errors are detected, this field is left empty; in failure cases, it contains a textual description of the error.

For example, the column composition-api_claude_polite_prompt contains the polite prompt text used with Claude for generating a Composition API component, composition-api_claude_polite_output stores the corresponding generated code and composition-api_claude_polite_error contains any errors if detected.

Overall, the tabular format enables direct integration with data analysis tools (e.g., pandas or R), simplifies cross-condition comparisons, and complements the hierarchical structure by providing a flat, analysis-ready representation of the dataset.

3. Methods

This section describes the methodological framework used to collect the PromptTone dataset, from technological context to experimental design and data collection procedure. The dataset was collected through a structured educational experiment embedded in a web development course, ensuring that the data reflects realistic usage scenarios while maintaining rigorous control over experimental variables.

3.1. Technological Context and Scope

This study focuses on code generation in Vue.js, a widely adopted progressive JavaScript framework for building user interfaces [20]. Vue.js was selected due to the coexistence of two distinct programming paradigms within the same ecosystem, which enables a controlled comparison of how LLMs adapt to different architectural styles while solving the same task.

The first paradigm, the Options API, represents the traditional approach to Vue development and is based on an object-oriented structure that organizes component logic into predefined sections such as data, methods, and computed. This style emphasizes readability and separation of concerns and is typically more accessible to beginners. The second paradigm, the Composition API, introduced in Vue 3, adopts a more flexible and functional approach centered around the setup() function and reactive primitives such as ref and reactive. This paradigm enables improved code reuse and scalability, but requires a different mental model and often leads to more compact yet less immediately interpretable code.

The comparison between these two paradigms provides a meaningful testbed for evaluating whether and how LLMs adapt their generation strategies to different programming abstractions within the same framework.

In addition to the programming paradigm, the study considers three state-of-the-art commercial LLMs, selected to represent diverse model families and deployment ecosystems. Specifically, the dataset includes GPT (OpenAI), Claude (Anthropic), and Gemini (Google). The automatic model selection feature was enabled for all platforms that provide it, allowing the system to choose the most appropriate model based on the prompt and task requirements. This approach reflects real-world usage patterns, where users often rely on platform defaults rather than manually selecting specific models. At the time of data collection, the default models were Instant and Thinking for GPT-5.1 Auto, while Sonnet and Haiku for Claude 4.5, depending on the prompt tone.

All models were accessed through their respective web interfaces to ensure a consistent and controlled interaction setting. While modern developer workflows increasingly rely on AI-integrated IDEs or no-code agentic platforms, these primarily provide a more user-friendly interface and orchestration layer over LLM interactions. In our case, the experimental design is based on self-contained, end-to-end request-response tasks, where all necessary context is explicitly provided within each prompt. Under these conditions, the interaction modality does not affect the outcome, as no external tooling, iterative context building, or agentic behavior is required. Therefore, we focus on direct web-based interactions, using each platform’s default model selection at the time of data collection. A summary of the models is provided in Table 2.

Table 2. Large Language Models included in the dataset, with provider and release date.

Model	Provider	Release Date
GPT-5.1 Auto	OpenAI	November 2025
Claude 4.5 Auto	Anthropic	September 2025
Gemini 2.5 Flash	Google	July 2025

These models were chosen because they represent the current generation of high-performance LLMs for code generation tasks, each characterized by different training methodologies, alignment strategies, and interface designs. By including multiple providers, the study aims to capture variability not only in model outputs but also in how different systems interpret and respond to identical prompts under controlled conditions.

3.2. Experimental Design and Data Collection Procedure

The dataset combines three independent variables: the Vue programming paradigm, the LLM provider, and the communicative tone of the prompt. The combination of these factors results in 2 api_type × 3 model × 3 tones = 18 distinct experimental conditions. Each condition corresponds to a unique configuration in which a specific model is prompted, using a specific tone, to generate a Vue.js implementation following a specific API style.

Each student contributed exactly one implementation per condition. Given the 22 participants involved in the study, the final dataset contains 396 independent code generation instances. This balanced design ensures that all factors are uniformly represented and enables controlled comparisons across dimensions, as well as the analysis of interaction effects between variables.

Given the structured nature of the assignment, the dataset was collected within a web development course at the University of Bologna during the first semester of the academic year 2025–2026. Each student was required to select a non-trivial form from an existing website, such as a registration, booking, or contact form, detailed in Table 3. The selected form served as the reference target for all subsequent implementations. A screenshot and a textual description of each selected form were also provided to ensure that all students had a clear and consistent understanding of the task requirements.

Table 3. Forms included in the dataset.

Delivery	Form Description	Reference
delivery-01	AirChina form to book a flight	[21]
delivery-02	Request form for RossoNero card of ACMilan	[22]
delivery-03	Google calendar event creation form	[23]
delivery-04	GitHub login form	[24]
delivery-05	OpenCV contact form	[25]
delivery-06	Registration form for fidelity program in a supermarket	[26]
delivery-07	Facebook registration form	[27]
delivery-08	StreamYard customer support form	[28]
delivery-09	Data entry form for sports club membership	[29]
delivery-10	Neilpatel contact form	[30]
delivery-11	Tsunami Nutrition customer support form	[31]
delivery-12	Reddit customer support form	[32]
delivery-13	Municipality service request form	[33]
delivery-14	SkyScanner flight search form	[34]
delivery-15	FITP membership request form	[35]
delivery-16	PassioneBeauty contact form	[36]
delivery-17	Europass assistance request form	[37]
delivery-18	Bartolini BRT shipment complaint form	[38]
delivery-19	MyAnimeList customer support form	[39]
delivery-20	Pre-registration form for blood/plasma donors	[40]
delivery-21	WeTransfer customer support form	[41]
delivery-22	Allbeauty e-commerce registration form	[42]

Once the reference form was defined, students were instructed to generate 18 independent Vue.js implementations of the same form, one for each experimental condition. All implementations had to be produced using only native Vue.js features, without relying on external libraries or User Interface (UI) frameworks. This constraint ensured that differences in the generated code could be attributed to the LLM and prompting conditions rather than to external dependencies.

A key methodological aspect of the data collection process is the single-prompt constraint. Each implementation had to be generated using exactly one prompt, without any iterative refinement, follow-up questions, or manual corrections. This constraint isolates the first-shot generation capability of the models and prevents confounding effects introduced by interactive prompt engineering. As a result, each generated component represents a direct mapping from prompt to output.

To further ensure consistency, all prompts were written in Italian. This choice eliminates variability due to language differences and allows the dataset to also serve as a resource for studying multilingual aspects of code generation.

Students were instructed to produce detailed prompts that explicitly described the structure of the form, including input fields, validation rules, expected behaviors, and relevant visual elements. This requirement ensures that the generation task is sufficiently constrained and comparable across different implementations.

3.3. Prompt Tone Manipulation

A central variable in the study is the communicative tone of the prompt. Each prompt was crafted according to one of three predefined tone categories: impolite, neutral, and polite. These categories were designed to capture different interaction styles that users may adopt when communicating with LLMs.

Impolite prompts are characterized by aggressive or impatient language, including commands, expressions of frustration, or dismissive wording. Neutral prompts adopt a purely technical and factual tone, focusing on clarity and completeness without any social markers. Polite prompts, on the other hand, include courteous expressions such as greetings, requests, and acknowledgments, reflecting a more formal and respectful interaction style.

Importantly, while the tone varied, the semantic content of the prompts was required to remain equivalent across conditions. That is, each prompt described the same form and the same requirements, differing only in phrasing and communicative style. This ensures that any observed differences in the generated outputs can be attributed to tone rather than to differences in task specification.

To reduce ambiguity during data collection, students were instructed to preserve a shared informational core across the three tone variants for each condition. In practice, each variant had to specify the same form fields, constraints, and expected behaviors, while changing only pragmatic and stylistic markers (e.g., imperative phrasing for impolite prompts, plain technical wording for neutral prompts, and courtesy markers for polite prompts). This operational rule was reinforced during assignment briefings to limit content drift between tone versions.

The tone manipulation should therefore be interpreted as a controlled pragmatic perturbation rather than a semantic reformulation of the task. Although this protocol improves comparability, it does not eliminate all residual linguistic variation: lexical and syntactic choices may still differ across prompts, especially in natural language authored by different participants. For this reason, the dataset is suitable for analyzing broad, condition-level differences in model behavior, while fine-grained causal attribution should be treated with caution.

3.4. Data Validation and Curation

Following the initial submission phase, all student implementations were reviewed by the instructor to verify completeness, adherence to the experimental constraints, and correctness of the prescribed folder structure. This review ensured that each submission contained the full set of 18 implementations, that all required materials were provided, and that the organization of files followed the expected hierarchy.

After this initial verification, a manual normalization process was carried out to standardize file organization, naming conventions, and formatting across all submissions. This step was necessary due to minor inconsistencies introduced by students during the collection phase. During normalization, personally identifiable information was removed to ensure anonymity and compliance with data protection requirements.

Subsequently, a validation phase was performed to assess the correctness of the generated Vue.js components. First, a static syntactic check was conducted to verify whether each Form.vue file could be successfully parsed by the Vue compiler. Out of the 396 generated implementations, only 3 were found to contain syntax errors. These errors were recorded and included in the dataset as structured metadata, allowing researchers to explicitly identify and analyze failure cases.

In addition to syntactic validation, a semantic consistency check was performed to verify whether the generated components adhered to the requested Vue API paradigm. This check was implemented using a simple heuristic based on the presence of characteristic structural markers in the code, specifically detecting the <script setup> block as an indicator of the Composition API. Using this approach, among the 198 implementations generated under the Composition API condition, 7 were identified as inconsistent, as they did not include the expected <script setup> structure. Similarly, among the 198 implementations generated under the Options API condition, 15 were found to be inconsistent, typically due to the use of Composition API block header. Although heuristic, this method provides a consistent and scalable proxy for assessing paradigm adherence, and all detected inconsistencies were recorded as structured metadata for further analysis. In a limited number of cases, the same component can receive more than one error label (e.g., syntax and API-consistency), so error annotations are multi-label rather than mutually exclusive.

Figure 1 provides a visual summary of these validation results through two pie charts, illustrating the distribution of syntax errors and API usage inconsistencies across the dataset.

Figure 1. Summary of validation results. Panel (a) shows the distribution of syntax errors across all implementations, while panel (b) shows the distribution of API usage inconsistencies for both Composition API and Options API implementations.

Table 4 reports the corresponding descriptive counts and rates. In aggregate, the dataset contains a limited number of failures, with most generated components passing the adopted syntactic and API-consistency checks.

Table 4. Validation outcomes across the 396 generated components.

Validation Outcome	Count	Rate
Compilation errors	3	0.8% (on 396)
Composition API mismatch (missing <script setup>)	7	3.5% (on 198)
Options API mismatch (contains <script setup>)	15	7.6% (on 198)

The final curated dataset was then prepared in both hierarchical and tabular formats and published on GitHub [43] and Zenodo [44] to ensure accessibility and reproducibility.

3.5. Analysis and Observations

From a structural perspective, the dataset is highly regular and balanced, reflecting the experimental design described in the previous section. The uniform distribution of samples across all experimental conditions is particularly advantageous for comparative analysis, as it avoids biases introduced by uneven data representation.

A first quantitative analysis focused on the length of the generated outputs, measured as the number of characters in each Form.vue file. This metric provides a coarse but informative proxy for code verbosity, complexity, and level of detail in the generated implementations across different LLMs and prompt tones. Figure 2 presents the average output length across different LLMs and prompt tones.

Figure 2. Average output length (in characters) across different LLMs and prompt tones.

The results reveal distinct behaviors across models. GPT exhibits relatively stable output lengths across all tone conditions, suggesting limited sensitivity to prompt politeness in terms of verbosity. Gemini shows a moderate variation, with slightly longer outputs for polite prompts and the longest outputs observed for neutral prompts. In contrast, Claude demonstrates a clear and systematic trend, with output length increasing proportionally with the level of politeness (impolite < neutral < polite). This pattern suggests that Claude may be more responsive to social cues in prompts, potentially generating more elaborate or structured code when prompted in a polite manner.

To complement the visual analysis, Table 5 reports descriptive output-length statistics (mean and standard deviation) for each model–tone condition, aggregated over both API paradigms. The table confirms the same qualitative pattern observed in Figure 2: GPT remains comparatively stable, Gemini peaks in the neutral condition, and Claude shows a monotonic increase from impolite to polite prompts.

Table 5. Descriptive summary of output length (characters) by model and prompt tone (aggregated over both API paradigms).

Model	Tone	Mean Chars	Std. Dev.
GPT	impolite	6168.7	3627.1
GPT	neutral	6206.5	3522.8
GPT	polite	5999.1	3245.4
Claude	impolite	10,021.8	3754.9
Claude	neutral	10,861.0	5075.3
Claude	polite	11,905.8	4985.3
Gemini	impolite	7317.0	3211.7
Gemini	neutral	8657.6	4370.9
Gemini	polite	8256.9	3845.3

To further investigate the interaction between programming paradigm and prompt tone, Figure 3 shows a heat map of the average output length comparing API type and tone. A clear divergence emerges between the two paradigms. For the Options API, the output length increases linearly from impolite to polite prompts, indicating that more courteous phrasing tends to produce more verbose implementations. For the Composition API, the pattern is less straightforward: while impolite prompts consistently produce shorter outputs, neutral prompts result in the longest implementations, with polite prompts yielding slightly shorter outputs than neutral ones. This suggests that the relationship between prompt tone and generated code length is not uniform across paradigms and may depend on how LLMs internally represents and generates different architectural styles.

Figure 3. Heat map of average output length (in characters) across API type and prompt tone.

These observations are intentionally reported as descriptive and exploratory. The present manuscript focuses on documenting the dataset and illustrating representative condition-level trends, not on providing definitive inferential claims about model behavior. Statistical hypothesis testing and effect-size estimation are therefore deferred to follow-up analyses built on the same released data.

3.6. Dataset Usage Example

To illustrate how the dataset can be programmatically accessed and prepared for analysis, we provide a concise Python 3.11 example based on the pandas library. Additional examples are available in the accompanying repository [43] and in Zenodo [44].

Listing 1 shows how to load the dataset, identify the relevant columns based on the naming convention, and transform the data into a long format suitable for analysis.

Listing 1. Example Python workflow for loading and restructuring the dataset.

The code first loads the CSV file into a pandas DataFrame, showing basic information. It then exploits the structured naming convention of the columns to automatically select only those containing prompts and outputs.

Subsequently, the dataset is transformed from a wide format to a long format using the melt() operation. This step is particularly useful for analysis, as it consolidates multiple experimental conditions into a single column, making it easier to perform grouping and aggregation operations.

Finally, the structured information encoded in the column names is extracted into explicit variables, namely the Vue API type, the LLM provider, the prompt tone, and the content type (prompt or output). The resulting long_df representation provides a clean and analysis-ready format, while the printed previews offer a quick inspection of both the original and transformed data.

4. User Notes

4.1. Compliance with FAIR Principles

The dataset adheres to the FAIR principles–Findability, Accessibility, Interoperability, and Reusability [45]:

• Findability: The dataset is assigned a unique Digital Object Identifier (DOI) through Zenodo, ensuring persistent identification and discoverability through standard academic search engines and data repositories.
• Accessibility: The dataset is hosted on Zenodo, a CERN-backed repository committed to long-term preservation. All files are freely downloadable without authentication barriers.
• Interoperability: The dataset consists entirely of plain text files (.vue, .txt, .json, .csv). Vue SFCs are standard web development artifacts that can be processed by any text editor, parsed programmatically, or executed in a Vue development environment. The dataset is also provided in two formats: a hierarchical file structure and a tabular CSV format, both of which are widely supported and facilitate integration with various analysis tools. To ensure reproducibility and compatibility with execution environments, each deliverable includes its own package.json file, which explicitly specifies the versions of the programming language runtime and all associated dependencies (e.g., Vue and related libraries). This guarantees that the exact software environment can be reconstructed. Instead, the CSV representation is provided as raw text without version constraints, as it follows a universally supported format that can be processed independently of specific software versions.
• Reusability: The dataset is released under a Creative Commons Attribution 4.0 International (CC BY 4.0) license, permitting unrestricted use, modification, and redistribution with appropriate attribution.

4.2. Dataset Limitations

While the PromptTone dataset provides a structured and controlled environment for studying the impact of prompt tone on LLM-based code generation, several limitations must be acknowledged when interpreting the results and generalizing the findings.

First, the dataset reflects an educational setting, as all implementations were produced by students within the context of a university assignment. This setup improves consistency and experimental control but may not reflect professional development practices, where prompts are often iterative, exploratory, and shaped by domain experience. The imposed single-prompt constraint, which restricts each generation to one interaction without refinement, further limits ecological validity because it captures only first-shot model behavior rather than realistic multi-turn human–LLM workflows.

Second, the scope of the dataset is limited both technologically and functionally. The study focuses exclusively on Vue.js and, more specifically, on form-based components. Although the inclusion of both Composition API and Options API introduces architectural variability, the dataset does not cover other frameworks, programming languages, or more complex application scenarios. Additionally, the selected tasks are relatively homogeneous, which may constrain variability in generated outputs and limit insights into more diverse or creative coding tasks.

Third, the dataset exhibits constraints related to linguistic and temporal generalizability. All prompts were written in Italian to ensure consistency, but this choice may influence how politeness and tone are interpreted by LLMs, potentially limiting the applicability of findings to other languages and cultural contexts. Moreover, the dataset reflects specific model configurations available at the time of data collection. Given the rapid evolution of these systems, including updates in capabilities and alignment strategies, the results may not be directly reproducible or fully comparable over time.

Finally, the validation and annotation processes introduce additional limitations. While syntactic correctness and API consistency were assessed, the validation relies partly on heuristic methods and does not fully capture deeper aspects of code quality, such as behavioral correctness, usability, accessibility, security, or maintainability. Furthermore, the dataset does not include rich qualitative annotations, which restricts the ability to perform fine-grained evaluations without additional manual analysis.

Despite these limitations, the dataset remains a valuable and well-controlled benchmark for investigating the relationship between prompt tone and LLM code generation behavior, while also highlighting opportunities for future extensions in scope, realism, and evaluation depth.

5. Conclusions and Future Work

This paper presented the PromptTone dataset, a controlled benchmark for studying the effect of prompt tone on LLM-based code generation in Vue.js. By combining three variables, such as programming paradigm, model provider, and prompt tone, the dataset enables systematic comparisons across 396 generation instances. Its dual representation (hierarchical and CSV), together with prompt-output pairs and error annotations, supports both qualitative and quantitative analyses of code generation behavior.

The dataset contributes to the growing body of research on human–LLM interaction by explicitly isolating the role of communicative tone, a factor often overlooked in code generation studies. Preliminary observations suggest that tone can influence properties such as verbosity and structural organization of the generated code, although these effects vary across models and APIs. This highlights the importance of considering linguistic nuances when designing and evaluating prompting strategies.

Future work can extend this contribution along several directions:

• Dataset expansion: include more diverse UI components and additional frameworks or programming languages to increase variability and external validity.
• Richer evaluation: integrate automated metrics (e.g., accessibility audits, code quality indicators) and manual annotations to enable more comprehensive assessments.
• Multi-turn interactions: move beyond the single-prompt setting to capture iterative and conversational development workflows.
• Multilingual analysis: extend prompts to other languages to study cross-linguistic differences in tone and model responses.

Overall, the PromptTone dataset provides a novel foundation for exploring how subtle variations in human communication affect LLM outputs, with implications for prompt engineering, evaluation methodologies, and the design of more effective AI-assisted development tools.