A Decision Support System Integrating Extended Reality and Conversational AI for Participatory Urban Planning

Abstract

Urban planning increasingly depends on methods capable of capturing citizen perspectives in forms that are both inclusive and analytically useful for decision-making. Conventional participation mechanisms, such as public meetings, paper questionnaires, and online platforms, often suffer from low reach, strong self-selection effects, and weak suitability for structured comparative analysis. This paper presents XRCity, a decision support system that combines extended reality, conversational artificial intelligence, and a planner-side backend to support participatory urban planning in public spaces. The system is centered on Olivia, a life-sized virtual assistant deployed on outdoor interactive screens, and on a backend environment that enables planners to prepare knowledge resources, configure interaction scripts, validate conversational behavior, process transcripts, and analyze elicited opinions. The contribution of the paper is not just the presentation of an XR interface, but the description and validation of a complete decision-support pipeline that connects campaign design, citizen interaction, opinion structuring, and planner-side analytics. The system was validated through real-world deployment in Torres Vedras, Portugal. Across more than 250 interactions and over 740 min of conversation, 191 usable sessions were analyzed, showing an average of 6.7 messages per user and 2.8 min per interaction. Of these sessions, 14.7% produced at least one structured response to an urban planning question, exceeding the project target of 10%. These results indicate the operational feasibility of using public-space conversational XR to elicit analyzable planning input, while a formal validation of the opinion-matching step remains future work.

1. Introduction

Evidence-based urban planning depends not only on technical analyses and expert judgment, but also on the systematic inclusion of citizen perspectives in the decision process. In practice, however, the mechanisms commonly used to gather these perspectives remain problematic. Public meetings tend to attract only a limited and often self-selected subset of the population, while surveys and online platforms can produce more structured data but frequently exclude citizens with lower digital literacy, limited connectivity, or weak motivation to engage in formal consultation processes [1,2,3,4,5]. As a result, planners face a persistent tension between inclusiveness and analytical usefulness: the more open and natural the participation format, the harder it becomes to convert the resulting material into decision-relevant evidence.

Recent advances in extended reality (XR) and conversational AI suggest a possible way to reduce this tension. XR technologies can improve the accessibility and immediacy of interaction in urban contexts, particularly when deployed in public spaces instead of confined to specialized devices or private online access [6,7,8,9,10]. Conversational agents, in turn, can lower the entry barrier to participation by replacing forms and menus with natural language dialogue [11,12,13,14]. Yet the mere coexistence of XR and conversational AI does not by itself yield a decision support system. Existing XR-oriented approaches to public engagement and broader metaverse-oriented visions for urban planning still tend to privilege visualization, immersion, or interaction itself, instead of a planner-side workflow for structuring and analyzing citizen input [15,16]. What is still often missing is a planner-side backend capable of turning a public interaction into a structured participatory instrument: one that can be configured around actual planning alternatives, tested before deployment, and translated afterwards into analyzable evidence for decision-makers [17,18].

XRCity is proposed in response to this gap. The project does not treat the avatar only as a communication device, nor the interaction just as a demonstration of immersive technology. Its core proposition is that citizen engagement can be organized as a decision-support loop. In this loop, planners define the decision context, configure the information and questions to be presented, validate how the conversational system behaves, deploy the interaction in public space, and then inspect the resulting opinions through aggregated analytics. The avatar Olivia, displayed on a weatherproof outdoor digital MUPI (an acronym for the French Mobilier Urbain pour l’Information or informational street furniture), is therefore only the visible element of a broader socio-technical arrangement whose purpose is to support urban planners in eliciting, structuring, and interpreting citizen input.

Related closed-loop decision pipelines have also been reported in other large-scale cyber-physical domains, such as adaptive Connected and Automated Vehicles control in mixed traffic, where sensing, modelling, control, and performance feedback are integrated in an iterative operational loop [19].

The paper examines XRCity primarily as a decision support system for participatory urban planning. It therefore focuses on the planner-side architecture, on the orchestration of the interaction so that free conversation can still yield structured opinions, and on the empirical evidence obtained during the Torres Vedras deployment.

Accordingly, the paper addressed two linked evaluative objectives. The first was technological: to examine whether public-space XR, conversational AI, and a planner-side backend could be integrated into a coherent decision-support workflow. The second was planning-methodological: to examine whether that workflow could support participatory urban planning by turning spontaneous citizen interaction into analyzable input for planner-side interpretation under real deployment conditions. In this sense, the technological innovation lies in the integration of the system components, while the planning-methodological innovation lies in using that integration as a structured participatory instrument connecting campaign design, elicitation, and analytics.

The main contribution is thus twofold. First, the paper describes a complete decision-support pipeline linking campaign configuration, conversational elicitation, transcript processing, and opinion analytics. Second, it reports validation results that indicate both the practical feasibility and the current limits of this approach in a real urban setting. In this sense, the paper is less about demonstrating that XR avatars can attract attention and more about showing how they can be embedded in a workflow that can produce decision-relevant outputs, even though the formal validity of the opinion-matching step is not established here.

The remainder of the paper is organized as follows. Section 2 reviews the literature and identifies the research gap addressed by XRCity. Section 3 presents the system overview, with particular attention to the relationship between the public-facing XR installation and the backend orchestration logic. Section 4 describes the XRCity decision support system from the perspective of campaign design, conversational control, transcript processing, and planner-side analytics. Section 5 reports the validation setting and the main empirical results. Section 6 discusses the significance of the approach for participatory urban decision-making, its practical strengths and limitations, and the main lessons drawn from the deployment. Section 7 discusses future work and concludes the paper.

2. Related Work

XRCity draws on four adjacent bodies of work that rarely meet in a single system: XR in urban contexts, conversational agents for citizen engagement, decision support systems for urban planning, and digital participation platforms. These strands contribute differently to the problem addressed here. XR work contributes accessibility, visibility, and immersion in public space, but usually stops at visualization or interaction. Conversational-agent research contributes natural-language engagement but is usually tied to web or mobile channels and not to planner-side decision workflows. Urban DSS research contributes analytical structure but generally operates on existing data sources instead of direct citizen dialogue. Digital participation platforms contribute organized channels for consultation, but often depend on personal devices, connectivity, and digital literacy. For the present paper, the key question is whether these capabilities can be combined into a single workflow that supports public interaction while also producing structured and usable evidence for urban decision-making. The following subsections review these strands and clarify the specific gap to which XRCity responds.

2.1. XR Technologies in Public Spaces and Urban Contexts

Extended Reality technologies have gained increasing attention in urban planning and public space design. CityScopeAR, developed by Noyman et al., introduced a computational-tangible mixed-reality (MR) platform for collaborative urban design, extending traditional design processes with tangible interfaces augmented on demand [6]. The system demonstrated potential for broad, decentralized community engagement through AR/MR devices and Web-Based AR frameworks, lowering entry barriers for nonprofessional participants. However, CityScopeAR focused primarily on design visualization and collaboration among stakeholders instead of structured feedback collection from general citizens.

ARspace, presented by Díaz et al., utilized Augmented Reality and Visual Analytics to improve public space quality in Bogotá, enabling citizens to create proposals with AR and facilitating decision-making through visual analytics [10]. While stakeholder acceptance was high, the system relied on predefined 3D models, limiting flexibility in citizen-generated proposals. Reaver’s study of AR as a participation tool for youth in Oslo’s urban planning found that AR increased understanding and confidence in displaying design intentions, but highlighted technical limitations including imprecise location tracking and the continued need for expert verification [8].

Recent work by Pajani et al. explored AR and digital placemaking for enhancing urban public spaces, demonstrating how interactive markers allow residents to influence planning and transform environments into smart cities [9]. Paraschivoiu et al. introduced “City Craft,” an AR application for co-located collaborative urban design, which fostered creativity and collaborative mindsets among 33 participants [7]. These studies demonstrate XR’s potential for urban engagement but focus primarily on visualization and design collaboration instead of systematic feedback collection for decision support.

2.2. Conversational Agents for Citizen Engagement

Conversational AI has emerged as a promising tool for citizen engagement in urban planning. Hasan et al. developed a specialized urban planning chatbot trained on planning text and code, finding that 65% of respondents emphasized its suitability for planning tasks [11]. The system demonstrated potential for city planners to collect input and suggest solutions, but operated through traditional digital interfaces instead of public-space deployments.

Condeelis et al. critically examined conversational AI in French public administration, finding that chatbots force administrations to reorganize internal knowledge and provide clear, unified answers, improving efficiency and citizen access to services [12]. However, their work focused on administrative service delivery instead of participatory planning. Borchers et al. explored AI-supported citizen argumentation on urban participation platforms, developing an AI-based feedback system that led to more argumentative and comprehensible contributions [2]. Their subsequent work on supporting citizens to create comprehensible contributions identified design principles for AI-based feedback on digital platforms [13].

While these studies demonstrate the value of conversational AI for urban engagement, they typically operate through web or mobile interfaces, limiting accessibility to digitally connected populations. None integrate conversational AI with XR technology in public-space deployments, nor do they provide comprehensive backend decision support systems for urban planners to design and analyze engagement campaigns.

2.3. Decision Support Systems for Urban Planning

Decision Support Systems have been developed to enhance urban planning processes. The CRISALIDE system, implemented in Rostov-on-Don, utilized urban ontology and artificial intelligence to facilitate brownfield regeneration and public participation [20]. However, the authors noted that new initiatives often focus too heavily on technological solutions while lacking comprehensive understanding of smart development, suggesting a gap in bottom-up, place-based approaches.

Shulajkovska et al. developed an open-source AI framework for smart cities through the Urbanite H2020 project, including a robust DSS for urban policymakers to conduct cost–benefit analyses of traffic management changes [16]. The system integrated simulation tools, Dexi for decision support, and Orange for machine learning, accelerating decision-making by three orders of magnitude. Kalyuzhnaya et al. investigated LLM agents within multi-agent AI systems for smart city management, achieving 94–99% pipeline selection accuracy and 17–55% improved response accuracy with RAG technology [15].

These DSS implementations demonstrate sophisticated analytical capabilities but focus primarily on processing existing data sources instead of facilitating direct citizen engagement. They lack mechanisms for structured feedback collection through natural conversation and do not integrate XR technologies for public-space accessibility.

2.4. Digital Participation Platforms

Digital participation platforms have proliferated in recent years, offering various approaches to citizen engagement. Fegert et al. presented the Take Part app utilizing AR/VR visualizations to motivate citizen participation in construction planning, demonstrating how immersive technologies can enhance imagination and foster common knowledge bases [4]. However, German building law limitations prevented full deployment for public projects.

Marshall et al. examined an experimental online participatory platform for urban regeneration in London, developing a “matrix of participative space” that identified cases of “participative deficit” and “democratic deficit” [1]. Tian et al. designed a digital collaborative platform for urban renewal in China, but encountered challenges including resident willingness, capacity, and credibility due to lack of institutionalized arrangements [3]. Stelzle et al. developed a “minimal viable process” for massive citizen participation, fulfilling good practice standards with minimal technical means [21].

Recent work by Sabitha et al. integrated AR, cloud computing, and machine learning (SVM) to enhance public engagement in sustainable city planning, allowing citizens to experience proposed developments in virtual cityscapes [17]. While these platforms demonstrate diverse approaches to digital participation, most require internet access and digital literacy, potentially excluding vulnerable populations. Furthermore, they typically lack integrated backend systems that enable planners to design structured engagement campaigns and analyze feedback systematically.

2.5. Research Gap

To the best of our knowledge, the reviewed literature provides limited evidence of systems that combine, within a single participatory urban-planning workflow, the four capabilities that XRCity brings together: public-space accessibility, natural-language citizen interaction, structured elicitation around planner-side alternatives, and planner-side analytics. XR studies mainly contribute visualization and co-located engagement [6,7,8,9,10], conversational-agent studies mainly contribute language-based interaction [2,11,12,13], urban DSS studies mainly contribute analytical processing [15,16], and digital participation platforms mainly contribute organized participation channels [1,3,4,17,21]. What remains weakly represented in combination is a system that links these elements into a single loop from campaign design to citizen interaction and back to planner-side interpretation.

XRCity addresses this gap by integrating three critical components: (1) XR technology deployed in public spaces for inclusive accessibility, (2) conversational AI that enables natural language interaction while collecting structured data, and (3) a comprehensive backend DSS that empowers urban planners to design engagement campaigns, validate interaction scripts, process transcripts, and analyze citizen opinions. Within the scope of the literature reviewed here, this integration suggests a distinct approach to participatory urban planning by combining accessible public-space XR, conversational interaction, and planner-side decision support.

3. System Overview

XRCity combines a public-facing XR installation with a backend decision support environment aimed at urban planners. The public-facing component is centered on Olivia, a life-sized virtual assistant displayed on a weatherproof outdoor MUPI. The choice of this deployment format is not incidental. By locating the interaction in a public urban setting, the project seeks to reduce dependence on personal devices, internet access, and prior digital fluency. In this respect, the hardware infrastructure supports a participation strategy: accessibility is treated not as an interface detail but as a design condition intended to broaden reach and lower participation barriers in the data eventually used for planning decisions.

The MUPI infrastructure was designed for robust outdoor operation and for relatively low-friction interaction. It incorporates a 55-inch display, environmental protection suitable for external deployment, embedded computing resources, audio input and output, visual sensors, and 5G connectivity. These features matter not simply as engineering specifications, but because they determine whether the system can sustain usable conversations in real public environments. The large display and life-sized presence of Olivia reinforce social visibility and ease of approach, while the voice-based interaction reduces the need for textual literacy or device familiarity. In other words, the hardware layer is part of the system’s decision-support rationale because it shapes who can participate and under which conditions [6,10].

At the software level, the system integrates three main elements. The SERMAS toolkit [22] is responsible for the XR orchestration of the avatar, including the synchronization of gesture, expression, and verbal output. A customized ChatGPT (Model GPT-4o) instance provides the conversational intelligence, operating over curated knowledge resources and planner-side interaction scripts. The XRCity backend DSS coordinates these components through APIs and manages the planner-side workflow, from campaign preparation to data analysis. The overall result is not a generic chatbot placed in an urban screen, but a coordinated architecture in which conversational behavior is linked to decision-oriented configuration and post-interaction analysis.

Conversational-System Configuration and Control Logic

The conversational component was implemented through a customized ChatGPT instance configured with planner-side knowledge resources and interaction scripts. The system prompt established Olivia’s role, the distinction between informational and elicitation modes, and the constraints governing how planning questions could be introduced. The knowledge base combined municipal and project-specific material prepared in advance through document upload, website-derived content, and manually curated entries. These resources were selected and organized by topic so that the assistant could answer local-context questions while remaining aligned with the campaign under deployment.

The manuscript does not aim to document a software-release-level configuration of the conversational model, and the deployment records do not preserve every parameter needed for exact replication at that level. However, the main operational logic can be stated. Retrieval was grounded in the planner-side knowledge resources prepared for the active campaign, so that the model answered within a bounded municipal and project-specific context instead of relying on unconstrained open-ended generation. Hallucination risk was addressed through this bounded knowledge base, through prompt-level constraints forbidding the invention of planning alternatives not previously defined by planners, and through the script-validator stage, which was used to expose problematic transitions, ambiguous interpretations, or responses that departed from the intended interaction frame. In this sense, the system combined generative flexibility with a controlled decision frame rather than relying on unrestricted conversational generation.

The conversational logic was not left fully open-ended. Guardrails were introduced through prompt-level instructions and script structure so that the model would remain within the intended domain of interaction, avoid inventing planning alternatives not previously defined by the planners, and preserve the distinction between free interaction and structured elicitation. In operational terms, the system was designed to begin in informational mode and to shift to elicitation mode only when the conversation had developed sufficiently in relation to the configured topic. This transition depended on the relevance of the interaction to the campaign context and on the depth of the exchange, as approximated by the progression of the dialogue.

The script validator provided a pre-deployment simulation environment in which planners could test the configured interaction, inspect how the system interpreted responses, and identify problematic or ambiguous transitions before deployment. Its role was not to prove correctness in a formal sense, but to support iterative refinement of the prompts, alternatives, and conversational flow before exposure to public use.

A key part of this architecture is the orchestration layer that manages the transition between two different but connected interaction modes. In one mode, Olivia provides information about local events, cultural heritage, municipal services, or context-specific topics. In the other, Olivia shifts toward the elicitation of opinions on urban interventions. The decision support relevance of the system lies precisely in this controlled transition: the same interaction that attracts citizens through useful conversation can also, when conditions are appropriate, produce structured input on planning alternatives.

Figure 1 summarizes the architecture that connects the public-facing XR installation with the planner-side decision support backend and the orchestration logic between informational and elicitation modes.

4. XRCity as a Decision Support System for Urban Planning

Table 1. Functional summary of the XRCity DSS modules.

Module	Main Input	Main Process	Main Output	Function in the DSS
Knowledge provision	Municipal/project documents, websites, manually curated content	Preparation and organization of domain knowledge	Structured knowledge resources for Olivia	Supports context-aware informational interaction
Interaction script designer	Planner-side intervention, rationale, questions, alternatives	Configuration of participatory campaign logic	Structured elicitation script	Defines the decision frame and available alternatives
Script validator	Configured script and example user inputs	Simulation and inspection of conversational behavior	Refined prompts, transitions, and alternatives	Supports pre-deployment quality control
Transcript processor	Recorded interaction text and session metadata	Anonymization, extraction, flagging of uncertain cases	Structured analyzable interaction records	Converts conversation into usable evidence
Opinion analytics	Structured responses and metadata	Aggregation, segmentation, visualization	Planner-side analytical views	Supports interpretation of citizen input

summarizes the main DSS modules in terms of their input, process, output, and decision-support role.

The most distinctive contribution of XRCity is the way it organizes citizen engagement as a decision-support process instead of as a stand-alone conversational experience. The backend is structured around five modules—knowledge provision, interaction script design, script validation, transcript processing, and opinion analytics—but the real value of the architecture lies in the way these modules support a coherent planning workflow. What matters is not only that each function exists, but that planners can move from problem framing to analyzable citizen input without leaving the system.

4.1. Decision-Oriented Design Rationale and Conversational Choreography

The design of the DSS is based on three assumptions. First, participatory interaction must remain natural enough to encourage engagement. A rigid questionnaire embedded in a public installation would quickly become artificial and discouraging. Second, open conversation alone is not sufficient for decision support, because planners eventually need comparable and aggregable outputs. Third, participation only becomes meaningful in planning terms when citizen input can be connected back to decision alternatives and interpreted by planners in a usable form. These assumptions were already visible in the prototype as constraints imposed by the need to support actual planning decisions.

To deal with these constraints, XRCity uses a conversational choreography that alternates between open interaction and structured elicitation. Citizens may begin by asking Olivia questions about local topics, events, services, or urban context. This free interaction serves an important function: it creates rapport, gives the interaction a public-service dimension, and avoids presenting the system as a disguised questionnaire. At suitable points, however, the system can move toward a decision-oriented segment in which Olivia introduces a planning issue, explains the rationale for the intervention, and asks one or more configured questions. The citizen still answers in natural language, but the answer is interpreted relative to a set of predefined alternatives established in advance by planners.

This mechanism is what makes the interaction useful for decision support. The system does not only record opinions; it maps spontaneous language onto explicit decision alternatives. The mapping is performed through AI-guided opinion matching, which compares the citizen’s utterance with the predefined options associated with the question. The result is stored as structured evidence while preserving the original wording for later inspection. This makes it possible to reconcile two normally conflicting requirements: conversational naturalness during interaction and structured comparability afterwards. The choreography is implemented through explicit state control over the conversation. Interactions begin in informational mode, in which Olivia answers local-context questions and establishes engagement. A transition to elicitation mode is permitted only when the interaction remains sufficiently aligned with the configured campaign topic and has reached a minimum conversational depth. Once in elicitation mode, the system introduces the planning issue, poses the configured question or questions, and avoids repeated prompting beyond the predefined script logic. If the interaction becomes off-topic or too fragmentary, the conversation remains in informational mode and no structured opinion is recorded.

Figure 2 illustrates this decision-support loop and the conversational choreography through which open interaction is converted into structured planning evidence.

4.2. Planner-Side Configuration: Knowledge, Scripts, and Validation

From the planner’s perspective, the first requirement is to control what the system knows and how it frames the participatory dialogue. The knowledge provision module (see Figure 1) supports this by allowing planners to populate Olivia’s knowledge base with materials drawn from documents, websites, and manual content. This ensures that the informational part of the conversation is not generic but tied to the municipal and urban context in which the deployment takes place. More importantly, it lets planners prepare the background information that gives meaning to later opinion questions, which is essential if citizen responses are to be interpreted as informed instead of simply reactive.

The interaction script designer module is the point at which a planning issue is translated into a conversational instrument. Here, planners define the intervention zone or urban object under discussion, articulate the rationale for the intervention, formulate the questions to be posed, and specify the set of predefined answer options that represent the decision alternatives under consideration. This is a crucial design choice: the alternatives are not produced by the conversational model itself; they are determined in advance by planners, which preserves institutional control over the decision frame while still allowing citizens to express themselves in free language. Optional demographic capture can also be configured so that later analyses can explore whether preferences differ across population segments. Methodologically, the system supports a move from loosely formulated participation to explicit preference elicitation grounded in planning alternatives.

Because conversational systems can behave in unexpected ways, XRCity includes a script validator module instead of assuming that a configured script will perform correctly in the field. Through a chatbot-style simulation interface, planners can test the designed interaction, role-play as citizens, observe how responses are classified, and inspect relevant diagnostics such as matched alternatives, confidence levels, or problematic transitions. In practice, this allowed planners to verify whether the system respected the intended transition from open dialogue to elicitation, whether the predefined alternatives were sufficiently distinct, and whether ambiguous responses were being surfaced for inspection instead of silently forced into a category. This module is important from a decision-support viewpoint because it reduces the risk that field deployment will produce unusable evidence due to ambiguity, overlap among alternatives, or poor phrasing. It effectively introduces a pre-deployment quality-control stage into participatory campaign design, which is uncommon in more conventional citizen-engagement platforms.

Taken together, these three modules already reveal that XRCity is an environment in which planners prepare the decision context, structure the elicitation logic, and verify the expected conversational behavior before asking the public to interact with the system. The front-end avatar is therefore only one element of a broader planning instrument.

4.3. From Conversation to Analyzable Evidence: Transcript Processing and Opinion Analytics

Once deployed, the system has to convert live interaction into evidence that can support planning decisions. This is the role of the interaction transcript processor and the opinion analytics modules (see Figure 1).

Table 2. Auditability elements of the AI-guided opinion-matching process.

Element	Description in XRCity
Input to matching	Citizen’s free-form natural-language response to a planner-side elicitation question
Reference set	Predefined answer options established in advance by planners for that specific question
Matching objective	Associate the citizen response with the predefined option judged to be the closest semantic fit
Output stored	Original response text, matched alternative, session metadata, and confidence indication when available
Decision frame	The model selects relative to alternatives predefined by planners
Ambiguous responses	Responses that did not align clearly with any predefined option were flagged for closer inspection
Low-quality interactions	Interactions could be flagged when the session was too short, the speech-recognition quality was poor, or the opinion mapping was ambiguous
Human/expert verification	No formal human-coded ground-truth validation or inter-rater reliability analysis was conducted in the present study
Evidential status	The present paper reports operational feasibility and structured-response yield, not formal classification accuracy

summarizes the opinion-matching process in auditable terms, including what is specified in the current implementation and what remains outside the scope of the present validation.

For the study reported in this paper, the analyzable dataset consisted only of anonymized conversation transcripts and associated session metadata from interactions for which explicit authorization was obtained at the beginning of the conversation. No original audio, image, or biometric data were retained as part of the study dataset. Because no original audio or image data were retained, non-participating bystanders were not included in the analyzed material as identifiable subjects. The system did not perform age identification; accordingly, only interactions for which explicit authorization was obtained at the beginning of the conversation were registered in the analyzable dataset. Access to the anonymized transcripts was restricted to researchers, and retention followed project data-management procedures.

In the implemented system, the mapping from free-form responses to predefined alternatives was carried out through the LLM-based conversational layer operating within the planner-defined decision frame, not through a separate rule-based classifier or a standalone matching module. Confidence indications, when available in the operational interface, were treated as practical signals for inspection. Responses whose relation to the available alternatives remained indirect, mixed, or weakly expressed were not treated as automatically reliable classifications, but were flagged as ambiguous for closer inspection.

Some examples drawn from the transcripts and classified as ambiguous help clarify what this meant in practice. In one deployment question, translated into English as “What do you think of this year’s St Peter’s Fair?”, planners defined four alternatives equivalent to “I am very satisfied with the fair”, “It is reasonable”, “I have visited better editions before”, and “I am not satisfied at all.” One response, “I prefer to visit during the day because it is less crowded,” was treated as ambiguous because it expressed a preference about visiting conditions rather than a direct evaluation of the fair itself. Likewise, “The fair lost some momentum during the pandemic, but now it is still better than it was before” was also treated as ambiguous because it combined a negative comparison with a positive recovery assessment and did not align cleanly with any single predefined alternative. A third response, “I am not much of a fair person,” was likewise treated as ambiguous because it referred primarily to the speaker’s general disposition toward fairs rather than to a sufficiently direct judgment about this specific fair. These examples show how responses with an indirect or mixed semantic relation to the available alternatives were handled more cautiously.

The transcript processor records the dialogue, associates it with time and session metadata, removes personally identifiable information, and extracts the elements needed for later analysis. These include the original natural-language response of the citizen, the matched alternative, the confidence of the match, and any associated demographic information configured in the script. It also flags low-quality interactions, for example when the session is too short, the speech recognition quality is poor, or the opinion mapping is ambiguous. This means that the system does not treat every communication as equally valid evidence; it includes a basic quality filter within the decision-support pipeline. When the mapping between a citizen response and the available alternatives was uncertain, the interaction was flagged for closer inspection instead of being treated as unproblematic structured evidence.

The opinion analytics module is where the decision-oriented value of the system becomes visible to planners. Instead of requiring manual inspection of raw transcripts, the module provides aggregated views of response distributions across alternatives, segmentation by demographic attributes, temporal evolution across the deployment period, and, where relevant, geographic patterns. The original prototype also refers to sentiment analysis and exportable reporting functions, which reinforce the idea that the objective is not only to gather participation traces but to transform them into operational material for planning discussion, reporting, or presentation to municipal bodies.

In practice, the system links the initial definition of alternatives to their later aggregation and interpretation. It begins with planners defining intervention alternatives and ends with planners receiving aggregated and interpretable evidence on how citizens positioned themselves relative to those alternatives (as depicted in Figure 2). This differs from many digital participation tools in which citizen contributions remain largely qualitative and require substantial manual coding before they can inform a decision process. XRCity does not eliminate qualitative richness, because the original responses are preserved, but it adds a structuring layer that makes the material operational more quickly.

The main planner-side functions of the DSS are summarized in Table 1, which presents the modules in terms of input, process, output, and decision-support role.

4.4. Technical Implementation and Operational Workflow

The DSS was implemented with a conventional but practical web stack based on Java Servlets on the backend, HTML/CSS/JavaScript on the frontend, and MySQL for persistent data management, running on a Linux VPS with Apache Tomcat. REST APIs handle communication with the SERMAS layer and the conversational model. The deployed system preserved complete conversation logs with turn-level timestamps, which supported post hoc analysis of session duration, message counts, and conversational response timing. These logs also made it possible to identify whether a session reached the elicitation stage when the configured planning prompts appeared in the interaction. However, these logs did not amount to a complete technical monitoring system. As a result, they did not provide full uptime statistics or a detailed breakdown of network problems, audio-hardware issues, or runtime errors. The architecture is modular and was sufficiently stable to support real-world deployment and iterative refinement. The use of a mainstream web architecture suggests that the backend software stack is not dependent on exotic infrastructure and can, in principle, be reproduced with widely available web technologies. This should not be conflated with operational replication of the full public-space installation, which also depends on outdoor hardware, connectivity, sensors, audio robustness, XR-toolkit integration, AI-service access, public-space maintenance, and data-governance arrangements.

Operationally, the workflow described in the prototype can be read as a participatory decision cycle. Planners first define the intervention and its rationale, then populate the knowledge base, configure the questions and alternatives, validate the script, deploy the MUPI in a relevant public location, collect interactions, process the resulting transcripts, and finally inspect the aggregated opinions. What matters is that each step feeds the next, from campaign design to interpretation of results, and the system is organized so that the eventual analytics remain traceable to the initial decision framing established by planners. This strengthens the interpretation of XRCity as a coherent decision-support workflow linking campaign design, interaction, and analysis.

5. Validation and Results

Table 3. Summary of validation phases and analyzed data.

Phase	Context and Period	Main Issue(s) Identified	Adjustment(s)/ Outcome	Data Considered in This Paper
Preliminary laboratory testing	Controlled pre-deployment testing, before public deployment	Conversational drift, microphone sensitivity, limitations in topic handling	Prompt and interaction refinements before field use	Not part of the analyzed field dataset
Public deployment at St. Peter’s Fair	Public event setting, Torres Vedras, 26 June–6 July 2025	Outdoor acoustic instability, especially due to loudspeakers and music in the surrounding area	Real-world deployment under high-footfall conditions; 250+ interactions and 740+ min of conversation	Included in the empirical basis of the paper
Final street installation	Ordinary urban public-space setting in Torres Vedras, after the fair deployment	Lower-footfall but more ordinary operating conditions	Complementary operational observation in non-event context	Reported as part of the deployment context
Final analyzed subset	Derived from the real deployment data	Not all observed interactions yielded transcript material suitable for analysis	191 usable sessions retained; 28 produced at least one structured planning response	Main analytical dataset used in Section 5.2 and Section 5.3

summarizes the main validation phases, their context, the main issues identified, the adjustments introduced, and the data considered in the present paper.

5.1. Deployment Context and Iterative Validation

The system was validated through public deployment during St. Peter’s Fair in Torres Vedras, Portugal, from 26 June to 6 July 2025 (Figure 3), followed by its final installation on a street in Torres Vedras (Figure 4). This context was appropriate for two reasons. First, the fair provided a high-footfall public setting, reportedly attracting around 233 thousand visitors over the event period, and therefore offered favorable conditions to observe whether the installation could attract and sustain spontaneous engagement. Second, the subsequent urban deployment offered a complementary, less event-driven context in which the system could be observed in a more ordinary public setting. In both cases, the relevant question was not only whether citizens approached the avatar, but whether the interaction could generate material usable in a planning-oriented decision pipeline.

The validation followed an iterative process in which preliminary testing was used to identify and correct weaknesses such as conversational drift, microphone sensitivity, and limitations in topic handling. Before public deployment at St. Peter’s Fair, the system had already undergone laboratory testing. The main gap observed in the fair setting was not a change in the intended conversational logic itself, but the effect of the outdoor acoustic environment, particularly the presence of loudspeakers and music performances at some distance, which made speech capture less stable than under controlled conditions. The final deployment therefore did not assess a static prototype, but a refined version of the system shaped by successive adjustments. This is relevant because the reported results reflect not only public interaction in the field, but also an effort to improve the reliability of the decision-support process before large-scale use.

5.2. Engagement, Structured Responses, and KPI Achievement

For clarity, three related but distinct units are used in this section. An interaction refers to any observed instance of citizen engagement with the system in the field, including brief “Hello!”. A session refers to a registered conversational exchange captured by the system as a discrete interaction record. A usable session refers to a registered session that contained sufficient transcript material to support analysis in the present study. The distinction matters because not every observed interaction resulted in a session record suitable for later analytical treatment.

Across more than 40 h of activity, over 11 days, the deployment generated over 250 citizen interactions and more than 740 min of conversation. From these, 191 sessions were considered usable for analysis. The study preserved the analyzed subset in a way that supports transparent reporting of the usable sessions and their outcomes. However, the recorded material does not support a fully granular reconstruction of all non-usable cases by exclusion cause. At the session level, the interaction averaged 6.7 (back-and-forward) messages per user and 2.8 min in duration, with sessions ranging from brief exchanges to extended multi-turn dialogues. These figures indicate that engagement was not limited to a momentary curiosity effect. Citizens were often willing to maintain a conversation long enough for the system to move beyond greeting or information provision and, in some cases, into structured opinion elicitation.

The most relevant result from the perspective of urban decision support is that 14.7% of usable sessions produced structured responses to urban planning questions. In absolute terms this means that 28 of the 191 sessions analyzed yielded directly classifiable opinions on planning-related alternatives. The proportion is modest by survey standards, but the comparison is imperfect because the interaction was voluntary, situated in open public space, and often approached for informational or exploratory reasons rather than with prior motivation to discuss urban interventions. Under those conditions, the fact that the threshold of 10% defined for the project was exceeded is meaningful because it shows that the system can indeed convert a share of spontaneous public interaction into structured planning input. This 10% threshold was a project-defined operational target used as a feasibility benchmark for the demonstrator, not a literature-based success criterion or a comparative performance standard. The 14.7% figure should therefore be read relative to the 191 usable sessions analyzed in this study, not as a percentage of all observed field interactions. For the same reason, the present study cannot estimate the conversion rate from all observed interactions to structured opinion production, but only the yield within the usable subset retained for analysis. Representativeness of the participating population was not assessed in the present study.

The present study did not include a controlled comparison with a conventional survey, QR-based questionnaire, or other standard participation instrument using the same questions and alternatives. For that reason, the reported results should be interpreted as evidence of feasibility under real deployment conditions, not as evidence of comparative superiority over established methods.

The validation confirms that the system attracted substantial public interaction and was also subjected to formal social acceptance assessment, to be reported in a different article. More importantly, a measurable share of usable sessions produced structured responses to urban planning questions. For the purposes of this paper, the key result is that a measurable share of public interactions yielded evidence that planners could interpret directly.

5.3. DSS Performance and Observed Limitations

The validation also provided evidence on how the decision-support pipeline behaved under real deployment conditions, particularly with respect to conversational control, opinion capture, transcript processing, and planner-side use of the results. More than suggesting a fully stabilized system, the deployment showed a pipeline that was already operational and useful, but still sensitive to some known limitations of public-space conversational interaction.

The deployment preserved complete conversation logs with timestamps, allowing analysis of session duration, message counts, and the occurrence of elicitation prompts within the interaction. However, the present study did not include a full infrastructure-level logging layer or a formal annotation scheme for all intermediate conversational states, so finer-grained indicators such as validated drift rates, complete confidence distributions, or lower-level runtime diagnostics are only partially available.

With regard to conversational control, the structured interaction logic was generally maintained across the recorded sessions, allowing the system to move from open dialogue to planning-oriented elicitation when the conversation developed appropriately. At the same time, some deviations from the intended script flow were observed, especially in longer interactions. These cases were associated with conversational drift, in which the accumulated context led the model away from the predefined structure. Early prototype testing had already exposed this issue and led to targeted refinements in prompt design, which improved behavior but did not eliminate the problem entirely. This remains an important limitation, because the reliability of the decision-support process depends not only on attracting interaction, but also on preserving the link between the live conversation and the planning alternatives defined in advance.

The opinion-matching mechanism was examined qualitatively as a means of translating free-form citizen responses into structured alternatives usable for later analysis. The strongest quantitative indication available in this study is that 28 of the 191 analyzed sessions (14.7%) yielded at least one structured response to an urban planning question, exceeding the project threshold of 10%. This result does not establish classification accuracy in a strict sense, since the present study does not include a ground-truth-labelled dataset, inter-rater reliability analysis, or a formal comparison between automated assignment and independent human coding. Accordingly, the present manuscript should not be read as providing a post hoc human-validation study of the matching outcomes, but as documenting the operational use of the mechanism under field conditions. It does, however, show that the system was capable of converting a meaningful share of spontaneous public interaction into analyzable planning input. Cases in which responses were ambiguous or did not align clearly with any predefined option required closer inspection, reinforcing the importance of carefully framed alternatives during script design.

The transcript processing component also proved operationally relevant. Conversations were captured and anonymized successfully, and no leakage of personally identifiable information was detected during inspection of the recorded material. No sessions were excluded specifically on the basis of acoustic noise, microphone sensitivity, network latency, or screen placement. However, these factors influenced the smoothness of interaction in the field. The most evident limitation was environmental noise during the fair, particularly due to loudspeakers and music performances in the surrounding area, which reduced speech-recognition stability and made progression to the elicitation stage less reliable in some interactions. This effect was observed qualitatively during deployment, but it was not instrumented in a way that would allow its exact impact on elicitation probability to be quantified. The mitigations applied in practice included prior laboratory testing, iterative adjustment of the system before final deployment, and attention to deployment conditions in the field.

Finally, the analytics component was operationally available to planners and enabled inspection of the collected responses through aggregated and segmented views. The resulting visualizations and segmented views allowed the collected responses to be inspected without requiring technical mediation, which is an essential condition for describing XRCity as a decision support system rather than merely as a data collection interface. Even so, the current validation should be read as evidence of practical feasibility rather than final maturity. The deployment confirms that the pipeline can support planner-side interpretation of citizen input, while also making clear where further technical refinement is still needed. The planner-side evidence reported here remains qualitative and should not be interpreted as a formal usability, task-based, or expert-review study of the analytics component.

For this reason, the current results should be interpreted as evidence of field feasibility and structured-response yield, not as a complete operational reliability assessment of each stage of the conversational pipeline.

Figure 5 synthesizes the main empirical results discussed in Section 5.2 and Section 5.3, bringing together engagement indicators and the structured-response yield used in this paper as evidence of field feasibility. The 14.7% value corresponds to 28 of the 191 usable sessions analyzed, not to all observed interactions.

6. Discussion

6.1. Contribution to Participatory Urban Decision-Making

The main value of XRCity is that it treats participatory interaction as part of a decision process instead of as an isolated communication event. Many participation tools succeed in collecting comments, but fewer succeed in connecting those comments to explicit alternatives, preserving conversational naturalness, and returning the result to planners in a form that can support interpretation. XRCity was designed with that objective. The system gives planners control over the decision frame, lets citizens respond in ordinary language, and then translates those responses into structured evidence through a transparent pipeline of matching, processing, and aggregation. In that sense, the project can be read as an effort to operationalize a human-in-the-loop decision cycle in urban planning instead of simply to digitize citizen engagement.

Seen from this perspective, the deployment results are easier to interpret. Three levels should be distinguished in interpreting these results. First, operational feasibility concerns whether the system can function in public space and sustain interactions under real deployment conditions. Second, capture rate concerns whether a meaningful share of those interactions yields structured planning-related responses, which in this study corresponds to 28 of 191 analyzed sessions (14.7%). Third, classification validity would concern whether the assigned alternative correctly reflects the citizen’s intended opinion. The present study provides evidence for the first two levels, but not for the third, which would require dedicated validation against independently coded ground truth.

The most important outcome is not that the avatar attracted public attention, although it did, nor that the interaction was socially accepted, although that matters. The most important outcome is that a public-space conversational installation was able to produce a non-trivial amount of structured responses linked to planning questions and to do so within a workflow controlled by planners. This is a stronger and more precise claim, especially from the decision support perspective, than stating that the project is innovative in XR participation.

6.2. Comparison with Established Participation Methods

The comparison below is analytical and functional in nature, not the result of a controlled empirical benchmark conducted within this study. This positioning is nevertheless consistent with the practical experience of the municipal partner, for whom formal street inquiries have often proved difficult to conduct and to translate into usable planning input, although this was not assessed here through a controlled comparison.

Compared with town hall meetings, XRCity offers broader situational accessibility and a more systematic path from interaction to analyzable outputs, but it does not replace the deliberate richness of face-to-face collective discussion. Compared with online surveys, it is less rigid and potentially more inclusive, yet it also yields a lower proportion of fully structured responses because the interaction remains voluntary, situated, and conversational. Compared with digital participation platforms, its potential advantage lies in lowering some of the barriers associated with personal-device-dependent online consultation, although this should be understood as a design affordance rather than as an empirically demonstrated inclusion outcome in the present study. Compared with formal public comment procedures, their outputs are more immediate and potentially more diverse, but they do not automatically possess the same legal or procedural status. The appropriate conclusion is therefore not that XRCity should substitute these mechanisms, but that it can complement them by occupying a space they cover poorly: low-threshold, in-situ, structured conversational elicitation in public space.

6.3. Practical Lessons and Future Directions

Several lessons follow from the prototype and its validation. First, hardware and conversational logic cannot be separated in public-space deployments. Acoustic quality, microphone range, latency, and screen presence all condition whether the decision-support sequence will unfold successfully. Second, the quality of the planner-side alternatives is central. Because the system maps free responses to predefined options, the analytical value of the final evidence depends heavily on how clearly these alternatives are framed during script design and validation. Third, the script validator is not a convenience feature but an essential part of methodological quality assurance, since it reduces the chance that flawed conversational logic will contaminate the collected evidence. A related lesson concerns ethics and data governance in public-space deployment: when conversational systems operate in open urban environments, methodological quality also depends on transparent consent, careful delimitation of what data are stored, anonymization of analyzable material, and clear institutional responsibility for how citizen input is processed and used.

The limitations identified in the prototype point to a clear future agenda. Multilingual support is needed if the system is to claim broader inclusiveness. More robust state management and possibly stronger retrieval support are needed to reduce drift in long conversations. Real-time analytics would make the platform more useful for live campaign steering. Integration with GIS and other municipal systems would strengthen the downstream use of the data in planning practice. Finally, longer and explicitly comparative studies are required to determine how this approach performs over time and relative to more conventional participation mechanisms, including controlled comparisons with short on-site or QR-based surveys using the same questions and alternatives, assessments of representativeness and cost per usable structured response, and richer operational logging to support the analysis of conversational drift, transitions to elicitation mode, question-completion patterns, and other intermediate indicators of system performance.

7. Conclusions

This paper analyzed XRCity as a decision support system for participatory urban planning. The paper framed XRCity as a complete pipeline connecting campaign design, conversational elicitation, transcript processing, and planner-side analytics. In this formulation, the central contribution of XRCity is not the conversational interface alone, but the integration of that interface within a workflow capable of eliciting and structuring citizen input for planner-side analysis and decision.

The real-world deployment in Torres Vedras indicates that this approach is feasible and practically meaningful. The system attracted substantial spontaneous engagement, sustained multi-turn interactions, and converted a portion of those interactions into structured responses to planning questions, surpassing the defined project threshold. At the same time, the results make clear that the approach remains sensitive to conversational drift, acoustic conditions, and the quality of the predefined alternatives used for opinion matching. These are not marginal technical issues; they define the reliability of the evidence generated for decision-making.

At the same time, the present study demonstrates operational feasibility rather than formal validity of the opinion-matching mechanism, which still requires dedicated evaluation against human-coded ground truth.

Beyond technical reliability, the long-term value of systems such as XRCity will also depend on whether use can be sustained after the initial novelty of the interaction declines. In that respect, factors such as usability, enjoyment, and perceived purpose are likely to matter for continued engagement. If municipalities do not provide visible feedback on how citizen input is considered or used, participants may gradually stop seeing the point of contributing. Future developments should therefore address not only the robustness of the decision-support pipeline, but also the renewal of the interaction experience and the communication of how citizen contributions inform municipal decisions.

Overall, XRCity is best seen as an early but functional contribution to participatory urban decision support. The main implication is that public-space XR and conversational AI can be configured to support a structured decision loop in which planners define issues, citizens respond in natural language, and the resulting material is returned to planners in an analyzable form. That is the contribution on which future work should build.