Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review

Abstract

Rapid advancements in foundation models, including Large Language Models, Vision-Language Models, Multimodal Large Language Models, and Vision-Language-Action models, have opened new avenues for embodied AI in mobile service robotics. By combining foundation models with the principles of embodied AI, where intelligent systems perceive, reason, and act through physical interaction, mobile service robots can achieve more flexible understanding, adaptive behavior, and robust task execution in dynamic real-world environments. Despite this progress, embodied AI for mobile service robots continues to face fundamental challenges related to the translation of natural language instructions into executable robot actions, multimodal perception in human-centered environments, uncertainty estimation for safe decision-making, and computational constraints for real-time onboard deployment. In this paper, we present the first systematic review of foundation models in mobile service robotics, following the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines. Using an OpenAlex literature search, we considered 7506 papers for the years spanning 1968–2025. Our detailed analysis identified four main challenges and how recent advances in foundation models, related to the translation of natural language instructions into executable robot actions, multimodal perception in human-centered environments, uncertainty estimation for safe decision-making, and computational constraints for real-time onboard deployment, have addressed these challenges. We further examine real-world applications in domestic assistance, healthcare, and service automation, highlighting how foundation models enable context-aware, socially responsive, and generalizable robot behaviors. Beyond technical considerations, we discuss ethical, societal, human-interaction, and physical design and ergonomic implications associated with deploying foundation-model-enabled service robots in human environments. Finally, we outline future research directions emphasizing reliability and lifelong adaptation, privacy-aware and resource-constrained deployment, as well as the governance and human-in-the-loop frameworks required for safe, scalable, and trustworthy mobile service robotics.

1. Introduction

In recent years, there have been rapid advancements in foundation models, including Large Language Models (LLMs) [1,2,3,4,5,6], Vision Language Models (VLMs) [7,8,9], Multimodal Large Language Models (MLLMs) [10,11], and Vision Language Action Models (VLAs) [12,13,14]. These models are significantly changing the overall field of robotics and its numerous applications. In particular, there have been notable improvements to personalized task management and context-aware smart assistant capabilities for domestic assistance [15,16,17], healthcare [18,19,20,21], and service automation [22,23]. Foundation models are trained on vast datasets to understand and generate human-like text and speech, as well as interpret visual information and reason. For example, language-guided mobile service robots utilizing foundation models can carry out complex instructions [24,25,26], perform fetch and carry tasks [27,28,29,30] and/or socially interact [31,32,33] with human users.

As robots are deployed in real-world environments to increasingly interact with humans, the integration of language-based intelligence has become crucial for intuitive and effective communications. In particular, mobile service robots can provide adaptable, context-aware assistance in dynamic and unpredictable environments to achieve various goals, from navigating person-centered environments to object or person detection and identification. Their ability to autonomously interpret human instructions, reason about tasks, and execute actions has enabled them to be deployed in numerous environments, from warehouses [34,35] to hospitals [12,36] and personal homes [26,37,38].

By leveraging the advancements in foundation models, service robots can bridge the gap between AI-driven perception and human-like decision-making, leading to natural human–robot interactions (HRIs) and collaborations in everyday scenarios. For mobile service robots, success also depends on dexterous, compliant physical interactions with people and objects, adherence to HRI constraints (e.g., personal space, idiosyncrasies, behavior predictability), and long-term autonomy in human-centered environments. However, to fully integrate the promising capabilities of foundation models, the open challenges in the development and deployment of embodied AI mobile service robots must be addressed. These open challenges include:

(1)

Translation of natural language instructions into executable robot actions: Mobile service robots need to handle ambiguous, incomplete, or colloquial instructions (“bring me that from the other room”) while grounding these instructions in physical navigation and manipulation tasks [39,40,41,42,43]. This can be challenging in domestic and healthcare settings, where instructions may be highly contextualized by the environment or a user’s state (e.g., a patient who is resting, in pain, or under medication may give shortened or unclear commands that require contextual interpretation by the robot). They also require long-horizon, context-aware planning across multiple rooms, dynamic layouts, and human activities [44,45,46].

(2)

Multi-modal perception: Mobile service robots must integrate multimodal inputs such as vision, speech, and touch while operating in human-centered environments such as homes and hospitals. As they navigate from region to region, they need to be able to adapt to: (1) lighting variations, (2) crowds, (3) occlusions by dynamic people, furniture, and/or medical equipment, and 4) noisy environments with background conversations or sounds including alarms [47,48,49,50].

(3)

Uncertainty estimation: As mobile service robots operate in safety-critical, human-facing environments, decisions must be made under partial observability, unpredictable human behaviors, and socially acceptable norms [51,52,53]. Namely, they need to estimate both aleatoric (sensor/environmental) and epistemic (model) uncertainty while ensuring their actions remain socially acceptable. However, current embodied AI methods struggle with uncertainty estimation, leading to overconfident predictions that can risk user safety. For example, a hospital delivery robot might proceed through a crowded hallway despite incomplete LiDAR or vision data, misjudging the proximity of patients or equipment and causing potential collisions [54,55,56,57].

(4)

Computational capabilities: Mobile service robots are constrained by onboard hardware, as continuous reliance on remote servers or cloud computation is often infeasible due to latency, connectivity instability, and data privacy concerns in hospitals and homes [58,59,60]. Sensing, perception, and decision-making need to be executed locally on embedded GPUs, NPUs, or other onboard accelerators close to the data source. The challenge lies within the tight compute, energy, and latency budgets of embedded platforms while ensuring safe operation around humans.

In general, foundation models can address the above challenges. They can bridge the gap between high-level human instruction, language understanding and low-level robotic control to enable more intelligent, adaptable, and interactive robots for diverse real-world applications, as shown in Figure 1.

To date, existing surveys have primarily examined either broad, non-task-specific applications in general-purpose robotics [61,62] or language-conditioned manipulation tasks for stationary robotic arms [63]. These surveys have not yet investigated the role of mobility in enabling robots to assist with human-centered tasks. In this paper, we present the first systematic review and analysis on the integration of foundation models in mobile service robotics, examining their role in advancing embodied AI. We introduce a unified architecture view in Figure 2 that illustrates how foundation models can integrate perception, planning, and control within mobile robots operating in human environments. Building on this integrated perspective, we identify research challenges and discuss how foundation models can enable task generalization, dynamic scene understanding, and social compliance. In particular, we focus on the promising applications of domestic assistance, healthcare and service automation for mobile service robots.

2. Methods Used for Identifying the Research Challenges

To quantify how research effort in mobile service robotics has been distributed to date, we used the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines [64]. Namely, we conducted a literature review analysis using OpenAlex [65], a large-scale scientific indexing platform. We queried and filtered papers related to mobile service robotics using title- and abstract-level keyword searches. The inclusion criteria comprised: (1) papers between 1968 to 2025 to capture the historical development of embodied robotic intelligence; (2) English-language papers; and (3) papers addressing embodied robotic systems intended for deployment in mobile or mobile-manipulation contexts. Exclusion criteria included papers focused primarily on: (1) autonomous driving and vehicular systems, (2) aerial robotics (e.g., UAVs and drones), (3) surgical robots, (4) underwater robotics, and/or (5) biological systems; and (6) papers lacking embodied robotics. A final total of 7506 papers were included in our systematic review. The identified literature was then grouped by recurring technical themes that consistently emerged across decades of research. A comprehensive description of the PRISMA [64] checklist mapping is provided in Appendix A.

2.1. Search Strategy

The search strategy was designed to systematically capture research relevant to embodied and embedded artificial intelligence in mobile service robotics. The process consisted of three stages: (1) keywords to define the robotics scope, (2) domain-level filtering to remove unrelated robots and application areas, and (3) embodied robot intelligence screening. Each stage involved title- and abstract-level keyword filtering to progressively refine the corpus toward embodied mobile service robotics.

(1)

Keyword query: An initial keyword query targeting general autonomous robots from a constrained timeline of 1968 to 2025 was performed to constrain the search space from the beginning of robotic autonomy to the present. This query returned 23,210 papers.

(2)

Domain-level filtering: Papers were filtered to remove research domains categorically outside the scope of mobile service robotics using the exclusion criteria. This stage ensured the corpus was restricted to ground-based mobile robotic systems. A total of 4530 papers were removed, leaving 18,680 papers.

(3)

Embodied robot intelligence screening: The remaining papers were then examined to ensure that they substantively addressed embodied robotic intelligence within the previously defined mobile robotics scope. At this stage, we retained papers that engaged core embodied AI functions in physical robotic systems (e.g., perception-action coupling, language grounding, planning, control, or computational deployment considerations). This refinement step removed 11,174 papers, resulting in a final corpus of 7506 papers included in this systematic mapping review.

The resulting corpus represents the primary high-level research challenges in embodied mobile service robotics, spanning perception, language interaction, planning, control, and computational intelligence. These areas reflect broad technical concentrations observed in the existing research.

Please note that for this study, Nano Banana Pro AI Image Generator was used for the sole purpose of generating illustrative conceptual images in selected figures presented, as explicitly disclosed in the respective figure captions and in the Acknowledgments section.

3. Open Challenges of Embodied AI for Mobile Service Robots

We analyzed the final corpus to identify recurring open challenges in mobile service robot research. Based on the systematic mapping described in Section 2, we identified four recurring research challenges that encompass open challenges in embodied AI for mobile service robots.

Table 1. Ranking of the four core challenges in mobile service robotics.

Rank	Challenge	Paper Count N	Percent of Papers (N/7506 × 100%)	Conclusions
#1	Challenge #1 Language-to-Action Mapping	2190	29%	Largest research focus: semantic grounding, task sequencing, navigation & manipulation logic
#2	Challenge #2 Multimodal Perception	2164	29%	Strong support for cross-modal sensing and perception alignment
#3	Challenge #3 Uncertainty Estimation	2005	27%	Strong emphasis on safety, robustness, and decision-making under uncertainty
#4	Challenge #4 Computational Capabilities	1147	15%	Focus on real-time execution, embedded efficiency, and resource constraints

ranks these challenges according to the proportion of papers in the 7506-paper corpus. Language-to-action mapping ranked the highest (29.18%), representing an emphasis on grounding instructions into executable behavior through planning, navigation, and manipulation. Multimodal Perception ranked second (28.83%), highlighting the challenge of integrating heterogeneous sensory inputs for varying environments. Uncertainty Estimation was ranked third (26.71%) due to the concerns related to robustness, safety, and decision-making under ambiguity in human-centered settings. Computational capabilities, while ranked fourth (15.28%), still remains a limitation to real-time execution and embedded deployment. Below, we discuss each of the four challenges in more details, while defining specific subchallenges for each.

3.1. Challenge #1: Translation of Natural Language Instructions into Executable Robot Actions

Enabling language-guided mobile service robots to follow high-level natural language commands provided by humans remains a challenge. Key limitations include:

(1)

Symbolic-to-embodied mapping and instruction ambiguity: Natural language instructions are typically provided by non-expert human users. These commands are often expressed at a high level (“clean the room,” “get my walking cane”) and may include colloquial, ambiguous, or underspecified phrasing, lacking precise object identifiers, spatial context, or temporal constraints. While classical planners such as STRIPS [39] and SHOP2 [40,41], or logic-based systems (e.g., PDDL) [42,43] attempt to decompose commands into discrete subgoals, they rely on hand-engineered operators that are domain-specific and brittle [66,67,68,69]. This challenge is particularly critical for mobile service robots engaging in HRI with people in different roles who may provide incomplete requests (“bring me that”) or might use gestures and speech simultaneously, requiring robots to adapt ambiguous instructions into executable actions.

(2)

Lack of embodied commonsense and awareness of physical constraints: Mobile service robots must not only interpret commands but also reason about constraints in human-centered environments. Many symbolic planners, grounding models, and PDDL-based models lack awareness of robot embodiment and environmental variability, according to refs. [70,71]. While language-only systems or semantic parsers trained on demonstrations can capture basic kinematics or affordances, they are not able to generalize across diverse service contexts [44,45,46]. For example, a patient may ask a robot to “place the food tray on the bed,” but without understanding that the bed is already occupied, the robot could attempt an unsafe or socially inappropriate action. Similarly, commands such as “open the door for my grandson” may be physically infeasible if the robot lacks the dexterity to open a door. In these cases, failing to account for physical constraints can result not only in task failure but also in loss of user trust.

(3)

Failure in long-horizon task planning: Many mobile service tasks, such as medication delivery, multi-room navigation, or guided patient assistance unfold over extended time horizons, requiring robots to reason across multiple steps, maintain working memory, and adapt to evolving conditions [72]. Classical planners such as Simple Task Network (STN) Planning [73] or Fast Forward (FF) [74] perform well in static domains, but they are not able to handle dynamic, human-centered environments, in which unexpected subgoals may emerge due to users’ needs or environmental changes [75,76]. Learning-based models such as Neural Task Graphs (NTGs) [77] and Option-Critic [78] offer flexibility in sequencing actions, yet they can suffer from policy drift, compounding errors, and limited memory retention when deployed in real-world service contexts [79,80,81]. As a result, mobile service robots are unable to maintain task continuity, recover from errors, or adapt their plans in response to user interruptions or environmental changes, leading to cascading task failures during long-horizon service execution.

(4)

Inability to handle mid-task requirement changes: Mobile service robots operating in human-centered environments need to be able to respond to mid-execution requirement changes, such as users: (1) changing goals (e.g., “bring me the red cup instead”), (2) reprioritizing actions (e.g., “stop cleaning and answer the door first”), or (3) interrupting tasks (e.g., “leave it and come here now”). These change requests have been addressed in classical STRIPS [39]- or PDDL [42,43]-based planners using execution monitoring frameworks that trigger either full replanning or plan repair within deliberative-reactive control architectures [82]. Classical three-layer deliberative-reactive architectures (e.g., planning, sequencing, and feedback control) allow robots to pause action execution, update the symbolic goal state and task constraints in the planning model, and recompute an updated action sequence. However, they rely heavily on hand-engineered domain operators and brittle state estimators [83]. Plan repair methods were introduced to avoid recomputing an entire plan from scratch when goals or constraints change, instead modifying only the affected portion of the remaining plan while preserving valid steps, thereby reducing computational cost and maintaining behavioral continuity [84]. Incremental and anytime planners (e.g., D*-style algorithms [85]) used in navigation and delivery tasks (be it within homes, hospitals, or public spaces) support responsiveness under changing goals or obstacle configurations by reusing previous search trees and updating only impacted graph regions; however, they are limited primarily to spatial objectives and do not generalize to high-level task restructuring [86]. Task-and-motion planning (TAMP) [87] frameworks for mobile manipulation provide joint symbolic–geometric replanning under revised task constraints, yet can incur computational overhead for real-time onboard deployment [87]. Reactive controllers and behavior-tree-based methods enable low-latency interruption handling and fallback behaviors but can only accommodate requirement changes expressible through pre-coded branches, limiting flexibility to designer-specified contingencies [82]. Probabilistic interaction models such as Partially Observable Markov Decision Processes (POMDPs) [80] for dialogue and intent tracking allow belief state updates and clarification under noisy or changing user intent but can scale poorly with state and action complexity and are often simplified for practical use [88]. As a result, these classical approaches struggle to robustly handle frequent, open-ended goal supersession and dynamic intent revision in unconstrained domestic service settings, often requiring conservative task structures or manual intervention.

3.2. Challenge #2: Multimodal Perception

Multimodal perception is essential for mobile service robots to interpret complex environments. These modalities often differ in spatial resolution, sampling frequency, and noise characteristics, making real-time sensor fusion and alignment difficult in human-centered environments. We discuss four core limitations below.

(1)

Cross-modal representation: Sensory data from various robotic sensors need to be fused into a shared latent space to support real-time reasoning in mobile service robots. Fusion architectures are commonly classified into [89]: (i) early fusion, where raw inputs are concatenated or merged at the pixel level; (ii) late fusion, which aggregates decisions from different modalities; and/or (iii) intermediate fusion, where features from each modality are encoded and combined using attention mechanisms, joint embeddings, or cross-modal transformers. However, for mobile service robots, each fusion type presents unique challenges. Early fusion is sensitive to cluttered environments, background conversations, and variations in lighting [90], since it combines raw, low-level sensory inputs (e.g., pixel intensities or waveform amplitudes) before feature extraction. This means that any noise or occlusion in one modality, such as poor lighting in images or overlapping human speech in audio, can potentially corrupt the entire fused representation, leading to degraded perception accuracy and unstable behavior in mobile service robots. Late fusion can present delayed or inconsistent results in the fused perception output and downstream decision-making when different modalities provide conflicting cues, such as audio indicating a person’s presence while vision temporarily loses track of the person due to occlusions in crowded human spaces, leading to uncertainty for situational awareness [91,92]. Intermediate fusion, which combines encoded feature representations from each modality using attention or joint embeddings, often leads to temporal gaps when aligning robot data streams with different sampling rates, resolutions, and noise characteristics. This can cause failures, for example, in gesture–speech association during HRI [48,49].

The aforementioned challenges are compounded by spatial misalignment and temporal desynchronization when robots (i) follow or guide users across different rooms or (ii) interact with multiple people simultaneously. When a caregiver both speaks (“pass me that medication”) and simultaneously points toward a medicine cart, the robot must associate the speech command with the pointing gesture. If the audio (speech) and visual (gesture) signals are not synchronized, the robot may misinterpret which object the caregiver is referring to, resulting in ineffective assistance. In practice, fusion inconsistencies can cause object association errors, scene parsing failures, and HRI breakdowns. Such failures can directly undermine fine-grained context interpretation, and a service robot’s ability to disambiguate human intent and environmental cues to provide task assistance.

(2)

Latency issues: Mobile service robots often rely on heterogeneous sensor streams that operate at different sampling rates, leading to temporal desynchronization [93,94,95]. These robots need to maintain situational awareness in dynamic, human-centered environments where people can have unpredictable motion, and tasks require real-time social responsiveness. Unlike fusion errors, latency stems from delays in updating perception and control loops. For example, a domestic assistant robot may take an unnecessarily long detour, as obstacle information was updated too late during navigation [93]. These delays reduce interaction responsiveness and fluidity, making robots appear slow or hesitant in environments where people expect robots to adapt in real time.

(3)

Uncertainty propagation across modalities: Sensor quality may degrade due to lighting changes, occlusions due to people, cluttered rooms, and environmental noise [96,97,98]. Uncertainty propagation occurs when sensor noise or degradation in one modality affects the reliability of fused estimates across multiple modalities. For example, in joint estimation pipelines, this problem is magnified for service robots as it can cause cascading perception errors that directly affect safe and effective HRI. Yet, many fusion methods assume stationary sensor noise models and fixed modality trust, which can result in overconfident predictions or action outputs, refs. [99,100]. Overconfidence can lead to unsafe behaviors for mobile service robots. For example, a robot might continue to navigate confidently through a dim hallway, despite losing its visual information. Adaptive confidence weighting, which assigns modality-specific weights based on online reliability metrics [101,102], remains underdeveloped for real-world deployment of mobile service robots due to such key limitations as: (i) high computational overhead incompatible with embedded edge hardware, (ii) assuming static environments, and (iii) lacking robustness to abrupt sensor failures and scene changes.

(4)

Domain adaptation and transferability issues: Early, late, and intermediate fusion models are often trained in static, well-lit indoor datasets and degrade in deployment across the cluttered households, dim hospital corridors, or dynamic outdoor and public venues which mobile service robots work in [103,104]. This domain gap makes it difficult for models to generalize once deployed in cluttered dynamic environments with multiple surface types (hospitals with glass doors and walls). Domain adaptation methods such as domain randomization [105], self-supervised online learning [106], and adversarial style transfer [107] face limitations in mobile service robot deployments. Transferability is further hindered when models are overfit to specific training conditions. Mobile service robots cannot retrain models in real time during operation, making them vulnerable to perceptual drift. Thus, robots can lose accuracy when exposed to novel environment setups: different layouts in long-term care homes, outdoor seasonal or weather-related lighting variations, or different human movement patterns.

3.3. Challenge #3: Uncertainty Estimation

Uncertainty is inherent in language-guided mobile service robotics, arising from sensor noise, human ambiguity, partial observability, and/or dynamic environments [54,57,108,109]. Uncertainty estimation in mobile service robots has three core limitations: lack of explicit uncertainty quantification, failure in long-horizon estimation, and uncertainty in HRI.

(1)

Lack of explicit uncertainty quantification: Mobile service robots often lack the ability to estimate confidence in their decisions. While probabilistic methods consider modeling confidence [54,108], they are computationally intensive for real-time robotic applications. As a result, many service-oriented models rely on rule-based pipelines [55] or standard deep learning architectures [56,57] that do not provide calibrated measures of uncertainty [110]. This limitation can cause implementation failures.

(2)

Failure in long-horizon uncertainty estimation: Reasoning about how uncertainty accumulates over extended time horizons is critical for service tasks. Traditional state estimation and planning methods, such as Kalman Filters [111,112], belief-space planners, and sampling-based approaches [113], do not model how uncertainty accumulates [114,115], which may lead to underestimating uncertainty as prediction horizons increase [116,117]. Belief-space motion planners [118] and sampling-based approaches such as Partially Observable Sparse Samplers [113] can become computationally intractable as the planning horizon increases [79,80,81]. For mobile service robots, this limitation directly impacts their ability to anticipate cascading errors, recover from uncertain states, or adapt task plans over time. In practice, robots may overcommit to long navigation paths that are not valid in dynamic or crowded environments.

(3)

Uncertainty in HRI: A central challenge for service robots is managing uncertainty in user-facing decisions, where ambiguous verbal commands, nonverbal gestures, or multimodal cues must be interpreted in real time. To date, the majority of robots engaged in HRI lack the ability to communicate uncertainty or proactively request clarifications, which often results in social errors and not adhering to social etiquette rules [119,120,121,122]. These failures typically stem from misaligned intent interpretation, where the robot’s confidence is overestimated despite incomplete or noisy human input. Traditional approaches, including plan-based dialog management frameworks [51], behavior-tree-based HRI controllers [52], and POMDP-based dialog managers [53], often assume fixed interaction protocols and lack the ability to model, quantify, or express uncertainty in real time [123]. Recent studies stress that effective HRI requires not only technical uncertainty quantification but also the ability to express uncertainty in socially meaningful ways, for example, pausing, seeking clarification, or adapting conservatively when unsure [124,125].

3.4. Challenge #4: Computational Capabilities

Mobile service robots have limited onboard computational power, making real-time inference with large-scale symbolic and deep learning models challenging. Key limitations include:

(1)

Perception and planning computational overhead: Large-scale deep learning models often require significant memory [126,127] and computational resources [128]. For example, models with convolutional backbones [126] or 3D segmentation networks [127] can exceed the real-time processing capacity of edge computing units commonly used in mobile service robots. Classical planning frameworks [129] and the Dynamic Window Approach (DWA) planner [130] were designed for lightweight CPU-based inference with simplified 2D cost maps. While efficient, they are not able to scale to the rich semantic representations needed in human-centered environments, such as multimodal feature maps or RGB-D affordance graphs [131,132]. Addressing this overhead requires combining efficient model compression and on-chip physical AI with selective edge-cloud collaboration, while maintaining the responsiveness necessary for safe, trustworthy HRI.

(2)

Lack of adaptive resource allocation for real-time inference: Most classical mobile service robot frameworks such as traditional navigation stacks [133] and behavior-based control architectures [134] process sensor data at fixed input resolutions and allocate static CPU or GPU budgets, regardless of changing scene complexity or service robot task urgency [135,136]. This can lead to onboard resources not being used effectively during routine tasks; however, it risks overloading resources when service robots face applications that are interaction-heavy or in cluttered environments [137]. Adaptive techniques have been explored in visual pipelines [138,139] and mapping frameworks such as ElasticFusion [140] to dynamically adjust computation based on current sensory load or scene complexity. However, their approach is heuristic, and they do not generalize across the full perception-planning-control stack required for continuous operation in dynamic human-centered environments. They typically optimize a single subsystem (e.g., perception), without coordinating resource allocation across downstream planning and control modules that must respond to the same real-time constraints [141,142,143]. This results not only in technical inefficiency but also a reduction in service quality. For example, in cluttered domestic environments, increased perception load from dense visual scenes can monopolize computational resources, leaving insufficient capacity for robot motion planning or safety monitoring, leading to hesitations or unsafe navigation behaviors.

Figure 3 situates these four challenges within the broader evolution of embodied AI for mobile service robots. The left portion of the timeline (“Pre-Foundation Model Era”) highlights how classical pipelines, ranging from STRIPS [39] and PDDL [42,43] planners to DWA [130], LSD-SLAM [98], and Move Base [133], addressed multimodal perception, language-to-action translation, uncertainty estimation, and computational constraints with task-specific and brittle solutions. The middle segment (“Rise of Foundation Models”) represents the emergence of large pre-trained models such as GPT [144], BERT [145], CLIP [146], and early robotics integrations (e.g., DeiT [147], SayCan [12], OpenVINS [94]), where scaling laws and transfer learning began to influence service robotics. The right segment (“Foundation Model Era”) shows recent LLMs, VLMs, MLLMs, and VLAs (e.g., GPT-4 [148], DeepSeek-R1 [149], Magma [150], OpenVLA [151], Aether [152], ${\pi }_0$ [153,154]) that directly target the four challenges, as indicated by the color-coded arrows for challenges #1–#4. This progression motivates Section 4, where we discuss and analyze how foundation models systematically address the challenges and their limitations in more detail.

4. Opportunities in Mobile Service Robots for Foundation Models

This section outlines how foundation models can address the four core challenges identified in Section 3.

To contextualize how foundation models have contributed to embodied intelligence, Figure 4 presents existing mobile service robot capabilities into three functional domains: (1) language communication, (2) vision-language navigation, and (3) manipulation and organization. Language communication capabilities (green box) enable robots to interact with users through language grounding and interactive question answering, supporting clarification, intent inference, and context-aware dialogue. Vision-language navigation capabilities (red box) support multimodal spatial reasoning for both autonomous visual goal reaching and language-guided navigation in complex environments. Manipulation and rearrangement capabilities (orange box) capture object-centric physical interaction, including object and scene manipulation (e.g., grasping, tool use, placing), as well as spatial rearrangement of objects within an environment (e.g., tidying, organizing, or reconfiguring layouts according to task goals). These capabilities go beyond isolated grasps to include sequencing, spatial reasoning, and task-level organization of objects, which are essential for everyday service tasks such as fetching, cleaning, and household reorganization. Together, these three domains provide a practical behavioral decomposition of how foundation models enable mobile service robots to perform meaningful, real-world assistance.

4.1. Addressing Challenge #1: Translation of Natural Language Instructions into Executable Robot Actions

Understanding and executing natural language instructions remains a core challenge for mobile service robots. In response, foundation models leverage language-conditioned policies [12], vision-language alignment [155,156], and multimodal reasoning [157,158] to bridge the gap between symbolic language and embodied robotic control. In particular, recent foundation models have advanced instruction-following in mobile service robots by addressing the symbolic-to-embodied gap [146,148,157,158,159,160,161,162,163,164,165], incorporating physical commonsense [166,167], and supporting long-horizon planning and adaptation [168,169,170]. We discuss these key advances below.

(1)

Symbolic-to-embodied mapping and instruction ambiguity: Non-expert users can provide vague or underspecified requests that mobile service robots must translate into precise, embodied actions. Unlike classical planners that depend on brittle rule sets [171], foundation models learn shared semantic spaces where language, perception, and control are jointly represented. This enables robots to resolve ambiguities in context-sensitive ways, for instance, interpreting a parent’s request to “put this somewhere safe” in a home by linking it to spatial cues like a shelf or cabinet. Models such as LIMO [159], a reasoning-focused language model, extends this capability of resolving ambiguities by retaining semantic context across steps, allowing robots to adapt dynamically when users clarify or modify instructions mid-task. VLMs and MLLMs further support disambiguation by grounding colloquial or multi-modal inputs in real-world layouts, ensuring that actions remain compliant with HRI norms such as safe distances, legibility of motion, and responsiveness to feedback during task implementation.

(2)

Lack of embodied commonsense and awareness of physical constraints: In general, mobile service robots can use foundation models to address the absence of physical commonsense, by incorporating: (1) physics-based priors [172], (2) affordance reasoning [166], and (3) constraint-aware planning [173] into language-grounded policy architectures. These capabilities address the embodied AI gap where robots often lack real-world intuition about object properties, support stability, or task feasibility which is critical in mobile service contexts. For example, Genesis [172], a generative physics simulation foundation model, encodes differentiable physics into object-centric representations, allowing robots to mentally simulate stacking, collision, or deformation outcomes before acting. This prevents unsafe task errors, such as placing medical supplies on unstable surfaces. By combining affordance reasoning with constraint-aware planning, foundation models enable mobile service robots to anticipate whether actions such as “reposition the IV pole beside the bed without tipping it” or “push the luggage cart through a narrow gate” are physically realizable given their embodiment. This awareness is especially critical in caregiving and domestic contexts, where failure to respect physical and social constraints can endanger others.

(3)

Failure in long-horizon task planning: Long-horizon service tasks require mobile service robots to maintain memory, adapt to interruptions, and replan under uncertainty. Foundation models address these challenges by: (1) decomposing high-level language into structured subtasks [168,169], (2) retaining goal-conditioned histories to preserve context [169,170], and (3) revising plans in response to dynamic sensory input. For instance, Code-as-Policies [168], a code-writing LLM, can translate robot assistive commands into modular control routines that allow task branching, while LLM-Planner [169], an LLM-based planner, adapts ongoing navigation when new verbal clarifications arise. Similarly, s1 [170], a reasoning-optimized LLM, dynamically adapts reasoning depth to prevent cascading errors in extended sequences. As shown in

Table 2. Comparison of major foundation models with respect to the core challenges for mobile service robots.

Model	Language- to-Action (Success Rate)	Perception (Accuracy/ Latency)	Uncertainty (Calibration Error)	Computation (GFLOPs/FPS)	Notes (Strengths (+)/ Limitations (−))	Best-Fit Scenarios
Magma [150]	52.3% (Google Robot)	Vision QA: 88.6% ~220–300 ms	—	≈45 GFLOPs/~30 FPS	+ Strong multimodal perception − Limited robotics action success; partial spatial reasoning	Healthcare: Vision-Language Navigation (Section 4.2): Multimodal gesture-speech grounding for mobile service robots operating in crowded hospital wards, enabling robust intent disambiguation under visual occlusion and acoustic interference.
DeepSeek-R1 [149]	—	MLLU: 84% ~300–400 ms	81.8%	≈90 GFLOPs/~18 FPS	+ Best uncertainty-aware reasoning − No action grounding or real-robot evaluation	Healthcare: Language Communication with Uncertainty Awareness (Section 4.3): Dialogue-driven task execution under ambiguous goal specification, with calibrated confidence estimation to ensure safe and trustworthy human–robot interaction.
SAM 2 [174]	—	Segmentation mIoU: 86% ~50 ms CPU	—	260–310 GFLOPs/25–44 FPS	+ Highest visual perception accuracy − Cannot produce or ground action commands	Healthcare: Vision-Language Navigation (Section 4.2): Real-time scene segmentation to support safe mobile service robot navigation through hospital corridors with dense human motion and dynamic obstacles.
CLIP-CAP [168]	71% across 7 tasks (Everyday Robot)	Goal recognition: 83% ~200 ms	—	175 GFLOPs/3–5 FPS	+ Best physical manipulation success − Low FPS hinders onboard deployment	Domestic Assistance: Manipulation and Organization (Section 4.1): Language-conditioned mobile manipulation in cluttered kitchens, translating natural language commands into executable control programs for object retrieval and rearrangement.
Perceiver IO [175] for Perceiver-Actor [176]	70% across 18 tasks (Franka Panda Robot)	Perceiver-IO scene encoding: 84.5% ~250–350 ms	—	9–10 GFLOPs/2–4 FPS	+ Lowest computational cost with multi-task robustness − Uncertainty not explicitly modeled	Domestic Assistance: Computationally Efficient Manipulation (Section 4.4): Edge-constrained multi-task visuomotor control for home service robots performing navigation and object interaction under strict onboard compute and latency limitations.
GPT [144]	85–92% structured question and answering (Franka Panda Robot)	MMLU: 90% ~700 ms multimodal	92.0%	≈800 GFLOPs/token → 1–3 FPS	+ Strongest high-level reasoning under ambiguity − Computationally prohibitive for onboard guidance	Service Automation: Long-Horizon Language Communication and Planning (Section 4.1 and Section 4.4): High-level delivery instruction decomposition for multi-room navigation and task sequencing with remote inference support under non-real-time operational constraints.
LLaMA [177]	~91% symbolic planning (Google Robot)	MMLU: 83.6% 250–350 ms	92.1%	$\approx 8\times {10}^3$ GFLOPs/token → ~0.7 Hz	+ Effective symbolic planner; strong long-context utility − Slow inference; lacks action grounding	Service Automation; High-Level Semantic Planning (Section 4.1): Pre-execution semantic plan synthesis within hybrid symbolic-LLM pipelines for navigation and manipulation tasks prior to embodied grounding.
Top Performer Per Challenge	CLIP-CAP (best success rate in manipulation tasks)	SAM-2 (highest perception accuracy & real-time latency)	DeepSeek-R1 (lowest reasoning error on HLE text-only tasks)	Perceiver-Actor (lowest GFLOPs with viable FPS on-board)	Demonstrates clear trade-offs; no single model excels across all four core challenges

Note: “—” means not available.

, These advances allow mobile service robots to sustain coherent, user-aligned task progression in complex environments, whether completing multi-room domestic routines (e.g., fetch-and-deliver sequences that span kitchen → living room → bedroom) or replanning mid-shift in hospitals during time-sensitive medical delivery under corridor congestion and changing staff priorities.

(4)

Inability to handle mid-task requirement changes: With the emergence of LLM-, VLM-, VLA-, and MLLM-based systems, mid-task requirement changes are increasingly modeled as conversational constraint updates that trigger closed-loop replanning rather than full task restarts. Instead of assuming a fixed goal specification, recent language-guided robot frameworks maintain an explicit dialogue-conditioned task state, allowing revised user intent to supersede earlier constraints during task execution [178,179,180]. Skill-based architectures such as SayCan [12] dynamically rescore or reselect low-level skills when user priorities change, enabling robots to adapt action sequences online without invalidating the full plan. Code-synthesis approaches (e.g., CLIP-based Code-as-Policies (CLIP-CAPs) [168]) further support mid-task revisions by regenerating reactive policy code that embeds perception, feedback, and control logic, allowing updated requirements to be recompiled into executable programs during execution. Furthermore, feedback-driven planners such as Inner Monologue-style reasoning frameworks [25] extend their reasoning capability by incorporating execution outcomes, scene descriptions, and user corrections into iterative reasoning loops, enabling continuous plan refinement under evolving constraints [180]. Building upon this closed-loop paradigm, recent corrective planning frameworks (e.g., CoPAL [181]) categorize failures into logical, semantic, or grounding errors and apply targeted repair strategies during execution, while VLM-grounded planners (e.g RePlanVLM [182]) re-evaluate the environment state to replan long-horizon tasks when goals or conditions change. These advances allow mobile service robots to maintain task continuity, safely revise goals during execution, and remain aligned with the user’s latest intent without requiring full task restarts, thereby improving robustness in dynamic domestic, healthcare, and service automation environments.

Prior approaches to language-guided robot planning such as STRIPS [39], PDDL [42,43], STN [73], and FF [74] relied predominantly on hand-engineered symbolic planners and task-graph formulations, and classical heuristic planning. While effective in static and well-specified domains, these methods were brittle to linguistic ambiguity, underspecified user instructions, and dynamic task variation [39]. Learning-based sequencing methods such as NTGs [77] and option-based policies [78] introduced greater flexibility, but still suffered from limited memory, compounding errors, and weak grounding when deployed in long-horizon mobile service tasks. Recent foundation-model approaches have begun to bridge the gap between symbolic reasoning and embodied execution, as shown in Table 2. In particular, CLIP-CAP [168] explicitly links vision-language representations to executable control programs, allowing natural language commands to be translated into modular, interpretable robot code that directly invokes perception, navigation, and manipulation primitives. Through this design, CLIP-CAP [168] achieves the highest reported real robot manipulation success (71% across real-robot manipulation tasks) among the evaluated models, while also supporting task branching and online replanning during execution. This capability marks a shift beyond earlier brittle planning pipelines towards more adaptive, language-driven mobile service robot behaviors.

To provide interpretability for deployment, Table 2 includes a ‘Suitable Scenarios’ column that maps each foundation model to the service environments where its architectural trade-offs are most appropriate (e.g., domestic assistance, healthcare, and/or service automation).

4.2. Addressing Challenge #2: Multimodal Perception

Robust multimodal perception is essential for interpreting complex environments. Particularly, multimodal VLMs and MLLMs can address this challenge by providing pre-trained multimodal representations of visual, textual and spatial features that enable the integration of heterogeneous sensory data through unified representation learning, temporal synchronization, uncertainty-aware fusion, and domain generalization [152,183]. We discuss below the key advances of leveraging foundation models for this challenge.

(1)

Cross-modal representation gap: Foundation models mitigate the fragmentation of sensory streams in mobile service robots by aligning multimodal data into unified latent spaces using either (1) cross-modal tokenization [146,150], (2) spatial encoding [152], and/or (3) attention-based fusion [184] to create representations that not only synchronize perception but also remain socially meaningful. Unlike robots that operate in structured factories or warehouses, mobile service robots must integrate heterogeneous inputs, vision, speech, LiDAR, tactile, and physiological signals in real time, while interacting with humans in cluttered or crowded environments. Magma [150] is a transformer-based multimodal foundation model designed to bridge perceptual understanding and embodied actions through Set-of-Mark (SoM) and Trace-of-Mark (ToM) prompting, which convert visual scenes into actionable spatial tokens and predict future motion trajectories. By integrating images, language commands, and robot traces into a joint token space, Magma enables unified cross-modal grounding and planning. This allows a service robot deployed in a hospital to correctly associate a nurse’s spoken instruction with a patient’s movement, while in retail environments the same model can interpret gestures such as pointing and ground expressions like “that one over there” on crowded shelves. By unifying perception and interaction modes, foundation models can reduce representational misalignment that can result in failures in object association, gesture-speech coordination, and/or scene parsing during real-time HRI.

(2)

Latency issues: Strategies for reducing latency in the service robot context include: (1) learning time-aware representations [185], (2) aligning asynchronous inputs [186], and (3) dynamically adjusting modality-specific update rates [187]. For learning time-aware representations, video foundation models such as VideoJAM [185] and InternVideo2 [186] jointly model visual appearance and optical-flow motion across time, enabling coherent spatiotemporal grounding of asynchronous frames. In a healthcare environment, for example, such models allow a robot to detect in real time when a patient begins to stand up from a chair, triggering immediate stabilization assistance or verbal support, before a fall risk escalates. In office environments, object-centric physical modeling frameworks further help synchronize sensor inputs during long hallway robot navigation when groups of employees are sharing the space while having unpredictable movements, enabling timely monitoring and smooth flow management. These advances allow mobile service robots to have real-time responsiveness in environments where even small delays can undermine safety, trust, and quality of assistance.

(3)

Uncertainty propagation across modalities: Mobile service robots must assess the reliability of multimodal inputs in real time, since sensor degradation in one channel can otherwise cascade into unsafe actions. Foundation models enable uncertainty-aware fusion, where each modality’s contribution is dynamically weighted by estimated confidence [188]. For example, DeepSeek-R1 [149], a reinforcement-trained LLM, and the Segment Anything Model (SAM) [164] and SAM-2 [174], vision foundation models, incorporate confidence-driven suppression of low-quality visual regions and reinforcement-based reasoning to avoid acting on uncertain observations. These mechanisms can be directly leveraged in service environments: in restaurant service contexts, confidence-aware fusion can prevent unsafe misinterpretation of overlapping table orders and simultaneous calls. In service automation, such as busy airports or malls, adaptive reweighting allows robots to filter noisy crowd signals while sustaining situational awareness. These strategies enable mobile service robots to maintain robust perception and socially safe interaction even when modalities degrade.

(4)

Domain adaptation and transferability: Service robots need to generalize across high variability, from clutter and lighting variations to reflective areas, reconfigurable layouts, and dense crowds. Unlike factory or lab-trained models, foundation model-based perception systems deployed in service robots cannot be continuously retrained during operation due to privacy, compute, and real-time performance constraints. Therefore, strong domain transfer capabilities are essential. Foundation models address this need through (1) self-supervised pretraining [189,190], which allows perception systems to adapt without large labeled datasets; (2) cross-domain token alignment [183,191], which stabilizes multimodal features when sensor inputs vary across environments; and (3) few-shot generalization [173,192], which dynamically adjusts tokenization for domain-flexible perception. These capabilities help mobile service robots obtain reliable recognition and reasoning under changing environments. This reduces perceptual drift and ensures continuity of safe, trustworthy interaction over long-term deployments.

Prior to foundation models, mobile service robots struggled with cross-modal alignment, noisy sensing, and limited generalization across objects and environments [47,48,49]. The perception results shown in Table 2 demonstrate that foundation models can address these concerns. Namely, SAM-2 [174] achieves the highest visual perception accuracy with real-time performance, while Magma [150] improves semantic grounding through unified image-language embeddings maintain competitive sensing at low compute cost.

4.3. Addressing Challenge #3: Uncertainty Estimation

Robust uncertainty estimation for mobile service robots enables such robots to work in unpredictable environments, where sensor noise, ambiguous input, and unexpected events can compromise decision-making. Foundation models address this challenge by supporting explicit confidence modeling, prediction long-horizon risk prediction, and uncertainty-aware HRI [144,175,193,194,195,196,197,198,199,200,201,202,203]. We discuss these below.

(1)

Lack of explicit uncertainty quantification: In human-facing environments, robots must be able to avoid unsafe or socially awkward actions. Foundation models support this through (1) reinforcement-guided value modeling [149], (2) adaptive attention-based reweighting [144,175,193], and (3) temporal uncertainty tracking [194]. The LLM DeepSeek-R1 [149] calibrates confidence using reinforcement signals, while the MLLM GPT-4o [144] models token-level reliability. These capabilities allow mobile service robots to cautiously perform actions in everyday contexts. Benchmarks such as Humanity’s Last Exam (HLE) [204] and MANNERS-DB [205] further highlight the importance of expressing uncertainty in socially acceptable ways, such as apologizing before asking for clarification or signaling doubt through tone and timing, which is essential for maintaining trust in domestic and public service interactions.

(2)

Failure in long-horizon uncertainty estimation: Planning over extended time horizons presents a major challenge for mobile service robots, as perception or reasoning errors can accumulate into significant risks during prolonged operation. Foundation models mitigate this through latent world modeling and temporal abstraction [195,196,197], which simulate outcomes in compressed latent space to anticipate risks such as localization drift, occlusion, or sensor degradation before unsafe actions are executed. Vision-based self-supervised models such as MAE [198] and Swin Transformer [199] further enhance robustness by reconstructing corrupted spatiotemporal patches, and improving confidence tracking under partial observability. In service applications, this enables long-horizon replanning for robotic tasks such as guiding visitors through multi-floor buildings where the availability of elevators or access restrictions can vary dynamically. By modeling uncertainty accumulation explicitly, these approaches support continuity of assistance and maintain reliability during extended, socially embedded tasks.

(3)

Uncertainty in human–robot interactions (HRIs): Effective HRI requires mobile service robots to detect uncertain or ambiguous human behaviors such as vague speech, inconsistent gestures, or shifting gaze, and to communicate their own uncertainty clearly during task execution. Foundation models support this by integrating behavior forecasting, social intent grounding, and natural language grounding. For example, Lumiere [200], a diffusion-based generative video model, generates motion-consistent forecasts of human activity, enabling robots to anticipate possible human actions and avoid socially disruptive responses. In a domestic setting, this may involve the robot briefly pausing to signal uncertainty and then proactively asking the user a clarifying question, for example, “Did you mean the unwashed bowls on the table or the ones by the sink?” thereby preventing incorrect actions while maintaining smooth interaction flow.

Traditional mobile service robots often did not account for uncertainty, leading to unsafe behaviors and brittle recovery when unexpected human interactions occurred [54,55]. In contrast, recent foundation models demonstrate measurable progress toward calibrated reasoning under uncertainty. For instance, DeepSeek-R1 [149] reports the lowest calibration error (~81.8%) on the HLE benchmark [204], indicating closer alignment between predicted confidence and actual correctness, whereas GPT [144] and LLaMA [177] class models generally exhibit higher calibration error values (~92%), demonstrating overconfidence. As summarized in Table 2, this improvement suggests that certain foundation models are beginning to quantify and communicate uncertainty more reliably, an ability that is essential for safe, trust-preserving service robot deployment in real-world environments.

4.4. Addressing Challenge #4: Computational Capabilities

Mobile service robots often operate under constrained budgets, making real-time perception and planning with large deep learning and symbolic models difficult. Foundation models address this challenge by enabling scalable model compression [206,207], efficient attention distillation [179,208], adaptive token processing [209,210], and on-device execution [211,212]. These advances allow mobile robots to maintain real-time responsiveness, energy efficiency, and task throughput even under strict power, memory, and latency constraints. We discuss these key advances below.

(1)

Perception and planning computational overhead: Real-time perception and planning can often exceed the computational limits of embedded mobile service robot systems. This creates latency bottlenecks that impair decision-making during navigation, interaction, or manipulation. Foundation models address computation limits through (1) architectural streamlining [151,213], (2) model compression [206,207], and (3) efficient attention mechanisms tailored for edge devices [179,208]. The LLM EdgeFormer [213] applies dynamic token control to sustain responsiveness during dialogue-driven task execution. The VLM DeiT [147,206], a distilled vision transformer, compresses large transformer architectures into compact models suitable for identifying small objects in cluttered environments. The MLLM VisionLLaMA [207] adapts LLaMA-style transformers for lightweight multimodal reasoning. These advances reduce inference latency while preserving reliability, allowing mobile service robots to operate efficiently under power and memory constraints while maintaining safe, socially responsive interactions.

(2)

Lack of adaptive resource allocation for real-time inference: Efficient task execution under fluctuating computational loads is essential for mobile service robots, which must remain responsive despite strict latency and energy limits. Foundation models address this challenge through adaptive computation strategies that allocate resources in proportion to service robot task complexity. For example, the LLMs Switch Transformer [209] and GShard [210] use Mixture-of-Experts (MoE) architectures to selectively activate the modules required for the current context, conserving energy during routine operations while scaling up capacity during socially or perceptually demanding interactions. This dynamic allocation directly enhances service quality: a domestic assistant robot can transition from monitoring to full-capacity multimodal parsing of requests. By dynamically rebalancing inference loads, foundation models enable mobile service robots to maintain timely, safe, and socially fluent interactions.

Onboard processors and battery constraints restricted mobile service robots to simplified models with reduced perception and reasoning abilities. Meaningful improvements have been achieved with foundation models. Namely, Perceiver-Actor offers the lowest compute cost (~9–10 GFLOPs) while still achieving multi-task visuomotor control, while SAM-2 and Magma deliver strong perception with higher computation budgets.

4.5. Lessons Learned and Failure Patterns Across Foundation Model Architectures

While Table 2 summarizes the performance trade-offs of major foundation models across language-to-action, perception, uncertainty, and computation, these results also reveal recurring failure patterns that highlight why certain foundation model architectures can underperform in specific service environments. Namely, these patterns identify architectural mismatches between model assumptions and the operational constraints of domestic assistance, healthcare, and service automation settings. We discuss these key limitations in more detail below.

(1)

High-level reasoning and executable control implementation gaps: General-purpose LLM/MLLM planners can generate coherent task decompositions and recovery strategies under ambiguous instructions; however, they can often fail when required to convert language into precise, physically feasible actions. This failure is evident in domestic manipulation tasks in cluttered household environments, where underspecified commands (“put this away”) require fine-grained object disambiguation, grasp feasibility checks, and contact-aware execution. For example, high-level reasoning-oriented models such as GPT [144], LLaMA [177] and DeepSeek-R1 [149] demonstrate strong abstract reasoning and planning performance. However, as these architectures do not natively integrate perception outputs with low-level motor control or physically validated action primitives (as reflected in Table 2 by the absence of real-robot grounding metrics), they risk producing plausible plans that omit embodied constraints. This leads to unsafe or infeasible steps, such as selecting an occluded or already-grasped object, attempting a grasp outside the robot’s reachable workspace, placing objects on unstable or non-supportive surfaces, colliding with nearby obstacles during arm motion, or issuing navigation commands that ignore local clearance and kinematic limits. In contrast, architectures that explicitly bind language inputs to perception outputs and predefined robot action primitives, such as CLIP-CAP [168] and Perceiver-Actor [176], reduce this type of failure by converting natural language instructions into executable control sequences that directly reference detected objects, grasp points, and motion policies. For example, CLIP-CAP [168] translates a command such as “pick up the red shirt and place it in the laundry basket” into structured program steps that: (1) call a vision module to localize the object, (2) compute grasp parameters based on object pose, and (3) execute validated manipulation actions using predefined robot control primitives. This explicit perception-to-action linkage is reflected in its higher real-robot manipulation success rate (71% across seven real-world tasks, Table 2). Similarly, Perceiver-Actor [176] directly maps multimodal scene encodings to low-level visuomotor policies (70% success across 18 tasks, Table 2), demonstrating stronger embodied integration compared to purely reasoning-oriented models.

(2)

Overconfidence and miscalibration under uncertainty: A dominant failure pattern for foundation models in healthcare and public environments is miscalibrated confidence estimation. In safety-critical settings, failure often occurs when foundation models provide confident but incorrect outputs, resulting in downstream planners providing unsafe decisions (e.g., misidentifying objects, misunderstanding priorities, or failing to ask for clarification). Table 2 highlights the comparatively stronger calibrated reasoning under uncertainty of DeepSeek-R1 [149], while the GPT-class [144] and the LLaMA-class [177] models exhibited higher calibration errors, indicating overconfidence. In practice, such failure can be amplified in hospitals, where ambiguity is common (multiple staff instructions, object/tool similarities, changing workflows) and where uncertainty should trigger conservative behaviors (verification, asking, or deferring), rather than decisive incorrect actions.

(3)

Latency and computation bottlenecks: In mobile service robotics, many failures can manifest as delayed responses rather than incorrect predictions. High-capacity models may succeed on static benchmarks, however, can become unusable in real deployments due to timing constraints imposed by HRI, navigation, and manipulation in dynamic settings. This is particularly critical in healthcare assistance (e.g., bedside monitoring, corridor navigation) and service automation (e.g., guidance and wayfinding) where responsiveness directly affects safety and trust. For example, GPT-style multimodal reasoning models [144] require substantial computation per token and operate at low inference speeds, making them unsuitable for onboard closed-loop control in time-sensitive environments. Similarly, CLIP-CAP [168] achieves strong real-robot manipulation performance but operates at low frame rates, which can limit responsiveness during continuous perception-action cycles. In contrast, Perceiver-Actor [176] demonstrates lower computational cost suitable for embedded deployment; however, this efficiency may come at the expense of richer reasoning capacity or explicit uncertainty modeling. This illustrates the broader trade-off between reasoning depth and real-time responsiveness on resource-constrained mobile platforms.

(4)

Failure amplification across domains: Across domestic and healthcare assistance, and service automation, the same architectural limitations can be amplified by different operational constraints. Namely, domestic settings exacerbate language-to-perception and perception-to-action mapping errors, where a model can incorrectly associate a linguistic reference (e.g., “that bowl”) with the wrong object instance or can select an action that does not account for occlusion, reachability, or grasp feasibility. Hospitals can amplify uncertainty and safety failures due to high-stakes decisions, vulnerable populations, and multi-user conflicts. Furthermore, public service scenarios amplify compute and latency failures due to scale, crowds, and continuous interaction demands. Perception backbones, including SAM-2 [174], can generalize broadly across different domains, while reasoning-heavy models such as GPT [144] and DeepSeek-R1 [149] are best suited for supervisory or cloud-assisted components. The latter models can potentially be paired with efficient control pipelines that directly translate model outputs into validated perception modules, motion planners, and low-level motor commands with collision checking and feasibility verification. Examples of frameworks that implement this reasoning-to-control grounding include code-generation and skill-selection frameworks such as CLIP-CAP [168], which compiles language instructions into executable robot programs, and SayCan [12], which grounds high-level language reasoning into scored low-level skill primitives prior to execution.

Addressing the core challenges of multimodal perception, language grounding, task generalization, uncertainty estimation, and computational efficiency, foundation models offer scalable solutions for mobile service robotics, while exposing new design trade-offs that must be managed across deployment domains.

5. Real-World Applications for Mobile Service Robots with Embedded Foundation Models

The global service robotics market is projected to experience rapid growth, approximately doubling from $47.10 billion USD in 2024 to $98.65 billion USD by 2029, with an average compound annual growth rate of 15.9% [214]. This surge reflects broader trends toward integrating intelligent service robots into homes, healthcare facilities, and public spaces [215]. As foundation models are increasingly being embedded into mobile service robots, we examine how these models address real-world needs and enable safer, more adaptive, and socially intelligent robot behaviors across these domains.

We categorize applications into three primary domains, domestic assistance, healthcare assistance, and service automation, with each decomposed into representative sub-tasks (Figure 5).

Domestic assistance consists of fetch-and-retrieval, cleaning, childcare, and cooking activities; healthcare assistance includes medical delivery, bedside monitoring, assistive mobility, and hygiene maintenance; service automation encompasses customer assistance, guidance and wayfinding, as well as service setup and maintenance tasks such as venue preparation, layout arrangement, equipment positioning, and post-event cleanup in public spaces. Representative frameworks and robot platforms in each application domain are presented in

Table 3. Foundation model applications across several robot domains.

Domain	Task	Representative Frameworks	Robots
Domestic Assistance	Fetch and Carry Tasks	Ok-Robot [216]	Stretch RE-1 Robot
		Octo [217]	Franka Emika Panda Robot
		CogNav [30]	Fetch mobile manipulator
		PARTNR [218]	Franka Emika Panda Robot Spot + 7-DoF arm
	Cleaning	TidyBot [37]	Holonomic mobile base and a Kinova Gen3 arm
		RT-2 [14]	Everyday mobile manipulator robots
	Childcare	Smart Help [28]	Custom mobile manipulator robot with Kinova Gen3 arm
		SAFE [219]	Franka Emika Panda robot
	Cooking	OVQSRFRW [220]	Everyday mobile manipulator robots
		TPLC [221]	6-DoF Universal Robots UR5e arm
		CTGN [222]	Dual-arm Universal Robots UR5e
Healthcare Assistance	Delivery of Medical Supplies	ProgPrompt [223]	Franka Emika Panda robot
		LLM-Grounder [224]	Franka Emika Panda robot
		VLM-Social-Nav [201]	Clearpath Jackal robot
	Bedside Patient Monitoring	MoMa-LLM [201]	Fetch mobile manipulator
	Assistive Mobility	OLiVia-Nav [202]	Clearpath Jackal robot
		GSON [225]	Custom differential-drive robot
	Hygiene Maintenance and Infection Control	AutoRT [226]	Everyday mobile manipulator robots
		SayPlan [227]	Franka Emika Panda robot
Service Automation	Customer Assistance, Guidance and Wayfinding	OVSG [228]	Ackermann Steering robot
		LM-Nav [229]	Clearpath Jackal robot
		ViNG [230]	Vizbot, Unitree Go1, Clearpath Jackal, and LoCoBot
	Service Setup and Maintenance	Hi Robot [231]	Dual-arm mobile manipulator Mobile ALOHA robot
		${\pi }_{0.5}$ [154]	Dual-arm mobile manipulator Mobile ALOHA robot
		PhysObjects [232]	Franka Emika Panda robot

, linking existing robotic systems (e.g., Stretch RE-1, Fetch, Everyday Robot, Jackal, and mobile manipulation platforms) to the tasks they perform. Together, Figure 5 and Table 3 provide a mapping from application domains and sub-tasks to specific foundation-model-enabled mobile service robot implementations that we examine in the following subsections.

5.1. Domestic Assistance

Domestic assistance refers to everyday tasks to support comfort, safety, and quality of life [233,234]. These tasks include the fetch and retrieval of household objects [217,218,235,236], cleaning [37,237], childcare assistance [153,238], and cooking activities [239,240,241].

(1)

Fetch and retrieval tasks: These tasks, often triggered by natural language, gestures, or contextual cues, involve multimodal perception, ambiguous instruction grounding, uncertainty management, and efficient planning. Fetch and retrieval tasks decompose into sub-tasks including: (1) object localization, (2) grasp planning and action refinement, (3) navigation and multi-step delivery, and 4) handover and placement reasoning. Using foundation models such as CLIP [146] and CLIP-Fields [242] for spatial-language grounding, Octo [217] for visuomotor control, GPT-4V [163] and CogNav [30] for high-level reasoning and route sequencing, and LLaMA3.1-8B [177] or CodeLLaMA-70B [243] for symbolic task decomposition, mobile service robots can achieve end-to-end perception and manipulation across these sub-tasks. VLM CLIP-Fields [242] support object localization by constructing a continuous 3D semantic field from RGB-D images and odometry. Each point in the environment is encoded using CLIP’s text-vision embedding space, allowing a robot to search for objects using natural language descriptions (e.g., “the yellow sponge near the sink, go get it and bring it to me so I can clean these dishes”) without predefined labels. This method has been deployed on the Stretch RE-1 [216], a commercial mobile manipulator equipped with an RGB-D sensor, telescoping mast, and a compliant gripper. The Stretch uses the semantic field to resolve which surfaces and object clusters have the highest likelihood to match the user’s query, enabling reliable localization in typical household scenes where objects may be partially occluded, moved, or surrounded by visually similar items. Grasp execution and refinement are produced by the VLA model Octo [217], which blends a T5 language encoder with a diffusion-based visuomotor policy. On a Franka Emika Panda Robot arm, Octo generates stable grasp trajectories when object poses are uncertain or cluttered, refining motions based on both visual input and the language-specified goal. High-level instruction reasoning and route sequencing are modeled through GPT-4 [148], GPT-4V [163], and segmentation backbones such as SAM [164]. Integrated onto the Fetch mobile manipulator, these models interpret user requests, segment relevant regions of the scene, and adjust navigation plans as environmental obstacles or room layouts change. The CogNav system [30] further extends spatial reasoning by using GPT-4V [163] to decide when to explore, when to navigate, and how to map new rooms during longer fetch-and-deliver routines. Context-sensitive handover and placement actions are enabled through the PARTNR framework [218], which applies LLaMA3.1-8B [177] and CodeLLaMA-70B [243] to infer appropriate approach angles, placement surfaces, and human-aware delivery strategies. PARTNR demonstrated these behaviors on both a Franka Panda and a Boston Dynamics Spot robot equipped with a 7-degrees-of-freedom (DoF) arm, enabling the retrieval and socially compliant delivery of household items during domestic assistance tasks.

(2)

Cleaning: Cleaning tasks involve a robot autonomously organizing, decluttering, and sorting household items into designated locations [244]. Unlike object-specific fetch tasks, cleaning often requires generalizing abstract commands such as “put everything away” into sequential, context-aware behaviors across many objects and categories. Cleaning tasks can be decomposed into the following sub-tasks: (1) user preference inference and object categorization, (2) object localization and sorting, and (3) task execution with multimodal grounding and action control. Foundation models such as GPT-3.5 [173] or LLaMA-based LLMs [177,243,245] for preference reasoning, VLM CLIP [146] for visual-semantic association, and embodied VLA models such as RT-2 can be used for physical execution. These models collectively support the subtasks in real-world cleaning robots. User preference inference and category reasoning are demonstrated in TidyBot [37], which integrates LLM GPT-3.5 [173] with a holonomic mobile base and a Kinova Gen3 arm. GPT-3.5 [173] is prompted with a small set of examples (e.g., “put light clothes in the drawer,” “place toys in the bin”) and infers general placement rules that align with household organization habits. The model predicts appropriate storage locations for previously unseen objects, enabling the robot to follow human-specific conventions rather than relying on rigid predefined templates. Object localization and sorting are enabled through the combination of GPT-3.5 and CLIP [146] on the same TidyBot robot, which together match visual observations to semantic categories. CLIP provides an open-vocabulary visual embedding space, allowing the TidyBot to recognize a wide range of everyday objects, even those not present in its training set, and associate them with the inferred categories generated by the LLM. This supports reliable sorting in typical home scenes where objects vary in appearance, lighting, or placement.

Task execution and multimodal control can be supported by VLA models such as RT-2 [14], which unify vision, language, and motor control. RT-2 policies map high-level cleaning commands (e.g., “clear the table,” “place everything neatly on the shelf”) into low-level robot actions while maintaining awareness of object poses, workspace boundaries, and user-defined organization rules. This method has been deployed on the Everyday Robot [14], a mobile manipulator with a single 6-DoF arm. The VLA model enables the Everyday Robot to carry out multi-step cleaning routines with consistency and adaptability.

(3)

Childcare: Childcare tasks require a mobile service robot to assist with daily routines, object retrievals, and monitoring of young children [246]. Example tasks include responding to prompts such as “bring me my blanket,” guiding hygiene routines like teeth brushing after meals, and detecting safety risks such as toddlers approaching stairs or handling small objects. Childcare can be decomposed into following sub-tasks: (1) behavioral intent grounding, (2) dynamic instruction decomposition, and (3) real-time multimodal control and safety adaptation. Using foundation models such as GPT-3.5/3.5-turbo-instruct [173] for intent interpretation and multi-step reasoning, and VLA safety-adapter frameworks such as SAFE [219] for embodied risk detection, mobile service robots can integrate perceptual cues, language inputs, and feedback signals to support safe and context-aware childcare routines. LLM GPT-3.5-turbo-instruct [173] supports language-grounded behavioral reasoning by learning probabilistic mappings between behavioral cues and task goals, allowing a robot to interpret vague requests such as “help me” in context-specific ways; for example, in retrieving a dropped toy, handing over a comfort item, or alerting a caregiver. This has been demonstrated on the Smart Help system [28] which uses a custom mobile manipulator robot with an onboard camera and a Kinova Gen3 arm. Smart Help uses GPT-3.5-turbo-instruct [173] to infer user intent from short or underspecified prompts. The LLM supports dynamic instruction decomposition, where, for example, longer caregiver commands (e.g., “check if the baby is asleep and then turn off the light”) can be translated into ordered subgoals that coordinate perception, navigation, and verification steps, allowing a robot to maintain situational context and align timing with child behaviors. Real-time multimodal control and safety adaptation are provided by SAFE [219], which integrates a multimodal safety classifier atop pretrained LLM- and VLM-based policies. The Franka Emika Panda robot has used SAFE to continuously analyze vision, proprioception, and trajectory predictions to halt unsafe grasps and prevent collisions with fragile objects. SAFE gives a timely alert such that the robot can stop, backtrack, or ask for help, enabling early intervention before failures cause harm to humans, adults and children, around.

(4)

Cooking: Cooking tasks involve mobile service robots executing multi-step procedures to assist elderly individuals, busy families, or users with disabilities. Cooking includes such subtasks as: (1) ingredient and utensil localization, (2) sequential cooking task decomposition and planning, and (3) bimanual manipulation and coordinated action execution. Foundation models such as LLM PaLM 540B [6] for visual-language grounding, LLMs BERT [145] and DistilBERT [247] for instruction parsing, and VLM CLIP [146] for visual-semantic correspondence, can map natural language cooking requests to perception-driven subgoals and physically executable motion plans in the home. In particular, the LLM PaLM 540B supports ingredient and utensil localization by visually identifying objects such as spoons, milk, eggs, or bread from natural language cues. This allows a robot to ground ingredient or utensil references directly to camera observations and infer their spatial context within the kitchen. This was demonstrated on the Everyday Robots mobile manipulator [220], which integrated the PaLM 540B LLM. Sequential task decomposition and planning have been demonstrated using the fixed-base 6-DoF Universal Robots UR5e arm system [221]. The system used the BERT and DistilBERT as well as CLIP to transform high-level goals such as “in kitchen, make a sandwich” into grounded action sequences like “pick bread from cabinet above” or “retrieve cooked meat from microwave.” These models decompose abstract cooking goals into structured subtasks linked to corresponding visual targets. Coordinated bimanual manipulation are enabled through the combination of GPT-3 [2] and a graph-based control network to synchronize grasping, tool use, ingredient placement, and force-sensitive interactions. This method has been deployed on a dual-arm Universal Robots UR5e configuration [222] utilizing GPT-3 as the high-level planner taking language instruction and RGB camera data as inputs. This integration allows the robot to perform LLM-guided actions such as stabilizing a bowl with one arm while stirring or transferring ingredients with the other, all while maintaining real-time perceptual grounding and safety constraints.

5.2. Healthcare Assistance

Healthcare assistance tasks involve supporting clinical workflows, assisting people, and infection control in dynamic hospital environments. These tasks include the delivery of medical supplies [223,224], bedside patient monitoring, assistive mobility [225,248], and hygiene maintenance and infection control [226,227,249].

(1)

Delivery of medical supplies: Delivery of medical supplies involves mobile service robots autonomously transporting medications, IV fluids, lab samples, and surgical instruments throughout hospitals and clinics. These deliveries are essential to maintaining smooth clinical workflows, reducing staff workload, and minimizing human error in time-sensitive medical tasks. Robotic delivery tasks can be decomposed into the following sub-tasks: (1) symbolic-to-action mapping and long-horizon planning, (2) spatial grounding of objects and target locations, (3) feasibility evaluation and adaptive plan selection, and 4) social navigation in dynamic clinical settings. LLMs such as Codex [250], GPT-3 [2], and GPT-4 [148] have been used for symbolic program synthesis and task decomposition, and VLMs such as OpenScene [251] for spatial grounding, and GPT-4V [163] combined with YOLO [252] for human-aware navigation. These models allow robots to translate requests into structured action sequences, infer 3D target locations, and adapt their motion to human activity in real time. Codex [250] and GPT-3 [2] support symbolic-to-action reasoning by converting instructions such as “take these surgical tools to the OR room” into structured control sequences (e.g., functionally decomposed like grab (kit) → goto (prep_room) → handover) allowing a robot to autonomously generate interpretable, executable programs using an LLM-API and predefined spatial primitives to produce interpretable, long-horizon delivery routines. This is demonstrated on a custom mobile manipulator base equipped with a Franka Emika Panda arm [223]. Spatial grounding was enabled by a combination of models such as GPT-4 [148] and OpenScene [251]. Demonstrated on the same custom mobile manipulator base in [224], GPT-4 and OpenScene together resolves spatial expressions such as “on the left tray by the door” into 3D coordinates suitable for pick-and-place execution, enabling accurate supply localization and placement even when trays, carts, or equipment partially occlude the scene. GPT-4V [163] and YOLO [252] supports social navigation in dynamic environments as was shown with VLM-Social-Nav [201] on a Clearpath Jackal mobile robot equipped with RGB cameras and wheel odometry. VLM-Social-Nav merged GPT-4V and YOLO to interpret real-time human gestures and motion cues to infer social intent, dynamically adjusting its navigation policy to yield, reroute, or proceed safely through crowded corridors.

(2)

Bedside patient monitoring: Bedside patient monitoring focuses on understanding a patient’s state, intent, and safety within the immediate vicinity of the bed, under strict clinical constraints on proximity, etiquette, and timely response. These tasks decompose into: (1) patient intent and state interpretation, and (2) bedside assistive decision-making. LLMs such as GPT-4 [148] and GPT-3.5-turbo-1106 [173] support these capabilities by reasoning over verbal cues, contextual room information, and clinical routines. This functionality is demonstrated in MoMa-LLM [248] on the Fetch robot, where GPT-4 [148] performs high-level reasoning over requests and situational context, while GPT-3.5-turbo-1106 [173] provides real-time room and object context classification. Together, these models enable robot bedside monitoring behaviors such as recognizing requests for assistance, detecting unsafe situations (e.g., a patient attempting to reach beyond safe range), and deciding when to reposition objects, alert staff, or initiate assistive actions, all while remaining localized to the bedside environment.

(3)

Assistive mobility: Assistive mobility focuses on safely navigating dynamic hospital environments while anticipating human motion and proactively adapting behavior. These tasks decompose into: (1) safe co-navigation and obstacle adaptation and (2) multi-agent forecasting and proactive intervention. Vision-language models such as GPT-4V [163] and SC-CLIP [253] enable socially compliant navigation by grounding human motion cues, spatial layouts, and interaction affordances in real time. This approach is demonstrated in OLiVia-Nav [202] on a mobile Jackal robot, which combines GPT-4V (offline) [163] for social-context captioning with SC-CLIP (real-time) [253] for responsive perception, allowing the robot to maneuver around people while maintaining appropriate spacing. For longer-horizon anticipation, GPT-4o [163] supports multi-agent behavior forecasting by analyzing pedestrian formations and predicting future motion patterns. This capability is demonstrated on a custom differential-drive robot equipped with RGB cameras and 2D LiDAR [225], where GPT-4o [163] enable proactive actions such as yielding to people, avoiding emerging congestion.

(4)

Hygiene maintenance and infection control: In hospitals, hygiene maintenance tasks involve disinfecting high-touch surfaces, restocking personal protective equipment (PPE), managing biomedical waste, and monitoring air quality in areas such as ICUs, isolation wards, and operating rooms. Hygiene maintenance and infection control robot tasks decompose into sub-tasks: (1) spatial perception and contamination zone identification, (2) hygiene task grounding and low-level action planning, and (3) real-time PPE tool handling and proactive environmental monitoring. LLM GPT-4 [148], MLLMs such as AutoRT [226], and VLAs such as SayPlan [227] enable robots to detect contamination risks, interpret abstract cleaning commands, and execute safe, context-aware disinfection procedures in clinical environments. AutoRT in [226] supports spatial perception and contamination zone identification by interpreting environment descriptions and linking them to actionable cleaning targets. AutoRT was integrated into the Everyday Robots mobile manipulator. AutoRT’s MLLM is able to parse high-level natural language queries (e.g., “disinfect all handles near patient beds”) under a robot-constitution safety framework and convert them into targeted manipulation instructions. This enables autonomous exploration of clinical rooms and prioritization of high-touch contamination zones. LLMs such as GPT-4 [148] support hygiene task grounding and low-level action planning by performing semantic reasoning over structured scene representations and generating long-horizon disinfection plans. This capability was demonstrated in SayPlan [227] on a Franka Emika Panda arm mounted on an Omron LD-60 mobile base, where GPT-4 [148] performed hierarchical semantic search over 3D scene graphs, which refine feasible disinfection steps across multi-room ICU layouts. This allows the robot to convert abstract instructions such as “sanitize the entire ICU wing” into sequential, spatially grounded actions. Real-time tool handling and proactive environmental monitoring is provided by VLA systems such as PaLM-E [13] by jointly coordinating visual perception, language reasoning, and manipulation. PaLM-E [13] is integrated with mobile manipulators to restock, manage tools, and with additional sensors can monitor indicators such as air quality, UV sanitation cycles, or filter status while avoiding contaminated regions during operation. PaLM-E’s multimodal grounding allows a robot to adapt actions when new hazards emerge, such as when staff flow alters the environment.

5.3. Service Automation

Service automation tasks involve helping customers, managing events, and navigating dynamic public venues such as malls, airports, and museums. These tasks include customer assistance, guidance, and wayfinding [228,229,230,254,255] as well as service setup and maintenance [231,232].

(1)

Customer assistance, guidance and wayfinding: Mobile service robots in public venues such as malls, airports, and museums assist customers by interpreting requests, navigating large and dynamic spaces, and supporting multi-stage guidance under social constraints. These tasks decompose into sub-tasks: (1) language-to-goal spatial grounding, (2) multi-stage waypoint sequencing, and (3) symbolic-spatial planning for real-time urgency-based redirection. LLMs such as GPT-3 [2], GPT-3.5 [173], LLaMA [245], and VLMs such as CLIP [146] and ViNG [230] enable robots to interpret open-ended user queries, associate them with real-world spatial layouts, and generate adaptive navigation plans that remain interpretable and socially aware in complex public environments. Language-to-goal grounding is achieved using GPT-3.5 [173] or LLaMA [245], which convert natural language queries into spatial targets and semantic constraints. This approach was used in OVSG [228] on the Ackermann-steered mobile platform ROSMASTER R2 equipped with RGB-D cameras. OVSG fuses GPT-3.5 [173]/LLaMA [245] with Detic-CLIP embeddings to build open-vocabulary 3D scene graphs encoding object labels, relations, and spatial geometry. As a result, the robot can interpret queries such as “find the nearest pharmacy across from the information desk” and resolve them into precise navigation goals in complex public spaces. LLMs such as GPT-3 [2] combined with VLMs CLIP and ViNG support multi-stage waypoint sequencing by breaking down language instructions into landmark-based routes and grounding those landmarks visually. This was shown in LM-Nav [229] on a Clearpath Jackal robot. GPT-3 [2] parsed instructions into ordered landmark sequences, CLIP matched these landmarks to visual observations, and ViNG predicted traversability between them via a topological connectivity graph. This enables robust execution of multi-stop instructions such as “first go to security, then continue to Gate A12,” even when crowds, occlusions, or temporary barriers appear. Real-time symbolic-spatial redirection is provided by models such as GPT-3.5 [173], which can re-evaluate priorities and update navigation plans as conditions change. This functionality was demonstrated in MLLM-Search [256] and HAM-Nav [257] on a Jackal, where GPT-3.5 reprioritizes goals based on new constraints, such as flight gate changes, service desk relocations, or temporary crowding. This integration of symbolic reasoning and live spatial cues allows the robot to dynamically reroute while preserving safety and interpretability in busy public environments.

(2)

Service setup and maintenance: Service setup and maintenance tasks involve preparing venues, arranging equipment, adjusting layouts, and cleaning up during conferences, trade shows, or public spaces. Robots must interpret vague and evolving instructions such as “set up chairs near the left projector,” “tidy this booth area,” or “build this structure based on the manual.” These sub-tasks decompose into: (1) adaptive motion skill chaining, (2) generalized layout planning and setup, and (3) spatial safety validation and manipulation. Using foundation models such as PaliGemma-3B [258] for multimodal grounding, ${\pi }_0$ [153] and ${\pi }_{0.5}$ [154] for language-conditioned visuomotor control, and PG-InstructBLIP [259] for physical attribute reasoning, mobile service robots perform adaptive skill sequencing, layout planning, and safe manipulation across these service setup and maintenance sub-tasks. LLM PaliGemma-3B [258] and VLA ${\pi }_0$ [153] support adaptive motion skill chaining, which together can translate vague service-setup instructions into structured sequences of low-level actions. This capability was demonstrated in Hi-Robot [231] using the dual-arm mobile manipulator Mobile ALOHA robot [241]. PaliGemma-3B interprets vague commands such as “tidy this booth area” or “organize the display materials,” while ${\pi }_0$ sequences grasping, placing, folding, or lining motions. Human-in-the-loop feedback allows the model to correct or refine behaviors online, which enables robots to adapt skill chains as service conditions change. Generalized layout planning and setup is achieved through the VLA ${\pi }_{0.5}$ [154], which extends LLM [258] by co-training across heterogeneous tasks and spatial rearrangement demonstrations. This generalization is also shown on the Mobile ALOHA platform, where ${\pi }_{0.5}$ can learn to interpret commands such as “set up chairs near the left projector” or “assemble the booth according to this layout” even in unfamiliar rooms. By grounding natural language against perceived furniture, landmarks, and geometry, the robot can infer feasible arrangements and execute multi-step venue setup procedures. Spatial safety validation and manipulation are enabled by PG-InstructBLIP [259], a fine-tuned VLM capable of inferring physical attributes such as fragility, density, or structural integrity from images. PhysObjects [232] used a Franka Emika Panda robot, where PG-InstructBLIP conditioned the robot’s motion policy on object properties. For example, slowing movement near glass displays, adapting grip force for heavy metal stands, or selecting compliant strategies when handling sensitive electronics. This allows a robot to manipulate equipment safely and reliably.

An overview of the three-level framework is presented in Figure 6, which connects application domains to embodied AI challenges and shared foundation model capabilities. At Level 1, the framework spans the three aforementioned domains of domestic assistance, healthcare, and service automation. Level 2 organizes specific systems according to their perception, reasoning, and action components, highlighting how different frameworks (e.g., CLIP-Fields [242], Octo [21], MoMa-LLM [248], OVSG [228], Hi-Robot [231]) combine vision, language, and control. Level 3 captures shared model capabilities across domains, such as CLIP-style VLMs for perception, GPT-family LLMs for reasoning, and VLA models such as RT-2 [14] and SayCan [12] for action. This shows that many foundation models serve as reusable building blocks rather than domain-specific one-offs. This taxonomy clarifies how embedded foundation models generalize across diverse service settings while still supporting domain-specialized behaviors.

6. Ethical, Societal, Human-Interaction, and Physical Design Implications for Mobile Service Robots with Embedded Foundation Models

Foundation models help close key technical gaps for mobile service robots (Section 3), but their use also creates deployment-time risks and responsibilities. The implications below should be read as requirements for safe adoption, not as arguments against foundation models. Each subsection pairs a risk with concrete mitigation hooks that align with the solutions discussed in Section 4 and Section 5. Figure 7 provides an overview of these four interconnected domains of responsibility, ethical, societal human–robot interaction and physical design and ergonomics implications. The circular structure emphasizes that risks in one domain often propagate into the others, motivating integrated mitigation strategies throughout Section 6.1, Section 6.2, Section 6.3 and Section 6.4.

6.1. Ethical Implications

Ethical implications concern the responsible use of foundation models in mobile service robots, focusing on protecting sensitive user data, ensuring accountability for autonomous decisions, and aligning robot behavior with human values.

(1)

Privacy and data governance: Mobile service robots deployed in homes or hospitals collect sensitive multimodal data (visual, audio, and physiological) [260]. When coupled with foundation models trained on web-scale corpora, risks of leakage [261,262], re-identification, or misuse increase [263]. Ethical deployment requires privacy-preserving architectures (on-device inference; selective edge-cloud collaboration), data minimization and purpose limitation, and auditable data-handling protocols aligned with the target domain.

(2)

Accountability and value alignment: When foundation models generate autonomous action plans, it becomes unclear who bears responsibility for harmful outcomes: the developer, operator, or model provider [264,265]. Ethical frameworks should mandate action provenance (logging how language was grounded into plans), human-over-the-loop approval for high-impact tasks, and verifiable policy constraints that encode domain values (e.g., clinical safety norms).

6.2. Societal Implications

Societal implications address how foundation model-powered service robots shape access, labor, and public trust, highlighting equity of deployment, transformation of human work roles, and the cultural acceptance of robots in daily life.

(1)

Accessibility and equity of deployment: If foundation model-powered service robots remain expensive, access will skew toward wealthier institutions [266], leaving vulnerable populations (elderly people and rural communities) behind. Equitable deployment favors cost-efficient, compressed models (Section 4.4), and procurement or reimbursement mechanisms that extend access to underserved homes and care facilities [267].

(2)

Labor and workspace transformation: As foundation-model-enabled robots become capable of handling service automation (hospital deliveries, retail stocking, and cleaning), societal concerns arise about job displacement [268,269]. At the same time, they may augment human workers by offloading repetitive or physically demanding tasks. Policy and design should prioritize augmentation (robotic workflows and role redesign) rather than substitution, with training pathways for clinicians and staff to oversee foundation-model-enabled service robots.

(3)

Trust and public acceptance: Societal trust in robots depends on transparency, reliability, and cultural sensitivity. Public acceptance will require mobile service robots which adapt to cultural norms [270,271] and institutional standards, transparent behavior rationales (“why this action and is this action a socially non-discriminatory one?”), calibrated self-expression, and certification regimes appropriate to domestic and healthcare contexts [272].

6.3. Implications for Human–Robot Interaction

Human–robot interaction implications capture how foundation model-equipped service robots reshape user expectations, emotions, and trust, requiring careful design of emotional, psychological impact management, transparency and conflict resolution.

(1)

Emotional and cognitive impact on users: Prolonged interaction with foundation model-equipped robots can influence user psychology [273]. While service robots may enhance companionship and reduce isolation in eldercare, they can also induce over-reliance, anxiety, or emotional confusion if their social behaviors mimic humans too closely without reliability [274]. Designers should tune anthropomorphism and disclosure (e.g., system status and capabilities) to reduce over-reliance and confusion, especially in eldercare.

(2)

Ethical persuasion and influence: LLM-based dialogue of mobile service robots enables persuasive interactions, reminding an elder to take medication or guiding a child to safe play. However, persuasion risks manipulation [275,276] if not carefully bounded. Research is needed on ethical persuasion policies that define limits for motivational dialogue in health, safety, or childcare contexts.

(3)

Multi-user interaction and conflict resolution: Unlike lab settings, service robots often face competing instructions from multiple people (e.g., family members, nurses). Foundation models excel at multi-turn reasoning but lack arbitration mechanisms [277]. Research should explore multi-user dialogue management and conflict-resolution strategies for embodied agents.

6.4. Physical Design and Ergonomic Implications

Physical design and ergonomics consider the embodiment of mobile service robots to promote safety and effectively support foundation-model-enabled perception, reasoning, and interactions in real-world environments. While foundation models enhance high-level planning, multimodal understanding, and dialogue capabilities, their deployment in domestic and healthcare settings require adaptive hardware configurations, morphology, and easy-to-use interfaces to meet user needs. Therefore, robot design plays a practical role in ensuring that advanced AI capabilities translate into safe, interpretable, and socially appropriate physical behaviors. Below, we discuss three main areas with respect to robot physical design and ergonomic implications.

(1)

Embodiment legibility and intent communication: Foundation models enable robots to generate multi-step, context-sensitive action plans. However, without clear physical signaling, such as visible pre-motion orientation changes (e.g., rotating the base or turning the head toward the target direction before moving), projected path indicators on the floor, auditory intent announcements (e.g., “moving to the kitchen”), such actions may appear abrupt, unpredictable, or unsafe to potential users. Robot design should incorporate legibility mechanisms, including gradual acceleration and deceleration profiles, directional pre-motion cues (e.g., torso or head orientation before navigation), light-based or display-based status indicators, and deliberate motion segmentation between task phases [278,279]. These physical cues allow users to anticipate robot movements and recognize a robot’s intent before behavior execution. This can improve perceived reliability during close-proximity operations.

(2)

Safety-oriented morphology and physical constraints: The increased autonomy enabled by foundation models requires hardware-level safeguards that reduce the severity and likelihood of physical harm. This includes compliant actuations (e.g., series elastic elements or torque-controlled joints); rounded external geometries to reduce injury risk upon accidental contact; conservative force and speed limits attuned for shared human environments, and passive mechanical limits that restrict joint ranges in high-risk zones [280]. Workspace-aware kinematic design, such as constraining the reach of a manipulator near a user’s head, or incorporating soft end-effectors for object transfer, can further reduce risk during assistive tasks. These design strategies ensure that even if high-level plans are imperfect or misinterpreted, the robot’s physical behavior remains bounded within safe operating parameters.

(3)

Ergonomics for multi-user and accessibility contexts: Mobile service robots frequently operate in environments with diverse users, including older adults, children, and individuals with varying demographic characteristics. Thus, ergonomic design should accommodate varying physical abilities and interaction styles. This includes height-adjustable or angled displays for seated and standing users, multimodal input interfaces (touch, voice, gesture), clear visual feedback visible from multiple viewing angles, and intuitive emergency-stop accessibility [281]. In healthcare environments, physical interface placement should avoid interfering with clinical workflows or obstructing equipment. By optimizing for accessibility and shared-space usability, hardware design supports inclusive interaction across heterogeneous user groups.

(4)

Hardware—foundation model co-design constraints: Foundation models impose computational, sensing, and thermal requirements that influence physical architecture. Sensor placements must support reliable multimodal perception (e.g., camera height aligned with seated and standing users), while onboard computation modules require appropriate thermal management without compromising form factor [282]. Power budgeting affects battery placement and weight distribution, influencing stability during robot navigation and manipulation. Co-design between hardware and model capabilities ensures that perception, language grounding, and uncertainty handling can operate reliably within the physical constraints of mobile platforms deployed in human-centered environments.

The same properties that make foundation models powerful for perception, language-to-action mapping, uncertainty handling, and efficient execution also necessitate explicit governance, human–robot interaction design, and embodiment-aware physical constraints, so that the benefits detailed in Section 4 are realized without incurring the harms outlined here.

7. Future Research Directions

Building on the challenges identified in Section 3, Section 4, Section 5 and Section 6, we outline four forward-looking research directions that define a research agenda for advancing foundation-model-enabled mobile service robots: (1) reliability and lifelong adaptation; (2) privacy-aware and resource-constrained inference; (3) governance, standards, and human-in-the-loop frameworks; and (4) multi-robot coordination and fleet-level reasoning. Figure 8 presents these research directions in a multi-stage capability-oriented milestone framework. The vertical axis represents increasing research maturity and verification depth, progressing from controlled research validation (Stage 1) to field-validated autonomy (Stage 2) and ultimately to trustworthy and governed system-level robustness (Stage 3). Across each stage, we define explicit capability milestones that correspond to verifiable system-level properties, such as closed-loop hallucination rejection in real environments, catastrophic forgetting across long-term service tasks, formal latency and safety-response guarantees under live human–robot interaction, privacy-certified federated adaptation without raw data transmission, auditable decision traces suitable for regulatory review, and scalable fleet-level coordination without central bottlenecks.

7.1. Reliability and Lifelong Adaptation

Reliability and lifelong adaptation address the need for mobile service robots with embedded foundation models to operate safely over time, avoiding hallucinations and preserving knowledge as environments and tasks evolve.

(1)

Mitigating hallucinations in safety-critical domains: LLMs and MLLMs are prone to generating plausible but incorrect outputs. In mobile service robots, hallucinations can cause serious risks [283], such as falsely identifying medical equipment or proposing unsafe navigation paths. Future research must develop hallucination-detection pipelines that ground foundation model outputs against real-time sensory consistency checks, ensuring that every proposed action is validated before execution [284].

(2)

Addressing catastrophic forgetting during continual learning: Mobile service robots deployed in hospitals or public spaces must adapt to evolving environments (e.g., service space rearrangements, new medical routines). Yet, online fine-tuning of foundation models risks catastrophic forgetting of previously learned tasks. Future work should explore continual adaptation methods [285,286] with domain-specific rehearsal buffers, elastic weight consolidation at the edge, or modular adapters that preserve core reasoning while integrating local updates.

7.2. Privacy-Aware and Further Resource-Constrained Research and Inference

Privacy-aware and resource-constrained research inference highlights the challenge of adapting foundation models to sensitive domains such as homes and hospitals, where strict data governance and limited onboard compute demand new approaches for secure, efficient, and context-specific execution.

(1)

Data scarcity and privacy in sensitive domains: Unlike web-scale training, robots in homes and hospitals face data scarcity and strict privacy constraints. Foundation models cannot rely on large-scale collection of in-situ data [287]. Future directions include federated foundation model adaptation, synthetic scene generation grounded in domestic and clinical layouts, and on-device fine-tuning under limited compute and memory budgets.

(2)

Efficient scaling for embedded execution: Although Section 4.4 showed advances in compressed and efficient architectures, mobile service robot research still faces energy, latency, and throughput ceilings [288]. Further research is required on task-aware scaling laws, where model complexity dynamically better adapts to robot workload (e.g., low-power monitoring vs. high-bandwidth multi-person dialogue), ensuring safe operation without exhausting onboard resources. There is always room for optimizing computation load.

(3)

Sustainability and energy consumption: Beyond latency and throughput, long-horizon operations of mobile service robots are fundamentally constrained by energy autonomy. Foundation models are energy-intensive, and sustained perception, reasoning, and dialogue can rapidly deplete onboard power, making frequent recharging impractical. Future research should investigate model-aware energy management mechanisms [289] that explicitly couple reasoning depth, perception frequency, and planning fidelity to the robot’s battery state and mission horizon [290]. Such mechanisms may include adaptive inference modes and energy-aware task scheduling. Under low-power conditions, hierarchical activation policies that prioritize safety-critical functions [291] can further ensure context-appropriate and reliable operation over extended deployments.

(4)

Performance trade-offs under onboard computational constraints: Adapting foundation models for onboard execution in mobile service robots introduces critical trade-offs between computational load and key performance properties, including reasoning depth, response latency, perceptual fidelity, and recovery robustness. Techniques such as model compression [292], reduced-precision inference [293], or early-exit reasoning [294] can lower latency and energy consumption, however, they may degrade interaction-level performance, particularly during mid-task corrections or recovery from execution errors [295]. When computational load is aggressively constrained, models may truncate reasoning chains, skip verification passes, or operate with lower-resolution sensory inputs to meet latency budgets. This directly affects safety-relevant behaviors: removal of verification stages prevents cross-checking of predicted actions; reduced reasoning depth weakens long-horizon task consistency; and lower perceptual fidelity increases the likelihood of misidentifying objects or human intent [296]. In healthcare and assisted living settings, such compromises can lead to executing incorrect or unsafe actions, even when average task success metrics remain acceptable. Future research must define explicit deployment constraints (e.g., maximum allowable response latency and required verification depth), below which models should not be compressed or simplified [297].

7.3. Governance, Standards, and Human-in-the-Loop Frameworks

Governance, standards, and human-in-the-loop frameworks emphasize the need for systematic evaluation environments, regulatory guidance, and supervisory protocols to ensure that foundation-model-enabled service robots operate safely, transparently, and under accountable oversight in human-centered domains.

(1)

Standardized benchmarks for mobile service robots: Existing foundation model benchmarks such as Habitat provide strong environments for testing navigation and embodied perception, but they remain limited in capturing the complexity of human-centered service tasks [31]. Current frameworks often evaluate vision, language, or manipulation in isolation, without integrating multimodal grounding, safety norms, or real-time human interaction. Future research should focus on developing multi-room, human-in-the-loop benchmarks that embed realistic human agents, unpredictable activities, and task-level evaluation metrics relevant to homes, hospitals, and service venues. Such benchmarks would enable systematic testing of foundation model-powered service robots under conditions that better approximate real-world domestic assistance, patient care, and service automation.

(2)

Human-over-the-loop interaction protocols: Even with strong autonomy, mobile service robots must allow human supervisors to override or confirm foundation model-generated plans in safety-critical domains [298,299]. Research should explore interaction protocols for seamless fallback control, combining foundation model reasoning with predictable human-in-the-loop interfaces.

7.4. Multi-Robot Coordination and Fleet-Level Reasoning

Multi-robot coordination addresses the challenge of scaling foundation-model-enabled service robots beyond single robots to collaborative teams operating in shared human environments, such as hospitals, airports, and large public facilities. To date, the majority of embedded foundation model research has focused on single-robot perception and decision-making [300]. However, real-world large service applications require groups of robots coordinating tasks with shared spatial and semantic understanding, while resolving conflicts [301].

(1)

Conflict-aware multi-agent planning: When multiple mobile service robots operate concurrently, uncoordinated foundation model reasoning may lead to spatial conflicts, redundant actions, or unsafe congestion around humans. Future research should explore multi-agent extensions of vision-language-action reasoning that incorporate shared world models, explicit conflict prediction, and negotiation mechanisms between agents [302]. This includes grounding high-level language goals (e.g., “assist patients on this floor”) into coordinated task allocations that account for robot proximity, workload balance, and human presence.

(2)

Communication and knowledge sharing across robot fleets: Foundation models enable rich semantic reasoning, however, fleet-level performance depends on how knowledge is shared across robots. Future work should investigate lightweight inter-robot communication protocols that allow robots to exchange task states, uncertainty estimates, and environment updates without excessive bandwidth or privacy leakage [303]. Techniques such as shared semantic maps, event-based updates, or selectively synchronized foundation model adapters could enable collective situational awareness while preserving onboard autonomy.

(3)

Hierarchical and supervisory fleet control: Large-scale service environments often require a hierarchical structure, where local robot autonomy is balanced with fleet-level supervision. Research is still needed on hierarchical orchestration frameworks in which foundation models operate at different abstraction levels: at the local level, individual robots perform perception, grounding, and task execution; at the global level, supervisory planners coordinate task allocation, prioritization, traffic regulation, and escalation to human operators when conflicts arise [304]. By explicitly separating local embodied reasoning from fleet-level orchestration, such hierarchical architectures support scalable coordination while maintaining human oversight in safety-critical domains.

8. Conclusions

This systematic review examined the integration of foundation models into mobile service robotics, focusing on how they address four core challenges: multimodal perception, translation of natural language into executable actions, uncertainty estimation across perception and planning, and computational efficiency for real-time operation. We analyzed how recent advances in LLMs, VLMs, MLLMs, and VLAs provide technical solutions to these challenges, such as cross-modal fusion, instruction grounding, confidence-weighted reasoning, and lightweight scalable architectures. We also discussed real-world applications in domestic assistance, healthcare, and service automation, highlighting how foundation models enable context-aware, socially responsive, and generalizable robot behaviors for service robots. While significant progress has been made in bridging semantic reasoning and embodied robot actions in real-world environments, critical limitations remain for achieving reliable and scalable deployment, specifically in model reliability and lifelong adaptation, privacy-aware and resource-constrained deployment, and the development of regulatory guidance and supervisory protocols for accountable operation in human-centered domains. Research in these areas will allow mobile service robots to safely, adaptively, and collaboratively interact in dynamic human environments to become proactive assistants in everyday life.