Deformable Object Manipulation in Caregiving Scenarios: A Review

Abstract

This paper reviews the robotic manipulation of deformable objects in caregiving scenarios. Deformable objects like clothing, food, and medical supplies are ubiquitous in care tasks, yet pose modeling, control, and sensing challenges. This paper categorises caregiving deformable objects and analyses their distinct properties influencing manipulation. Key sections examine progress in simulation, perception, planning, control, and system designs for deformable object manipulation, along with end-to-end deep learning’s potential. Hybrid analytical data-driven modeling shows promise. While laboratory successes have been achieved, real-world caregiving applications lag behind. Enhancing safety, speed, generalisation, and human compatibility is crucial for adoption. The review synthesises critical technologies, capabilities, and limitations, while also pointing to open challenges in deformable object manipulation for robotic caregiving. It provides a comprehensive reference for researchers tackling this socially valuable domain. In conclusion, multi-disciplinary innovations combining analytical and data-driven methods are needed to advance real-world robot performance and safety in deformable object manipulation for patient care.

1. Introduction

The growing importance of deformable object manipulation (DOM) in caregiving settings can be attributed to various factors. One significant driving force is the aging population, which has led to increasing demand for caregiving services [1]. In addition to supporting the elderly, deformable object manipulation technologies can assist individuals with disabilities and special needs, improving their quality of life and promoting independence [2]. Another essential aspect of deformable object manipulation in caregiving scenarios is the potential to enhance precision and safety in medical procedures. These technologies can reduce the physical strain experienced by human caregivers, preventing injuries and enabling them to provide better patient care [2,3,4]. Despite the promising applications of deformable object manipulation in caregiving settings, numerous challenges must be addressed to harness its potential fully. Deformable objects exhibit complex and high-dimensional behaviour, making them difficult to model and control [5]. Additionally, their non-linear behaviour and interactions with the environment further complicate the development of effective manipulation strategies [6]. Furthermore, robustness and real-time performance are crucial in caregiving scenarios, as unpredictable situations and time-sensitive tasks demand prompt and reliable responses from robotic systems [7].

1.1. The Scope of the Review

This review delves deep into the mechanisms of manipulating deformable objects in caregiving contexts. It examines fundamental operations and the intricate simulation techniques used to model these objects accurately. Additionally, it explores the subtle strategies of perception, planning, and control for precise object manipulation and their application in purpose-built robots and assistive technologies across various caregiving scenarios.

1.2. The Outline of This Review

The outline diagram of this article is represented by Figure 1.

Section 2 summarises the methodology, statistics, and trends in the literature reviewed. Key publication years, research areas, and technology evolutions are analysed.

Section 3 provides an outlook on deformable object manipulation in caregiving over time. Applications are classified and analysed based on utility, frequency, complexity, criticality, and maturity. A multi-factor prioritisation method is also proposed. Key challenges are discussed.

Section 4 examines core technical elements for deformable object manipulation in caregiving. This includes modeling and simulation methods, perception and sensing, planning and control strategies, end-to-end learning frameworks, and purpose-built manipulator and assistive device designs.

Section 5 discusses critical considerations in safety and the responsible integration of technologies into caregiving settings. Mechanical, control, perceptual, and human interaction factors for safe operation are reviewed. Social acceptance barriers and effects on human caregivers are also analysed.

Section 6 concludes by summarising key findings, limitations, and directions needed for continued progress in deformable object manipulation for robotic caregiving assistance.

1.3. Related Works

Previous surveys have examined various aspects of deformable object manipulation, including sensing and control [8], vision-based perception [9], tactile sensing [10], and dynamic manipulation [11]. Recent reviews have focused on learning-based manipulation [12], continuum manipulators [13], and applications in surgery [14]. However, there has been limited synthesis of advances specifically for caregiving contexts. This survey provides a comprehensive overview tailored to caregiving needs by analyzing suitable sensing modalities, control strategies, system designs, and human–robot interaction factors. It uniquely covers the categorisation of objects based on caregiving activities and the integration of manipulation capabilities into assistive robots. By thoroughly examining the latest technological and human-centric considerations, this review offers novel insights to guide further research on deformable object manipulation for improved caregiving.

2. Methodology and Statistics

This literature review was conducted according to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement [15].

An in-depth search was conducted using the Google Scholar search engine on 13 August 2023. The scope was restricted to articles and reviews, exclusively in the English language, using the following search terms: ((“deformable” OR “soft”) AND “object manipulation” AND “modeling” AND “robot”) OR (“caregiving robot” AND (“deformable” OR “soft”) AND “manipulation”) OR (“assistive robot” AND (“deformable” OR “soft”) AND “manipulation”). The retrieved time range was set from 1983 to 2023. This initial search identified 7130 records. After removing duplicates, 6821 records remained.

The study selection process is outlined in the PRISMA flow diagram in Figure 2. The titles and abstracts of the 6821 records were screened for relevance. A total of 5641 records were excluded during this screening, leaving 1180 potentially relevant abstracts. The 1180 abstracts underwent full-text screening, which led to the exclusion of an additional 808 abstracts. The most common reasons for exclusion were irrelevant focus, outdated technologies, lack of ongoing research, or weak relevance to caregiving scenarios. This screening process resulted in 372 eligible full-text articles for full-text review. After acquiring and assessing the full-text articles, 231 were excluded primarily due to lacking focus on deformable object manipulation in caregiving contexts. Ultimately, 141 studies met the inclusion criteria and were included in this review. The included studies comprised 128 articles and 13 systematic reviews. They were sourced from several databases integrated within Google Scholar, including Web of Science, Scopus, IEEE Xplore, and PubMed.

Statistical analysis, as depicted in Figure 3, revealed that the primary sources of references are journal articles followed by conference proceedings, with other types of sources being relatively rare, constituting less than 10 percent.

The foundational references for this sweeping review arc over a significant historical span, ranging from the technologically formative year of 1984 up to the present cutting-edge developments in 2023. However, the spotlight primarily illuminates the period from 2018 to 2022, reflecting the vibrant crescendo of innovation and discovery in this sphere, as shown in Figure 4. The mainstay of these references hails from scholarly journals rich with meticulous studies and groundbreaking findings, whilst conference papers encapsulating spirited discourse and emerging trends within the academic community represent a substantial contingent. This review serves as a panoramic lens, capturing nearly four decades of academic achievement and technological evolution, with a heightened focus on the dazzling array of research breakthroughs witnessed in the past decade.

As shown in Figure 5, early work in the 1980s–1990s focused on mathematical modeling techniques like finite element methods [16] and mass-spring systems [17] to represent deformable object behavior. These allowed some simulation and analysis but were limited in real-world applications. In the 2000s–2010s, vision-based sensing and model-based control strategies started emerging as key areas of research for the perception and manipulation of deformable objects. However, these relied heavily on accurate models and could not handle uncertainties well. Over the last 5 years or so, data-driven techniques like deep learning have become prominent for DOM. Convolutional and recurrent neural networks have been applied for perception and control learning using large datasets. End-to-end learning frameworks combining deep neural networks with reinforcement learning have recently demonstrated promise for learning complex DOM policies directly from sensor inputs. Hybrid approaches are also gaining traction, integrating physics-based modeling with data-driven methods to get the best of both worlds. Advanced simulation environments that leverage differentiable physics have further accelerated DOM research. Overall, the trend has been towards more data-driven methods and learning-based techniques to overcome the challenges of mathematical modeling and achieve more adaptive policies for real-world DOM tasks and applications. The integration of deep neural networks, reinforcement learning, and differentiable physics simulations has been at the forefront of recent advances.

3. DOM in Caregiving: Outlook, Analysis, and Challenges

The growing aging population primarily drives the increasing demand for caregiving robots over the past two decades, the need for cost-effective healthcare solutions, and technological advancements in robotics [1]. This section reviews the critical milestones in the development and deployment of caregiving robots, focusing on real-world cases and the evolving landscape of their applications.

3.1. An Outlook by Timeline

As shown in Figure 6, since the 1980s, deformable object manipulation in caregiving has progressed from early simulation work [16,17] to real-world assistive robot systems, enabled by advances in analytical modeling, computer vision [18], and AI techniques. Key applications that emerged over the decades include robotic dressing [19], bedding [20,21,22], feeding [23,24,25,26], personal hygiene [27], and medical care assistance [28]. While early research in the 2000s relied on model-based methods [29,30,31], recent breakthroughs leveraged data-driven deep learning [32,33] for more flexible policies, like robotic knot-tying in 2021 [34,35]. The timeline illustrates milestones from early vision sensing in the 2000s [18] to robotic meal assistance in 2020 [24] and sway adaptation for bandaging in 2023 [28]. The overarching trend is the translation of deformable object manipulation from simulation environments to real-world caregiving tasks, with increasing autonomy and intelligence. This promises continued improvement in robotic assistance for the elderly and disabled through handling real-world deformable objects.

3.2. Classification of Applications

Over the past decade, several cases of DOM in caregiving scenarios have been documented, showcasing advancements in robotic capabilities, methodologies, and applications in the caregiving domain. User perspectives have also been studied, such as factors influencing the adoption of caregiving robots by the elderly and attitudes towards assistive systems [36]. In this critical outlook, we classify the cases based on tasks and functions and calculated the proportion of each application in the references in this review in Figure 7.

• Dressing assistance is an essential application of DOM in caregiving scenarios, particularly for users with limited mobility or dexterity. Researchers have developed robots capable of assisting users in putting on various clothing items, including T-shirts [19], pants [37], and footwear [38]. These studies have demonstrated the potential of DOM in addressing the challenges faced by individuals with disabilities and the elderly in performing daily dressing tasks.
• Bedding and Cloth Management is another crucial application of DOM in caregiving scenarios, as it involves handling large, deformable objects and ensuring user comfort and hygiene. Researchers have developed robotic bed-making systems to grasp tension and smooth fitted sheets [20]. Robots capable of managing blankets [21] and pillows [22] have also been developed. The success of these works highlights the potential of DOM in addressing the challenges associated with bedsheet management and the need for continued research and development in this area. Cloth folding is also essential in caregiving settings, particularly for maintaining order and cleanliness. Researchers have developed robots capable of folding clothes, such as towels, shirts, and ropes [19,34,39,40,41,42,43].
• Personal Hygiene Support is another critical application of DOM in caregiving scenarios. Researchers have developed robots that handle soft materials such as gauze [27] and diapers [44]. These studies have highlighted the importance of integrating various sensory modalities and control techniques for effective soft material handling in caregiving scenarios.
• Meal Assistance is another important application of DOM in caregiving scenarios, particularly for users with limited mobility or dexterity. Researchers have developed robots capable of manipulating deformable objects such as food items and utensils [23,24,25,26]. These studies have demonstrated the potential of DOM in addressing the challenges faced by individuals with disabilities and the elderly in performing daily meal assistance tasks.
• Daily Medical Care. In the context of bandaging, deformable object manipulation systems offer improved precision and control, enabling more effective and efficient wound-dressing procedures. These systems can adapt to the varying shapes and contours of the human body, as well as the patient’s involuntary swaying, ensuring proper bandage placement and tension for optimal healing [28]. Deformable object manipulation technologies assist patients during therapeutic exercises and activities in rehabilitation. They provide real-time feedback, support, and guidance, enhancing repair and promoting faster recovery [45,46,47].
• Some other Applications. DOM has also been explored in other caregiving applications, such as housekeeping tasks like laundry management [48,49], window cleaning [50], and dishwashing [51].

The past decade has witnessed significant advancements in deformable object manipulation in caregiving scenarios. The increasing demand for caregiving robots highlights the importance of continued research and development to advance their capabilities in DOM and ensure their responsible deployment in various care settings.

Deformable objects are ubiquitous across the diverse caregiving applications discussed, highlighting the importance of continued advancements in deformable object manipulation capabilities. In the following section, we will delve deeper into categorising deformable objects based on operational modes relevant to caregiving tasks and analyse the unique challenges they present.

3.3. A New Method to Classify and Analyse

In past research, the classification of deformable objects primarily adopted methodologies such as Sanchez’s [27], which emphasised the physical properties and morphology of the objects. The main objective of these techniques was to facilitate precise modelling of the objects in question. However, a paradigm shift in recent research trends, particularly in Artificial Intelligence, has changed this approach considerably.

The introduction of innovative learning methodologies like imitation learning and reinforcement learning has accelerated the capabilities of robotic arm operations. Consequently, the necessity to model different deformable objects separately has become less prominent. This study presents a unique classification framework focusing on distinct material behaviors and properties that influence manipulation strategies, as shown in Figure 8. This classification takes into account both the functionality of deformable items in caregiving scenarios and the physical characteristics of the items, which facilitates further analysis of the operating objects faced by the caregiving robot/robotic arm.

3.3.1. Common Types of Deformable Objects in Caregiving Scenarios

• Textiles: This category would cover all cloth and fabric objects like clothing, sheets, towels, etc. Key properties are flexibility, drape, and shear.
• Elastomers: Includes stretchable/elastic materials like bandages, tubing, and exercise bands. Key properties are elongation and elasticity.
• Fluids: Encompasses materials like water, shampoo, and creams that flow and conform to containers. Key behaviors are pourability and viscosity.
• Aggregates/Granular: Covers aggregated materials like rice, beans, and tablets. Flows but maintains loose particulate structure.
• Gels: Highly viscous/elastic fluids like food gels, slime, and putty. Resist flow due to cross-linked molecular structure.
• Cellular/Porous: Materials with internal voids like sponges and soft foams. Compressible and exhibit springback.
• Composite/Hybrid: Combinations of the above categories, like stuffed animals and packaged goods. Display complex interactions of properties.

This review summarises the proportions of research objects found in the reference literature, as depicted in Figure 9, where textiles stand out as the most popular deformable objects for research, whereas studies on operations involving cellular/porous and aggregates/granular objects are currently quite limited, which is a scarcity that could indicate deficiencies in practical applications.

3.3.2. A Multi-Factor Analysis Method

While assigning arbitrary subjective scores lacks rigor, a multi-factor analysis can systematically prioritise deformable object manipulation research needs for caregiving. Several key factors should be considered:

• Application Utility (U)—Potential to reduce caregiver burden by automating tasks
• Object Frequency (F)—How often the object occurs in caregiving activities
• Task Complexity (C)—Technical challenges posed by physical properties and handling difficulties
• Safety Criticality (S)—Risks of injury or harm during object manipulation
• Research Maturity (M)—Existing state of manipulation methods for the object

To apply this analysis, quantitative metrics for each factor are gathered via:

• Surveys, interviews, and activity logging (for Application Utility)
• Workflow observations and activity logging (for Object Frequency)
• Material testing, caregiver surveys, and interviews (for Task Complexity)
• Incident data and healthcare professional feedback (for Safety Criticality)
• Literature review (for Research Maturity)

The metrics are then combined as follows:

• Each metric is normalised on a 0–1 scale based on the maximum value observed. This transforms metrics to a common scale.
• Criteria weights are assigned to each factor based on the caregiving context. For example, Safety Criticality may be weighted higher for hospital settings compared to home care.
• Weighted sums are calculated by multiplying each normalised metric by the criteria weight.
• The weighted sums are aggregated to derive an overall priority score P for each deformable object, where U, F, C, S, and M are the normalised metrics and w1 to w5 are the criteria weights.:

  $$\begin{array}{cc}\hfill P& =w1\times U+w2\times F+w3\times C+w4\times S-w5\times M\hfill \end{array}$$

  (1)
• Objects are ranked by priority score P, identifying high-impact areas needing research innovations.

Tuning the criteria weights for specific caregiving applications allows us to systematically prioritise the most promising research directions via this data-driven analysis. Multi-factor analysis indicates general priorities, while subjective scoring adds nuanced use-case customisation. Using both techniques together provides rigor and versatility in determining the key deformable object manipulation research needs for a maximal real-world impact.

However, our exploration continues further. In the next part, we will delve into the many challenges of handling these deformable objects, particularly when considering their high-dimensional, non-linear, and time-varying behaviour. Whether it is the need for the real-time perception of the object’s shape and interactions or the development of novel control frameworks, we are faced with a complex task that requires a multi-disciplinary approach. This highlights the necessity of moving beyond traditional methodologies and stepping into the realm of data-driven, hybrid models that can more adequately cater to the demands of real-world caregiving scenarios.

3.4. Challenges of DOM in Caregiving

Accurately modelling deformable objects is challenging due to their high-dimensional, non-linear, and time-varying behaviour [16]. While mathematical models, such as the Finite Element Method (FEM) and Mass-Spring models, can capture some of these properties [17], they may not always be sufficient for real-world caregiving scenarios. This necessitates exploring data-driven and hybrid modelling approaches, such as deep learning-based models [41,43,52,53,54,55] and Gaussian process regression [56,57,58,59,60].

Deformable object manipulation often requires real-time, robust, and precise perception of the object’s shape, deformation, and environmental interactions [61]. This can be challenging due to occlusions, sensor noise, and varying environmental conditions. Multi-modal sensing techniques, such as vision and tactile sensing, have enhanced perception capabilities [62,63,64]. Recent advancements in soft tactile sensors and RGB-D cameras have further improved the accuracy and reliability of deformable object perception [65].

Developing effective control and planning algorithms for deformable object manipulation is challenging due to the non-linear dynamics, uncertainties, and constraints associated with these objects [66]. Traditional robotic control strategies may not be suitable, leading to the investigation of novel control frameworks, including learning-based, adaptive control methods [67] and impedance control [68].

In caregiving scenarios, ensuring the safety of patients and human caregivers during deformable object manipulation is of paramount importance. This requires the development of control strategies that can adapt to uncertainties and unforeseen events, as well as methods for effective communication and collaboration between robots and humans. Recent research on shared autonomy, where a human operator and a robotic system jointly control the manipulation process, has shown promise in achieving safe and efficient deformable object manipulation in caregiving scenarios [69,70,71].

4. Key Technologies for DOM in Caregiving

As mentioned above, addressing these challenges requires a multi-disciplinary approach, combining advancements in modelling, sensing, control, and human–robot interaction to develop practical solutions for deformable object manipulation in caregiving scenarios.

Having explored the categories and complexities of deformable objects commonly encountered in caregiving scenarios, we now shift our focus to developing solutions to manage these challenges. In the upcoming sections, we will examine key technologies and strategies to enhance the robotic manipulation of delicate and unpredictable materials for assistive applications.

4.1. Modelling and Simulation

Robots and assistive systems are increasingly employed in various care applications; they must interact with complex, delicate, and flexible materials, such as human tissues, garments, and assistive devices [72,73,74]. Developing accurate models and simulations for these deformable objects is essential to understanding their behaviour and dynamics [27,75]. This knowledge allows robotic systems to plan and execute precise manipulations, ensuring patients’ safety and comfort [4,76]. In this context, modelling and simulation serve as indispensable tools for designing, testing, and optimising the performance of robotic and assistive systems in the ever-evolving field of caregiving [77,78,79]. This section will discuss mathematical, data-driven, and hybrid modelling for manipulating deformable objects and provide outstanding simulation environments.

4.1.1. Mathematical Models

Mathematical models are essential for understanding the complex behaviour of deformable objects in caregiving scenarios. Continuous models, such as those based on linear elasticity, nonlinear elasticity, and viscoelasticity, describe deformation using partial differential equations (PDEs) [75,80]. Discrete models, like the FEM and Mass-Spring models, use interconnected elements or particles to represent deformable objects, simplifying PDEs into ordinary differential equations (ODEs) or algebraic equations [81,82]. The appropriate mathematical model depends on accuracy, computational resources, and application requirements.

4.1.2. Data-Driven Models

Data-driven models offer an alternative approach to deformable object modelling, leveraging machine learning techniques to learn object behaviour from the data instead of explicit mathematical equations. These models can adapt to varying object properties and environmental conditions, making them suitable for many caregiving applications.

Deep learning-based models, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and the graph neural network (GNN), have shown great promise in modelling complex deformations and predicting the dynamics of deformable objects from visual and tactile data [32]. Researchers have used CNNs to learn mapping between tactile sensor measurements and object deformation, enabling robots to manipulate soft objects with high precision [83].

Gaussian process regression (GPR), a non-parametric Bayesian method, has been employed to model the deformation of soft objects in robotic grasping tasks. GPR provides a probabilistic representation of the object’s state, allowing uncertainty quantification and robust decision-making [60].

4.1.3. Hybrid Models

Hybrid models are an emerging class of deformable object models that combine the strengths of both mathematical and data-driven models. By integrating mathematical models with data-driven techniques, hybrid models can achieve improved accuracy, adaptability, and generalisation capabilities compared to standalone models. One example of a hybrid model is the combination of the FEM with neural networks, where FEM provides a physics-based prior and the neural network refines the model parameters or captures the residuals between model predictions and the observed data [84].

Researchers have also explored using GPR in conjunction with mathematical models in order to better capture the uncertainty in deformable object manipulation tasks. By incorporating GPR into a physics-based model, these hybrid models can provide probabilistic representations of the object’s state, allowing for uncertainty quantification and more robust decision-making [17].

4.1.4. Simulation Tools and Environments

Simulation tools and environments are crucial in developing, testing, and validating deformable object manipulation strategies. These tools provide a controlled virtual environment for evaluating the performance of various algorithms and models and generating synthetic data for training and testing data-driven models.

A popular physics engine for deformable object simulations is the Bullet Physics Library [85], which supports both rigid body dynamics and soft body dynamics, including FEM-based deformable models. The SOFA (Simulation Open Framework Architecture) framework is another widely used tool for simulating deformable objects, offering a modular architecture and support for various modelling and simulation techniques, such as FEM, mass-spring, and continuum mechanics models [86].

In recent years, developing sophisticated simulation environments incorporating diverse learning algorithms and heightened intelligence has dramatically enhanced the functional progress of deformable object manipulation. These advancements have not only improved the development efficiency but also reduced the associated costs. This section will name a few of these cutting-edge simulation environments and provide a critical analysis of their respective strengths and weaknesses. The simulation tools to be introduced below are displayed on a timeline in Figure 10.

• SoftGym [52] is a simulation environment focusing on soft-body manipulation tasks, providing researchers with a platform to develop and test algorithms for various applications such as robotic grasping and manipulation. While SoftGym may offer unique benefits and opportunities, it is essential to examine its limitations and potential areas for improvement critically. One possible drawback is that the simulation may need to fully represent the complex real-world conditions, which could lead to discrepancies when applying developed algorithms to actual tasks. Additionally, the simulation might only cover some possible soft-body objects and scenarios, potentially limiting its applicability to a narrower range of cases. Further research and development in SoftGym may be required to address these limitations and ensure the platform’s continued relevance and effectiveness.
• DeformableRavens [87] is an open-source simulated benchmark, with 12 tasks manipulating 1D, 2D, and 3D deformable objects to help accelerate research progress in the robotic manipulation of deformable materials. It creates an end-to-end target conditional transportation network that learns visual-based multi-step operations for deformable 1D, 2D, and 3D structures. However, the current scope of DeformableRavens is limited to a set of predefined tasks and objects, which may hinder its adaptability to more diverse scenarios.
• ReForm [88] is another simulation environment focusing on deformable objects like metal wires with elastic and plastic properties. While it addresses the limitations of SoftGym, more information about its usability, versatility, and performance in a broader range of applications would be needed to assess its overall effectiveness.
• PlasticineLab [77] is a simulation environment focusing on soft-body manipulation, utilising differentiable physics to optimise control policies for robotic manipulation tasks. While PlasticineLab offers a novel approach to solving soft-body manipulation problems, it is crucial to consider potential limitations and areas for improvement. One concern could be the accuracy of the differentiable physics models, which might need to capture the complex interactions between objects and their environment fully. Furthermore, the scalability of the simulation to more complex and diverse scenarios may be limited, which could affect its applicability to real-world situations.
• DefGraspSim [78] is a simulation environment focusing on grasping 3D deformable objects like fruits, vegetables, and internal organs. While the simulation provides valuable insights into robotic grasping strategies, it is essential to consider potential limitations and areas for improvement. For instance, the simulation may not account for all possible variations in object shape, material properties, and environmental factors, which could affect the performance of developed algorithms in real-world applications. Additionally, the simulation’s efficiency may be limited by the processing power of the GPU used, potentially restricting the ability to test a wide range of objects and scenarios in a reasonable time frame.
• RCareWorld [79] is a human-centric simulation environment designed to develop physical and social robotic caregiving. The simulation incorporates inputs from stakeholders such as care recipients, caregivers, occupational therapists, and roboticists. While RCareWorld offers a promising platform for developing robotic caregiving solutions, examining potential limitations and areas for improvement is essential. For example, the simulation may not fully capture the complexities of human–robot interaction in real-world caregiving settings, which could lead to discrepancies when applying developed algorithms to actual tasks. Additionally, the simulation might only cover some possible care scenarios and patient needs, potentially limiting its applicability to a narrower range of cases. Ongoing research and development in RCareWorld will be crucial to address these limitations and ensure the platform’s continued relevance and effectiveness.

Combining hybrid models [84] and advanced simulation tools and environments [86] facilitates the development and evaluation of deformable object manipulation strategies in caregiving scenarios. By leveraging the strengths of both mathematical and data-driven models and using comprehensive simulation environments, researchers can create more effective and robust techniques for handling deformable objects in a wide range of caregiving applications. These advances will ultimately improve the performance and safety for caregivers and care recipients in various healthcare and home care settings.

4.2. Perception and Sensing

Perception and sensing in deformable object manipulation within caregiving scenarios [1] is critical as it directly impacts the quality of care provided. Robots and assistive systems must handle delicate, flexible, and unpredictable materials in these settings, making accurate assessment and interaction essential. Adequate perception and sensing techniques allow these systems to gather crucial information about the objects’ properties and behaviours, enabling improved decision-making and action planning. By prioritising the development and integration of innovative perception and sensing methods, deformable object manipulation systems can achieve enhanced precision, adaptability, and safety in caregiving contexts, ultimately leading to better patient outcomes [2,3,4]. This section will explore vision-based [5,6,7,8], tactile-based [61], and sensor fusion approaches [12] for perception and sensing in deformable object manipulation.

4.2.1. Vision-Based Techniques

Vision-based techniques enable robots to perceive and manipulate deformable objects in caregiving scenarios. These techniques primarily involve processing and analysing camera image data to extract information about the object’s shape, deformation, and pose. Various vision-based methods have been developed to address the unique challenges associated with deformable object manipulation, such as occlusions, varying appearances, and non-rigid transformations.

One popular approach uses RGB-D sensors, which provide colour and depth information, allowing for more accurate and robust object segmentation and pose estimation [18]. Researchers have developed methods for fusing colour and depth data to improve the analysis of object deformation and grasp planning [89,90]. Deep learning-based techniques, such as CNNs, RNNs, and GNN, have been widely used for vision-based deformable object manipulation tasks, including object segmentation, pose estimation, and deformation prediction [43,83,91]. These techniques can learn complex, high-dimensional representations of deformable objects from large amounts of image data, enabling robots to handle various things with different shapes and properties.

Template matching and model-based methods are also employed to estimate the pose and deformation of known deformable objects [92,93]. These methods involve matching the observed image features to a pre-defined template or a 3D model, allowing for an accurate estimation of the object pose and deformation under varying conditions.

4.2.2. Tactile Sensing

Tactile sensing is another essential aspect of deformable object manipulation in caregiving scenarios as it allows robots to perceive contact forces, object properties, and local deformations during manipulation tasks. Tactile sensors can complement vision-based techniques by providing additional information that may not be easily accessible via visual observations, such as the distribution of contact forces, friction, and local compliance of the object [94].

Various types of tactile sensors have been developed for robotic manipulation tasks, including capacitive, piezoresistive, and piezoelectric sensors [95]. Each sensor type offers unique advantages and trade-offs regarding sensitivity, resolution, and robustness, making them suitable for different applications and environments.

Researchers have explored the integration of tactile sensing into deformable object manipulation tasks, such as object grasping, manipulation, and deformation control. For instance, tactile feedback has improved grasp stability and adapted to object deformation during manipulation [96]. Tactile sensing can also be utilised for real-time deformation control, enabling robots to monitor and adjust the applied forces to prevent excessive deformation or damage to the object [97].

Machine learning techniques, including deep learning and reinforcement learning, have been applied to process and interpret tactile sensor data for deformable object manipulation tasks [98,99]. These methods can learn complex relationships between tactile sensor readings and object properties or manipulation outcomes, allowing robots to make more informed decisions during manipulation tasks.

Recent studies show that robots can better understand the object’s properties and behaviour by combining tactile sensing with vision-based techniques, improving performance and safety during manipulation tasks.

4.2.3. Sensor Fusion

Sensor fusion integrates multiple sensor data types, such as vision, tactile, and force sensing, and has emerged as a promising approach for deformable object manipulation in caregiving scenarios. By combining information from various sources, sensor fusion can enhance robotic systems’ accuracy, robustness, and adaptability, enabling them to better handle the complexities associated with deformable objects [100].

Vision-based sensing is widely used in robotic manipulation, providing rich information about the object’s shape, colour, and texture [101]. However, vision-based methods may need help with occlusions, specular reflections, or poor lighting conditions. Tactile sensing, on the other hand, can complement vision by providing direct contact information, such as pressure distribution, contact location, and object compliance [102,103]. Force sensing further augments the system’s capabilities by measuring the forces exerted during manipulation, which can be essential for ensuring patient safety and comfort [95].

Recent studies have demonstrated the benefits of combining these sensing modalities in various caregiving scenarios. For instance, in surgical applications, integrating vision and force sensing has proven effective in providing more accurate feedback during delicate tissue manipulations [104]. In rehabilitation settings, fusing tactile and force information has improved the control of soft robotic exoskeletons, leading to better patient outcomes [72]. One example of sensor fusion in deformable object manipulation is the work of Luo et al. [105], which describes a framework that fuses vision and forces feedback to control highly deformable objects. This approach demonstrated an improved performance compared to using vision or force sensing alone, highlighting the potential benefits of sensor fusion in handling deformable objects.

Another example can be found in the work of Choi et al. [106], who developed a multi-fingered robotic system for dexterously manipulating deformable objects using visual and tactile sensing. Their approach used a combination of visual features extracted from RGB-D images and tactile measurements from an array of pressure sensors to model the object’s deformation. This information was then used to control a robotic manipulator to achieve the desired deformations, demonstrating the effectiveness of sensor fusion in controlling deformable objects.

Moreover, researchers have explored deep learning techniques for sensor fusion in deformable object manipulation. For example, Yang et al. [32] proposed a deep learning-based framework for tactile and visual sensing fusion in robotic grasping. Their method combined CNNs to extract features from tactile and visual data, fusing them using a multimodal fusion layer. This approach led to an improved grasping performance, showing the potential of deep learning for sensor fusion in deformable object manipulation.

There are also efforts to develop general frameworks for sensor fusion in robotic manipulation. One such example is the work of Inceoglu et al. [107], who proposed a Deep Multimodal framework for integrating multiple sensor modalities, including vision, sonar, microphone, and tactile sensor data. Their approach aimed to account for the uncertainties associated with each sensing modality and provided a principled way to fuse the information for an improved object manipulation performance in order to make robots more aware of the unintended outcomes of their actions for ensuring safety.

4.3. Planning and Control

The significance of planning and control in deformable object manipulation within caregiving scenarios cannot be overstated [72,105,106]. In these contexts, robots and assistive systems must interact with delicate and often unpredictable objects, such as human tissues, textiles, or prosthetics, making precise planning and control crucial for patient safety and comfort [32]. Efficient planning enables the system to navigate complex interactions with deformable objects by predicting their behaviour and adjusting actions accordingly [108]. This reduces the risk of unintended consequences and enhances the system’s adaptability and overall performance [18]. Furthermore, robust control techniques ensure that manipulations are executed accurately and smoothly, accounting for the inherent uncertainties and complexities associated with deformable objects [29]. By integrating advanced planning and control strategies, deformable object manipulation systems in caregiving scenarios can achieve a higher level of dexterity, responsiveness, and reliability, ultimately improving the quality of care and the overall patient experience [72]. This section examines model-based, model-free, and hybrid control strategies for manipulating deformable objects.

4.3.1. Model-Based Control

Model-based control strategies for deformable object manipulation rely on accurate models of the object’s geometry, material properties, and dynamics to predict its behaviour and plan appropriate control actions [29]. These approaches typically involve using mathematical models, such as finite element models, mass-spring systems, or continuum mechanics models, to represent the object’s deformations and dynamics [30,31]. Model-based control methods can offer high performance and accuracy. Still, they may require extensive prior knowledge of the object’s properties and can be computationally expensive for complex objects or real-time applications [16,17].

One advantage of model-based control strategies is that they allow us to explicitly consider constraints, such as object deformation limits, contact forces, or actuator limitations, during the planning and control process [11,29]. This can lead to safer and more effective manipulation strategies [30], particularly in caregiving scenarios where the object’s integrity and patient safety are paramount [109,110].

Various model-based control algorithms have been proposed for deformable object manipulation tasks, such as force control, impedance control, and model predictive control (MPC) [108]. These methods can adapt to varying object properties and environmental conditions, enabling a robust and reliable manipulation performance.

4.3.2. Model-Free Control

Model-free control strategies for deformable object manipulation do not rely on explicit object models but instead learn control policies directly from the sensor data and interaction experiences [111,112]. These approaches can be particularly advantageous when the object’s properties are unknown, difficult to model, or subject to change over time. Model-free control methods often employ machine learning techniques, such as reinforcement learning or imitation learning, to learn control policies from the data.

Reinforcement learning algorithms enable robots to learn control policies by interacting with the environment and receiving feedback as rewards or penalties [113]. These methods can discover effective manipulation strategies via trial-and-error exploration without explicit object models or human supervision. Recent advances in deep reinforcement learning have shown promising results in learning complex control policies for deformable object manipulation tasks [35].

Imitation learning techniques allow robots to learn control policies by observing and mimicking human demonstrations or other expert behaviour. These methods can leverage human expertise to bootstrap the learning process and accelerate the acquisition of effective manipulation strategies. Imitation learning has been successfully applied to various deformable object manipulation tasks, such as knot tying, cloth folding, and surgical suturing [35,114,115].

4.3.3. Hybrid Control Strategies

Hybrid control strategies combine the advantages of both model-based and model-free approaches to achieve a more effective and robust deformable object manipulation performance [60,74,77,78]. By integrating explicit object models with learning-based methods, hybrid control strategies can leverage the benefits of prior knowledge while also adapting to uncertainties or changes in the object’s properties and environment [35,108,114]. One common approach in hybrid control strategies is using model-based controllers, such as MPC or impedance control, in conjunction with reinforcement learning or imitation learning algorithms [116,117,118]. The model-based controller provides an initial control policy based on the object’s known properties. At the same time, the learning-based algorithm refines and adapts the approach through interaction with the environment or observation of expert demonstrations [74,115]. The control techniques discussed enable the precise and adaptive manipulation of deformable objects. Building on these methods, the integration of end-to-end learning promises more dynamic and flexible solutions by learning complex mappings directly from the sensor data [33,119,120]. Next, we will discuss the application of end-to-end learning to deformable object manipulation.

4.4. End-to-End Learning

End-to-end learning has emerged as a powerful approach for addressing complex robotics and artificial intelligence tasks. This paradigm aims to learn direct mapping from the raw sensory inputs to the control outputs, bypassing the need for explicit intermediate representations or handcrafted features. In deformable object manipulation, end-to-end learning has shown great potential for enabling robots to perform tasks with high levels of complexity, adaptability, and robustness.

One of the critical advantages of end-to-end learning is its ability to leverage deep learning techniques, such as CNNs and RNNs, to extract relevant features and representations from the raw sensory data automatically. This allows robots to learn more effectively from large amounts of data and generalise the data to new situations. Several studies have demonstrated the effectiveness of deep learning-based approaches for deformable object manipulation, such as cloth folding, knot tying, and surgical suturing [33,119].

Reinforcement learning is another critical component of end-to-end learning in deformable object manipulation [111]. Reinforcement learning algorithms enable robots to learn control policies by interacting with their environment and receiving feedback in the form of rewards or penalties [113]. This trial-and-error learning process allows robots to discover effective manipulation strategies without requiring explicit object models or human supervision. Recent advances in deep reinforcement learning have shown promising results in learning complex control policies for deformable object manipulation tasks [112,120].

Imitation learning is another end-to-end learning technique successfully applied to deformable object manipulation. Robots can learn control policies that leverage human expertise by observing and mimicking human demonstrations or other expert behaviour [115]. This approach can accelerate learning and enable robots to perform challenging tasks more efficiently, such as cloth folding, knot tying, and surgical suturing.

One of the critical challenges in end-to-end learning for deformable object manipulation is the need for large amounts of training data, as deep learning techniques typically require a considerable amount of data to achieve a good generalisation performance [121]. Collecting this data can be time-consuming and labour-intensive, especially in real-world caregiving scenarios.

Researchers have explored various data augmentation techniques to address this challenge, such as domain randomisation and synthetic data generation, to generate more diverse and representative training data [122].

Another challenge in end-to-end learning is specifying appropriate reward functions for reinforcement learning algorithms. Defining a suitable reward function that accurately captures task objectives and constraints can be challenging in many deformable object manipulation tasks. One promising solution is to use inverse reinforcement learning techniques, which learn reward functions from demonstrations or expert behaviour. By combining inverse reinforcement learning with deep reinforcement learning, robots can learn more effective control policies for deformable object manipulation tasks [116,117,118,123].

4.5. Manipulator and Assistive Device Designs

The focus of the following sections is on discussing practical robotic manipulator and assistive device designs for handling deformable objects in caregiving scenarios, rather than simulation techniques [77,78,79]. This provides an overview of recent hardware innovations that can directly improve real-world caregiving tasks and quality of life [21,22,74]. The aim is to highlight advancements in robotic arms, grippers, and assistive technologies that address specific requirements and challenges in deformable object manipulation during service provision [8,73].

In service provision, caregiving robot actuator designs can be categorised into manipulator and assistive device designs [13,124]. This section examines the design characteristics of robots handling deformable objects within these two service modalities, highlighting the distinct creation focuses for each type.

4.5.1. Manipulator Design

Robotic arms and grippers are essential components in deformable object manipulation within caregiving scenarios. In recent years, there have been significant advancements in robotic arm and gripper designs, aiming to address the specific requirements and challenges of various caregiving tasks.

For instance, lightweight robotic arms with enhanced dexterity and adaptability have been developed, which are particularly suitable for tasks such as dressing assistance [74], food feeding [26], or personal hygiene assistance [109]. These robotic arms often incorporate advanced sensing and control technologies to ensure safe and efficient human–robot interaction. Collaborative robots (cobots) have also been explored for caregiving applications, emphasising human–robot collaboration while maintaining safety standards [125].

Regarding grippers, many recent designs have been proposed to handle deformable objects in caregiving tasks. Soft robotic grippers have gained significant attention due to their ability to conform to the shape of deformable objects and provide gentle yet firm grasping. Recent advancements in soft robotic grippers include adaptability, sensing capabilities, and control strategies [73].

Another noteworthy development in gripper technology is the use of AI-enabled multi-fingered hands which are capable of learning and adapting grasping strategies based on sensory feedback, making them particularly suitable for handling deformable objects with complex geometries [126]. These grippers often employ advanced machine learning techniques, such as deep learning and reinforcement learning, to acquire efficient grasping strategies from the data [33].

In addition, researchers have investigated the use of underactuated grippers, which can adapt to the shape of deformable objects using fewer actuators than the number of degrees of freedom [127]. These grippers balance complexity, weight, and adaptability, making them suitable for specific caregiving tasks. Recent research in robotic arms and grippers for deformable object manipulation in caregiving scenarios has focused on enhancing dexterity, adaptability, and human–robot interaction and incorporating advanced sensing and learning capabilities. This progress in robotics and assistive technology promises to improve the quality of life and independence for individuals requiring caregiving assistance.

4.5.2. Assistive Device Design

Assistive technologies are essential components of robotic systems for deformable object manipulation in caregiving as they provide the necessary support, guidance, and augmentation of the robot’s capabilities to perform complex tasks safely and effectively [128]. These technologies encompass many solutions, such as sensor-based guidance systems, haptic interfaces, teleoperation, and shared control [128].

Sensor-based guidance systems can enhance the robot’s perception and understanding of deformable objects and their environment, enabling a more accurate and reliable manipulation performance. These systems often integrate multiple sensing modalities, such as vision, tactile, and force sensing, to comprehensively represent the object’s shape, deformation, and interaction forces [129].

Haptic interfaces can provide intuitive and effective human–robot interaction in caregiving scenarios, allowing the human operator to feel and manipulate the deformable object via force feedback and tactile sensations. These interfaces can facilitate precise control of the robot’s actions, improve the operator’s situational awareness, and enable more natural and intuitive communication between humans and robots [130].

Teleoperation and shared control methods enable human operators to guide or supervise the robot’s actions during deformable object manipulation tasks, leveraging their expertise and cognitive capabilities [131]. Teleoperation allows the operator to control the robot’s motions and activities directly. In contrast, shared control methods distribute the control responsibilities between humans and robots based on their strengths and limitations.

5. Safety Ensurance and Social Challenges

While advances in modeling, sensing, control, and learning show great promise for improving the robotic manipulation of deformable objects, ensuring these technologies are safely and responsibly integrated into caregiving contexts remains critically important. As robots and assistive devices interact with patients in sensitive healthcare and home environments, maintaining safety, trust, and acceptance by users must be prioritised. Seamless collaboration between humans and robots is needed to leverage their complementary strengths. Additionally, ethical considerations regarding privacy, autonomy, and effects on caregiver employment must be proactively addressed.

The following section examines key factors in ensuring safety during deformable object manipulation and overcoming societal barriers to enable patient-centered integration of robots in caregiving roles. A holistic approach considering mechanical design, adaptive control, robust perception, intuitive interfaces, and transparent data practices can lead to the responsible adoption of robotic assistance in domains like elderly care and medicine. By coupling technological innovation with human-centric practices, deformable object manipulation in caregiving can be advanced to improve care quality while mitigating risks.

5.1. Safety Ensurance

Ensuring safety during the robotic manipulation of deformable objects in caregiving settings requires addressing risks across multiple areas: mechanical design, control strategies, perception, and human–robot interaction. By considering safety holistically across these domains, robotic systems can safely assist with care tasks involving deformable objects.

• Mechanical design plays a key role in mitigating safety risks. Soft robotic systems built with compliant materials can conform to objects and distribute forces more evenly during manipulation. This reduces risks of damage or injury compared to traditional rigid robots [132]. Compliant joints and actuators also help absorb impacts from collisions.
• Control strategies must adapt in real time to deformable objects’ changing shapes and material properties. Model-based methods like impedance control allow us to adjust robotic stiffness and damping [133]. Incorporating adaptive, safety-aware algorithms enables them to respond appropriately to variations in the environment and task [73]. This helps maintain stability and prevent unsafe interactions.
• Accurate sensing and perception provide the feedback needed for safe control. Estimating object poses, deformations, and material properties enable better manipulation by detecting possible risks [124]. Vision, tactile sensors, and sensor fusion give comprehensive data to guide actions [134].
• Effective human–robot communication establishes shared understanding between the robot and user, resolving uncertainties that may lead to unsafe conditions [135]. Intuitive interfaces, legible motion, and recognising human actions facilitate safe collaboration when assisting with care tasks [76].

By holistically addressing safety across these areas, robotic systems can reliably and responsibly manipulate deformable objects to assist with caregiving needs. The integration of compliant designs, adaptive control, robust perception, and human-aware interaction enables safe, effective assistance.

5.2. Social and Psychological Challenges

As the development and application of robotics in caregiving scenarios continue to grow, there is an increasing need to address the social and psychological challenges associated with integrating robots into the caregiving process. This section discusses several critical social and psychological issues, including user acceptance, human–robot interaction, privacy concerns, and the potential impact on the caregiver workforce [136].

User acceptance is crucial for successfully implementing robotic systems in caregiving environments [137]. Several factors can influence user acceptance, such as perceived usefulness, ease of use, and the degree to which the robot’s appearance and behaviour conform to user expectations and cultural norms. Studies have shown that the elderly and people with disabilities generally have positive attitudes toward assistive robots [138]. Still, concerns about the robot’s reliability, safety, and potential impact on personal autonomy and human contact may hinder acceptance [139]. Research indicates that considerations like perceived usefulness and ease of use influence user acceptance of caregiving robots [36], which should be priorities in design. To address these concerns, researchers and designers should actively involve end-users in the development process, conducting user studies and evaluations to ensure that robotic systems meet their needs and preferences.

Human–robot interaction (HRI) is a critical aspect of robotic caregiving systems, as effective communication and collaboration between humans and robots are necessary to complete caregiving tasks successfully. HRI research in the context of caregiving has focused on developing natural and intuitive interfaces, such as speech recognition, gesture recognition, and touch-based interaction, to facilitate seamless communication between humans and robots. Moreover, research on social robotics has explored the development of robots with socially appropriate behaviours, emotional intelligence, and the ability to recognise and respond to human emotions and social cues [140]. These advances can help build trust, rapport, and a sense of companionship between humans and robots, which is essential in caregiving scenarios where emotional support and empathy are crucial [141].

Privacy is a significant concern in deploying robotic systems in caregiving environments, as the collection and processing of sensitive personal data, such as health information and daily routines, could be misused or compromised [142]. Researchers have proposed various strategies to address these privacy concerns, including using privacy-preserving algorithms, secure data storage and transmission protocols, and transparent data management policies that respect user consent and preferences [143]. It is essential to balance the need for data to enable practical robotic assistance and the protection of user privacy.

Caregiver workforce. As robotic systems become more capable and autonomous, there are concerns that human caregivers may need to be replaced or marginalised, leading to job loss, deskilling, and reduced quality of care.

Some studies have also found that using robots in nursing environments can reduce the physical and mental needs of caregivers, but in reality, they can increase the workload of caregivers [2]. However, by taking over repetitive, physically demanding, or time-consuming tasks, robots can free human caregivers to focus on more complex, emotionally engaging, and personalised aspects of care. To ensure that integrating robots into caregiving environments positively impacts caregivers and care recipients, it is crucial to develop collaborative approaches that leverage the complementary strengths of humans and robots.

6. Summary

This paper provides a comprehensive review of deformable object manipulation technologies for robotic caregiving applications. The growing demand for assistive robots highlights the needs of advancing robotic capabilities in handling real-world deformable objects.

Common materials like textiles, gels, and aggregates pose modeling, control, and sensing challenges due to their complex dynamics and properties. Representing behaviours, planning manipulations, and perceiving states accurately remain as difficult, open problems.

Covered innovations include analytical modeling, data-driven learning, sensor fusion, and end-to-end policy learning. Hybrid analytical-learned approaches are promising. Tactile, visual, and depth sensing enable key perception. Recent laboratory successes exhibit potential, but real-world deployment lags. Enhancing speed, safety, adaptability, and human compatibility is vital for adoption. Cost, reliability, and workflow integration also remain open challenges.

This review has some limitations. It focuses on technical approaches without deeply analyzing ethical, legal, and social implications. It is also limited to published academic literature, excluding proprietary innovations. Additionally, it takes an engineering perspective without emphasising caregiver and healthcare professional perspectives.

Furthermore, Section 3.3.2 provides a qualitative multi-factor analysis for prioritising research directions. However, it lacks specific quantitative data and only provides analytical dimensions and theoretical methods. Future work should gather quantitative data via surveys or field visits to conduct more rigorous statistical analysis and draw informed, data-driven conclusions.

Further research should evaluate technologies with patients and caregivers for safe, ethical, human-compatible solutions. Collaboration between engineers, computer scientists, and healthcare researchers is critical for understanding the needs from all stakeholder viewpoints. Studies into practical implementation factors like cost, reliability, and integration are also needed to drive real-world adoption and impact.

In conclusion, this review synthesises critical work and indicates areas needing further multi-disciplinary innovation. Analytical methods, data-driven learning, and human factors considerations should be combined moving forward. The aim is to guide the advancement of real-world capabilities for improved robotic caregiving.