1. Introduction
Over 5.5 million children in the United States have a cognitive or physical disability that results in at least some difficulty with activities, including play [
1,
2,
3]. Example developmental disabilities include cerebral palsy (CP), autism spectrum disorder (ASD), muscular dystrophy (MD), and Down’s syndrome, etc. [
4], and are often accompanied by physical impairments such as spasticity, muscle contracture, bone deformity, muscle weakness, and coordination disorders. These conditions cause deficits in day-to-day activities, such as grabbing or holding objects [
5,
6]. In turn, these children may lose their ability to play with their family, friends, or primary caregivers, and are less able to express themselves and make meaningful connections with others [
1,
7].
According to The American Academy of Pediatrics, play is essential to development because it contributes to the cognitive, physical, social and emotional wellbeing of children and youth [
8]. Play has also been recognized by the United Nations High Commission for Human Rights as a right of every child [
9]. Research in [
10,
11,
12,
13,
14,
15,
16] indicates how cognitive abilities such as language, memory, self-regulation, and the ability to plan, focus and execute tasks can be improved with gaming. To date, several digital games have been developed for children with CP, ASD and other disabilities, but they are either single player or rely on a virtual assistant. For example, ‘A Sunny Day: Ann and Ron’s World’ uses an iPad game application [
17], while ‘TeachTown: Basics’ adopts computer-assisted instructions and gamifies traditional treatment exercises into rewards to motivate learning [
18]. Unfortunately, the concept of single-player games undermines the idea of collaborative gameplay. By contrast, projects like ‘Invasion of the Wrong Planet’ [
19], ‘Collaborative Puzzle Game’ [
20], and ‘SIDES’ [
21] encourage collaboration (e.g., to defend a planet or solve a jigsaw puzzle). Nevertheless, they are only suitable for children with mild disabilities. For moderate developmental delays, commercial gaming platforms (X-box [
22], Nintendo [
23], etc.) are taking initiatives to adapt their consoles. However, these gaming consoles are usually challenging. Finally, for children with severe disabilities, communities like The AbleGamers [
24], Special Effect [
25] and Warfighter Engaged [
26] take major steps, as does eye-gaze-based interaction integrated with Digital Games-Based Learning [
27]. Nevertheless, games designed for children with severe disabilities tend to target learning instead of bonding with a parent or caregiver, which is an aspect that is critical to the child’s development [
28].
In order to address shortcomings in the state-of-the-art, we propose collaborative games that rely on touch between fully-abled and disabled players, i.e., physical touch between the players is sent as an input for the gameplay. Wearable sensors capable of detecting touch play a key role in this regard. Here, we focus on FSRs as a reliable, cost-effective, and flexible solution [
29]. We also focus on embroidered FSRs, as embroidered surfaces are known to be mechanically robust, tolerant to repetitive deformations, and washable. A key requirement for this FSR design is to be highly sensitive to low forces applied upon the sensor by individuals with disabilities; a performance metric that previously reported FSRs failed to meet. More specifically, Ref. [
29] describes a significant decrease in gross and fine finger dexterity in children with ASD, Ref. [
30] reports lower peak grasp forces for children with ASD compared to typically developing children, and Ref. [
31] shows that children with ASD and MD have impaired lower-hand symmetry. With the touch force applied by healthy individuals ranging from 1.27 N to 3.22 N [
32], the forces applied by children with disabilities are expected to be even lower. Referring to
Table 1, most of the previously reported embroidered FSRs focus on wide dynamic ranges of employed forces (e.g., up to 20 N [
33], 30 N [
34], or 56.7 N [
35]) and exhibit poor sensitivity in the detection of small forces. The work in [
36] studies a narrow dynamic range of 0–5 N, but the resulting sensitivity is poor, which was attributed to the high resistivity of the employed threads. Relatively newer techniques other than FSR sensors highlighted in [
37,
38] use hydrogel elastomer ionic sensors for hand motion monitoring. However, the fabrication is sophisticated and expensive compared to FSR sensors. Likewise, there are potential problems with hydrogel dehydration and limited temperatures of operation. The previous solutions are, thus, unsuitable for the application under consideration.
In this paper, we report new classes of embroidered FSR sensors that are optimized for collaborative gameplay between individuals of various physical abilities, and which outperform the state-of-the-art in terms of sensitivity to low forces (<5 N). In order to optimize the experience of gameplay, we study the ergonomics of the proposed sensors by exploring various fabrics and various locations upon the human body.
Section 2 presents the system architecture and provides details on the employed materials and methods.
Section 3 reports our results, while
Section 4 discusses our findings. The paper concludes in
Section 5.
4. Discussion
The proposed wearable sensors functionalized with FSRs provide a promising solution to enable collaborative digital gaming and other touch-based solutions. The results indicate that textile-based FSRs can replace off-the-shelf FSRs in this regard to make the sensor more seamless and durable. In particular, embroidered FSRs provide a high level of control over the resulting conductivity and mechanical performance, enabling optimized sensors with minimal false positives in the detection of ‘touch’ vs. ‘no touch’. Regardless of the FSR selection, a need was demonstrated to separate the sensing element from the human skin. Fabrics were explored to this end that considered thickness and stretchability factors to optimize performance. It was found that thick and stretchable fabrics work the best. It can be noted that the studies reported in
Table 2 and
Table 3 were performed using off-the-shelf FSRs, which account for the worst-case scenario. In-house FSRs (woven fabric/embroidered) can be designed to be conformal to the arm, i.e., the forearm, the palmar/dorsal side and so on, as per the requirements of the user, thus reducing the errors seen in off the shelf FSRs. Fabric 4 was chosen in the same way because of the skin isolation it provided. As a result, the performance of any type of FSR on a similar fabric type would be identical.
The resulting armband can then be designed in a form factor that fits the application needs under consideration. For example, we selected a small width of 2 cm for the armband of
Figure 2a to minimize the fabric coverage upon the arm and enhance skin-to-skin contact between the players. Similarly, the sensor placement upon the human body may vary per the application needs, though the performance was shown to improve upon flat/uniform areas. Various types were also explored, and the robustness of the idea was confirmed in all cases. Overall, multiple possibilities can be explored should the designer have a specific game application and target demographic in hand.
On a system level, ‘touch’ and ‘no touch’ inputs can be registered on an ESP32 microcontroller and transmitted wirelessly via Bluetooth to a remote mobile device (e.g., a tablet). ESP32 in the deep-sleep mode and BLE mode of operation are ideal for reducing power consumption and increasing the time of play. Though we have experimented with a power bank, Li-Ion or Li-Po batteries are also suitable. These batteries are rechargeable, and range from 150 mAh to 2500 mAh.
In the future, we can explore more seamless sensor designs by implementing shunt-based FSRs on embroidered e-threads, and by designing in-house electronics. Textile-based piezo-resistive materials can also be explored to replace the Velostat, and to enable fully-textile substitutes for the FSR sensors. The mechanical/thermal performance and launderability will also be explored for the sensors. Finally, we have the option to use multiple FSR sensors to further expand the sensing area and potentially improve the resulting sensitivity.
5. Conclusions
New classes of wearable sensors functionalized with FSRs were reported for touch-based collaborative gaming. An off-the-shelf FSR was originally selected and connected to an ESP32 microcontroller to ultimately transfer data to a remote tablet in a wireless manner. The placement of the sensor directly on the skin was found to compromise performance, and necessitated fabrics to be placed in between. Four different types of fabrics were tested in this regard, indicating that thick and stretchable options were the most suitable. Multiple on-body locations were analyzed for the sensor, and placement on the palm side of the hand was identified as optimal, followed by placement on the forearm. Depending on the application and target audience, different placements can be considered. Different touch scenarios were also explored to consider players with physical disabilities. Finally, textile-based FSRs were explored, including an embroidered version that was shown to considerably outperform the rest in terms of sensitivity to low applied forces. The stitching density was altered during the embroidery process to identify an optimal value for effective gameplay.
As an example application, this work intends to enhance the bonding between children with disabilities and their parents without disabilities. In the future, we plan to use the FSR sensors in a real-world game environment, connected to an iPad or other Bluetooth-enabled device. The study aims to evaluate the performance of the fabricated FSR sensors compared to commercial sensing systems. However, numerous other applications may be considered for diverse age groups and/or medical conditions. Our sensors can also be expanded to textile-based force-sensing alternatives, such as pressure sensing mats for bed-bound patients, or pressure sensitive socks for sprinters and marathon runners.