Reminder Objects in the Connected Home of the Future and Beyond ^†

Abstract

This paper presents best practices on how to design reminder objects. Reminder objects are digitally augmented everyday objects that break down and communicate complex information via sensory input and output. In the first section of this paper, we introduce reminder objects—what they are and how they work. We then describe our understanding of computing and the interaction between humans and technology, in detail, in the second section. We then present our vision on how reminder objects can enrich the user’s experience by combining different objects. We then use the three following sections to showcase corresponding examples of our work in the previously described field. Next, three scenarios—people with dementia living an independent life at home, checking the weather, and navigating the city—will illustrate our understanding of reminder objects. The paper then concludes with a discussion of the presented work and an outlook regarding the future of reminder objects.

1. Introduction

Reminder objects are connected everyday objects, which turn abstract digital information into intuitive sensory output. They help users to make decisions in certain situations—for example, “Do I need an umbrella—will it rain today?” Information for these situations can be found online or through mobile apps, but in both cases the user has to actively search in order to find it.

Reminder objects aim to enrich a person’s life by communicating information in a subtle, unobtrusive, and sensual way that is appropriate for the situation at hand. By breaking complex data down to the core information, and communicating this information with an appropriate sensory stimulus, people can make everyday decisions much faster and with a lower cognitive load. This communication of digestible information gives people “hidden superpowers”.

Reminder objects target different senses to include people with limitations—from children who cannot read yet to elderly people with physical limitations. By focusing on sensory I/O, playful implementations, and universal design [1], solutions can be created which function intuitively regardless of age.

2. Reminder Objects in Everyday Life

Highly diverse devices in the home and in people’s everyday lives will be connected to the Internet in the near future. This interconnectivity raises questions in areas ranging from ethics to privacy. In this context, we focus on the user experience of people interacting with technology because we believe that the key to acceptance and actual utility lies in the experience that users have with products or services. Therefore, key aims of our research are

• to reduce the cognitive load and sensory overload with which today’s computers, tablets, and smartphones present us,
• to make information intuitively and universally accessible in a given environment,
• to augment everyday objects with digital information and to express that information through sensory stimuli, and
• to design the objects’ behavior, interaction, and communication with respect to users and other objects around them.

2.1. Objects as Extensions of a Person

People encounter around 20,000–30,000 objects in their everyday life [2]—from the alarm clock in the morning, to the office chair they sit on during the day, and to the keys with which they open the door to their house. On top of this, people regularly come into contact with all kinds of technologies that affect their lives each day—from the electricity that powers their lives to the Internet, which makes it possible to access and share knowledge. All of these technologies are extensions of human capabilities according to McLuhan—both physically and psychologically [3].

With reminder objects, we examine how different extensions—the Internet as an extension of the brain and objects as an extension of physical capabilities—can be combined. In various nationally or EU-funded research projects [4,5,6,7,8,9], we work to provide universal design solutions by transforming insights about people with specific needs (e.g., elderly people) into solutions that work for everyone.

In these projects, we explore how everyday objects can be augmented with digital information in a way that enriches the user’s life. This information should be easily accessible for diverse target groups and should not compromise the usual interaction with the object—therefore not impairing the object’s original use. The following examples serve as best practices.

2.2. Processing Sensory Information

Humans process their surroundings and everything that happens with their senses and then react to the input that those different senses provide. Complex data, e.g., text on a screen, need to be absorbed, processed, and analyzed. This process requires people to be able to see, read, and be able to understand the language in which the text is written. If the aforementioned data is encoded in sensory information that addresses the corresponding senses, people can process it faster and take action more quickly.

The idea behind the following projects is to match the input and output of different technologies to human perception. Our goal was to select one or more senses and make information graspable in a sensorial way, because we believe that this will lead to a higher rate of acceptance of technology by a large variety of target groups. Currently, target groups, such as young children and people who are visually impaired or elderly, might be reluctant or unable to use common screen-based interfaces like computers or smartphones. By turning the underlying data into a form that these target groups can understand, in a sensual and intuitive way, we can improve their lives.

2.3. Technology Serving the People

From the weather report to your account balance at the end of the month—this information is available online. However, looking for information online can be very uncomfortable or even stressful. The trend to replace natural interaction with a screen-based interface (e.g., opening a car or a door with a mobile app) usually makes this task harder for the user. Instead of humans catering to the needs of computer, technology should be designed to serve humans by supporting their everyday needs without requiring (too much) input and attention from the user [10].

In 1991, Weiser already set the goal for how technology should serve human needs: “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it” [11]. By building upon this understanding of technology and computation, we enrich existing everyday objects with digital information to create new hybrid objects—reminder objects.

3. The Ecosystem of Reminder Objects

Reminder objects can be used in every aspect of life: They can be part of the home, integrated into the environment, or even embedded into the things people carry around with them, such as clothing. If interconnected, they can form a network of objects that can support every aspect of people’s daily lives. The goal is not to cram as much functionality into one central object, but rather to assign the right functionality to corresponding objects. Having this goal in mind improves usability and avoids the creation of the “Swiss army knife problem”, where the replacement of a tool with a certain intended function, crammed into a single object with many other functions, always works more poorly than a tool optimized specifically for that function alone [12].

The following examples show best practices of how reminder objects work. These objects are very easy to use and work very well for the respective situations that they are created for, because they were designed to provide one core functionality. In the spirit of the Unix philosophy [13], they can extend each other’s functionality when combined and be more than the sum of their individual parts.

4. MeMoTray: Helping People with Dementia Stay Independent

Most people have a spot in their homes where they leave important things, such as keys, purses, or reminder notes for organizing their daily lives. The “MeMoTray” connects these analog everyday objects with the Internet and assigns them to tasks and activities in the digital world (Figure 1). We developed this prototype as part of the EU research project CCE (Connected Care for Elderly People Suffering from Dementia) where we focused on the needs of elderly people, especially those in the early stages of dementia.

Equipped with RFID and ambient sensors, the MeMoTray identifies and analyzes objects and environmental data, such as the keys that will be needed to lock a specific door. By analyzing the environment, the internal system of the MeMoTray can decide when reminders are necessary. If a person opens a door while the keys to that door are still in the MeMoTray, the system makes a warning sound.

Furthermore, the MeMoTray has access to the user’s calendar and is able to support him or her in planning daily activities. Visual, acoustic, and haptic output ensures that the MeMoTray’s reminders will be noticed. Connecting the MeMoTray to other smart household devices expands its capacity to provide feedback in every location where it might be needed to do so.

The MeMoTray is ideally suited for people suffering from dementia. By following their behavior, and knowing what to remind them of, the MeMoTray allows people with dementia to regain some normality, be independent, and be treated at home longer—instead of at special care facilities.

4.1. Methodology

During the development process, we used various research methods to understand how people with early stages of dementia live. We conducted focus groups and interviews with experts, care givers, and family members. Two separate interviews were then conducted, the first of which was with a representative of an association for people with dementia, and the second with an expert on interior design for accessibility concerning the challenges for people with dementia. This was followed by a group interview with experts and other stakeholders concerning problem areas, privacy aspects, and existing AAL technologies. The participants of this interview were a manager of a residential care home, a gerontology professor, a manager of a nursing and social service, a representative of a health insurance, and a member of a support group for senior citizens. Additionally, two focus groups were conducted with nine people who each were caring for a relative with early or mid-stage dementia. Two more groups, consisting of a total of 12 caregivers responsible for people with dementia in their homes, were then interviewed [14].

In a shadowing study [15], we talked to people with dementia in their familiar environment and observed their daily activities. Four people with dementia were accompanied individually, for 4.5 h each, in their daily lives. This study was conducted first and foremost at the homes of these people. Additionally, we conducted interviews with three people with early or mid-stage dementia.

The insights from this research helped us to identify different problem areas. Additionally, in a scenario-based design [16] process, we developed various scenarios that visualized the current problems as well as design ideas and solutions. These scenarios aided us in realizing the MeMoTray.

In the framework of this project, ELSI guidelines were followed during all recruiting, research and testing stages (Horizon 2020 Ethical principals http://ec.europa.eu/research/participants/data/ref/h2020/legal_basis/fp/h2020-eu-establact_en.pdf). All subjects or their legal guardians gave their informed consent for inclusion before they participated in the study. Sorokin & Limbach developed guidelines on how to conduct research with people with dementia as part of the project [14]. The project partners Fraunhofer IGD and Fraunhofer IESE were responsible for ethical, political, social, and legal aspects of the project. They conducted research on privacy and handling of personal information—transparency, control, and trust—which was documented and shared with the other project partners.

4.2. Results

The previously mentioned interviews with experts helped identify target groups and frame the scope of problem areas: The target groups were limited to people with early- to mid-stage dementia because the goal of the project was to help people stay at home and have more independence. Therefore, people with severe dementia were excluded from the scope.

The research phase then showed that people with dementia have specific needs and fears that are sometimes hard for other people to understand. These people with dementia complained about feeling disoriented, having low self-esteem, and not being in control of their lives [17].

The conversations which followed, with people with dementia and their care givers, also highlighted that a purely technical solution would not solve the problems. On the contrary, this research indicated that anything technological should fade into the background as much as possible because the participants were not willing or able to use technology.

Visiting people with dementia in their homes then revealed a key insight into their lives: There was a central location in every home where people with dementia, family members, or care givers would keep important things (e.g., medication) or leave messages and to do lists (Figure 2).

These locations were not created artificially but existed prior to the diagnosis of dementia. After the diagnosis, these locations became even more important for safekeeping and finding important objects. This insight helped us design the MeMoTray; it was built to be used where important objects are generally collected in certain locations, and its bowl is shaped so as to invite people to place objects therein. Its aesthetic design is meant to evoke positive emotions and prevent people with dementia from being confronted with their deficits.

We tested the MeMoTray in a usability test with various people with dementia and the participants rated the MeMoTray’s usefulness and aesthetics very positively.

5. Rainminder: The Weather Forecast Reimagined

With the “Rainminder”, we explored how digital information could be transformed into sensory attributes and how this can be communicated effectively for fast decision-making.

The doormat is an Internet of Things installation for the front entrance of a home. If the residents step on the Rainminder on a rainy day, every step sounds like stepping into a puddle. This reminds them to take an umbrella with them on their way out. On a sunny day, however, the mat remains silent.

5.1. Methodology

The insights that led to the Rainminder’s design came out of the previous project (MeMoTray). We took the insights that we gathered, in the specific context of people with dementia living at home, and abstracted new universal design solutions that would work for everyone. By focusing on sensory input and output and playful interaction, the Rainminder is a universal design solution that works for people regardless of their age (e.g., for school kids, adults, and seniors).

Through explorative prototyping, we refined the Rainminder’s concept and people’s interaction with it. The first prototype was based on wooden panels equipped with pressure sensors, that transmit changes in resistance, to a connected Arduino Yún Board. During the next iterations, the bulky wooden panels were replaced with Velostat-based pressure sensors. The advantages of these self-made sensors are the material’s flexibility and the thinness (Figure 3).

5.2. Results

As soon as somebody stepped on the mat, the resistance of the sensors changed and a signal was sent to the connected Arduino board. The board interpreted the values and concluded that the mat was being stepped on. Aware of its location, the mat then called up a real-time weather forecast via the API of openweathermap.org.

When the weather forecast reported rain, the connected loudspeakers played randomized water splashing sounds. To the user, this conveyed the feeling of walking through puddles on a rainy day (Figure 4).

We also considered combining two senses for sensory output: hearing and smell. In addition to the splash sounds that the Rainminder makes when it rains, the user would also be able to smell the scent of rain. This can be made possible with a special organic compound (geosmin), which contributes to the smell of rain (petrichor) [18]. In this way, the information (that it is going to rain today) will be perceived even faster—almost subconsciously. Spraying the compound in the air is not a problem but removing it again from the air remains problematic.

6. inDAgo: Keeping Seniors Active

In the project “inDAgo” [4], we looked at ways to improve the mobility of elderly people. Elderly people usually experience physical and mental limitations which come with age and reduce their mobility. This leads to social isolation and a fear of going outside. These people are afraid of accidents happening while outside, or of being faced with finding their way using public transportation without appearing vulnerable [19].

6.1. Methodology

Initial user research with 6 experts and 28 elderly people in 4 focus groups revealed that elderly people were not open towards technical gadgets like smartphones and did not want technology to interfere with their sense of independence [19].

Therefore, an ecosystem of different prototypes that connect to each other, and enrich one another, was developed: a smart armband, a smart pouch, and a smart jacket [20].

The participants for these focus groups were selected through the nursing home that was part of the project. In the recruiting process of selecting the 28 elderly people, it was important that the people chosen reflected a wide range of impairments: The participants included people with severe visual impairment, all kinds of motor impairments (people walking with canes or walkers, people in wheelchairs), hearing loss, and general mental decline that comes with age. The age of participants varied between 60 and 94 years of age, and there was no emphasis on gender for the recruiting. In the framework of the project, ELSI guidelines were followed during recruiting, research and testing. All subjects or their legal guardians gave their informed consent for inclusion before they participated in the study. An ethics workshop was conducted by Prof. Dr. theol. habil. Arne Manzeschke (professor for anthropology and ethics for health professions). In cooperation with the associated partner Seniorenrat Darmstadt e.V. (an association for senior citizen in the city of Darmstadt), interdisciplinary workshops about ethical aspects in research were conducted together psychologists, social scientists, and legal experts.

6.2. Results

Navigating unfamiliar environments become much easier and much more inconspicuous for elderly people by using a smart jacket (Figure 5). Such elderly people would have no need to hold a map and signal to the environment that they were unfamiliar with their surroundings, or to fiddle with a smartphone which would be hard for them to operate. Through soft vibrations on either shoulder, the smart jacket is also capable of indicating to the wearer where to go.

The smart jacket prototype, that was developed, has an NFC reader and vibration motors hidden inside. When an NFC tag for a certain destination is placed in the jacket’s pocket, it guides the wearer gently through the city by vibrating at the locations where the wearer should take turns. The jacket comes with a collection of various NFC tags that are sewn into cotton badges depicting certain destinations like the closest pharmacy, doctor or other points of interest in the city.

This invisible and hands-free navigation system is unobtrusive and easy to use—as much for seniors as for any other target group, e.g., children.

The functionality of the jacket can be enhanced with the other prototypes developed in the project: The smart armband (Figure 6) extends the navigation functionality of the jacket with more details and combines different stimuli: touch and sight.

Through the extension of the jacket’s functionality by connecting the smart armband prototype, a wearer’s navigation is more precise while still being inconspicuous to other people. The armband also shows additional information that the jacket itself cannot communicate on its own.

After developing these three prototypes, we conducted qualitative research by contextual observations in a real-life scenario with eight elderly people: The task was to navigate the city to points of interest by foot and trolley. The prototypes generated positive feedback and received good values concerning their acceptance. Integrating functionality into existing objects as shown was important to the participants, as they expressed that they were reluctant to carry additional devices with them [20].

7. Discussion

Through an iterative process and a focus on tangible results, we iteratively developed prototypes that people can experience firsthand. These people can put the MeMoTray in their home and hear the warning sound when they forget important objects when leaving the house. They can walk on the Rainminder and hear the “water” splashing if it is going to rain, or wear the inDAgo smart jacket and feel the vibration on the shoulder where they should take a turn. This approach enabled us to test the prototypes in real world scenarios.

The process of agile prototyping and evaluations with the end-users was the key to successfully implementing tangible solutions.

Based on these evaluations, we derived the following conclusions:

• The first challenge for reminder objects is to identify suitable objects that can support the users in their everyday tasks in a meaningful way.
• The second challenge is to translate the underlying data into useful sensory information that can be interpreted easily.
• Another issue is to create solutions which fade into the background, so that the user does not serve the object but the other way around: By using data that is available online or through input from integrated sensors, the reminder object should require as little user input as possible.
• The interaction with the object should not change fundamentally: Even a digitally augmented doormat is still an object that people expect to be able to walk on and not having to treat it like an electronic device (e.g., a computer) that needs to be handled with care.

The interplay of connected objects among each other with people still raises many questions—questions that Gold prompted in 1995 and are still relevant today [21]: Can you expect a computer-controlled house or object to still be functioning in 10 years? Does it evolve over time? What kind of relationship does it have with its users? Who controls the settings in an environment with diverse people and objects? Such questions concerning the social structure between people and objects are an exciting topic that demands further research.

8. Conclusions

The projects “MeMoTray”, “Rainminder”, and “inDAgo” are three best practice examples which show the interaction with digitally augmented everyday objects:

• The MeMoTray keeps important objects in one place and connects them to digital functionality without any user intervention or changes to the objects themselves. The keys become smart and will not be left at home.
• People of all ages can understand intuitively that it is going to rain by just walking on the Rainminder door mat and hearing the sound that each of their steps makes if it is raining outside.
• With the inDAgo wearables, elderly people can navigate unfamiliar environments with ease without attracting attention. This is achieved via tactile vibration in the smart jacket, which makes it easy to know when and where to turn. Choosing a destination is as easy as putting anything in your pocket.

The results of the project evaluations presented in this paper show that breaking down complex digital data into sensory stimuli is an appropriate way to communicate information: In the three scenarios—living at home independently, checking the weather forecast, and navigating the city—the information was easy to process, and decision-making was simplified in comparison to a screen-based user interface.