1. Introduction
Technologies shape the world in which we live. How to conceptualize the role they play in our world is an interesting question from a philosophy of technology perspective. Technologies, especially those based on computing, have changed our daily life intensively in the last decades. Ubiquitous computing is a computer science research field which especially addresses the relations between the user and the computing system in everyday life (e.g., domestic, work, leisure and social networks).
Wearable computing (e.g., smartphones or smart-watches) is one of these technological developments, which has changed everyday behavior in all of the mentioned settings from domestic to social networks. In this paper, we focus on wearable devices for activity recognition; it is part of the current computer science research and is about devices designed for detecting human actions with the purpose of monitoring these actions or improving the human-computer interaction. Human actions are thus seen as a measurable entity. We assume that activity recognition applications have the potential to influence or shape the everyday world. Especially interesting are applications for using wearable activity recognition as a tool for self-tracking or self-monitoring. In this work, we focus on this application field.
Using wearable activity recognition devices for self-tracking is already common, for example in devices for monitoring sport activities like jogging or fitness exercises [
1]. Commercial products—for example, the Nike Fuelband, Fitbit or Jawbone UP or apps on smartphones as well as smartwatches— are quite common in tracking activities, especially sport activities. People have a growing interest in tracking or logging their everyday experiences [
2], which can be seen for example in the lifestyle-oriented community of quantified self [
3]. A scientific investigation of self-tracking and life-logging with ubiquitous computing devices can be originally found in the work of Li
et al., who used the term “personal informatics” for it [
4]. There are also projects connected to the field of health behavior change and education. For example, self-tracking in order to change unhealthy behavior (smoking, eating the wrong food or not drinking enough) or to increase healthy activities (doing more sports). The so called persuasive computing field is a research area which is interested in general concepts for applying technical devices to motivate or persuade the user to change his behavior [
5]. Most of the applications we mentioned here are specialized and are only of interest to a smaller group of people. However, it can assumed that the “next step” to more serious and complex applications in mainstream settings will come soon [
6].
Regarding our interest in the relation between persons and technical systems, wearable devices are especially interesting because of the close relationship they provide. Close means here both that the device has the potential to be near the body and that it is designed for long-term usage with the goal of 24/7 assistance. These factors are also described as everywhere and anytime. Technically, this is possible because the sensors are small (can therefore be integrated everywhere) and can be worn a long time without the need of recharging them. This means they can be used day and night without noticing them as explicitly present. According to that, the activity can always be monitored in the background and can either be used as input for the interaction with the wearable device or for a retrospective evaluation of the user’s behavior. We think the phrase "in interaction" describes this close relation to the wearable device in which the user should not "step back" but rather fully integrate the device in the daily behavior.
To summarize, the idea that technology is shaping the world in which we live, will be analyzed in this paper focusing on the technology of wearable activity recognition. In the applications of self-tracking, a new perspective on the user’s actions is provided, which can be seen as an extended experience of the self. This new perspective is mediated by the technology, the interpretation algorithms and the way in which the detected activity data is visualized to the user.
In the interdisciplinary analysis, we follow stems on the one side from an ongoing research practice where we use results from two case studies and on the other side on philosophical concepts based on general thought about the mediating character of technology. We start with a scheme of a closed loop (as a first systematization), which illustrates the mediating character of wearable activity recognition in the way persons reflect on their actions.
2. In Interaction with Wearable Activity Recognition: Two Cases
In this section, we provide a first systematization on the mediating character of the technology of wearable activity recognition. We discuss the mediating role especially concerning the influences this technology has on the users’ decisions to act. The systematization is realized by a feedback loop scheme which is shown in
Figure 1 (a similar feedback loop was introduced in [
7]). Thereby, we give a deeper insight into the design of devices by introducing two cases which are both projects we are now or previously were related to. The cases should also be used to explain the feedback loop, which is shown in
Figure 1.
Figure 1.
Automated feedback of an activity recognition system as basis for reflection
Figure 1.
Automated feedback of an activity recognition system as basis for reflection
Case A (in
Table 1) is about using wearable activity recognition to detect smoking behavior with the further goal of increasing the awareness of the person on his smoking habits. A motions sensor (accelerometer) is used which can be worn on the wrist like a watch to detect the typical movement of the arm while smoking [
8]. The sensor makes a long term usage possible (low energy consumption) and so enables monitoring up to two weeks. The goal of such an automatic detection of smoking habits, is to combine it with information on the spend time and the amount of smoked cigarettes and visualize this information for the user. This visualized information can then be used by persons, whose smoking habits are tracked, to find out more about this behavior, potentially with the result of becoming more aware of it or finding out triggers and in the end change their behavior.
Table 1.
The two cases presented based on the structure in
Figure 1.
Table 1.
The two cases presented based on the structure in Figure 1.
| Case A | Case B |
---|
Recording Data | A wrist-worn device for motion tracking which can measure acceleration in three dimensions (raw sensor data). | A wrist-worn device for motion tracking which can measure acceleration in three dimensions (raw sensor data) [9]. |
Automated Classification | Specific features in the raw sensor data are seen as indicators for the activity smoking. Characteristic for smoking is the frequent movement of the arm following a certain curved line. | Specific features in the raw sensor data are seen as indicators for working steps of an experiment in the laboratory. For example pipetting is characterized by a specific position of the wrist. |
Feedback | Feedback for the user is given by a visualization-tool which shows a summery of the time, total amount of smoked cigarettes and the money spent for smoking. The information can be used to raise knowledge about the smoking behavior and reflect on it. | A protocol of the experiment could be exported which gives insights in the sequence of execution and durations of certain working steps. Then the results can be evaluated by comparing different executions and outputs of the experiment. |
Case B (in
Table 1) is about a recognition system supporting scientists who work in a biological laboratory. In this project, possibilities for detecting single working steps in different experimental settings in a biological laboratory and possibilities to visualize the data for the scientists are explored. Thereby, the same wrist-worn sensor was used to detect, for example, the activities “using a pipette”, “stirring” or “pouring”. The structure of the system and the ways of interpretation are equal to the smoking detection case. This automatic detection results in a protocol of the experiment, which makes it possible to compare different executions of the experiment and makes it easier to reflect on possible errors that were made during the experiment.
3. Method: The Interdisciplinary Approach
Our approach is interdisciplinary: on the one hand, it belongs to the research of computer science, more specific to the field of human-computer interaction (HCI) and ubiquitous computing, and, on the other hand, to philosophy of technology in its direction of understanding and conceptualizing the relations between technology and human beings. Ubiquitous computing focuses on a user-centric design and technical systems supporting everyday life. Thereby, it is the challenge of a good computer system design (interaction design) to include an understanding of social and individual needs for designing an efficient and functional system but also to design technologies which serve the user in a good, valuable way. With respect to the philosophical direction, it is one interest to understand how technology in general and, specifically, new technologies are shaping or mediating a person’s perception of the world and the self.
Such an interdisciplinary approach is appropriate and necessary when thinking about computing technologies in everyday environments in a scientific way. The ubiquitous computing research has been aware of these requirements since its beginning—for example, Weiser in his vision was also referring to that [
10]. That philosophical concepts can enrich the computer science research is common in this field. Phenomenological thoughts are especially discussed and widely known in the community [
11,
12,
13,
14]. However, this interdisciplinary direction only makes up a small part in ubiquitous computing, compared to the amount of classical work driven by feasibility aspects.
On the other side, the philosophical research which addresses technology, compared to the long philosophical tradition, is quite new. The philosophy of technology research includes theoretical approaches based on the question what technology is (essence) and the capacity of artificial intelligence (AI) but empirical approaches as well, focusing on concrete human-technology relations. Especially following the second line, philosophy of technology requires concrete cases and knowledge in the design practice to specify or even build the concepts.
Concluding, on the one side, concepts based on the philosophical theories can help to specify models and classifications in the design process. On the other side, it is interesting for philosophy of technology to include the current design practice and applications in its socio-technical concepts.
4. Background: The Mediating Character of Technology
We intend to show how the shaping or mediating character of wearable activity recognition can be conceptualized, especially the influences on the users’ actions.
In this section, we introduce two approaches, whereby the first belongs more to the philosophy of technology research but with a direction to the practices of design. The second approach is a quite popular set of concepts located in the HCI and Ubiquitous Computing which are discussed under the term embodied interaction. Starting with the philosophical background, we introduce Peter-Pauls Verbeek’s theory, who is providing a “systemic analysis of the relations between human beings and technological objects, wherein he discusses the connection to contemporary industrial design” [
15] (p. 3).
4.1. Verbeeks Thoughts on the Relation of Human Beings and Technology
In our approach, we are interested in the mediating character of technology especially technical systems of everyday use. Peter-Paul Verbeek is providing a theoretical concept on the mediating role of technology, which he discusses under the heading “philosophy of technical artifacts” [
15] (p. 9). Verbeek is thereby following the philosophical tradition of phenomenology but is enriching it with hermeneutic thoughts and concepts of the actor-network theory. It is important for him to understand the mediating role, not one-sided which he insinuates regarding the classical phenomenology theory (of technology). This theories are exclusively focusing on the limiting function of technology in the way technology is “alienating human beings from themselves and reality” [
15] (p. 4). Following Verbeek, in contrast, is to assume that there are both, possibilities and limits of technology use, and the task to find out these should be one topic of a philosophy of technology. He says: “Many forms of technological mediation are possible that transform our access to the world in myriad ways, some of which open up to us new ways of access unavailable to ‘naked-eye perception’, and some of which narrow this access” [
15] (p. 144).
He is therefore providing a richer framework which makes it possible to understand the mediating character in two ways. First, the user’s perception and interpretation of the real world can be mediated by the use of technology and second, the involvement with the technical device, in the way action and interaction is made possible, is mediated.
Mediated perception and interpretation describe how certain world-views are preferred through the use or involvement of a technical system. Following the involvement direction, it is about how technology affects in which way human existence is shaped that means “how it can make possible certain kinds of actions and inhibit others” [
15] (p. 191). Thus, perception and actions are mediated by technology in a way that it makes new perspectives and ways of behavior possible, not only concerning the concrete functions of the technical system, but also cultural and social circumstances. Technology is both changing the perception in scientific research, how we see the world in a macro-perspective, but also, and more interesting for our approach, how it changes the everyday perception (mirco-perspective) of people using technical systems. One famous example for a social impact is the round table as a technology where an authoritarian hierarchical order is reduced because it has, in contrary to the rectangular table, no head of the table where the most important person is sitting—or two other examples: the development of a raster-tunnel-microscope has changed the world-view in a physical understanding, the observing capacity of human beings is mediated in a way that new things are visible (in a new smaller sphere), but Facebook is also reshaping the social relationships of persons (because they are structured and visualized in a certain way). The goal of Verbeek’s investigation is not to show that there is a mediating effect, because it is clear for him there is always one, but he wants to show how this mediating role can be understood by including all possible levels of impact, from micro to macro and from individual to social or cultural.
Based on his theoretical thought, he suggests, a philosophy of technology approach should be done in a way that the possibilities and limits of technologies in all of its facets are addressed. Additionally, Verbeek focuses on the engineering design as a further direction, which he calls the practical value of his approach [
15] (p. 204). Thereby, he adapts some thoughts of a concept, called “material aesthetics”, which refers to a material oriented design approach. One aspect, Verbeek focuses on, is how for example objects in their material impression acquire meaning to the user. One research project he refers to shows that people find technical objects special for them, mostly because they have “memories that clung to them” [
15] (p. 223). Therefore, the material aesthetic of a device can be more relevant to the everyday use of technology than the pure function the technology was planned for. Verbeek says: “Technologies are not merely functional products that also have dimensions of style and meaning” [
15] (p. 235).
These examples of concrete design do not fit well to the technology and applications we like to address here, but Verbeek’s approach builds a good theoretical background and also contains the questions that should be addressed in philosophy of technology.
4.2. Embodied Interaction
From these thoughts, we go over to an interdisciplinary theory especially referring to HCI and Ubiquitous Computing. We focus on a concept of embodied interaction which was first introduced to the computer science research by Paul Dourish [
11].
He grounds his concept on thought of the phenomenological tradition in philosophy. Embodiment is extensively discussed in the philosophy, but we focus on the concepts which have already been applied to the field of HCI and ubiquitous computing. Linking phenomenological thoughts with computer science design by itself has a tradition in the well-known book of Winograd and Flores [
16]. In Dourish’s concept of embodied interaction, he focuses on the role of how we participate within the world. That means, how (inter)actions in the world are related to how we have a perception or create meaning of something and thus action and perception cannot be separated. “The world has meaning in how it is physically organized in relationship to our physical abilities, and in how it reflects a history of social practice” [
11] (p. 9). The body is embedded in our experiences within the world, and in terms of the environment(s) that it creates. The mediating character of technology is in a lot of aspects similar to the approach of Verbeek.
After this brief introduction, we will highlight the influences his thoughts had and still have in the current design practice (Interaction Design). One topic for Dourish applying his concepts is how the context-awareness of technical systems can be implemented. Context is a topic which is widely discussed in computer science concentrating on building everyday devices. It describes how situations (e.g., everyday settings) can be understood properly by an automatic interpretation system. Therefore, he argues that rationalistic approaches which try to hold on the world in an ontology of static contexts should be rejected and a dynamic construct of context should be preferred [
17]. With a dynamic construct he means that context is dependent on personal preferences, cultural and social meaning and can change over time. As a consequence for the interaction design, it can mean to include possibilities for the user to change labels of context in runtime. For example, a classic context type is location. Location can be understood in its GPS coordinates, its postal address or concerning the personal experience a person has with it (a cafe with the best espresso in town). Depending on the application, it can make sense due to different reasons to choose one of them. However, this classification can also change over time when the meaning changes (e.g., he has quit drinking espresso because of health reasons).
Although Dourish is naming his approach embodied interaction, he is not going very deep in the phenomenological thoughts on the understanding of the body. Svanaes captures this and refines the concept of the lived body for embodied interaction by especially referring to the philosopher Merleau-Ponty [
12]. Following him, the body has a dual nature: “on the one hand, we can see it as an object among other objects in the ‘external’ world. On the other hand, it exists to us as our experiencing/living body”. The lived body as a concept is arguing against the problematic classic perspective, separating the body as an object from the mind as sphere of subjectivity. Shusterman, who discuses the distinction in a deeper way, says: to the body belongs for example the bones, the inner organs or the neuronal structure of the brain. The lived body instead is the dynamic experience of the body, which is located in living feeling and perception [
18]. Following that, persons are meditated by the way the body is in the world, including it bodily perception (for example tactually) but also, for example, by the fact, that the body is vulnerable and mortal. For example, pain is neither a pure bodily phenomena or mere explainable with an interpretation by the cognitive system. Additionally, the way someone has experienced a situation both in a conscious or unconscious way with its emotions and feelings, influences the further perception. Related to that, we want to mention one direction in this lived body discussion that concerns the role of habits and routines in daily life. Shusterman focuses on the “unreflected spontaneity of acting and feeling” which manifests in habits and routines which are expressions of the lived body. In contrast to the measurement of pure bodily phenomena (e.g., heart rate, blood pressure), in the technology of wearable activity recognition the routines and habits become part of a reflection. This reflection on the “unreflected spontaneity” is an interesting aspect which should be a topic of a further discussion [
18].
4.3. Critical Conclusion of the Concepts
Both concepts provide a richer understanding of the relation between technology and persons with a direction to the role of artifact design. Richer means here, in simple terms, to take more aspects into account than the “obvious” functions of the technical system. How technology shapes the world was the term we used in the beginning. This understanding of technology is related to the research which can be found under the term of socio-technical systems. Classically, this research is focused on the cultural and social dimension of technology (e.g., the round table example) but not only in use, rather for example the social dimension of production (e.g., new conditions of mass-production) and disposal (e.g., radioactive martial). Verbeek is not only collecting different dimensions of effects that technologies have, rather he introduces a systematic order which is centrally oriented on the distinction between mediated perception and action. We think this distinction has its weaknesses because actions are depended on the perception which determines the decisions to and plans for actions. Referring to the concept of the lived body, which brings both dimensions of perception and action in a closer relation, is one possibility to refine this.
We have focused on theories, having an explicit direction towards the design of technical systems which fits to the interest of the interdisciplinary approach. One problem of both theories is that it remains to be seen to which further purpose such an analysis on the mediating character of technology will lead. The direction of engineering design, Verbeek provides, is quite vague and focuses only on one aspect, the material aesthetics. Additionally, by rejecting the dystopian judgment, what he does by criticizing the classical phenomenological concept about technology, his theory lacks a critical dimension.
However, these concepts, even better in combination, are a good starting point to analyze a new technology which can be predicted as very influential in the everyday world. According to that, the theoretical background can also be used as a basis for a judgmental or ethical investigation. Before we show such a perspective in a final section (towards a better design), we discuss the case studies in two directions and give an idea how the technology of wearable activity recognition is mediating.
5. How Is Wearable Activity Recognition Mediating?
Wearable Activity Recognition provides a new perspective on human actions, which is mediated by the technology. Analyzing this mediating character is the interest of this paper. We follow, therefore, the theoretical thoughts, especially of Verbeek for analyzing the possibilities and limits of technologies: he asks, therefore, which perceptions and actions a technology is opening up and which it is narrowing. Our analysis includes two approaches. First, the mediating character seen from an applications perspective, including the two case studies/projects, and second seen from the perspective of the design practice, which is also based on the experience gathered during the project work.
5.1. In the Use Practice: Mediated Reflection on Activity Data
Starting with the first, we focus on the applications of self-tracking and the possibilities for using the tracked data for further reflection and behavior change. In the closed loop scheme, shown in
Figure 1, we refer to this application whereby the reflection on activities with the possibility of changing them closes the loop.
In the named applications, the influences are part of the functions that means the technology is mediating the users self-perspective in a way that changes the behavior. However, there are also possibly unintended ways the technology is mediating the perception of the user—for example, in case B, in which the wearable sensor is used for detecting working steps in the laboratory with the purpose of documenting the experiments. One result can be that scientists who use this tool probably adapt their behavior by reflecting on the documentation delivered by the tool. This behavior change is hence intended.
When using such a tool, it is clear that only what is detectable becomes part of the documentation. Following that, it is predictable that when persons who use this tool in the laboratory slightly adapt to a certain way of doing an activity, because some performances were more or less successful regarding the detection. These influences are unintended or at least not part of the function. For example, observations in the biological laboratory have shown that there are different performances possible to complete a working step: stirring different fluids in a beaker can be done with a spoon or by moving the beaker and bringing the fluid to move. It can be expected that using such systems for documentation and reflection of results direct a general way of performing tasks or some types of doing it. So using such systems on the one side facilitates a comparison of activities in the laboratory but on the other side can also limit the diversity of performances. That may not be a problem in this application but should kept in mind regarding other applications.
In case A, the technology of wearable activity recognition is applied to detect smoking behavior with the goal of increasing the awareness of the smoking activity. This can probably help to decrease the amount of smoking and even aid quitting smoking. The technology is mediating the perspective on the smoking activity of a person by combining it with additional information about time of the day, duration and money spent. When the user reflects on this mediated smoking information, it is possible that the person becomes more conscious about the smoking behavior in the future or can find out triggers which normally led him to smoking. A simple trigger would be, for example, alcohol related socializing with friends, what a person can find out when he sees in his data that depending on the night—especially on Friday or Saturday—the amount of smoked cigarettes increases. This perspective on smoking behavior is intended and enables reflection and behavior changes e.g., being especially attentive when drinking alcohol on weekend nights. However, it is also possible that unintended behavior changes can result. In some situations, it is imaginable that users perform in such a way that the activity is not detectable. For example, in order to not feel guilty the next day when smoking too much on a night in a bar, users could use the other hand (without wrist sensor) for smoking instead.
We had the intention to present some examples which show the complexity of the mediating character of wearable activity recognition. Next, we will focus on investigating the mediating character out of a design practice perspective.
5.2. In the Design Practice: Getting the Everyday
One of the main technical challenges of wearable activity recognition is to design computing systems which are able to get an everyday understanding of human actions. This challenge of how to deal with everyday entities is a general problem in the HCI research and especially in the ubiquitous computing where the focus on everyday applications is intrinsic. The research, referring to this question about computer based detection of everyday entities, is related to context-aware computing. It is mostly about the computable understanding of situations in which the technical system is in interaction with users to make this interaction appropriate to these circumstances.
In this chapter, we want to show how an everyday understanding of action is included in wearable activity recognition technologies. It is an investigation based on the current practice with reference to the two cases. With focus on the cases, the question reads as follows: how do people usually smoke and how are single procedures in the laboratory performed, named and understood? For the development of such recognition systems, the learning-based approach is the most common. It is based on the idea that the computing system is learning from situations in which the activities are performed, so to say learning from realistic real world data. In this concept, the distinction between the training phase of a system and the runtime is important. The training influences how the system works at runtime. The training data consists of a raw data stream and its related information on the activities, the so-called ground truth. Situations in which these data are acquired should be as realistic as possible but always have an artificial component because there has always been an observational aspect which is required to produce the annotations for the data (ground truth). In case A, a special cigarette lighter was developed to get the annotations and, in case B, the annotation was done by video-observation. This training results in an ordinary understanding of doing things or different types of doing things, the so-called stereotypes. The stereotypes influence what is detectable and so mediate the user’s actions and perception.
Thus, the mediating character of the technology is predetermined mainly by the learning-based algorithms, including the chosen probabilistic models, classifications of labels and how the training data is acquired. Especially, the aspect on which real world data the systems should based, is discussed under the phrase “out of the lab into the wild” (of the everyday) [
19].
6. Conclusions
The technology of wearable activity recognition is providing a perspective on persons’ actions which can result in intended behavioral changes (e.g., smoking cessation) but also in unintended influences when, for example, persons adapt the way of doing things because the technology is only able to detect certain stereotypical performances. How these stereotypes are constructed in the design practice was shown in the last section. Bringing these dimensions of possible and existing use practices and the design practice together was the methodological orientation of this paper. A deeper philosophical discussion, how technology is mediating the relation of person to the world and self is grounded in Verbeeks work and the concepts of embodied interaction. This directs one to an understanding of how the perspective on persons’ actions is mediated, meaning how persons relate to their actions, which was the central goal of this paper.
Following that, we conclude: actions are an intrinsic part of how we experience the world, what we are aware of and how we see our role in social interaction. When certain activities are brought into view by wearable activity recognition devices, it prevalently influences how people see their actions and, following on that, how they experience similar situations in the future. The awareness regarding an activity (e.g., smoking) can change and also in which way certain activities are interpreted. Additionally, the characterization of wearable technologies, which provides a close everyday and anytime relation, intensifies the assumption about influences on behavior and the perception. What follows from that, especially out of an ethical perspective, will be briefly discussed in the last section.
7. Prospect: With Philosophy of Technology Towards a Better Design
In this section, we show some consequences which can be concluded out of the analysis of the mediating character of wearable activity recognition technologies. Towards a better design means to think about a computer science design which is valuable in an ethical and responsible way. Valuable design means that social responsibility, ethics and privacy should be an intrinsic feature of the technology. For that, it is useful to mention the prospective Verbeek gives in his theory: one goal of his analysis is to investigate the possibilities and limits of technologies, which perception and action can be made possible and which is narrowed. Thus, using wearable activity recognition tools makes it possible to detect certain activities and visualize them for the user. This means some activities are preferably perceived by the persons and other activities, which are not detectable, become less important. Shortly, only what is detectable comes into the point of view. In the design practice, for example, Rogers follows concepts of embodiment and is suggesting an integration of such a wider perspective. “Opportunities are created, interventions installed, and different ways of behaving are encouraged. A key concern is how people react, change, and integrate these into their everyday lives” [
19].
Regarding our applications, the question for a better design in a valuable way is how the system should be built to really support self-reflection. This can include the possibility of the user to influence the labeling process and the automatic interpretations. For example, having the possibility to reflect on the raw data itself when required, is increasing the trust to the results an activity recognition system is providing. Technologies which try persuading the user to act in a certain way can help to motivate, e.g., to do sports, but also used to manipulate to act in a certain way which is only useful for the provider e.g., buying behavior detection with mechanisms to motivate the user to buy more.
Talking about valuable design regarding technologies which detect and store personal data, privacy concerns have to be discussed. One central aspect regarding this technology is that the raw data often consist of a lot more information than is used for the specific function of the recognition system. Following that, not only the interpreted activities should underlie a strict personal privacy but also the raw data, for example the accelerometer data stream. Regarding existing applications of, for example, sport tracking or eating behavior tracking, the data can be potentially used by health insurances to provide the regular users cheaper offers. This should be discussed regarding privacy and justice concerns.
Wearable activity recognition is an ongoing research which is strongly related to questions of a philosophy of technology in its conceptual interest on the role that technology has in our world but also in its critical dimension. We aimed at showing in this paper that such an interdisciplinary approach referring to the computer science design is promising.