Realizing Loose Communication with Tangible Avatar to Facilitate Recipient’s Imagination ^†

Abstract

Social network services (SNSs) allow users to share their daily experiences and significant life events with family, friends, and colleagues. However, excessive use of SNSs or dependence upon them can cause a problem known as “SNS fatigue” that is associated with feelings of anxiety and loneliness. In other words, the tighter and stronger the social bonds are through SNSs, the more users feel anxiety and loneliness. We propose a method for providing users with a sense of security and connectedness with others by facilitating loose communication. Loose communication is defined by the presentation of abstract information and passive (one-way) communication. By focusing on the physicality and anthropomorphic characteristics of tangible avatars, we investigated a communication support system, Palco, that displays three types of contextual information with respect to the communication partner—emotional state, activity, and location—in a loose manner. Our approach contrasts with typical SNS interaction methods characterized by tight communication with interactivity and concrete information. This paper describes the design and implementation of Palco, as well as its usefulness as a communication tool. The emotional effects on the users are evaluated through a user study with 10 participants over four days. The results imply that Palco can effectively communicate the context of the communication partner, and provide a sense of security.

1. Introduction

Increasing numbers of individuals are using social networking services (SNSs) such as Facebook [1], Twitter [2], and Instagram [3]. This makes it possible for users to share information on events and enjoyable experiences with other users in a relatively easy manner. A characteristic of SNSs is that people can easily connect with each other by freely posting messages at any time [4]. According to survey data from the Ministry of Internal Affairs and Communications of Japan on individuals ranging in age from teenagers to those in their sixties [5], many people think that the “bonds between others have deepened” due to SNSs. In other words, users aim to share information on their own events and pleasant experiences in real time, while also sharing their feelings and time with others in order to feel a sense of connection with them.

However, mental fatigue due to excessive use of SNSs poses a increasing problem, which is exacerbated by the growth in the number of SNS users. According to a survey conducted by the Ministry of Internal Affairs and Communications of Japan with elementary, junior high, and high school students, as well as university students and social workers (up to age 25) [6], 56.9% of the respondents felt burdened by SNS usage. This fact suggests that approximately half of these users feel stressed by use of SNSs. In addition, another survey conducted by TimesCurrent involving users in their twenties and thirties who have used personal accounts on Facebook [7] showed that over 40% of Facebook users felt tired by the use of Facebook. Krasnova et al. argue that this fatigue is caused by constant checking for updates from other users, the loneliness and jealousy felt when users compare themselves to others, and anxiety about the impression of their own posts viewed by others [8]. People use SNSs frequently to feel and maintain a connection with others; however, fatigue increases as the communication becomes denser and the connection becomes tighter.

In this article, we investigate the communication support system Palco (Palco: a coined word from “Printable Avatar system that realizes Loose COmmunication”) (Figure 1) that enables to improve the sense of security between people who maintain loose connections. We pay special attention to the physicality and anthropomorphic features possessed by a tangible (embodied) avatar. A tangible avatar is designed as a communication tool, and is made of paper for easy deployment. The information of a communication partner such as current emotional state, activity and location are projected on the head, body and feet of the avatar, respectively. Once a user owns the basic projector-camera unit, Palco can easily be scaled up with respect to the number of avatars and variation of their appearances. The usefulness of Palco is validated through a user study, in which the emotional effects on users are investigated. The remainder of this article is organized as follows. In Section 2, we examine related work on SNS fatigue, communication support systems, and research on system design. Section 3 provides information on the design process of Palco, and a user study is described in Section 4. A discussion follows in Section 5. Finally, Section 6 concludes the article with future prospects. In a previous study [9], we presented the basic concept of a tangible avatar-based system to facilitate loose communication between users of SNSs. This article extends this work with respect to several points. Firstly, related work is throughly examined to clarify the uniqueness of the work (Section 2). Secondly, the design rationale and information on its implementation are presented in greater detail. For example, information on how the tangible avatar was created and how communication partner states are acquired using a dedicated Twitter client application (Section 3.4) is presented, and two research hypotheses are added. The number of subjects in the user study has been increased in order to gain an extensive view on the impact of the tangible avatar on users (Section 4). In addition, an extensive analysis of the usefulness of the design and the emotional effects on the users is undertaken based on the user study (Section 5).

2. Related Work

2.1. SNS Fatigue

Krasnova et al. investigated that the life satisfaction of SNS users and implicated factors using responses from 584 Facebook users [8]. The work suggested that long-term exposure to posts from others displaying self-expression and narcissistic behavior may cause negative emotional states, such as jealousy and envy. Such negative emotional states can degrade individual’s satisfaction with life and, as a result, lead to annoyance and fatigue. Maier et al. [10] argued that SNS users feel exhausted or unsatisfied with SNSs when they perceive that they are giving too much social support to their friends through this medium. Moreover, they claimed that the extent of usage, the number of friends, subjective social support norms, and the type of relationship (online-only friends vs. face-to-face friends) are factors that directly increase such social overload. Lee et al. [11] regard overload (i.e., stressors) as a core determinant of SNS fatigue (i.e., strain) and identified three dimensions of overload: information overload, communication overload, and system feature overload. In this article, we propose relaxing the bonds between individuals through loose communication to avoid excessive communication overload.

2.2. Communication Support Systems

In communication support systems, two axes exist with respect to the concreteness of information presentation and activeness of the communication. The axis of concreteness represents the level of abstraction of the presented information, while that of activeness describes the existence of interactivity, i.e., showing information only or accepting the recipient’s reaction. With respect to the concreteness of presentation, Chameleon Mask [12] allows telepresence using a face display, in which the remote user’s face streams in a real-time manner. The work is categorized as a form of concrete information presentation because the partner’s face is directly displayed. Messaging Kettle [13] aims at promoting communication with elderly people and relatives who live apart by using a kettle as a communication tool. The system is presented in the form of a kettle, enabling text and voice messaging to remote users directly. It also represents a form of concrete information display. Another system that has the characteristic of concrete information display is Rickety Bench [14]. Rickety Bench expresses the presence of the partner by vibration of the bench and a display that looks like a mirror. SyncDecor [15] is a system that allows strongly paired romantic couples to remotely synchronize augmented furnitures and provide awareness such as a feeling of living together of each other. The communication style “Tsunagari-kan” (which refers to a sense of being connected with each other in Japanese), is aimed at maintaining and building human relations [16]. In the framework of Tsunagari-kan, the FamilyPlanter system was developed, which constantly sends and receives cues of information such as tone of voice, walking patterns, etc. in order to feel the presence of partners. Furfur [17], a robot pet sharing system for long distance couples, reminds the users of the presence of their partner by sending contact patterns and sound patterns to the system, while a long-distance couple can share the responsibility of taking care of a pet robot. In the three examples above, the partner’s information is expressed in an abstract manner, rather than concretely, in communicating the presence or a feeling of connectedness with remote users. Therefore, we consider that they can be categorized into a communication system with the presentation of abstract information. In addition to these methods, Calmate [18], Cubble [19] and Breathing Flame [20] are also regarded as communication support systems that realize abstract presentation.

Chameleon Mask provides the transmission of active information in that the user talks to a remote partner via the face display. Messaging Kettle promotes communication through use of a kettle, acting as a medium for active communication through the exchange of messages in the form of text and voice messages. Other systems with the characteristics of explicit communication are Rickety Bench [14] and HeartChat [21]. The mobile chat application HeartChat integrates the heart rate as a cue to increase awareness and empathy. In the SyncDecor, FamilyPlanter, and Furfur systems, information recipients are actively involved with the system such that they can communicate through interaction with the same daily objects each other. These systems form part of the category of active communication, enabling bi-directional communication while allowing the users to interact with each other via the system. In addition to these systems, LumiTouch [22] and WearLove [23] are also categorized into active information transmission. On the other hand, the coffee table Habitat connects two remote locations [24]. The contact between the coffee table and objects is detected, and the object in contact is displayed on the partner’s coffee table using a projector. In Habitat, the images of objects on the table such as books and cups are sent to a partner’s coffee table in a one-way manner. This indicates that Habitat is categorized into passive communication in that it does not allow users to interact with the information displayed by the projector. A wearable robot suite iFeel_IM conveys emotional states of a partner to the wearer through a haptic channel [25]. The emotional states are recognized from the text from a chat application, and the recipient can react to the transmitted haptic feedback through the chat application if desired. We consider that iFeel_IM belongs to the passive communication category because the chat application is separated from the haptic feedback. In other words, the chat-based response is an optional one.

In summary, the combination of abstract presentation and active communication is often utilized in communication support systems. In order to communicate the presence of remote users and to provide a sense of connectedness with them, we consider that abstract information presentation is effective in terms of reducing social overload and fatigue on exposure to concrete information. An advantage of the active communication style is that the user can receive the reaction from his/her partner in active communication; however, the interaction allowed by such a style can be a trigger for fatigue and annoyance. In this article, we take abstract information presentation and passive communication in order to address the issue.

2.3. End-User Design of Daily Objects

The paradigm of digital fabrication/personal fabrication has drawn attention for research in recent years [26,27,28,29,30,31]. Costanzas et al. reported the impact of tangible user interfaces (TUIs) on users’ everyday environments using two types of low-cost simple tangible interfaces for a music player; the interfaces were made of printed paper provided in a website [28]. They showed that the method is mature for distributing tangible user interfaces in an inexpensive and democratic way. Gervais et al. developed a tangible avatar, TOBE, to allow users to reflect themselves in an avatar [27], while Palco shows the states of communication partner on its body. In TOBE, the user can customize the appearance of an avatar and the animation pattern of the projected images, and the avatar can be printed using a 3D printer. Palco also employs printer-based avatar creation; however, the body of the avatar in Palco is made of paper. This allows a user to create an avatar using an ordinary ink-jet printer, or even without a printer, i.e., drawing development diagram by hand.

3. The Design and Implementation of Palco

3.1. Key Features

3.1.1. Tangible Avatar-Based Communication

In the Palco system, information about a communication partner is delivered through a physical avatar. We focus on the effects of the physicality and anthropomorphic characteristics of the tangible avatar. Physical contact with an object provides the user with a real experience, and we consider this to have an effect on human materialism. Anthropomorphism also affects the user’s performance and subjective reactions. In particular, anthropomorphism contributes to facilitating friendly interactions such as expressions of personality and emotion [32,33]. Also, as shown by Mohd and Wills, nonverbal behaviors such as body language signals and facial expression provide the user with a positive impression, and anthropomorphic characters seem to have the effect of supporting long-term interaction [34]. Hence, we believe that the tangible avatar’s physicality and anthropomorphic appearance provide users with a sense of realism and familiarity, while amplifying the impression that the system’s outputs and real data combine together to create a rapport. Therefore, we decided to apply the characteristics of tangible avatar to remote communication.

3.1.2. Loose Communication

We introduced the term of “loose communication” through two characteristics: abstract presentation and one-way communication. Conventional communication services use texts and image/videos to communicate facts, thoughts, emotions, etc., building strong ties between users due to concreteness. However, an explicit provision of the partner’s information can cause envy and loneliness to the user. We consider that the partner’s information should be made abstract so that the user can imagine the situation of the partner by themselves. In Palco, the state of the communication partner is represented without a textual expression, such that the avatar talks to the user, but with still or animated images through an appropriate abstractness level conversion. Also, interactivity in conventional services such as “Like!” in Facebook and Twitter enables active involvement with the partner’s information; however, this may trigger fatigue and annoyance on the part of the user. We consider that passive involvement realized by one-way communication plays a role in relaxing the pressure of response. In Palco, the user just watches the partner’s information.

3.1.3. Being Printable

The avatar shows the communication partner’s information, which was transmitted without any electronic or mechanical components (for example servo motors, light emitting diodes (LEDs), or liquid crystal display (LCD)). Instead, a video projector presents the information on the avatar’s body and the ground as described in Section 3.3, which allows the avatar to be made of paper. The paper-based tangible avatar allows users to download the preferred templates from the Internet or customize the appearance by themselves. Palco can be easily scaled up in terms of the number of avatars and variation in their appearance once a user has a projector. We further expect that the handmade and customizable nature of the avatar will amplify the bonding of the users to their own avatars.

3.2. System Components

Based on the key feature analysis, the Palco system was designed. Figure 2 shows the schematic view of the system. Palco comprises one or more cardboard avatars and the corresponding partners (followees), a projector, a camera, and a controller. A remote partner provides his/her information either explicitly or implicitly. The camera is used to detect the projection area and correct the distortion of the area, and the user views the partner’s information from the projected image of an avatar. As described above, the avatar is made of paper. Hence, the avatars can be incrementally added to the system given that they appear within the camera’s view. The projection image is generated by taking into account the position and distortion of the projection surface.

3.3. Information to Be Presented

People desire to be with others, and this can be accomplished artificially through the sharing of their current states. The states communicated through Palco are the activity, the location, and the emotional state at a particular moment, which we consider to be the contextual information of the person. We expect that the user may feel at ease by sharing the contextual information that characterizes the person. These elements of the partner’s information are projected by taking into account the intuitiveness of the relationship. This means that the location, the activity, and the emotional states are projected at the foot, on the body, and on the head of the avatar, respectively (see Figure 3). The information about the location and activity is represented through illustrations that can abstract such information, rather than with textual description or pictures. The emotional states are presented with facial expressions. Additionally, for the purpose of increasing the rapport with the avatar as well as its charm, the emotional states are rendered with animations such as blinking, laughing, and so on.

3.4. Prototype System Implementation

A prototype system of Palco was implemented, with the minimum functionality for the user study as described in Section 4.

3.4.1. Apparatuses and Software Libraries

In our very first version prototype [9], the controller component ran on a Windows-based PC with a web camera; however, in this version, Raspberry Pi 3B (Jessie OS) with a high-definition camera module was utilized, which makes the system more portable while being more cost-effective. A tiny laser projector, the Smart Beam Laser LB-UH6CB from SK Telecom, was used to project information. Since laser projectors do not have any focal points, the user does not need to be concerned about the placement of an avatar. Image processing for camera–projector calibration, projection image correction, and superimposed image creation was performed using the library in OpenCV 3.1. We also used Twitter as the communication infrastructure. A Twitter account in order to receive tweets solely from the communication partner was implemented using Twitter 4J and was hosted on the controller device.

3.4.2. Acquiring the Partner’s States

The Palco system separates the method of acquiring the partner’s states from its implementation. Any method that generates a string consisting of the identifiers of current emotional state, activity, and location with a delimiter, e.g., 7/2/19, can be used. The acquisition process could be automated by applying sentiment analysis to extract emotional states from text, activity recognition from sensors in a smartphone, and location recognition from the Global Positioning System (GPS) receiver in a smartphone via reverse geo-coding. However, for simplicity, we developed a twitter client application for Android OS that allows the user to select only appropriate items from pull-down lists and press the Tweet button (Figure 4).

3.4.3. Projecting Images

In order to superimpose information on an appropriate part of the avatar surface even when the avatar is moved, the avatar should always be tracked, and the projection needs to be adjusted to the position of the avatar. Although a user can design the avatar based on his/her preferences, a constraint is that rectangles are drawn for the head and body to track the position and for distortion correction, as well as for camera-projector calibration. The avatar’s head and body are recognized as rectangles. The four vertex coordinates of these rectangles are acquired from the projection area on the camera image. By applying the camera-projector perspective transformation matrix to each coordinate, the projection area is calculated for the projection image. The four vertex coordinates of projected images that correspond to partner’s states are perspectively converted to coincide with these calculated coordinates, and a superimposed image is created. The projection area at the foot of the avatar is calculated relative to the head and body coordinates of the projection image. By applying the calibration to match the coordinates of the camera and projector, the projected image is successfully superimposed on the appropriate part of the avatar surface.

Figure 5 shows an example of avatar tracking, in which an avatar was moved from left to right. Strictly speaking, the avatar is not tracked in a real-time manner; the projection mapping process starts once an avatar is still to reduce the computational load. Hence, as seen in the center of Figure 5, the projected image appears in the previous position during movement, which we do not consider a major problem due to the short duration (< 2 s).

To increase rapport, information projected onto the head is rendered with animations such as blinking and laughing. As shown in Figure 6, the head is animated in a way such that four gradually changing images are projected continuously. The interval from one image to the next image was adjusted so that it could look like a natural animation, as shown in the figure.

4. User Study

4.1. Methodology

A user study was conducted to assess the avatar-based system’s usefulness as a communication tool and its emotional impact on users. Ten individuals participated in the experiment as users of Palco, “receiver” in other word, while one person participated as a communication partner with them, i.e., sender, and provided information through a dedicated Twitter client on a smartphone as input to Palco. In this case, three types of relationship with the communication partner existed. One individual represented a mother, five individuals were friends, and there were four colleagues. “Mother” is an attribute of the participant who watches the information about her own child through Palco. “Friend” and “colleague” indicates the relationship with the sender. The friends and the colleagues were aged in their twenties. They were asked to use Palco for more than six hours a day for four days, while being allowed to continue with their day-to-day work, leave the room for short periods, and switch the system on or off as they wanted. They could see the avatar and confirm their partner’s emotional state ($emo$), activity ($act$), and locations (two types): abstract ($lo{c}_a$) or detailed ($lo{c}_d$), at any time. The granularity of the projected information was changed from Level 1 (low) to Level 4 (high) by the combination of the elements of the status information as shown in Figure 7. The four types of information are presented for each subject in a random order to avoid order bias.

Meanwhile, the communication partner tweeted their current states using the dedicated client application described in Section 3.4.2.

Table 1. The projected image set used in the experiment.

	Type of Projected Images
emotion	neutral, happy, joyful, sad, angry, surprised, sleepy
activity	eating, sleeping, web browsing, reading a book, playing piano, conversation, bathing, drinking, listening to music, watching TV, using a smartphone, shopping, working on a PC, walking, watching movies, cram school teaching, drinking coffee, bowling, playing a game, calling, muscle training, exercise, cleaning, cooking, coffee serving
abstract location	facility, home, location of a part-time job, shop, within a train, university
detailed location	cinema, spa, bowling alley, concert hall, home, cram school, coffee shop, restaurant, bookstore, bar, tavern, coffee shop office (part-time job), event venue, supermarket, home appliance mass retailer, Keihin Tohoku line, Chuou line, laboratory, department store, campus cafeteria, campus library, Fuchu campus, gym

shows the possible states of the communication partner, which we assumed as appearing in their daily lives.

At the end of each day and a couple of days after the experiment, the subjects were asked to answer a questionnaire survey and have follow-up interviews. In addition to the subjective results, the system counted the number of movements of the avatar for later analysis.

4.2. Hypotheses

The following four hypotheses were set to evaluate the usefulness of the tangible avatar-based loose communication system as a communication tool, as well as the emotional impact on users.

Hypothesis 1

(H₁). A tangible avatar becomes a communication tool.

Hypothesis 2

(H₂). Loose communication is useful for maintaining relationships with a partner.

Hypothesis 3

(H₃). Conversation or communication using conventional media is triggered.

Hypothesis 4

(H₄). A daily relationship between the sender and the recipient affects the user’s response.

4.3. Result of Concept Validation

4.3.1. Impressions on the Tangible Avatar

To verify $H_1$, we counted the number of times that participants touched and moved the avatar from the system log. As a result, although there was a difference in the number of strikes for each participant, the avatar was moved multiple times each day (mean: 6.9, SD: 3.7). Then, the participants were asked about their sensations regarding the avatar, and the following were the reasons for touching it. Here, the letters in parentheses represent the IDs of the participant.

${\mathbf{O}}_{\mathbf{1}}$-1

Rectangle detection was unstable. (A, D, J)

${\mathbf{O}}_{\mathbf{1}}$-2

I was concerned about what it would be like when moving avatars to the left and right. (C, F)

${\mathbf{O}}_{\mathbf{1}}$-3

I experimented in the range visible through movement. (C, I)

${\mathbf{O}}_{\mathbf{1}}$-4

I touched it at break-time because I had nothing to do. (B, G)

${\mathbf{O}}_{\mathbf{1}}$-5

I think I did not touch it with a simple doll. (B)

${\mathbf{O}}_{\mathbf{1}}$-6

I touched it when I found something interesting in the projected image. (E, J)

The obtained opinions about the impression towards avatar are shown below.

${\mathbf{O}}_{\mathbf{1}}$-7

I felt a sense of familiarity due to the animated facial expressions.(A, D, E, G, H, I, J)

${\mathbf{O}}_{\mathbf{1}}$-8

Once I got used to Palco, I concentrated on my work and I did not mind the existence of the avatar. (B, C, E, I, J)

${\mathbf{O}}_{\mathbf{1}}$-9

I felt that the avatar stood there naturally. (C, F)

${\mathbf{O}}_{\mathbf{1}}$-10

Not much motivation was needed to switch my gaze to the avatar. (D)

${\mathbf{O}}_{\mathbf{1}}$-11

The partner was naturally reflected in the avatar. (E)

4.3.2. Impression on Passive Communication

To verify $H_2$, we asked participants about their impressions on passive communication, which is one of the elements of loose connection.

Table 2. Main opinions on passive communication.

Participant	Opinion About Passive Communication
A (mother) E (colleague) H (friend)	-I wanted to confirm the status of my partner by e-mail. -I wanted to show a reaction to partner.
F (friend) G (friend) J (colleague)	-I wanted to react when Palco was sorrowful. -When Palco showed a positive emotion such as laughing, it was enough to just look.
C (friend) D (colleague)	-It is sufficient for one-way communication. -I also liked active communication only when I felt strong negative feelings from Palco.
B (friend)	-I do not think I would like to interact with Palco. -It might be interesting to tell the partner that I touched the avatar.
I (colleague)	-If Palco can show a reaction, I may use it out of mere curiosity.

shows the opinion obtained from each participant, in which conflicting opinions were found such as “I wanted to react to my partner” and “I do not think I would like to interact with Palco”.

4.3.3. Impressions with Respect to the Presentation of Abstract Information

To verify $H_2$ as well, we asked participants about their impressions of the presentation of abstract information, another element of loose connection. The participants were asked to rate their levels of awareness or interest in their partner’s states per element through the projected images. This was to evaluate the effect of this abstract form of presentation for each level of abstraction. Here, the rating consists of five levels (1: not noticeable/not interested, 5: strongly noticeable/extremely interested).

Table 3. Average levels of awareness of and interest for each of the partner states (5 is the highest score).

	Level 1	Level 2	Level 3	Level 4
emotion	4	4	4	3.6
activity	-	-	3.9	3.6
location	2.9	3.4	3.2	2.6

shows the average ratings.

The ratings for emotion and activity are high among all subjects. On the other hand, the score for location tends to be relatively lower than the other two. The following are main opinions about the impression of abstract information presentation.

${\mathbf{O}}_{\mathbf{2}}$-1

I do not get tired of seeing dopey. (A, B, C, F, G, H, J)

${\mathbf{O}}_{\mathbf{2}}$-2

Facial expression is a cue to think of the partner. (A, B)

${\mathbf{O}}_{\mathbf{2}}$-3

First, I look at the information shown through the body and can see emotional state from there. (B)

${\mathbf{O}}_{\mathbf{2}}$-4

I can imagine the state more specifically with the activity information. (B, C, D, E, H)

${\mathbf{O}}_{\mathbf{2}}$-5

I felt relieved when I knew the location of the partner. (E, I, A, D, I)

${\mathbf{O}}_{\mathbf{2}}$-6

Just by knowing what the partner is doing now, I can guess a certain partner’s location. (C)

${\mathbf{O}}_{\mathbf{2}}$-7

It is better to combine the types of information. (A, E, J)

At the end of each day and a couple of days after the experiment, we asked about the emotional effects with respect to the partner in a range from −5 to 5 relative to the score before using Palco (0). Figure 8 shows the transition in the amount of change with respect to the emotional effects for each participant. Common to all subjects, the sense of annoyance towards the communication partner barely increased. On the other hand, positive emotions such as frequency of thinking about the partner and a sense of security were found to be improved by using the system. These positive emotions were also confirmed to be decreased in participants A, E, and J after the experiment.

4.3.4. Conversation Triggered by Palco

To verify $H_3$, we compared the frequency of communication before and after the experiment for each participant (rated from 1 “very few” to 5 “very often”). We focused on oral conversation and an SNS-based one. The results are shown in Figure 9, in which four people showed an increase in the frequency of communication, while two people showed a decrease, and for four people the frequency did not change at all.

We summarize the conversations triggered by information from Palco during system use. The topic of conversation is mainly with respect to the partner. Additionally, as seen in $O_3$-7, conversations with people who shared the information were triggered.

${\mathbf{O}}_{\mathbf{3}}$-1

It was different from the schedule heard in advance. Did you have something? (A)

${\mathbf{O}}_{\mathbf{3}}$-2

Did it not work very well when you were practicing the piano? (A)

${\mathbf{O}}_{\mathbf{3}}$-3

Why did you get angry yesterday? (H)

${\mathbf{O}}_{\mathbf{3}}$-4

Why were you crying at the part-time job? (B)

${\mathbf{O}}_{\mathbf{3}}$-5

Were you asleep while working at home? Or just sleepy? (D)

${\mathbf{O}}_{\mathbf{3}}$-6

Is it due to yesterday’s party that you were late today? (C)

${\mathbf{O}}_{\mathbf{3}}$-7

Conversation was triggered with not only the partners but also those who were seeing the same information. (B, E)

4.3.5. Attributes and Daily Relationships with the Communication Partner

To verify $H_4$, we asked each participant about the daily relationships and impressions of the communication partner using 5-scale Semantic Difference (SD) method (Figure 10). The scores are presented in Figure 10, in which participants can be divided into three groups: Group 1 {B (friend), C (friend), D (colleague) and I (colleague)}, Group 2 {A (mother), F (friend) and H (friend)}, and Group 3 {E (colleague), G (friend) and J (colleague)}. The characteristics of the groups are summarized in

Table 4. Grouping of the subjects by daily relationships and impressions of the partner.

Group	Characteristics/Descriptors
Group 1	carefree, friendly, unconcerned, reliable, reassured, pleasant, not connected
Group 2	carefree, friendly, concerned, reliable, a little worried, from pleasant to boring, connected
Group 3	carefree, friendly, a little concerned, reliable, a little worried, pleasant, connected/not connected

. By comparing the characteristics of the groups, we found that the effects on the users of Palco vary by groups, which depends on daily relationship between the user (receiver) and his/her communication partner (sender).

5. Discussion

5.1. Effect of a Tangible Avatar as a Communication Tool

As mentioned in Section 3.1.1, we focused on both physical and anthropomorphic aspects of the tangible avatar. According to Section 4.3.1, the subject moved the avatar more than once each day. By referring to the reason for moving the avatar indicated by $O_1$-1 to $O_1$-6, the avatar was moved due to a problem in the system implementation as argued by $O_1$-1. Meanwhile, $O_1$-2 and $O_1$-3 suggest that the fact that the user can touch and move the avatar actually attracted the participant’s interest. From $O_1$-4 and $O_1$-5, we consider that the avatar had a property that encouraged the participants to touch it. Additionally, as pointed by $O_1$-6, the presence of the communication partner was felt via the avatar.

As mentioned in $O_1$-7 and $O_1$-8, the change in the facial expression of the avatar attracted the user and amplified the affinity, rather than disturbing the user. Also, from $O_1$-9, the avatar adapted to the user’s living environment; however, as shown in $O_1$-10, a risk exists that the presence of avatar will fade away and not draw attention as it blends into the environment. The effect on long-term usage of the system will be investigated in the future.

These analyses indicate that physicality and anthropomorphic characteristics add a sense of presence or familiarity to the avatar. An effect of seeing their communication partner in the tangible avatar like $O_1$-11 was also confirmed. We consider that $H_1$ “tangible avatar become a communication tool” was supported.

5.2. Effect of Loose Communication on the Relationship between the User and the Partner

As mentioned in Section 3.1.2, we defined passive communication and abstract information presentation as the elements of loose communication. From the opinions about passive communication shown in Table 2, it appeared that some participants such as A, E, and H wanted to respond to their partner. Other participants such as F, G, and J tended to want to react when the avatar appeared to be crying or sad. The other four participants did not want to interact with the avatar and preferred one-way communication. Consequently, we grouped participants A, E, F, G, H and J into the “active group” $G_a$, while the other four participants were grouped into a “passive group” $G_p$. By referring to Figure 10 which represents daily relationships and impressions with the partner, we analyzed the difference between $G_a$ and $G_p$; the participants in $G_a$ were fully or a somewhat concerned about how their partner normally was. On the other hand, participants in $G_p$ answered that they were not concerned about how their partner was. We consider that the difference between the participants who liked active communication and those who liked passive communication was related whether they were concerned about the daily appearance of the communication partner or not. In contrast, the participants in $G_p$ also liked active communication but only when they felt strong negative feelings. For example, a user wanted to hit the avatar to express his annoyance on receiving the information that the partner was experiencing a state of enjoyment, while the user was very busy and stressed. A field test of SyncDecor reported that a couple had attempted to repair the relationship by synchronizing the furniture during a fight, but the devices had rather made them irritating each other [15]. Moreover, the movement and accompanying noise of Furfur often made the users hesitated to use Furfur to avoid interrupting the partner [17]. Similarly, Palco frustrated users when they received the information that the partner was experiencing a state of enjoyment, while the users were very busy and stressed. In these systems, negative emotions may be induced when people who got into a situation such that the sense of burden or stress is remarkably high used the system. In addition, SyncDecor synchronizes furniture while Furfur shares robot pets. SyncDecor sometimes caused the user to feel annoyance about the movement of furniture. By contrast in Furfur, such annoyance to the system was not reported. We consider that the embodiment of communication tool, i.e., pet robot, amplified the affinity and cuteness of the system, and thus suppressed the annoyance. Likewise in Palco, six out of ten subjects did not increase annoyance points in Figure 8. Even in the remaining four subjects, three of them increased points by +1 (maximum: +5), and the points did not increased in some days. Therefore, we consider that the subjects not always feel annoyance. By leveraging the findings in Furfur, the anthropomorphic avatar in Palco might amplify the affinity and cuteness, which we consider made the user rarely annoying.

With respect to the awareness of and interest in projection information (Table 3) and the opinions on the abstract information presentation ($O_2$-1 to $O_2$-7), the scores for emotion and activity were high for all participants. Moreover, expression is an element where the emotional state directly appears, which indicates that expression becomes a trigger to recall the partner’s presence as in $O_2$-1 and $O_2$-2. Also, the opinions such as $O_2$-3 and $O_2$-4 suggest that the activity information greatly contributed to the impression of the avatar by making the emotion information more impressive. In other words, the activity information supports the user in understanding the reason for the emotional states inferred from the avatar’s facial expression. Moreover, we confirmed the possibility of improving the sense of security using more detailed location information, as argued in $O_2$-5. However, the score for location was the lowest of the three information elements. This can be found in the opinion $O_2$-6. The reasons for this are that the most interesting information is with respect to the emotional state, and that the location information can be inferred from the activity. Furthermore, although projection with more finely grained information, i.e., detailed information, was preferred, the upper limit of the preferred granularity was not known. Regarding abstract presentation, Gaver et al. [35] argued that ambiguity plays as a resource for designing information presentation system, which can be used to encourage close personal engagement with systems. Ambiguous information requires users to fill in the gaps in information by finding answers by themselves. Such characteristic was suggested to be not only aesthetically attractive, but also conceptually appealing by supplementing the gap, i.e., ambiguity, with the user’s own interpretations and beliefs. Palco can also facilitate the user’s imagination by displaying abstract information, in which the communication partner is recalled based on the user’s (receiver’s) free interpretation.

In terms of the amount of change with respect to the emotional effects as shown in Figure 8, the emotional effects seemed to be improved in almost all participants during system use. However, participant J, who experienced reduced emotional effects commented “I was sorry when highly private information was seen”. In this case, the participant had seen the picture of activity “taking a bath” and considered bath-taking as privacy-sensitive information. Hence, private information needs to be carefully handled. In most subjects, the scores with respect to frequency of thinking about the partner, the sense of security, and feelings of connectedness were high. We consider that certain effects exist with respect to recognizing the partner’s state and providing a sense of security. However, emotional effects after the experiment, i.e., without the system, tend to decrease or are kept at the same level as compared to the level obtained during using Palco. We do not have sufficient evidence to conclude that the effects will last for a long time after the system is removed.

We consider that the second hypothesis $H_2$, “loose communication is useful for maintaining relationships with the partner” was not fully supported for all participants; however, the abstract information presentation was found to be useful for the imagination of the partner. Also, we confirmed that the preferred level of activeness depends on daily relationships with the communication partner. As mentioned in Section 2.1, long-term exposure to posts from others displaying self-expression and narcissistic behavior, social overload, information overload, communication overload, and system feature overload were considered as major factors of SNS fatigue [8,10,11]. The motivation behind the design of Palco is to address this issue by passive and abstract information presentation, i.e., loose communication, to reduce fatigue such as anxiety, envy, and loneliness caused by traditional straightforward communication media. By reflecting the subjective opinions, we consider that the concept of loose communication was supported for those people who were not so much concerned about daily states about their communication partner.

5.3. Communication Triggered by the Information on Avatar

Figure 9 indicates that the communication frequency before and after utilization of the system did not change much in most participants. Participant A increased the frequency of SNS communication, which was supported by the statement in Table 2 “I wanted to confirm the status of my partner by e-mail”. In participant C, the frequency of communication (either oral or SNS-based) with his partner was not actually changed, although C insisted that it had increased. In Figure 8, the emotional effects of C obtained after the system was removed did not differ from those was obtained during the experiment. We consider that participant C confused a feeling of getting along with the partner with that of increasing oral and SNS-based communication even after he stopped using the system, because they are friends. The opinions $O_3$-1 to $O_3$-7 summarize the conversation triggered by the system, in which the conversations where the participants were concerned about their partner’s state were mainly induced as shown from $O_3$-1 to $O_3$-6. Hence, the system contributed to motivating participants to talk with their partners when the projection information concerned them. Furthermore, $O_3$-7 suggests the possibility of inducing a conversation with other people who were watching the same information together. These analyses indicate that conversation can be induced in particular conditions although it is difficult to state that $H_3$, “conversation or communication using conventional media is triggered”, was supported for all subjects.

The result of a field test of SyncDecor showed that the presented information sometimes triggered calling or sending e-mail to the partner, while the system itself sometimes became a topic of conversation. The conversation by traditional media are thought to be increased because the target users of SyncDecor are long distance couples who cannot meet frequently and thus the traditional media are plays important roles. In Palco, on the other hand, the frequency of oral and SNS communication did not increased notably in most participants. However, we consider that topics related to projected information onto Palco let the users talk in daily face-to-face communication because the users frequently met with the partner. Assuming that Palco is used by long distance couples, online conversations such that the user is concerned about their partner’s current state are expected to increase like SyncDecor.

5.4. Impact of the Daily Relationship between the User and the Partner

As mentioned in Section 5.2, active communication through bi-directional communication was shown to be desirable for the subjects in Groups 2 and 3, while people in Group 1 liked passive (uni-directional) communication. We consider that the difference lies in whether the subjects are concerned about the situation of the communication partner’s everyday situation or not. Also, the participants who belonged to Groups 2 and 3 had frequent conversations triggered by the system. In Section 5.3, it was suggested that concerns about the partner triggered conversation. For these reasons, there appears to be a correlation between daily relationships and impressions, and the user’s response. On the other hand, there was a big difference in terms of a sense of security between participants B and C, who belong to the same group and have the same attributes, i.e., they are friends, as shown in Figure 10 and Table 4. Participant B argued, “I am not interested in what other people are doing. So, I do not get a sense of security from the information”. We consider that not only daily relationships and attributes but also the characteristics of individual users affect the user’s response. Based on the discussion, we consider that $H_4$, “a daily relationship between the sender and the recipient affects the user’s response”, was supported, although we also found other factors.

6. Conclusions and Future Work

In this article, we proposed a tangible avatar-based system, Palco, to address the SNS fatigue problem by facilitating loose communication between people. We expected that the physicality and anthropomorphic appearance of tangible avatars would allow the user to receive information on the communication partner through the avatar. The partner’s states, i.e., emotional state, activity, and location, represent his/her contextual information, which was projected onto the paper-crafted avatar in an abstract manner—the user just saw the information without any reaction. A user study was conducted to assess the avatar-based system’s usefulness as a communication tool and its emotional impact on users. The following findings were obtained:

• The tangible avatar became a communication tool through its physicality and anthropomorphic characteristics.
• A preference with respect to the level of activeness, i.e., active or passive, in communication varies depending on the daily relationship with the partner. Additionally, the preference may also be affected by the temporal impression on the partner, such that the user may want to reply to a partner who shows different states from the user.
• The presentation of abstract information by projecting current partner states is useful for maintaining relations with the communication partner.
• There is an effect of urging the users to talk with their partner when the projection information concerns them.
• Not only daily relationships but also the user’s interests in the communication partner affect the user’s attitude.

The number of the participants (10) is small in order to cover a wide range of attributes of people such as age, gender, the relationship with the sender, and culture; however, we consider that the results are still worth in terms of revealing the basic characteristics of loose communication using tangible avatar. In addition to the experiment with large population, investigating the effect of long-term use is a future research agenda.

In this article, we focused on understanding the effect of a tangible avatar as a means of loose communication. However, practical issues remain. The current implementation allows only one avatar to exist within the field of view of a camera. We plan to investigate the avatar identification method to handle multiple avatars with one camera, using an extensible tangible avatar system. Palco was designed to be independent of the method used to extract the three pieces of information, and a dedicated Twitter client was developed. However, to improve the user experience of the information provider, automatic recognition of the information elements should be realized. We hope that the implications obtained from the experiment will inspire future efforts to uncover the potential effects of loose communication for enhanced social connection without the negative aspects of SNSs.