Tangible Cooperative Gestures: Improving Control and Initiative in Digital Photo Sharing
1
Faculty of Design, Architecture and Building, University of Technology Sydney, City Campus (CB 06.06.69B), PO Box 123, Broadway, NSW 2007, Australia
2
Department of Industrial Design, Eindhoven University of Technology, Den Dolech 2 (LG 1.88b), PO Box 513, 5600 MB Eindhoven, The Netherlands
3
Duncan of Jordanstone College of Art & Design, University of Dundee, Nethergate, Dundee DD1 4HN, UK
4
Australian Research Council's Centre of Excellence in Cognition and its Disorders, Maquarie University, NSW 2109, Australia
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Machines 2015, 3(4), 268-295; https://doi.org/10.3390/machines3040268
Submission received: 26 August 2014
/
Accepted: 24 August 2015
/
Published: 30 September 2015
(This article belongs to the Special Issue Tangible meets Gestural)
Abstract
:This paper focuses on co-present digital photo sharing on a notebook and investigates how this could be supported. While analyzing the current digital photo sharing situation we noticed that there was a high threshold for visitors to take control of the personal computer of the photo owner, resulting in inequity of participation. It was assumed that visitors would have the opportunity to interact with the notebook more freely if this threshold was lowered by distributing the user interface and creating a more public, instead of personal, interaction space. This, in turn, could make them feel more involved and in control during a session, creating a more enjoyable experience. To test these assumptions a design prototype was created that stimulates participants to use tangible artifacts for cooperative gestures, a promising direction for the future of HCI. The situation with the cooperative gestures was compared with the regular digital photo sharing situation, which makes use of a keyboard. In dyads, visitors felt more involved and in control in the design prototype cooperative gestures condition (especially during storytelling), resulting in a more enjoyable digital photo sharing experience.
1. Introduction
Personal digital photo collections are growing rapidly as digital cameras are becoming cheaper and more widely available, for example because of the integration of cameras in mobile phones. At the same time digital storage capacity is becoming less relevant as hard drives and storage cards rapidly increase in capacity. People are therefore encouraged to take photos more often and to take more photos of the same situation (to make sure at least one of them will turn out okay), resulting in large personal photo collections [1]. Sharing these digital photographs on devices such as notebooks presents new problems and challenges.
One of the most obvious disadvantages when sharing digital photographs is the lack of direct manipulation possibilities, meaning you cannot physically grab the photographs to have a closer look or to hand them around, which makes sharing more difficult. Many research projects focus on this problem and try to reintroduce direct manipulation into digital photo sharing, for example by using emerging technologies such as multi-touch tables or tablets. What remains relatively unknown is how we can support the digital photo sharing situation on a notebook. Many people still share their digital photographs on personal computers, such as portable notebook and desktop computers, despite new developments such as tablets and remote sharing through online services and social networks. It is therefore important to understand this co-present digital photo sharing situation and to support it. This paper will focus on an elegant tangible user interface that allows for basic tangible gesture interaction [2].
In this paper we will give a brief overview of the most relevant work that has been done on how people manage (for example browsing and editing) and share photos (Section 2). An analysis of current digital photo-sharing practices can be found in Section 3 and Section 4. In Section 5 the research question will be explained. Based on the analysis results from Section 4, a design prototype was created whose aim was to support the current digital photo-sharing situation (Section 6), which was validated through an experiment with dyads comparing triad situations (Section 7, Section 8, Section 9 and Section 10). We will end this paper with a discussion (Section 11) and conclusions (Section 12).
2. Related Work
In Section 2.1 we will discuss some of the most relevant work that has been done around photo sharing and managing (both printed and digital). In Section 2.2 we will give a short overview of some example systems that have been made to support digital photo sharing.
2.1. Photo Sharing Practices
According to Sumi et al. [3] stored digital photos are not effectively used to share, eventually becoming dead storage. However, sharing is the main purpose of photographs [4,5,6] and is considered to be a very common and enjoyable activity associated with photos [7]. Rodden and Wood [1] have studied how people manage their digital photographs and conclude that digital photos are most often sorted by event. Individual photos can then be found by browsing through the thumbnails of all the photos of a particular event in chronological order. This technique works well and they state that browsing facilities are used far more often than search options. The reason for this is that people consider it to be too much of an effort to annotate individual photos and current image processing techniques cannot yet extract enough semantics, such as context and persons displayed in a photo, to be useful. Additionally, people often want to share a set of photos rather than a single photo [4,5] which makes sorting photos by event more suitable for sharing. The most recent photos are also the photos that are shared most often which makes it easier to remember where photos are located.
Frohlich et al. [7] and Crabtree et al. [8] have been studying how people share printed photographs in a home environment and made design recommendations for digital photo sharing devices based on their findings. Frohlich et al. [7] make distinctions between three different ways of co-present sharing. The first one is storytelling in which the owner of a set of photos (photo owner) shares photos with others who were not present at the event the photos were taken (visitor). In this situation the owner is telling a story to others using individual photos as input for his or her story. The photo owner gives most input during the session while the visitor’s contributions are minimal (occasionally pointing, commenting or asking questions). The second way of sharing is called reminiscing talk. In this situation multiple people share and discuss photos of an event where they were all present. All participants participate in an equally active manner. The third way of sharing is a mixed situation in which multiple people are present who were and were not present at the event the photos refer to. Storytelling and reminiscing talk complement each other during the sharing session. In all these situations all people actively participate while sharing. This makes sharing photos a collaborative and social activity where people sit together, face to face, to recollect memories or exchange experiences. Furthermore, printed photos can be browsed quickly by direct manipulation. They can be spread out, compared, annotated, shuffled, and handed around, creating opportunities for the participants to look back or ahead of the visual narrative and to seize control of the images or conversation [9]. These properties are important for conventional photo sharing and traditional interfaces (such as a notebook) are not well suited for this collaborative way of sharing. Often there is a single person that operates the interface while others stand or sit around him, looking at a small, central screen. Crabtree et al. [8] suggest combining collaborative interfaces with tangible interaction techniques to support the movement and manipulation of photographs. They also provide several suggestions to support remote sharing of photos, for example by proving telepointing and telepainting interaction techniques (remote sharing will not be further discussed in this paper).
Lindley and Monk [10] have studied two factors that influence the way people socially interact around digital photos; the seating arrangements and the distribution of control (in storytelling and reminiscing talk conditions). In the first condition it was noticed that people generally tend to hover behind the photo owner that is controlling the PC. If the seating arrangement was changed in a way that allows people to see each other’s reactions (huddling), the sharing session became more fun, equal and free. In the distributed control condition all the participants were able to give input to the session through individual remote controls. They stated that the photo owner is probably reluctant to give away control over the session, making the sharing of digital photos less equal compared to the sharing of printed photos. It was concluded that the distribution of control created an additional channel for communication between users, making the session less formal and more enjoyable. Equality and freedom were also rated higher when participants were able to give input to the session through their individual remote controls. The session became more like a sharing session instead of one person telling a story and more like a discussion instead of a question-and-answer situation. Different roles developed in some of the sessions (occasionally still resulting in a leader of the session) and there were some significant differences between storytelling and reminiscing talk scenarios. Reminiscing talk sessions were considered to be more equal, interactive and fluent, indicating a higher feeling of involvement among participants. These findings show the clear distinction between storytelling and reminiscing talk as made by Frohlich et al. [7].
2.2. Example Systems
Due to the shift from printed to digital photography, new technologies have been developed to support storage, sending and sharing of photos on-line. These new, second-generation offerings are referred to as photoware [7] or future photo sharing technologies. They distinguish four different types of photoware, from which an overview can be found in Table 1. Many new applications focus on remote sharing, archiving or sending as these are the most obvious ways in which technology can support the sharing of digital photographs. This study however focuses on co-present sharing.
Table 1.
Four different types of photoware [7], this study focuses on co-present sharing (shaded grey).
| Same Time | Different Time | |
|---|---|---|
| Same place | Co-present sharing (prints, photo viewing software) | Archiving (albums, website, CD-ROM) |
| Different place | Remote sharing (telephone, instant messaging) | Sending (mail, email) |
Overall, we found two main approaches that attempt to support digital photo sharing; through software, for example [3,6,11] or through new devices, including tangible interaction, for example [9,12,13,14,15,16,17]. These systems can all be placed inside one (or more) of the four categories of photoware [7]. PhotoChat [3] is a system which allows users on different locations to exchange and discuss digital photos real time; a clear example of remote sharing. Cueb [13] and 4Photos [17] are examples of co-present sharing which allow co-located users to explore digital photo collections together. Cueb specifically targets intimate photo sharing between parents and their teenage children and 4Photos aims at group photo sharing during dinner. Hilliges and Kirk [14] used their Photohelix tabletop interface to explore the notion of getting side-tracked; an interesting moment during photo-talk when people tend to actively engage with randomness in their narratives. They suggest that interfaces do not need to provide active support in the structuring of narratives but instead allow people to repurpose their intentions while browsing through digital photos. The Digital Photo Browser [15] uses physical souvenirs as remembering cues to start browsing through archived photos on a portable display, making it a combination of archiving and co-present sharing. The system of Nunes et al. [16] uses a similar approach. Hoven and Eggen [18] present a complete overview of augmented memory systems that can help people to recollect memories by browsing through photos or other types of media.
One of the above-mentioned systems not only uses tangible interaction as a paradigm, but also, in retrospect, the newly-identified tangible gesture interaction [2]. Hoven and Mazalek define tangible gesture interaction as “the use of physical devices for facilitating, supporting, enhancing, or tracking gestures people make for digital interaction purposes” while meeting tangible interaction criteria, including the ones stated by Ullmer and Ishii [19]. We could argue that Cueb [13], which allows for different physical hand movements to interact with the photo cubes, is an example of tangible gesture interaction. The prototype described in this paper, also makes use of a tangible gesture.
Summarizing, a considerable amount of research addresses how people share printed photos and several physical systems have been made to support the sharing of digital photos. However, research on how people currently share digital photos on their notebooks has remained fairly limited, which is what this paper will focus on.
3. Exploring Current Digital Photo Sharing Practices
Different (prototype) systems have been created to support the sharing of digital photos. It is assumed that these solutions improve the overall sharing experience, often based on how people share non-digital photographs [7]. However, sharing digital photos on a notebook is still a common activity and the question arises what actually is the current digital photo sharing situation when using a personal computer?
To understand the way people currently share personal digital photos, amateur digital photo camera users were observed during digital photo sharing sessions and interviewed afterwards. A distinction was made between storytelling and reminiscing talk sessions based on the findings from Frohlich et al. [7] and Lindley and Monk [10]. The goal of this analysis was to retrieve a rich amount of qualitative information about how people currently share digital photos on personal computers and to find opportunities for improvement (without knowing in advance what we would find). A confirmation of the knowledge was already generated by other studies (such as [7] and [10]), thus, we also hoped to find new topics that could use further investigation and serve as a starting point for this study.
3.1. Participants
For each session we invited one photo owner and one visitor (who knew each other well). In our case the photo owner is the person who owns the digital photos and owns the personal computer used to display the photos. The visitor is the person who the photo owner will share his or her photos with. We organized 10 sessions with 20 participants in total. The sample consisted of 13 females and seven males, ranging from 19 to 52 years of age.
3.2. Procedure
Each session was conducted in the home of the photo owner on his or her notebook or desktop PC. The photo owners were told to share personal photos of their choice with the visitor in the way they would normally do this. No time limit was provided; the participants could take as much time as they needed to browse through the photos from a single event. In total five sessions concerned storytelling photo sets and five sessions concerned reminiscing talk photo sets. Before each session some general questions about digital photo sharing were asked. During the session observations included communication between participants, the interactions with the system, the tools used to view the photos and the actions that were carried out during the session (rotating, zooming etc.). After the observations a short interview was conducted were the photo owner and visitor could answer and discuss questions together. Questions were asked about involvement, control, likes, and dislikes about the current way of sharing digital photographs and how they thought it compared to the sharing of printed photographs. There were no signs of people being hesitant to comment on subjects like control with the other person beside them (probably because the participants all knew each other well and the interviews were conducted in the comfort of a familiar environment). In general the observations and interviews were kept open in order to discover as many facets of digital photo sharing as possible.
4. Exploration Results
Each session lasted around five minutes; the average time it took to browse through all the photos from a single event (without the interviews). Combined, the interviews and observations provided a large amount of interesting data. The main results from this exploratory exercise were grouped into the following recurring themes.
4.1. Digital Photo Sharing: What, When, Why and with Whom?
A lot of similarities were found between participants. All of them indicated that they shared digital photos most often with family or friends. The main goal of this sharing was either to communicate experiences to others (storytelling) or to recollect memories together (reminiscing talk). 18 out of 20 participants indicated that for them the most common goal was storytelling. This was most often done after people had returned from a holiday or other special event and they wanted to share their experiences with others or others requested to see the photos. Reminiscing talk often concerned older sets of photos and required a trigger for people to go and recollect memories. For example, talking about a previous holiday or event could stimulate them to start browsing through older sets of photos. With storytelling the fact that you have been on a holiday or to an event was enough to start sharing photos and therefore occurred more often.
4.2. Inequity of Participation
One of the big advantages of sharing physical photo albums is that there is a low threshold for social interaction. The photo owner often gives the physical album to the visitor so that he or she can browse through it while the owner tells a story or gives comments [8]. For example, in case of a storytelling session, the visitor controls the “interface” and the photo owner tells the story. In case of a reminiscing talk session both participants are interacting with the “interface” and contributing to the story in an equally active manner [7]. In both conditions the participants are actively involved in the sessions [7], indicating equity of participation. The sharing of digital photos however is different. Eight participants mentioned that they would not take place behind the personal computer of someone else and browse through photos, even if they would know where to find them (some indicated it to be an invasion of the photo owner’s private space). There was only one input mechanism possible at the same time (mouse or keyboard) and they thought it was rude to use these controls if they did not own the personal computer. This meant the photo owner would control the device (start the session and browse through the photos) while the visitor acted much more passive, indicating inequity of participation and resulting in a less enjoyable experience. These findings differ slightly with the findings from Lindley and Monk [10] where it was stated that the photo owner was probably reluctant to give away control, whereas our findings indicate that there is a high threshold for visitors to take control over someone else’s personal computer, even if this was allowed by the photo owner. As mentioned by Marshall et al. [20], equity of participation is often considered to be a desirable state. Especially if certain participants dominate during an activity, there is a risk that relevant information is not shared properly. They concluded that interactive participation on tabletop interfaces is more equal with touch input and multiple entry points than with mice or single input, but verbal participation is not.
4.3. Being in Control
In eight out of ten sessions people indicated that there was a clear separation of photo owner and visitor, stating that one person was clearly in control during the session. This was the case in both storytelling and reminiscing talk sessions, resulting in inequity of participation while sharing digital photos on a notebook or desktop computer. Despite this inequity, in most sessions (four storytelling sessions and four reminiscing talk sessions) the photo selection was adapted by the photo owner to the visitor’s preferences or assumed preferences, confirming [7]. Not all the visitors (four out of ten sessions) indicated that they would like to have more control over the session. This also had to do with the way the photos were shown. If the photo owner used a slide show to show the photos, then the potential influence was already limited. If the photo owner was more active by going through the photos one by one, more people indicated that they would like to have some way of giving input in to the session, for example by looking at a photo longer or going back to a previous one. This last way of actively browsing photos (opening the first photo of a set of photos and browse through them one by one using the back and forward keys) was the most common way of sharing photos (eight out of ten sessions). Additional interactions with the software (such as zooming) were rarely used during the sessions.
Factors that can have an influence on the distribution of control include the visitor’s attitude towards the session (e.g., a willingness to be more active or passive), the purpose of the session (e.g., for fun or searching for a particular photograph), the location and atmosphere of the room (e.g., a formal or informal environment), the content of the pictures (e.g., photos concerning reminiscing talk or storytelling events), the design and layout of the interface (e.g., the location and design of the mouse, track pad and keyboard) and the personalities of the participants (e.g., a dominant or reserved person). Some of these factors are predetermined or specific to a particular situation. Others are determined at the start of a session and can be of great influence during the rest of the session. For example, we noticed that the photo owner always took place on the seat directly in front of the personal computer, hereby gaining easy access to the available input mechanisms (mouse and keyboard). The visitor always sat on an additional seat left or right from the central seating position which made it harder to seize control of the device (and correspondingly the digital photos stored on it). Control or procedures were never discussed. During the sharing sessions factors such as personality and attitude became more relevant. However, decisions taken at the start of a session were not adjusted. The initialization of a sharing session is therefore of great importance as it influences the distribution of control for the remainder of the session.
4.4. Being Comfortable
Eleven participants indicated that they preferred the sharing of physical photos over digital photos because this allowed them to take a comfortable sharing position (on a couch for example). When sharing photos behind a computer you often have to sit on uncomfortable chairs or remain standing. Moreover the screen’s environment and display characteristics such as resolution, brightness and viewing angle influence the way photos are perceived. This viewing distance is often relatively large and the screen cannot easily be picked up to have a closer look (assuming the photo owner owns a desktop PC and not a notebook). This made viewing digital photos less comfortable than viewing printed photos, a conclusion which was also drawn by [10].
4.5. Storytelling vs. Reminiscing Talk during Digital Photo Sharing
Although there might be a clear difference between storytelling and reminiscing talk when sharing printed photos [7], storytelling and reminiscing talk when sharing digital photos turned out to be much more similar. During reminiscing talk both the photo owner and the visitor were talking a lot, discussing the photos and the experiences relating to them. This made reminiscing talk more equal than storytelling in terms of communication between participants. With storytelling, the photo owner did most of the talking and the visitor provided some occasional input. However, in both conditions the control over the personal computer (and, thus, the control over the digital photos) was still in the hands of the photo owner. In contrast to the sharing of printed photos the visitor did not have any direct influence on the session when sharing digital photos. Lindley and Monk [10] also concluded that the distribution of control can have a significant influence on the way photos are shared.
5. Research Question and Hypothesis
The overall aim of this study was to find out how we could support co-present digital photo sharing on a personal computer. During our initial exploration of current digital photo sharing practices we discovered several interesting aspects that could serve as a starting point for our scientific experiment. To us, the most interesting aspect was the observation that visitors experienced a high threshold to take control of the notebook or desktop PC of the photo owner (and correspondingly the digital photos stored on these devices), resulting in inequity of participation. Therefore, we asked ourselves the following question: Can we support existing digital photo sharing practices on a personal computer (during both storytelling and reminiscing talk conditions) while enabling more equitable participation?
Despite the fact that not all visitors in our initial exploration had the desire to have more control during a sharing session, we hypothesized digital photo sharing would benefit from a more democratic distribution of control. We reasoned that visitors would have the opportunity to interact with the personal computer more freely if the threshold for providing input was lowered by distributing the user interface and creating a more public, instead of personal, interaction space around the device. In turn, this could make them feel more involved and in control during a session, creating a more enjoyable experience. Additionally, we assumed that the photo owner would also enjoy a photo sharing session more when the visitor is actively involved. We also expected storytelling sessions to benefit more from an equal distribution of control compared to reminiscing talk sessions. To test these assumptions a design prototype was created that could be used on the table together with a notebook.
6. Design Concept
In this section we will describe the design concept that was created based on the observed inequity of participation.
6.1. Approach
The most interesting result from the our initial exploration of digital photo sharing was the fact that there was a high threshold for visitors to take control over the personal computer that belonged to the photo owner, resulting in a large amount of control for the photo owner. Therefore we decided to design a tangible user interface that could lower the threshold for user-system interaction by making the interaction space around a notebook public instead of personal. The idea was that the visitor does not feel like he or she is invading the private space of someone’s personal computer, making it possible to interact with the system more freely. This would also make them feel more involved and in control during a session, just like the photo owner. Tangible objects were chosen because they can be spread around, giving each participant the opportunity to interact freely with the system without the risk of invading somebody else’s private space. Additionally, tangible objects have a high affordance to be touched, which in turn should lower the threshold to use them. It should be possible for both participants to interact with the system in a way in which they both have an equal amount of influence and opportunities during a session. For interaction with the tangible objects we considered using tangible gesture interaction, an innovative interaction style we wanted to explore. The focus of the design would not be to test tangible interaction, but to support collaboration through an innovative interface.
We choose to work out a concept where multiple artifacts have to be used cooperatively. The next photo is only shown if all the participants agree with this, eliminating situations like “Could you go back?” or “What was displayed on that last photo? You were going past it too quickly”. It is also a way to guarantee that all the participants are actively involved in the session and because the input from the visitor is needed, he or she might feel less like an invader of the photo owner’s private space. Each participant has a separate artifact to control a digital photo sharing application. However, no single artifact is completely in control. Therefore, none of the participants can have a direct influence on the session, but are equally incapable. These kinds of interactions are referred to as cooperative gestures [21]. Cooperative gestures are interactions where the system interprets the gestures of more than one user as a contribution to a single, combined command. Cooperative gestures can be used to enhance users’ sense of teamwork, increase awareness of important system events, facilitate access and control on shared displays, or add a unique touch to an entertainment oriented activity [21]. We, therefore, believe it is a good way to increase involvement and enjoyment during a digital photo sharing session and to distribute control, hereby hopefully reaching equity of participation. This approach has never been implemented in a photo sharing context before although it has with photo manipulation, for example, as described by Morris et al. [21]. It also differs from the approach used in studies by Lindley and Monk [10] and Marshall et al. [20] where one participant could have complete control over the session and no cooperation was required.
6.2. Final Design
During the initial observations it was noticed that the most commonly used interaction with a portable or desktop personal computer was to open the first photo of a set of photos and then browse through them using the back and forward arrow keys; a simple but effective and fast way of looking through photos from a particular event (other control options such as zooming were rarely used). Therefore it was decided to copy these simple back and forward commands to the design concept as basic tangible gesture interactions so the current, existing situation could easily be compared with the newly designed situation. Two balancing artifacts (or tumblers) were created out of plywood (see Figure 1) and lead weights were added to prevent them from falling over. The idea of balancing artifacts came from the fact that we were also trying to create a balance between the photo owner and the visitor in terms of the interactions they have with a notebook while browsing through digital photographs. The artifacts can be tilted or gestured to the left or right (limited by their shape) and when released they will automatically return to their upright position, which allows for the user to go of the object whenever he wants. Due to this playful interaction, the physical artifacts have a large affordance to be used, which should also lower the threshold to use them. These artifacts will, from now on, be referred to as collaborative artifacts.
Figure 1.
The collaborative artifacts that were used by the participants to browse through the photos in a cooperative manner.
Figure 1.
The collaborative artifacts that were used by the participants to browse through the photos in a cooperative manner.
Tilting an artifact to the right indicates that you want to go to the next photo, which will scroll on-screen coming from the right. Tilting it to the left indicates that you want to go the previous photo (see Figure 2). Holding the artifact upright indicates you want to keep looking at the current photo. The principle of additivity as described by Morris et al. [21] was used to browse through the photos. Additivity means that an action carried out by a single user is only slightly meaningful, but this meaning is amplified when simultaneously performed by other users. In our application this means that if a single user wants to go the next photo by tilting his artifact, the current photo will only move slightly to the left. It is only when the other user also wants to go the next photo by tilting his artifact as well that the complete command is executed in which case the current photo scrolls away and is replaced with the next one. Individual commands therefore have no direct influence on the photo on display as you can only indicate that you want to move forward or backward, but it shows your intention to the other photo observer. Both artifacts need to be positioned upright again before the next photo can be selected to move into view, hereby preventing the photos from scrolling too fast. Phidget accelerometers were used to measure the tilting motion and send the data directly to a notebook (through a USB cable) running an Adobe Flash application.
Figure 2.
Tilt the collaborative artifacts to go to the previous or next photo.
We decided to work with real photo collections of the individual participants in order to approach a real-life situation. Therefore the photo sharing application was designed to support this by loading in photos dynamically. The application also shows reflections of the photos on the bottom of the screen and smooth transitions when scrolling from one photo to another. The photos are loaded like a movie strip; a long scroll of photos. If you are looking at a photo you can catch a glimpse of the previous and next photo displayed on the edge of the screen (see Figure 3).
Figure 3.
A screenshot of the GUI that was used by the participants to share personal digital photos.
Figure 3.
A screenshot of the GUI that was used by the participants to share personal digital photos.
Another version of the Adobe Flash application was created with only one modification, namely the arrow keys were used to browse through the photos, making it possible to compare it to the condition with the collaborative artifacts. The application was programmed in a way that allowed for quick switching between the regular and collaborative setup and the different sets of photos required for the storytelling and reminiscing talk conditions.
7. Dyad Experiment
In this section we will describe a dyad experiment which was done to find out whether we have been able to make the sharing of digital photos on a notebook more equitable through the use of collaborative artifacts (as described in the previous section) and if this supports the sharing process.
7.1. Experiment Design
In total, four different types of sessions were compared, indicated by Table 2. The regular sessions utilized the regular arrow keys on the keyboard to browse through the photos. The collaborative sessions utilized the collaborative artifacts to browse through the photos. Each couple of participants (one photo owner and one visitor) participated in these four sessions. The order of the sessions was counterbalanced throughout the study. During each session involvement, control, smoothness, ease, enjoyment, initiative, and time were measured and compared with other relevant situations. Lindley and Monk [10] found no relations between verbal behavior and enjoyment. Therefore it was decided not to measure the verbal communication between participants, but focus more on the user-system interaction. We expected to see an increase in the feeling of involvement and the feeling of control for the visitor in the collaborative conditions, hereby creating a more enjoyable experience. We also assumed that if control is shared the next photo is only shown if both participants are ready for this, possibly resulting in a smoother conversation. This could also help to make sharing digital photos easier. Lastly, we expected the photo owner to enjoy the session more when the visitor is actively involved, even if this would mean less control for the photo owner. The initiative taken by participants was measured to see how absolute control would compare to the perceived feeling of control and time was measured to find out how the collaborative artifacts would influence the duration of storytelling and reminiscing talk sessions.
| Regular Storytelling | Regular Reminiscing Talk |
|---|---|
| Collaborative Storytelling | Collaborative Reminiscing Talk |
The setup for this analysis is similar to [10], where important elements like enjoyment, equality and involvement were measured through conversation analysis, questionnaires and interviews. In our study questionnaires were used that consisted of the following five items (based on the questionnaires used by [10]) that participants had to rate on a five point Likert scale:
- -
- I felt involved in the session
- -
- I felt in control during the session
- -
- The session felt smooth
- -
- It was easy to share photos
- -
- Sharing the photos was enjoyable
After each session every participant had to fill out this questionnaire individually. The Likert scale was chosen because it allowed the experimenter to easily compare the data from each participant between different conditions. During each session it was noted which participant (photo owner or visitor) took the initiative that lead to another photo being displayed. In the regular sharing sessions this initiative could either be spoken words or direct control by pushing a button. In the collaborative sharing sessions the initiative was measured by looking at which person tilted their artifact first (if that would lead to another photo being displayed) or spoken words.
7.2. Participants
The sample of participants consisted of 20 students (17 men and three women) from the Eindhoven University of Technology with an average age of 23.5 (ranging from 22 to 26 years). On average each participant estimated to have over 5000 (ranging from 750 to over 10000) personal digital photos stored on his or her personal computer and had been taking digital photos for over five and a half years. Each dyad consisted of one photo owner and one visitor, who knew each other because one requirement was that the photo owner should have at least 50 photos of an event they both visited to be used as input for the reminiscing talk conditions. In addition the photo owner also needed to have 50 or more photos, of an event he or she visited without the visitor, to show to the other person in the storytelling conditions. None of the participants participated both as photo owner and visitor to approach a real-life photo-sharing situation.
7.3. Procedure
The four sessions (regular storytelling, regular reminiscing talk, collaborative storytelling, and collaborative reminiscing talk) each consisted of a different set of 25 photos that the participants had to browse through. The photos used in the storytelling sessions all came from the same event, meaning 50 photos from the same event were shared. The same applies to the reminiscing talk sessions. In total 100 unique photos were shared during each experiment, consisting of four sessions of 25 photos each. Participants were told to share the photos in the same way as they would normally do this and they could take as much time as they needed. For the sake of consistency visitors were always told to sit on the left side of the photo owner. No additional constraints were provided. For example, it was not indicated by the experimenter who should be in control during the sessions. After each session the participants filled in their own questionnaire. Between the sessions the photo sharing application was manipulated by the experimenter to make sure the correct photos were loaded into the application for the right sessions (because the sessions were counterbalanced throughout the study). After finishing all four sessions participants were rewarded with a snack. A pilot test proved both the procedure and application worked properly.
7.4. Materials
All the sessions were done in a private room on campus. Two tables were present in the room; one table was used by the experimenter and the other table was used by the participants to share their photos on an HP Compaq 8510w notebook running the photo sharing application. The notebook had the required software installed to interface between the application and the Phidget accelerometers that were inside the collaborative artifacts. All the personal photos the participants supplied were stored on the notebook so they could be dynamically loaded into the application when necessary. Two chairs were placed in front of the notebook; the photo owner would be sitting in the right chair and the visitor in the left chair (see Figure 4). The observer sat behind the other table to monitor the actions taken by the participants. A Canon EOS 350D digital camera was placed on a tripod to take occasional photos and a Canon Hv20 HD video camera was used in some of the sessions to record video for later reviewing. A stopwatch was used to record times for each session.
Figure 4.
The dyad experiment setup.
7.5. Data Analysis
The gathered data were analyzed using four different methods. The answers from the questionnaires consisted of numbers between one and five on a non-parametric scale. To see if differences between conditions within subjects were significant (for example a significant difference between regular storytelling and collaborative storytelling for the visitor) a Wilcoxon Signed Rank Test was used. To see whether differences between conditions between subjects were significant (for example a significant difference between regular storytelling for the visitor and regular storytelling for the photo owner) a Mann-Whitney U Test was used. For each question this resulted in eight different comparisons, four being within subjects and four being between subjects. The numbers of observed initiatives were numbers on a parametric scale. They were analyzed in the same way as the questionnaires, only because of the parametric instead of non-parametric scale, the Wilcoxon Signed Rank Test was replaced with a paired-samples t-test and the Mann-Whitney U Test was replaced with an independent samples t-test. An overview of all the methods used can be found in Table 3.
Table 3.
Test methods used for comparing different conditions. The Wilcoxon Signed Rank Test and the Mann-Whitney U Test were used for the non-parametric data (the questionnaires), while the paired-samples t-test and the independent samples t-test were used for parametric data (the observed initiatives).
| Wilcoxon/Paired T | Mann-Whitney/Independent T | |
|---|---|---|
| Regular Storytelling vs. Collaborative Storytelling Photo owner | x | |
| Regular Storytelling vs. Collaborative Storytelling Visitor | x | |
| Regular Reminiscing vs. Collaborative Reminiscing Photo owner | x | |
| Regular Reminiscing vs. Collaborative Reminiscing Visitor | x | |
| Regular Storytelling Photo owner vs. Regular Storytelling Visitor | x | |
| Collaborative Storytelling Photo owner vs. Collaborative Storytelling Visitor | x | |
| Regular Reminiscing Photo owner vs. Regular Reminiscing Visitor | x | |
| Collaborative Reminiscing Photo owner vs. Collaborative Reminiscing Visitor | x |
8. Dyad Experiment Results
During each session the time taken to view and talk about the 25 photos was recorded with a stopwatch. The average time of a session was 205 s (three minutes and 25 s). The storytelling sessions took 217 s on average whereas the reminiscing talk sessions took 194 s. Overall the storytelling sessions took longer than the reminiscing talk sessions in both the regular and collaborative conditions.
Involvement, control, smoothness, ease, and enjoyment were all rated on a five point Likert scale. As can be seen in Figure 5a, the collaborative sessions made participants feel more involved than the regular sessions. The collaborative sessions scored higher in both storytelling and reminiscing talk conditions, but especially in the storytelling conditions involvement was rated much higher. It turned out to be significant for both the photo owner and the visitor (µphoto owner-regular-storytelling = 4, µphoto owner-collaborative-storytelling = 4.6, p < 0.05, µvisitor-regular-storytelling = 3.3, µvisitor-collaborative-storytelling = 4.4, p < 0.05). Especially, the involvement of the visitor increased significantly. The differences between reminiscing talk sessions did not reach significant levels. The participants indicated that for the control condition they already felt more involved in the reminiscing talk sessions compared to the storytelling sessions (µphoto owner-regular-reminiscing = 4.2, µphoto owner-regular-storytelling = 4, µvisitor-regular-reminiscing = 4.3, µvisitor-regular-storytelling = 3.3). The differences in involvement between photo owner and visitor decreased when using the collaborative artifacts, even though the differences were not significant to begin with. The involvement was more equally balanced between participants in the collaborative sessions, especially in the storytelling case.
Figure 5.
The graphs show the average scores on a particular question from the dyad experiment. The vertical axes of the graphs (a–e) represent scores on a five point Likert scale. Graph (f) shows the number of initiatives taken by the photo owner and visitor during different conditions. The different conditions are indicated by the legend on the bottom of this figure. Standard errors are indicated for each condition.
Figure 5.
The graphs show the average scores on a particular question from the dyad experiment. The vertical axes of the graphs (a–e) represent scores on a five point Likert scale. Graph (f) shows the number of initiatives taken by the photo owner and visitor during different conditions. The different conditions are indicated by the legend on the bottom of this figure. Standard errors are indicated for each condition.
Figure 5b shows the ratings for the feeling of control. The photo owner felt less in control during the collaborative conditions compared to the regular conditions (during both storytelling and reminiscing talk). On the other hand, the feeling of control from the visitor was higher in the collaborative conditions (during both storytelling and reminiscing talk). In the collaborative conditions the feeling of control was more equally balanced among the participants. In the regular storytelling condition there was a significant difference between the visitor and the photo owner (µvisitor-regular-storytelling = 1.9, µphoto owner-regular-storytelling = 4.6, p < 0.01), which disappeared in the collaborative condition. The differences between the regular and collaborative storytelling conditions for the visitor were also significant (µvisitor-regular-storytelling = 1.9, µvisitor-collaborative-storytelling = 3.9, p < 0.01), showing a large increase in the feeling of control. It should be noted that in some of the regular reminiscing talk sessions (three out of 10), the control was given almost completely to the visitor. This meant that there was no significant difference between the photo owner and the visitor in the regular reminiscing talk sessions. Although the averages seem relatively similar, control in the individual sessions was still unequally distributed where one person had almost full control. When discarding these “abnormal” sessions the differences between photo owner and visitor, in the regular reminiscing talk sessions, stay significant.
As can be seen in Figure 5c, the results did not reveal any significant differences in smoothness within or between subjects. Even though the differences between regular storytelling and collaborative storytelling for the photo owner almost reached significant levels, it was not completely clear what this would mean for the overall picture. In all collaborative sessions the feeling of smoothness increased without reaching significant levels.
The results as can be seen in Figure 5d did not show any significant differences in ease (did the interface allow for easy browsing through the photos) within or between subjects. In most cases the ease decreased in the collaborative sessions, although without reaching any significant levels. The ease of sharing photos was more equally balanced among the participants in most of the collaborative sessions compared to the regular sessions. Especially during collaborative storytelling conditions the ease of sharing photos was equally balanced among participants.
Figure 5e shows that enjoyment was rated higher in all the collaborative sessions compared to the regular sessions. Especially with storytelling the differences were large, reaching significant levels for the photo owner (µphoto owner-regular-storytelling = 3.6, µphoto owner-collaborative-storytelling = 4.4, p < 0.05). This indicates that the photo owner preferred it when the visitor was actively involved in the session. The differences for reminiscing talk sessions were smaller and did not reach significant levels. By using the collaborative artifacts the level of enjoyment during storytelling sessions was brought closer to that of reminiscing talk sessions for both photo owners and visitors. The differences in enjoyment between photo owner and visitor were small in both the regular and collaborative conditions.
During each session the experimenter noted which participant took the initiative to go to the next or previous photo. Figure 5f shows that the initiative in the regular sessions was completely dominated by the photo owner, resulting in a significant difference between the photo owner and the visitor in the regular storytelling conditions (µphoto owner-regular-storytelling = 28.9, µvisitor-regular-storytelling = 0.2, p < 0.01). It should again be noted that in some regular reminiscing talk sessions (three out of 10) the control was purposely given to the visitor by the photo owner. This meant that the visitor took a lot more initiative than would normally be the case, making it seem (on average) like the visitors took a reasonable number of initiatives in the regular reminiscing talk sessions. The initiatives in the individual sessions however were still unequally distributed where one person took almost all of the initiative and no real collaboration was seen. If the “abnormal” sessions were to be discarded, the differences between the photo owner and the visitor would also be significant for the regular reminiscing talk sessions. The initiative was much more equally balanced in the collaborative sessions, although the photo owner still had a significant advantage in the collaborative storytelling sessions (µphoto owner-collaborative-storytelling = 16, µvisitor-collaborative-storytelling = 11.2, p < 0.01). This was probably due to the fact that the photo owner knew which pictures were present in a particular photo set and wanted to drive the session based on their content. The initiative in the collaborative reminiscing talk conditions was almost the same for the photo owner and the visitor, indicating equity of participation.
To help interpret the results, we suggest looking at Figure 5 which gives a visual overview of all the different variables. An overview of all the significant results can be found in Table 4.
An overall observation during the experiment was that visitors were hesitant to take away control from the photo owner during regular sharing sessions. The photo owners always took control during the first regular sharing session (without verbal consent from the visitor), even though this was not indicated by the experimenter. At the beginning of the second regular sharing session control was sometimes discussed and occasionally handed over to the visitor. If only one event would have been shared this verbal discussion would not have taken place, resulting in full control for the photo owner during regular sharing conditions. We also noticed visitors were still asking questions like “Could you go back?” during regular conditions, whereas this was replaced by a gesture with the artifact during the collaborative conditions (meanwhile the conversation about the pictures themselves continued).
After the experiment, many participants indicated the collaborative artifacts were fun and easy to use due to their basic functionality and playful interaction characteristics. Many participants started playing with the artifacts, indicating the design had a high affordance to be used.
| p | Sign. | |
|---|---|---|
| Involvement | ||
| Photo Owner Regular Storytelling vs. Photo Owner Collaborative Storytelling | 0.034 | < 0.05 |
| Visitor Regular Storytelling vs. Visitor Collaborative Storytelling | 0.026 | < 0.05 |
| Control | ||
| Visitor Regular Storytelling vs. Visitor Collaborative Storytelling | 0.007 | < 0.01 |
| Photo Owner Regular Storytelling vs. Visitor Regular Storytelling | 0.000 | < 0.01 |
| Enjoyment | ||
| Photo Owner Regular Storytelling vs. Photo Owner Collaborative Storytelling | 0.011 | < 0.05 |
| Initiative | ||
| Photo Owner Regular Storytelling vs. Photo Owner Collaborative Storytelling | 0.000 | < 0.01 |
| Visitor Regular Storytelling vs. Visitor Collaborative Storytelling | 0.000 | < 0.01 |
| Photo Owner Regular Storytelling vs. Visitor Regular Storytelling | 0.000 | < 0.01 |
| Photo Owner Collaborative Storytelling vs. Visitor Collaborative Storytelling | 0.010 | < 0.01 |
9. Triad Experiment
To find out how generalizable the results were from our experiment with two participants per session, a similar experiment was conducted with three participants per session, one photo owner and two visitors. The goal was to observe how larger groups of three participants would interact with two collaborative artifacts and to find out what effect this would have on the photo sharing session.
9.1. Method
The experiment, procedure, and analysis were copied from our dyad experiment as described in Section 7. In this triad experiment, one photo owner and two visitors shared their personal photographs from two different events (50 photos from each event, replicating storytelling and reminiscing talk conditions) using arrow keys in the regular conditions and two collaborative artifacts in the collaborative conditions. Like in the dyad experiment, the visitors would always be sitting on the left side of the photo owner. The visitor who sat next to the photo owner (in the middle) will from now on be referred to as visitor 1 and the visitor who sat on the outer left will from now on be referred to as visitor 2. No additional constraints were provided besides an indication of the seating positions. The participants were told to share the digital photographs in the same way as they would do normally and fill in an evaluation form (the same form as used in the dyad experiment) after each sharing session.
9.2. Participants
A total of seven experiments were conducted with three participants per session, giving a total of 21 participants (five men and 16 women) with an average age of 23.33 (ranging from 19 to 27 years). The participants were all students (or recently graduated) with different educational backgrounds (having at least a Bachelor degree). On average each participant estimated to have over 4400 (ranging from 300 to over 10,000) personal digital photos stored on his or her personal computer and had been taking digital photos for over seven years. The experiments were conducted in the home of the photo owner using the notebook of the experimenter, with the same photo sharing application as used in the dyad experiment.
10. Triad Experiment Results
During each session the time taken to view and talk about the 25 photos was recorded with a stopwatch. The average time of a session was 277 s (4 min and 37 s). The storytelling sessions took 268 s on average whereas the reminiscing talk sessions took 287 s. In contrast to the dyad experiment, the reminiscing talk sessions took longer than the storytelling sessions in collaborative conditions. We suspect this was caused by visitors being more interested in photos concerning reminiscing talk compared to photos from storytelling events. Because visitors had the opportunity to give input during collaborative conditions they could (partly) control how long photos would be displayed, favoring those from reminiscing talk conditions (perhaps also to involve the other visitor). The average time taken to go through all the photos during reminiscing talk conditions was significantly higher in the triad experiment compared to the dyad experiment (µdyad-reminiscing = 194, µtriad-reminiscing = 287, p < 0.05), suggesting that more participants equals more reminiscing photo talk.
The data from the questionnaires proved to be difficult to analyze statistically. Due to the fact that participants participated in several different, but successive (counter-balanced), photo sharing sessions, control was often (20 out of 28 sessions) handed over to the person who had the least amount of influence during the sessions so far. According to participants this was to keep things fair (for example, they used phrases such as “Why don’t you do it now, I have already done it” and “Don’t you want to have a go now?”). As mentioned in Section 8, this effect would normally not occur during a single sharing session. The phenomenon was a lot stronger in the triad experiment (which might have been influenced by a difference in gender balance compared to the dyad experiment) and affected all the different conditions (regular and collaborative, storytelling and reminiscing talk). This made it difficult to draw statistically-meaningful conclusions from the data. Patterns of control and initiative constantly fluctuated between participants in the different conditions due to the counter balanced order in which the sessions were conducted (e.g., starting with regular storytelling in one experiment and starting with collaborative reminiscing in another experiment). In other words, if a particular condition was placed at the beginning of an experiment, control and initiative were handled differently than when it was placed at the end of the experiment (because of the participants’ desire to keep things fair). Especially the ratings for control and initiative were therefore unsuitable for statistical analysis. Unfortunately, this also meant we could not make a clear statistical comparison between the dyad and the triad photo sharing experiment.
For example, the overall feeling of enjoyment for all participants seemed to go up in collaborative conditions (µregular = 4.0, µcollaborative = 4.2), as did the ratings for smoothness (µregular = 3.9, µcollaborative = 4.0) and involvement (µregular = 4.1, µcollaborative = 4.2) compared to the regular conditions. However, the ease of sharing photos seemed to go down in collaborative conditions (µregular = 4.2, µcollaborative = 4.1). Reminiscing talk sessions were considered to be more involving (µstorytelling = 4.0, µreminiscing = 4.3), smooth (µstorytelling = 3.9, µreminiscing = 4.0) and enjoyable (µstorytelling = 3.9, µreminiscing = 4.4) compared to storytelling, which also corresponds with our findings from the dyad experiment. However, many of these ratings did not reach significant levels (due to a large standard error).
For visitor 2 (sitting furthest from the photo owner) the feeling of control (µvisitor2-regular-reminiscing = 2.4, µvisitor2-collaborative-reminiscing = 4.2) and amount of initiatives taken (µvisitor2-regular-reminiscing = 0.2, µvisitor2-collaborative-reminiscing = 14.1) did increase significantly in the collaborative reminiscing talk conditions (p < 0.05 and p < 0.01 respectively) compared to the regular reminiscing condition.
Despite the lack of many quantitative results, we can provide additional qualitative information resulting from the observations made during each session. In collaborative conditions we used two collaborative artifacts instead of three. We anticipated the possibility that the two visitors would start to use a single artifact together (for example by handing it around). This did however not occur, resulting in two persons being in control instead of three (although this was an improvement compared to one person being in control). The collaborative artifacts were most often (10 times out of 14 sessions) used cooperatively by the photo owner and visitor 2. We believe this is due to the seating positions in combination with the positioning of the collaborative artifacts. The artifacts were placed on the front left and front right side of the notebook used in the experiment where they were closest to the photo owner and visitor 2 (although also easily accessible by visitor 1). We noticed visitor 1 would often take an intermediary role in the session by trying to synchronize the movements made with the artifacts by the other participants (for example with phrases such as “1, 2, 3…Go!” or “Next!”). Despite having no direct input device this might have helped to make the person without an artifact feel more involved and in control during a sharing session (though this could not be statistically proven during this study).
An interesting difference between the dyad and triad experiments was that control was more often discussed in the experiment with three participants per session, in particular during collaborative conditions. In regular conditions, when only one input device was available, control was always (with only one exception) seized by a single participant and not handed over during the session. Like in the dyad experiment it was always the photo owner who took control during the first of the regular sharing sessions (control in the second regular sharing session was sometimes discussed and/or handed over to “keep things fair”). In the collaborative conditions the participants often discussed who should be using the artifacts based on personal preferences and accessibility (the latter seemingly being the most important aspect). It suggests that having multiple input devices can create opportunities for the discussion of control (for example, allowing people who desire control to come forward), whereas this is often not made explicit when only a single input device is used. Seating positions and their relative distances to input devices seemed to be an important factor in establishing equal photo sharing sessions for larger groups as the feeling of control appeared to be directly related to the distance towards an input device. In the regular conditions the feeling of control was highest for the photo owner (closest to the arrow keys, µ = 4.1), then visitor 1 (next closest to the arrow keys, µ = 3.7) and then visitor 2 (furthest away from the arrow keys, µ = 2.4), resulting in a significant difference between the photo owner and visitor 2 (p < 0.05) in both storytelling and reminiscing talk conditions. In the collaborative conditions the feeling of control was highest for both the photo owner and visitor 2 (closest to the collaborative artifacts, µ = 3.8) and lowest for visitor 1 (furthest away from the collaborative artifacts, µ = 3.1), resulting in a significant difference between the photo owner/visitor 2 and visitor 1 in storytelling conditions (p < 0.05) and visitor 1 and visitor 2 in reminiscing talk conditions (p < 0.05). The same pattern also emerged in the feeling of involvement, enjoyment and initiatives taken, although without reaching significant levels.
11. Discussion
11.1. Reflections on the study
This study has shown that the use of collaborative tangible gesture artifacts can increase the feeling of control (for the visitor), involvement (for both photo owner and visitor), and enjoyment (for the photo owner) during a dyadic digital photo sharing storytelling situation and distribute the number of initiatives taken by participants. We believe that, in particular, the clear increase in initiative from the visitor adheres to more equity in participation when comparing photo owners and visitors. We did not find significant increases in the reminiscing talk conditions for the dyad experiment; however, we did in the collaborative triad experiment, where the feeling of control of visitor 2 (the visitor sitting closest to one of the two balancing artifacts) increased as well as the number of initiatives taken, but only in the reminiscing talk condition. In the regular photo sharing sessions visitor 2 had the least amount of control after the photo owner and visitor 1. This could be related to the physical distance to the keyboard. This caused bigger differences between the regular and collaborative sessions for visitor 2 and a significant increase in felt control. In the dyad experiments the visitor was sitting closer to the keyboard than visitor 2, thereby resulting in fewer differences in perceived control, therefore no significant difference. Perhaps the physical location of the visitors can explain the differences between the dyad and triad experiments, an interesting direction for further investigation.
The triad experiment turned out to be problematic in several respects. For comparison reasons we tried to stick to the dyad experiment setup as closely as possible in terms of having two collaborative artifacts and having the same seating arrangement. However this introduced some differences, such as the distance to the keyboard and/or artifacts as mentioned in the previous section. This is purely speculation, but perhaps the biggest impact was the different gender balance. It turned out to be easier to find groups of girlfriends who had shared events together in groups, therefore the triad participants were mostly female. This probably had an impact on the group dynamics, compared to the male dyads. What we learned from the triad experiment is that taking into account group dynamics, which is probably influenced by many factors including gender, the number of collaborative artifacts and seating arrangement, would make for an interesting direction for future research in photo sharing.
As indicated before, it should be noted that in some of the dyad regular reminiscing talk conditions the control was given almost completely to the visitor. This meant the overall distribution of control and initiative was affected, resulting in insignificant differences between photo owner and visitor in regular compared to collaborative conditions. However, control and initiative in each individual session were still unequally distributed, showing a clear difference between photo owner and visitor and regular and collaborative conditions. Due to the relatively small test sample this did not prove to be significant. We also did not observe this effect in our initial exploration of digital photo sharing practices, which makes us believe it was caused by the experimental setup. Due to the fact that participants participated in several different, but successive (counter-balanced), photo sharing sessions, some of the participants thought it would be fair to give control to a person who had the least amount of influence during the sessions so far. This effect was even stronger in the triad experiment. We noticed this through a verbal discussion between photo owner and visitor(s) at the beginning of some of the sessions. In everyday life, however, usually only a single event is shared during a photo sharing session which means the effect we observed in our experiments cannot occur. For future studies it could be helpful to execute these experiments without an observer in the room, or by having one session at a time.
Our initial objective with the collaborative artifacts was to increase the feeling of control, involvement, and enjoyment for visitors by providing an additional input channel during a digital photo sharing session. As expected, it turns out that the photo owner benefits as well (during dyadic sharing). Even though the photo owner’s feeling of control and the number of initiatives were lower in the collaborative conditions, he or she did feel more involved and considered the session to be more enjoyable (especially in the case of storytelling). This indicates that the photo owner prefers it if the visitor is more actively involved during a sharing session (to make sure all relevant information is communicated), even if this reduces the amount of control he or she can have on the session.
We would like to point out that some of the results from the dyad experiment are not as obvious as they might seem. One might argue that the visitor's increase in initiative in the collaborative conditions is obvious, since he or she could not control anything in the regular conditions. This is however not exactly true. In the regular conditions there was one input device available (keyboard), but the experimenter did not indicate who should be using it during the experiment. The photo owners all took control during the first regular sharing session. Moreover, this also meant that the visitor could control something in the regular conditions and take the initiative by simply reaching for the keyboard or by using speech. Apart from a few exceptions (see Figure 5f), this did not occur during the experiment. Additionally, having an input device does not automatically mean a person wants to provide input. Instead, it simply lowers the threshold to do so.
Similar to [10] our experiment utilized a standard screen (a notebook in our case) to display personal photographs of the participants. Even though the participants did not use their personal computer, the same kind of behavior emerged in terms of initiative taken by the photo owner and visitor during the dyad regular sharing sessions (compared to the results from our initial exploration of digital photo sharing practices). This indicates our experiment was capable of creating a close to real-life situation. It also suggests that the threshold to take over control visitors experienced was also due to the fact that they did not own the particular photographs. Both the photo owner and visitor did not own the notebook used in the experiment, yet the photo owner still took almost all of the initiative during the regular sharing sessions. Not owning the personal computer and invading someone else’s private space therefore only seem to be part of the reason why visitors were reluctant to take over control during the regular sharing sessions. Despite this, we have to acknowledge the fact that having an observer present throughout the experiment might have influenced the way the participants shared their personal photos, making them take particular care to be polite.
11.2. Reflections on Tangible Gesture Interaction
Even though the study presented in this paper did not aim to test individual elements of the user interface and their impact on user behavior we did make some observations that are worth investigating further to study the value of tangible gesture interaction. One aspect, which was not validated during this study, is the amount of influence the collaborative artifacts had on enjoyment compared to a regular keyboard and mouse. It is possible that the participants rated enjoyment higher in the collaborative conditions because they were using a new type of tangible interface and not because of the actual collaboration. In this respect the experiment would have produced more solid results if the collaborative artifacts would have been replaced with a double set of back and forward arrow keys, making the interactions in the collaborative conditions more similar to the interactions used in the regular conditions. However, we decided to design an innovative interaction that could facilitate different steps in the photo sharing process: indicating intentions and the actual transition from one photo to the next or previous one. We feel this would not have been so successful with arrow keys. We also reasoned that the use of arrow keys would too much resemble the current non-collaborative photo sharing situation and therefore might not have the desired effect. The artifacts allowed users to execute very basic commands without the risk of doing something inappropriate to the data stored on the computer (such as accidentally deleting a file). Additionally, the two conditions that were compared during the experiment (regular and collaborative photo sharing) had the same control options (browsing through photos by going forwards and backwards). Therefore, the collaborative artifacts did not make the process more or less rigid but added their own qualitative characteristics. Compared to keyboard and mouse, the graspable shape of the tangible artifacts and the playfulness of the gesture interaction using the tangible gestures seemed to lower the threshold to use them (for visitors), thereby facilitating cooperative gestures.
The collaborative artifacts used in this study were designed to be easily accessible and easy to use by all participants. The results show that they were no more difficult to use than the standard back and forward arrow keys while distributing control and increasing involvement. Their tangibility allowed for easy access and lowered the threshold to use them. This also resulted in a much more equally-balanced number of initiatives taken by the photo owner and visitors. The real world practicality of the designed system is debatable. However, it was not the aim of the study to find the most efficient or best-integrated photo sharing system. We believed that tangible artifacts would stimulate interaction with the system and therefore decided to create physical objects that subjects could interact with.
11.3. Reflections on Cooperative Gestures
The cooperative gestures required by both participants in the dyad collaborative conditions helped to make them feel more involved in the session. The applied principle of additivity meant that each participant could show his intentions without interrupting the session or requiring verbal consent. It was expected that this would make the photo sharing session more smooth, although no significant improvements in smoothness were found in both dyad and triad experiments. This may have something to do with the fact that browsing through photos using arrow keys is already relatively fast and easy. There might not be a difference in smoothness when comparing the two interaction styles.
The distinction between multiple individual artifacts (such as the collaborative artifacts used in this study) and a single shared artifact (for example a shared keyboard) while using cooperative gestures is an aspect which could use further research. At this point we do not know what the effects of a single shared artifact will be compared to multiple individual artifacts. Marshall et al. [20] already proved that two separate inputs will result in a higher equity of participation compared to one if cooperative gestures are not required. We also know that different social roles can develop within a photo sharing session if individuals can have full control over the session, occasionally still resulting in a leader of the session [10]. Our findings show that social roles were actually becoming more similar when using the collaborative artifacts (in the dyad experiment) as control, involvement, enjoyment and initiative were more equally balanced between photo owner and visitor, indicating equity of participation. However, during our triad experiment we noticed visitor 1 (sitting in the middle) would often take the role of mediator in collaborative conditions while the photo owner and visitor 2 would be using the collaborative artifacts. Providing each of the three participants with a collaborative artifact could again produce different results.
The use of collaborative artifacts in larger groups could also have interesting effects on a photo sharing session. In our experiment we limited the number of participants to two and three participants per session. Cooperation between four or more participants while browsing through digital photos has not yet been studied and can possibly provide new insights and present new challenges, even though finding participants for the reminiscing talk conditions will be a challenge. We expect that collaborative artifacts in larger groups will be more difficult to implement properly. Cooperation between multiple people also requires coordination which has to be provided through a central point or a leader (visitor 1 did sometimes provide this coordination in the triad experiment), but we believe a well-designed interface can provide this coordination, make collaborative artifacts available to larger groups and avoid for one of the participants to take the lead and also take control.
11.4. Reflections on Co-Present Digital Photo Sharing
In addition to the main focus of this study, we also found other interesting aspects, which resulted from our initial exploration of current digital photo sharing practices (described in Section 3 and Section 4), which turned out to correspond with the results from other studies. For example, we found that photo selections were adapted by the photo owner to the visitor's preferences or assumed preferences and that there was a clear distinction between storytelling and reminiscing talk conditions. Both aspects confirm [7]. Additionally we also found that being comfortable during a photo sharing session was considered to be important, something which was also concluded by Lindley and Monk [10]. A difference between [10] and ours was that they stated that the photo owner was probably reluctant to give away control during a photo sharing session, whereas our findings indicate that there is a high threshold for visitors to take control over someone else’s notebook or PC. It became clear to us that the distribution of control had an effect on the way photos were shared (which was also concluded by [10]) and can be seen as one of the main differences between the sharing of printed and the sharing of digital photos.New slate technologies (such as the Apple iPad) can provide new opportunities for digital photo sharing. These devices can easily be picked up and handed around to have a closer look, making them perfect for individual photo browsing. Although most slates support multiple user inputs at the same time, their size makes them unsuitable for collaboratively sharing digital photographs with groups. Besides, many people still view their digital photo collections on a personal computer, such as notebooks. Multi-touch tables (such as Microsoft Surface and some of the example systems mentioned in Section 2.2) are a more suitable solution for digital photo sharing. They allow for both individual and collaborative photo browsing and sharing. It will however take a while before such systems will find their ways into the homes of the average photo owner. Remote sharing of digital photographs through online services and social networks is becoming more popular, but we expect this will not replace co-present photo sharing. Therefore, the tangible collaborative gesture artifacts presented in this study could be used to support co-present digital photo sharing on a personal computer.
The results of this study can also be taken outside a photo-sharing context. It is possible that the conclusions drawn here also apply to other situations where control is limited to a single person due to accessibility or social reasons. Many modern products and systems are designed with a single user in mind, while in practice these devices are often used by multiple people, sometimes simultaneously. By stimulating people to collaborate, control is distributed and people feel more involved in what they are doing. This is an aspect which can, for example, be useful in situations where teamwork is important. Think of computer systems that only allow one input to be given at the same time or a TV set with a single remote control. These applications and many more can possibly benefit from collaborative interaction. Tangible interaction is an approach where collaborative interaction can be easily implemented, this also holds for collaborative tangible gesture interaction, which supports people’s natural tendency to mimic and mirror other people’s gestures.
12. Conclusions
The overall aim of this study was to find out how co-present digital photo sharing on a personal computer (both portable and desktop) could be supported through innovative tangible gesture interaction. We focused on improving equity of participation, such as taking control and initiative by both photo owner and visitor, through tangible artifacts supporting cooperative gestures. Therefore, we designed two dedicated balancing artifacts, one for the photo owner and another one for the visitor, and evaluated these artifacts in an experiment with groups of two (dyads) and three participants (triads). In our triad experiment, we noticed that participants were kind to each other by explicitly handing over control, when moving to a follow-up session, resulting in hardly any significant results. We did find that for the visitor sitting closest to one of the two balancing artifacts the feeling of control and the number of initiatives taken significantly increased in the reminiscing talk conditions. In our dyad experiment, control and initiative were much more equally balanced between visitor and photo owner, resulting in a more enjoyable and involving photo sharing experience in the collaborative storytelling sessions compared to the regular storytelling sessions. It was clear that during regular sharing sessions participants felt more involved in reminiscing talk conditions compared to storytelling conditions and considered them to be more enjoyable, confirming results of [7] and [10], but in our study with a tangible gesture user interface the differences between reminiscing talk and storytelling conditions became smaller in the collaborative sharing sessions, bringing the experience of storytelling closer to that of reminiscing talk (where the latter is considered to be more enjoyable).
Therefore we can conclude that the tangible gesture interface presented in this paper, which was specifically designed for ease of use on a horizontal surface as input for a personal computer, was successful in supporting and improving the co-present digital photo sharing experience for both owners and visitors.
Acknowledgments
The authors thank all the participants, in particular for sharing their personal digital photographs. Special thanks go to Siân Lindley and David Frohlich for their valuable input during this study.
This research was supported by STW VIDI grant number 016.128.303 of The Netherlands Organization for Scientific Research (NWO), awarded to Elise van den Hoven.
Author Contributions
First author, Elise van den Hoven, supervised second author, Tom van Bergen, during his individual design research project as part of the Industrial Design Master degree, Eindhoven University of Technology, the Netherlands. After the project had ended they co-wrote this paper.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Rodden, K.; Wood, K. How Do People Manage Their Digital Photographs? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, USA, 5–10 April 2003; pp. 409–416.
- Van den Hoven, E.; Mazalek, A. Grasping gestures: Gesturing with physical artifacts. Artif. Intell. Eng. Des. Anal. Manuf. 2011, 25, 255–271. [Google Scholar] [CrossRef]
- Sumi, Y.; Ito, J.; Nishida, T. PhotoChat: Communication Support System based on Sharing Photos and Notes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008; pp. 3237–3242.
- Cooper, M.; Foote, J.; Girgensohn, A.; Wilcox, L. Temporal event clustering for digital photo collections. In Proceedings of the IEEE International Conference on Multimedia & Expo (ICME), Baltimore, MD, USA, 6–9 July 2003; pp. 364–373.
- Cunningham, S.J.; Masoodian, M. Identifying Personal Photo Digital Library Features. In Proceedings of the 7th ACM/IEEE-CS joint conference on Digital Libraries, Vancouver, BC, Canada, 17–22 June 2007; pp. 400–401.
- Miller, A.D.; Edwards, W.K. Give and take: A study of consumer photo-sharing culture and practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 28 April–3 May 2007; pp. 347–356.
- Frohlich, D.M.; Kuchinsky, A.; Pering, C.; Don, A.; Ariss, S. Requirements for photoware. In Proceedings of the Computer-Supported Cooperative Work and Social Computing, New Orleans, LA, USA, 16–20 November 2002; pp. 166–175.
- Crabtree, A.; Rodden, T.; Mariani, J. Collaborating around collections: Informing the continued development of photoware. In Proceedings of the Computer-Supported Cooperative Work and Social Computing, Chicago, IL, USA, 6–10 November 2004; pp. 396–405.
- Frohlich, D.; Fennell, J. Sound, paper and memorabilia: Resources for a simpler digital photography. Pers. Ubiquit. Comput. 2007, 11, 107–116. [Google Scholar] [CrossRef]
- Lindley, S.; Monk, A. Social enjoyment with electronic photograph displays: Awareness and control. Int. J. Hum. Comput. Stud. 2008, 66, 587–604. [Google Scholar] [CrossRef]
- Shneiderman, B.; Kang, H. Direct annotation: A drag-and-drop strategy for labeling photos. In Proceedings of the IEEE International Conference on Information Visualization; 2000; pp. 88–95. [Google Scholar]
- Balabanović, M.; Chu, L.L.; Wolff, G.J. Storytelling with digital photographs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, London, UK, 19–21 July 2000; pp. 564–571.
- Golsteijn, C.; van den Hoven, E. Facilitating parent-teenager communication through interactive photo cubes. Pers. Ubiquit. Comput. 2013, 17, 273–286. [Google Scholar] [CrossRef]
- Hilliges, O.; Kirk, S. Getting Sidetracked: Display Design and Occasioning Photo-Talk with the Photohelix. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; pp. 1733–1736.
- Van den Hoven, E.; Eggen, B. Digital photo browsing with souvenirs. In Proceedings of the Interact 2003, Zurich, Switzerland, 1–5 September 2003; pp. 1000–1003.
- Nunes, M.; Greenberg, S.; Neustaedter, C. Using physical memorabilia as opportunities to move into collocated digital photo-sharing. Int. J. Hum. Comput. Stud. 2009, 67, 1087–1111. [Google Scholar] [CrossRef]
- O’Hara, K.; Helmes, J.; Sellen, A.; Harper, R.; ten Bhömer, M.; van den Hoven, E. Food for talk: Phototalk in the context of sharing a meal. Hum. Comput. Interact. 2012, 27, 124–150. [Google Scholar]
- Van den Hoven, E.; Eggen, B. Informing augmented memory system design through autobiographical memory theory. Pers. Ubiquit. Comput. 2008, 12, 433–443. [Google Scholar] [CrossRef]
- Ullmer, B.; Ishii, H. Emerging Frameworks for Tangible User Interfaces. IBM Syst. J. 2000, 39, 915–931. [Google Scholar] [CrossRef]
- Marshall, P.; Hornecker, E.; Morris, R.; Dalton, N.; Rogers, Y. When the fingers do the talking: A study of group participation with varying constraints to a tabletop interface. In Proceedings of the 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems (TABLETOP 2008), Amsterdam, the Netherlands, 1–3 October 2008; pp. 33–40.
- Morris, M.R.; Huang, A.; Paepcke, A.; Winograd, T. Cooperative Gestures: Multi-User Gestural Interaction for Co-located Groupware. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 22–27 April 2006; pp. 1201–1210.
© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Hoven, E.V.d.; Van Bergen, T. Tangible Cooperative Gestures: Improving Control and Initiative in Digital Photo Sharing. Machines 2015, 3, 268-295. https://doi.org/10.3390/machines3040268
AMA Style
Hoven EVd, Van Bergen T. Tangible Cooperative Gestures: Improving Control and Initiative in Digital Photo Sharing. Machines. 2015; 3(4):268-295. https://doi.org/10.3390/machines3040268
Chicago/Turabian StyleHoven, Elise Van den, and Tom Van Bergen. 2015. "Tangible Cooperative Gestures: Improving Control and Initiative in Digital Photo Sharing" Machines 3, no. 4: 268-295. https://doi.org/10.3390/machines3040268
APA StyleHoven, E. V. d., & Van Bergen, T. (2015). Tangible Cooperative Gestures: Improving Control and Initiative in Digital Photo Sharing. Machines, 3(4), 268-295. https://doi.org/10.3390/machines3040268
