Semantic Priming of Colored Signs in the Peripheral Vision during Electronic Text Reading

Abstract

This study explores if peripheral colored ambient light or colored light can be noticed and semantically associated with text while reading a novel from an Electronic Visual Display (EVD). Additionally, it is investigated if those colored signs are perceived as supplementary to the story and result in a more immersive experience. For that purpose, a user evaluation study was performed based on attention theory (selective attention, semantic priming, and divided attention), reading behavior, and semantics of colors. Participants received a reading assignment. While reading a novel from an e-book, dynamic colored ambient light or colored light was emitted. Throughout the reading assignment, the colors were adapted to the particular scenes the participant was reading. Findings suggest that attention can be divided between reading the novel from an EVD and the ambient color signs.

1. Introduction

The quality of Electronic Visual Displays (EVDs) has improved over the last decade, with developments in resolution, contrast, and flickering. However, the experience of reading on EVDs did not advance as extensively as for other entertainment devices such as televisions and game consoles. In particular, for other devices, researchers have investigated how supplementary visual information creates a more immersive experience. Information presented on surrounding displays, such as colors, light arrays, objects, movement, or sounds, enhance game and film experiences [1]. Colored lighting is also used for ambient television [2]. Findings from these studies demonstrate the potential of using peripheral vision to enhance media experiences. Ishii et al. [3] show that common appliances may be successfully extended with additional visual information. This gives participants a higher sense of ‘being there’ [4]. Peripheral lighting creates an increased sense of immersion and presence [1]. Additionally, subjective quality assessments indicate that added ambient light effects reduce eye strain and improve the overall viewing experience. Consequently, when reading a novel from an EVD with an ambient atmosphere of colored signs, reading could become more desirable and satisfactory. Nonetheless, in some cases (when all light sources and displays operate at the same time), additional visual information can result in concentration loss and an ‘information overload’ effect [3].

Previous research has not studied an ambient atmosphere of colored signs surrounding an EVD, with ambient light matching the content of the text. In order to investigate this in more detail, the current study focuses on the perception of colored signs (semantically congruent with the text) in the peripheral vision during electronic text reading. This research aims to reveal, by adopting user-centered design, if dynamic colored signs can be perceived while reading from an EVD, and if such colored signs are perceived as supplementary to the text.

2. Theoretical Background

Investigating the exposure, attention, and interpretation of colored signs when reading from an EVD requires insight in the human capability of perceiving information from the periphery. Previous studies on this matter typically used attention theory to investigate human attention management [5]. Thereby, the main focus was placed on selective attention including semantic priming [6] and divided attention [7,8,9]. Additionally, for the purpose of this research, the reading behavior [10,11] and the semantics of color [12] are elaborated upon in more detail.

2.1. Attention Theory and Priming

Reading is a controlled process where predominantly one primary task can be performed. If readers are immersed in a book, they may not even notice someone calling their own name [13]. Therefore, when reading from an EVD, humans cannot focus on all stimuli at once as the attention needs to be directed to the main stimuli. In the areas of cognitive psychology as well as neuropsychology, several models of this process of attention management have been developed throughout the past decades, in particular selective attention and divided attention [5,7,14].

2.1.1. Selective Attention and Priming

Selective attention is a way of cognition processing in which mental resources are only focused on the most significant stimulus, intentionally ignoring the others [15,16]. For example, focusing on one conversation in a crowded room or distinguishing voices of loved ones from the voices of surrounding people. This form of processing is performed at a subconscious level of awareness before the process of attention starts taking place [14].

Treisman [17] suggests that this process of selection is influenced by the relevance of the information. For example, when having a conversation about an event, words such as the location of the event or performing artists may be primed. When someone close by says any of these words, it would be more likely that these words will be recognized than when those words have no relevance. The related words, consciously or subconsciously, influence the perception of the information.

The most common type of priming in word recognition tasks is semantic priming [6]. The term arose in the 1970s from Meyer and Schvaneveldt [18], who asked participants to decide whether two simultaneously presented strings of letters were both words or not. Half of the word–word pairs were semantically related, and half were not (e.g., nurse–doctor or bread–door). People are faster in priming a semantically related word than a semantically unrelated word [19].

This filtering paradigm also enables relevant visual stimuli to be selected without intentional control [20]. The current study aims to uncover similar semantic priming between colored signs and e-text.

2.1.2. Divided Attention

Divided attention is a form of multitasking. Whereas selective attention and priming enable focusing on a limited set of stimuli, divided attention allows people to carefully divide attention over multiple tasks simultaneously [8]. To what extent a person can perform multiple tasks depends on the mental effort required for each task, which decreases with practice and experience [9]. A good example of divided attention is walking. At some point, walking becomes a highly trained process, an ‘automatic process’ that requires no particular attention. Conversely, ’controlled processes’ such as reading a book require continuous attention. When arousal increases, the attention to the controlled process of reading also increases [13].

Subsequently, only one task can be performed at a time. In other words, people can read and walk at the same time; therefore, a controlled and an automated process can occur simultaneously. However, when the reading task causes more arousal or requires deep thought, people often stop walking to focus more on what they are reading [21].

2.2. Reading Behavior

Baron [10] contrasts the continuous attention readers pay to a novel or a nonfiction book, also known as ‘deep reading’, to a more stop-start, goal-oriented kind of reading, which she called ‘reading on the prowl’ or, as outlined in this paper, ‘light reading’. Note that although the number of studies addressing sequential and continuous reading texts in print and on screen is limited [2,11], the issue was already raised before the introduction of digital devices [22]. Findings suggest that light reading may allow for more divided attention than deep reading, but also that light reading may occur when reading a novel.

2.3. Semantics of Color

In any graphical communication, color is a powerful emotional signal (or sign) because it draws upon associations from the past and present [23]. Caivano [12] divides semiotics of color into syntactic, semantic, and pragmatic dimensions. The syntactic dimension comprises color order systems. Colors can be arranged according to their similarities in hue, saturation, and lightness. The semantic dimension relates the color to what it may denote. This means that color is considered as a sign that substitutes other things, such as red for danger. The pragmatic dimension focuses on relations of signs to their interpreters or users and its practical application, such as the way color can be used to increase workers’ productivity.

The semantics of color can be divided into iconicity, indexicality, and symbolicity in color [12]. The iconicity in color is made by psychological associations (e.g., the colors orange, red, and yellow are commonly associated with fire, heat, and sun). As a result of this association, such colors are defined as warm colors, while blue and green-blue are defined as cold colors. Color is also an indexical sign; for example, the yellowish color of a person’s skin can be taken as a sign of illness. This is established by an indexical connection; the illness itself produces the pigmentation of the skin. The symbolicity in color can be defined as the meaning assigned to colors independently. For example, green is safe or go; yellow is warning or be careful, and red is stop or danger. However, the symbolicity in the color sign can be interpreted differently, depending on, for instance, cultural factors [24,25,26].

Combining theories of attention, color semantics and reading behavior, this study explores whether peripheral ambient colored light can be primed and semantically associated with the text while reading a novel from an EVD. Consequently, the dependent variable in this study is the perception of dynamic colored signs while reading from an EVD. More in particular (1) whether peripheral ambient colored light can be primed and (2) semantically associated with the text.

3. Method

3.1. Research Participants

Ten participants took part in the study and gave written informed consent before data collection. This approach and sample size is in line with similar qualitative user studies [1,2,3,11,27]. Subjects (6 females and 4 males) were in the age group 35–64 (M = 38.0, SD = 10.2). All participants were unaware of the aims of the experiment, had good eyesight (wearing glasses or lenses was accepted), and had normal color vision (not color-blind).

3.2. Setup and Procedures

To approximate an everyday real-world situation (ecological validity), the user test was executed in a natural home environment such that the participant could feel at ease and pleasant enough to sit back and enjoy the reading of an e-book. Within a household setting, a couch was positioned in front of a desk with a television (which was not turned on) and two light sources. A third light source was positioned beside the couch. Participants were sitting on the couch at a 3.5-m distance from the front two light sources, and at a 1 m distance from the light source adjacent to the couch (see Figure 1). They were reading an Apple iPad tablet, which they kept at a convenient reading distance. Apart from the ambient light sources, there was no other artificial light and no direct sunlight. Note that given this experimental design, it is impossible to isolate the effects of lighting from other possible effects, so it is assumed that if the lights are perceived, that is due to the lights itself and not due to other reasons.

All participants received the same reading assignment from the e-book novel ‘40 hours’, which is a thriller written by the German authoress Kathrin Lange. The novel ‘40 hours’ was chosen for the extensive change in characters and environments within the storyline of the book. The text consisted of four chapters of the book (including prologue) and was divided into three main scenes: namely, the suffering scene (prologue, chapter two), the daytime scene (chapter one), and the metro scene (chapter three). The storyline consists of different events with various characters at different locations. The characters intersect with each other during the four chapters in the story. Afterwards, a structured interview was taken to evaluate the test.

3.3. Materials

3.3.1. E-Book

The choice for the Apple tablet is twofold. Firstly, we chose the tablet for the use of Apple’s Airplay. AirPlay allows the wireless streaming of audio, video, photos and screens between devices on the same Wi-Fi network, which is required for this study. Secondly, the display of an Apple tablet already emits light, in contrast to an e-reader. Therefore, the brightness of the surrounding ambient light mattered less for the participants to read the e-text.

3.3.2. Light

During reading, dynamic colored ambient light was emitted with the use of three Philips Hue A19 light bulbs to transmit a colored sign (Figure 2). The content on the screen of the tablet was duplicated via AirPlay on an Apple TV, which was connected to a monitor (the observer station). The view to the observer station was blocked by a plant, so the observer would not distract the participant during reading. With the duplicated image of the Apple tablet, the observer monitored the page turns of the participant. In reaction to a page turn by the participant, the observer controlled the three Philips Hue A19 light bulbs in accordance with a predetermined light plan.

All color transitions of the ambient light had a 10 s cross-fade. The Philips Hue light bulbs connected through a bridge with an open standard protocol called ‘ZigBee Light Link’. The light bulbs create a mesh network with each other. This network enables each light to communicate and pass on a message to the next light. The professional-grade lighting control application ‘Luminair 3′ was connected to the bridge network and enabled the observer to control the lights.

In order to define optimal luminance settings while preserving ecological validity, contradictory claims have to be taken into consideration. Some studies indicate that higher luminance decreases visibility and visual comfort [28,29]. There is also evidence that a change of luminance over time results in comfortable reading [30]. Finally, higher luminance differences between text and background on LCD displays is claimed to improve legibility and reading speed [31,32]. In this user study, the luminance emitted by the Apple tablet had one level of intensity to ensure enough contrast between text, background, and ambient colored light. To ensure the maximum amount of ambient light was perceived in the peripheral vision, the e-text was displayed with a white text on a black background [33]. White luminance emitted from the Apple iPad with Retina display was approximately 421 cd/m², and black luminance (at full brightness setting) emitted from the Apple iPad with Retina display was approximately 0.48 cd/m² (contrast ratio 877:1) [34].

3.4. Stimuli

Two light plans were created (Figure 3), an ambiance light plan and a colored light plan. The light plans are consistent with the e-book storyline, taking culturally specific visual codes into account by setting the colors to common Western culture preferences [25]. For example, for the daytime scene, brightness and color were set to create an atmosphere that started from low orange toward a bright yellow light as the day dawns in the story.

The ambience light plan produced a colored atmosphere in the room with dissimilar colors in the main three lamps. This ambience light plan created a certain atmosphere with supplementary colors toward the storyline of the e-book. It was not showing one pure colored sign. For that purpose, a second light plan was created: the colored light plan.

The second light plan produced a colored atmosphere in the room with one main color in the three lamps (Figure 3). The colored signs were set to Western psychological associations i.e., red for sexy, love, romance, vigor, optimism, strength, and caution, and brown for masculinity or earth [25]. The second light plan is referred to in this report as the colored light plan.

3.5. Data Collection and Analysis

In order to reveal if the dynamic colored signs were perceived while reading from an EVD, and if such colored signs were perceived as supplementary to the text, a structured interview was taken after the reading assignment. To this end, a tailor-made questionnaire was developed. It consisted of 19 interview questions derived from literature on attention theory, priming, reading behavior, and semantics of color. The questions were grouped into four themes, namely general experience, reading, distraction, and priming (see

Table 1. Structured interview questions.

Questions		Themes
Q1:	What did you think of the story?	General
Q2:	Would you like to finish the story?	General
Q3:	Were you, during reading, immersed in the story?	Reading
Q4:	Were you distracted during reading?	Reading, Distraction
Q5:	Why were you distracted?	Reading, Distraction
Q6:	Could you, after being distracted, easily come back into the story?	Reading, Distraction
Q7:	What did you think of the environment to read in?	General, Reading, Distraction
Q8:	Did you feel at ease on a scale 1–5? 1 (very poor) to 5 (very good)	General, Reading, Distraction
Q9:	Was the text readable on your tablet?	Reading
Q10:	What did you think of the ambient light?	General
Q11:	Have you, during reading, been distracted by ambient light?	Reading, Distraction, Priming
Q12:	If yes, how many times were you distracted by the ambient light?	Reading, Distraction, Priming
Q13:	What did you think of the color(s) of the ambient light?	Reading, Priming
Q14:	Did the color(s) of the ambient light add to the story?	Reading, Priming
Q15:	Did you associate the color(s) of the ambient light with the story?	Reading, Priming
Q16:	Theorem: The color of the ambient light made me feel more connected to the story.	Reading, Priming
Q17:	Would you like to use the ambient light with dynamic color more often when reading?	General
Q18:	How much value does the ambient light with dynamic color has according to you while reading on a scale 1–5? 1 (little value) to 5 (great value)	General
Q19:	Would you recommend the ambient light with dynamic color to your friends?	General

).

General questions were designed to gather information about the appreciation of the book, environment, and ambient light. Questions about reading were formulated to discover the legibility of the e-text and give insights of immersion during reading. The questions about distraction were to determine if the participant was distracted during reading by the environment and ambient light and if the distraction had any influence on the reading experience (divided attention).

Both questions about reading and distraction give an indication of light reading as participants may have a stop–start or light reading experience. The questions about priming were designed to reveal if the colored signs were primed by the participant and if the participant perceived them as supplementary to the story. With this approach, the subjective opinions of the participants are collected and responses are elicited with additional explanations to discover new understandings.

The interviews were directed by the interview questions. All interviews were recorded. Audio recordings of the interviews were transcribed. The qualitative data of the transcribed interviews were scanned and analyzed. During thorough reading, quotes in each interview were identified that share the same topic. These were coded according to the themes general experience, reading, distraction, and priming. The full data corpus was examined, and correspondence between the categories and their indicators was verified.

4. Results

4.1. Story, Mood and Place

All participants read between 20 and 30 min. The choice of the book was appreciated by the participants (Q1, Q2). The reading experience during the user evaluation was rated as pleasant (Q7, Q8). Since the mood and place of the reader can influence meaning-making and the process of reception [35], it was important that the participants felt comfortable and at ease in their reading environment.

4.2. Legibility and Visual Fatigue

Nine participants indicated that the readability of the e-text was good (5 participants) or very good (4 participants). Still, in their additional remarks, three of these participants commented that it took them some effort to familiarize themselves with this type of reading, and two participants indicated that some effort was required to change focus during reading when the colored ambient light changed (Q9–10, Q17–19).

4.3. Immersion and Distraction

In the ambience light plan, three participants noted having been immersed (Q3) in the story during reading. One of these three participants stated to have been fully immersed, as the other two participants noted to have been immersed somewhere during the storyline. Partially immersed means that the participant was immersed somewhere during the storyline but not during the entire reading. The participant that was fully immersed did not experience any distraction nor did the participant experience any colored light.

In the colored light plan, four people noted having been immersed. One out of the four participants noted having been immersed somewhere during the storyline as the other three participants stated having been fully immersed. However, these three participants did also indicate having been distracted by the ambient light and other environmental stimuli (Q4, Q11–12).

Six out of the 10 participants were not fully immersed and clearly perceived the colored signs. This would indicate a stop–start kind of reading or light reading. Of the four participants who were fully immersed, three also perceived the colored signs.

In total, nine participants were showing divided attention, because they were sometimes distracted by the ambient sounds, ambient light, or having to refocus their eyes on the e-text during reading (Q4–6, Q11–12). The participants were distracted by sounds such as the traffic outside or pedestrians talking on the street. However, most of the participants did not have any problem getting back into the storyline after being distracted. The distractions and refocusing of the eyes for legibility resulted in more light reading.

4.4. Priming

Most participants of the ambience light plan perceived the difference in color and brightness, creating a sense of excitement during reading. However, they did not perceive semantics of the colors. Two of the five participants noted that the colored ambient light did add to the story, but they did not associate the colored signs with the storyline (Q13–16). Another participant was not aware of any colored light but did note that the differences in brightness of the ambient light, it being darker or lighter, emphasized the atmosphere of the storyline. The participant stated: “It is, of course, an exciting story [and] then the ambient light makes the story more intense.” For this participant, color adds little to the story. One other participant did not associate the lights with the storyline and did not find that any ambient colored light added to the story. Only one participant noted that the ambient colored light added to the content of the story and associated the connotation of colored signs with the storyline as it was intended. The participant stated: “During the metro scene, the light became brighter. Which complies with the perception of the story.” The participant also indicated having been more immersed in reading with the colored ambient light: “During the scene with emergency services [daytime scene] the ambient color turned blue, then you experience it as if you were in the square and all around you are flashing lights of emergency vehicles. In the part of the blood running down his face [the suffering scene] you see the red ambient light, you just experience it more”.

In the colored light plan, two participants noted that the ambient colored light added to the atmosphere of the story. As one participant stated: “The red color had some impact on my feelings, now the story is going to be exciting.” For one participant, the color added little to the story. Two other participants did not associate any colored signs to the storyline. They indicated that they would enjoy their reading more with a static white or yellow ambient light.

Furthermore, one participant may have unknowingly registered and related the red ambient color to the suffering scene. This participant stated that they were unaware of it during reading and was not entirely sure about it either but did relate the colored sign to the right part of the storyline afterwards. This might be explained by theory indicating that selective attention and priming can occur on a subconscious level [36].

The goal of the experiment was not told to the participants before reading. However, one participant stated: “I was not aware of it, but if I would read it, being more aware of it, then I probably would like it more.” Another participant stated: “If the hypothesis of this research came to me that this was the intention, I probably paid more attention during reading, if I know that it is part of the story, it might be less disturbing.” These remarks indicate that the colored ambient light could have been experienced differently if they were aware of it before they started reading.

5. Discussion

First, the results show that attention can be divided between reading a novel from an EVD and ambient color signs. The colored signs can be perceived while reading. As Kahneman [13] noted, if the arousal increases, the attention to controlled progress also increases. The same applies to ambient colored light. If someone is fully immersed in an e-book, they may not notice any distraction or colored signs, which implies that light reading is necessary for perceiving colored signs during e-text reading. However, the results of this study show that light reading may occur when reading a novel [10] even when the reader is fully immersed in the story. Nevertheless, it remains unclear how other environmental cues have distracted the participant, such as sounds in the room or traffic on the street.

Second, it is shown that the colored signs can be perceived as supplementary to the e-book story [17]. Even without explicitly associating the semantics of colors to the content of the story, changes in light color and brightness impact feelings and excitement while reading and reinforce the atmosphere of the story. For example, in this research, the color red emphasized an upcoming excitement as the tension in the storyline grows. Semantic priming theory suggests that people are faster in priming semantically related signals [19]. Therefore, it can be argued that when participants claim colors add to the text, even without having an explicit association between the semantics of the color and the content of the story, this is an indication of semantic priming [13,18].

These observations comply with theory of color semantics [12]. During exposure, attention and interpretation of colored signs, not only the symbolicity in color plays a role, emphasizing the meaning assigned to colors independently, but also the iconicity (psychological associations) and indexicality in color (color as an indexical sign) might be important. Additionally, it is interesting to note that besides color, the brightness of the lights emphasizes a certain tension, which adds to the atmosphere in the storyline. While reading, a change in the brightness of the lights also creates expectations that the storyline starts to turn and thereby generates excitement. This is an indication that not only the semantics of color, but also the syntactic dimension (hue, saturation, and lightness) played a significant role [12].

Third, only a very limited number of readers (1 of the 10) were able to explicitly associate the semantics of colors with the content of the story. This can be explained by the theory of selective attention, which suggests that reading a novel requires concentration, focusing mental resources mainly on the text [13]. For this same reason, a reader may stay completely immersed in the story without paying any attention to the ambient lights or other distractions. So, even if the color semantics is consistent with the content of the text, which would make the information more relevant for selective attention [17], the process of reading requires deep thought, leaving only limited attention to the ambient colors and brightness.

Furthermore, two participants stated that if they would have been aware of the intended meaning of the colored ambient light, they would have paid more attention to it. This could be an indication that when being mindful of the fact that the ambient light is congruent with the storyline, readers may accept light reading faster and appreciate the ambient colors as part of the story. Similarly, the need for awareness of the ambient colored signs shows that some readers may need to learn to pay attention to the colored lights during reading, as a reader needs to get used to light reading, distractions, and visual fatigue. Ishii et al. [3] already indicated that ambient media has a learning curve.

As to visual fatigue, reading with ambient colored light is not an appreciated experience for everyone. Although EVD devices have improved in terms of resolution, contrast, luminance and other technical factors [27,37], the readers’ eyes need to refocus when the ambient light colors change, which increases visual fatigue. Moreover, reading white text on a black background was not common for some readers. Other text and background color combinations to increase legibility were not tested during this study [33].

Despite its valuable findings, this study contains some limitations. First, the current study did not include a comparison with other forms of reading. To investigate the semantic priming of ambient colored light in more detail, an experiment comparing reading from an EVD with white light and a printed novel would be an interesting path to explore. Second, the generalizability of the current study to a broader population is low due to a small sample size and because source credibility is not investigated. In different words, no justified conclusion can be drawn on whether the participants were knowledgeable or trustworthy, other than that they have good eyesight and normal color vision and were requested to provide authentic and valid assertions. Therefore, the current findings need to be interpreted with caution, in particular when deducing specific conclusions from the user tests. Still, within its scope, this research produces a deeper understanding of the key theme, and the outcomes are instructive and valuable. Finally, not only the colored ambient light distracted the reader, also other external factors could have divided their attention, such as the sounds of traffic and pedestrians, and the presence of the observer. Even with these extraneous influences, the effects of the colored ambient lights were observed. Future studies should investigate the effects of these external factors on reader’s attention, for example in a laboratory setting with a tighter experimental control.

6. Conclusions

This study investigated if dynamic colored signs (semantically congruent with the text) can be perceived while reading a novel from an EVD, and if such colored signs are perceived as supplementary to the text. It can be concluded that with the use of colored ambient light, colored signs can be primed and can be perceived as supplementary to an e-book novel. However, the combination of dynamic colored light, reading a novel, legibility, and dividing attention, is highly delicate. Within the scope of this study, the main conclusions are:

• Colored ambient light can be primed while reading a novel from an EVD.

  ○

  Light reading may occur when reading a novel from an EVD.

• The colored signs can be perceived as supplementary to the e-book story.

  ○

  Even without explicitly associating the semantics of colors to the content of the story, changes in light color and brightness impact feelings and excitement while reading and reinforce the atmosphere of the story.

  ○

  Only a very limited number of readers are able to associate the semantics of the colors with the content of the text. Therefore, it would be premature to come to the conclusion that the ambient light can be semantically associated with the text.

  ○

  Some readers may be affected subconsciously by the colored lights.

• When readers are informed about the intended meaning of the colored ambient light before they start reading, they could appreciate the ambient colors more as part of the story.
• There is some evidence of a learning curve to perceive ambient colors while reading a novel from an EVD.
• Reading on and EVD with ambient colored light is likely to increase visual fatigue.