How Form and Structure of Chinese Characters Affect Eye Movement Control

Abstract

This study investigated the correlations between the form features and legibility of Chinese characters by employing the eye tracking method in two experiments: Experiment 1 examined factors affecting Chinese character legibility with character modules and identified the correlations between character form and legibility of crossing strokes; and Experiment 2 examined the effect of crossing strokes on subjective complicacy perception in both Chinese characters and English letters. This study determined that enclosed Chinese characters affect subjective complicacy perception and reduce saccadic amplitude. In addition, greater number of stroke crossings produced higher subjective complicacy perceived for both Chinese characters and English letters. The results of this study serve as a reference for predicting Chinese character legibility and assessing type design superiority.

Introduction

Chinese characters are logograms originated from pictograms where each one made up of a combination of components made up of strokes and forms, represents a word in the language. Studying Chinese characters involves studying the form, and finding patterns in the form makes it easier to understand the concepts associated with its meaning in the language. All characters have a fundamental form, or its contour, as well as structure, which refers to the arrangement of component blocks with a direct impact on how a Chinese character is written and viewed, affecting the entirety of the word’s features.

Designing a proper typography for Chinese characters naturally involves considering its readability, or meaning extraction, but should also involve legibility, or the ease with which a text can be read. Both the act of reading or simply seeing can affect the recognition of a character. From the perspective of ergonomics according to Sanders & McCormick (1987), the factors involved in how easy it is for the reader to recognize well designed characters involves visibility, legibility, and readability. Character legibility determines how easy it is to recognize the character and the distinction between strokes, structure, and shape can all determine how quickly characters are recognized without being confused with some other characters (Hsu, Peng, & Wu,1991). In the design of Japanese signage for the highway, in order for the words to be recognized within 5-6 s, signage typography was designed to simplify details and magnify stroke difference in order to achieve the objective of higher legibility.

Another crucial factor affecting Chinese character recognition lies in the message conveyed by the contour, or form, of the characters formed by character structure. Numerous studies have indicated that structure and legibility are interrelated (Ai, 1965; Cheng, 1981; Tsai, Huang, & Yu, 2001; Yu et al., 1990; Yu, Tsai, & Chen, 1996). Studies on Chinese character in the past, however, commonly focused on passage reading and understanding instead of the structure and recognition of individual characters. Orthographic study of Chinese character and its effect on character recognition is yet another favorite topic in this field (Chua, 1999; Feldman & Siok, 1999).

Chinese character structure is multifarious. Each character is arranged into blocks from one or more character components. The components were originally used for organizing words in the dictionary, and thus every character has one primary component, or root. Roots are typically not stand-alone components, such as “、”,“〡”, and “〦”, but can sometimes be their own character, such as “一”, “乙”, “金”, “龍”, and “龠”. Characters are composed of roots and their constituent components, and different arrangements can make different characters, such as “口” and “木” can become “ 困”, “ 杏”, or “ 呆”. One component can also be replicated to create other words of similar concepts, such as “ 木”, “ 林”, and “ 森”. Various other structural arrangements are possible.

As Yeh S.L. et al. pointed out, the primary pathway of character recognition involves the character’s form and structure rather than its strokes or blocks components. For example, enclosed characters such as ”園”, “圓”, and “固” have distinct symbolic implications from characters with other forms, and that difference between structural features is a crucial aspect of character recognition. (Yeh, Li, & Chen, 1997; Yeh, Li, & Chen, 1999; Yeh & Li, 2002). Although most researchers have examined the numerous structures, or types of module arrangement, of Chinese characters, these structures are never meaningfully standardized and are instead classified arbitrarily. Huang and Ma (2007), for example, classified Chinese character component arrangements into seven module types: unitary, enclosed, left to right, above to below, above to middle and below, left to middle and right, and L-shaped. Yeh et al. (1997), on the other hand, empirically classified Chinese characters into five module types of component arrangements: left to right, above to below, P-shaped, L-shaped, and enclosed. Because the classification of Chinese character structures remains unaffected regardless of font type, font scale, and reader, the character form and structure can be said to be solely responsible for affecting character recognition (Yeh et al., 1997; Yeh et al., 1999; Yeh, 2000; Yeh, Jing, Takeuchi, Sun, & Liu, 2003).

According to feature integration theory (Feature Integration Theory, proposed by Treisman, is a theoretical model on visual attention which states that the process of perception is separated into the two parallel processes of registration and feature-consolidative recognition of the stimulus. In this theory, every feature of the visual stimulus can be quantified as a vector value, such as the lines, shape, color, etc.), form identification is a process of identifying an object from basic visual features to the entire appearance (Treisman & Gelade, 1980; Treisman & Schmidt, 1982; Zheng, 1982), which coincides with recognition of English letters (i.e., vertical segment and horizontal segment). In other words, letters are identified from visual features to the entire form through the feature integration process (Gibson, 1969; Rumelhart & McClelland, 1982).

The microgenetic method (The microgenetic method is an effective scientific method used in the observation of changes in perceived detail that helps the researcher focus on the crucial changes that happens in the observation and learning process in the study of cognitive development) is a research technology directly reflecting the microgenetic process of graphics and letter recognition. In this method, an initially blurred visual target appears as target clarity gradually enhances for participants to incrementally perceive the shape of the target from a chaotic state through contour to the complete shape (Flavell & Draguns, 1957; Shen & Zhu, 1995). Peng and Yu (1983) conducted a form perception experiment of Chinese characters by adopting the microgenetic method to examine Chinese character features. The experiment results showed that in Chinese characters with a “十” formation, the “十” is the first to be perceived, and in enclosed Chinese characters, the contour is the first to be identified. Furthermore, Shen and Zhu (1997) observed that “十” and “口” are striking features of enclosed characters formed using them, and that form familiarity does not affect the identification of both features.

Japanese scholars, due to their use of many Chinese characters in Kanji, have also conducted extensive studies into Chinese character recognition, such as in Kunyomi and Onyomi, the study of character pronunciation in Chinese and Japanese, respectively (Verdonschot, La Heij, Tamaoka, Kiyama, You, & Schiller, 2013); (Verdonschot, Lai, Feng, Tamaoka, & Schiller, 2015), Tamaoka and Yamada (2000) Other studies have also shown that although stroke sequence reveals very little about the root characters, knowing the Chinese roots is fundamental and can be quite beneficial for understanding and the lexical meaning of each Kanji. Koga and Groner (1989) conducted a trans-cultural study of Kanji character recognition by studying the eye movements with eye tracking methods of those who have studied Kanji and those who have not and discovered that the fixation behavior in the eye-movement patters of the two groups differ.

In another experiment using eye tracking methods on Kanji, Hukuda and Watanabe (1996) observed that readers tend to focus on stroke crossings. In our experiment, we hypothesize based on this finding that the focusing on crossings is due to an effect known as Irradiation, where when two or more strokes cross or meet at a point forming a blotted area, the individual strokes become blurred like an expanding ink blot. This effect is especially prominent once the characters are shrunken, and therefore characters such as ”十、木、米、大”, or English letters ‘M’, ‘N’, and ‘W’ often need to adjust the stroke thickness at the cross points during typographical design, as shown in Figure 1. (Lin, 2004, Zhou, 2004)

In previous character form studies, designers have discussed the use and design of character form from a subjective design perspective without further explaining the correlations between design and the features and structure of form. Although studies on stroke features have been conducted through the use of visual evaluation, subjective questionnaires, and physical calculation, no objective quantified statistics are currently available, nor have there been eye movement studies conducted to observe the reader’s behavior when perceiving Chinese character forms.

According to several psychological studies, eye tracking is the most straightforward method to indicate attention distribution, and the duration of fixation reflects the complicacy of external information and depth of information processing of readers. In addition, eye tracking can directly reflect the attention distribution and cognition strategy of an individual (Just & Carpenter, 1976; Williams, Loughland, Green, Harris, & Gordon, 2003). For example, the fixation duration during the perceiving of text increases as the message contained in the text becomes more complex (Baker & Loeb, 1973; Mackworth & Morandi, 1967; Rayner & Pollatsek, 1989). Average fixation duration can also reflect task difficulty or the required mental workload of readers and increases as the difficulty of a task increases. Furthermore, Saccadic amplitude can be measured to indicate information density, and higher information density can lead to higher saccadic amplitude (Chen, Cheng, Tseng, Su, & Jhan, 2010). In addition, recurrence refers to the shifting fixation of the eye from a new fixation point back to the previous point, And recurrence rate is an indication of the condition under which an individual undergoes information reanalysis (Chaffin, Morris, & Seely, 2001).

This research conducted experiments of sight experiments and analysis on contemporary Chinese character forms and evaluation methods in attempt to establish a more objective perspective on character typographical design. Eye tracking methods were employed to gain insight into correlations between eye movement and character form complicacy, and to understand how readers decipher character forms and features and what its impact is on legibility. When applied to typographical design, this approach may aid in the reduction of form complicacy, allowing readers to identify characters with more clarity and perceive information more effectively.

Hypotheses

This research focuses on the effect of stroke-crossings on visual complicacy and eye movement and proposes the following hypotheses and experiment framework, as illustrated in Figure 2.

H1: 

A more complexly structured Chinese character can induce shorter saccadic amplitude

H2: 

Characters with cross-stroke structure can induce a higher number as well as longer durations of fixation occurrence

H3: 

The higher the number of stroke-crossings in the character, the higher its complicacy value

Experiment 1

Method

This experiment aims to identify the correlations between human cognitive complicacy and the features of Chinese characters as well as the effect of different character module types on eye movement information.

Participants. This study comprised 13 participants aged 18–25 years. All of the participants were native Chinese speakers. In the experimental design, stimuli were presented at random and modules were selected according to participant design. Each participant participated in all aspects of the experiment.

Materials. Based on the form classification research of Huang and Ma (2007) and Chao (2005), 49 representative characters from the 5731 Chinese characters in the Concise Chinese Dictionary Editing Data Word Frequency Statistics Report (1997) were used in the experiment and divided into seven categories according to word frequency, stroke, and module (

Table 1. Form classification of Chinese character (Huang and Ma 2007).

). The size of each experiment character was 600 × 600 pixels. After fully viewing each character, the eye tracking of the participants in the browsing process was recorded. The total testing frequency was 49 times. The duration of each time was 5 sec., and total testing time was 6 min. The experiment was manipulated with the module type as the independent variable and total number of fixations, total saccadic amplitude, average saccadic amplitude and recurrence as four dependent variables.

Procedure. Before the formal experiment commenced, the participants sat approximately 60 cm in front of a 21-in cathode-ray tube screen at a standard testing site. The screen and participants were aligned in a straight line. The screen width was approximately 36.8º of the viewing angle, and screen height was approximately 28.1º of the viewing angle. The participants then wore an eye tracker (EYELINK 1000, SR Research). Right eye tracking was recorded at a sampling frequency of 250 Hz. The official experiment began after calibration.

When the experiment began, the participants moved a mouse pointer to a cross sign at the center of the screen to ensure that it was the fixation point of the participants each time. The fixation point disappeared when the mouse pointer reached the cross sign, and the Chinese character appeared immediately for 5 s. The computer displayed the experimental characters at random and performed drift correction after displaying seven characters to ensure that the participants could clearly identify the characters. After displaying each experiment character, the system displayed three characters of similar form for the participants to confirm the experiment character. The procedure is shown in Figure 3.

Results

Chinese character form and structure and eye movement analysis. For the number of fixations, this study found that the main effect of module type on the total number of fixations (F_6,508 = 0.92, p > .05, η² = .07) was not significant. This suggests that the module type difference does not affect vision distribution in legibility. However, the effect was significant with the number of nodes and number of strokes as the covariate variables. The effect of both the number of nodes (F_1,507 = 7.82, p < .01, η² = 0.02) and the number of strokes (F_1,507 = 8.51, p < .01, η² = 0.02) on the total number of fixations were significant. This suggests that stroke features, including the number of nodes and the number of strokes, affect the number of eye movement fixations in character recognition: the number of fixations is higher when recognizing unitary characters and lower when recognizing enclosed-shaped characters, as shown in Figure 4.

The ANOVA of total saccade amplitude and average saccade amplitude showed that the main effects of module type on total saccade amplitude (F_6,508 = 4.05, p < .01, η² = .24) and average saccade amplitude (F_6,508 = 4.65, p < .01, η² = .27) were significant. This suggests that module type affects saccadic amplitude in character recognition. For example, saccadic amplitude is significantly shorter when recognizing enclosed and L- shaped characters, and saccadic amplitude is longer when recognizing left to right and above to below characters. The effect of module type in recurrence (F_6,438) = 4.30, p < 0.01, η² =.212) was significant, suggesting that different module types affected recurrence frequency to determine whether a difference from the previous characters could be observed, as shown in Figure 5.

Chinese character cross-stroke structures and eye movement analysis. Research on the correlations between the features and legibility of Chinese characters has indicated that legibility is highly correlated with form structure. Research on the form features of Chinese characters has determined that stroke crossings are first identified in Chinese characters formed using “十” and that readers tend to focus on stroke crossings. Stroke crossings may contain the most information. However, excessive visual foci and information may cause complicacy, affecting legibility. Therefore, this experiment conducted to investigate the effect of stroke crossings on Chinese character legibility to determine the correlations between the microgenetic cognitive process and eye movement information in Chinese character features.

The experiment was manipulated with the stroke crossings (comparison of characters with and without crossings) as shown in Figure 6, as the independent variable and total number of fixations, total fixation duration, and average fixation duration as three dependent variables.

The effect of module type on both total fixation duration (F_1,1169 = 922.078, p < 0.05, η² = .99) and average fixation duration (F_1,1169 = 727.087, p < 0.05, η² = .99) was significant. This result suggested that crossings and module type affect gaze duration. In general, the number of fixations, total fixation duration, and average fixation duration increase when a Chinese character consists of more crossings. By contrast, the number of fixations, total fixation duration, and average fixation duration decrease when a Chinese character consists of fewer crossings.

The mutual effect between crossing and module type was significant in total fixation duration (F_6,1169 = 16.955, p < 0.01, η² = .944), average fixation duration (F_6,1169 = 76.236, p < 0.001, η² = .987), and the number of fixations (F_6,1169 = 13.887, p < 0.01, η² = .933). This result suggested that in different module types, the difference between characters with more crossings and characters with fewer crossings was significant in the number of fixations, total fixation duration, and average fixation duration, suggesting that the effect of module type on vision distribution was significant in the inference of the number of crossings, as shown in Figure 7.

Experiment 2

Based on the outcomes of Experiment 1, if stroke crossings of Chinese characters are irradiative, the number of fixations and fixation duration increase as the number of crossings increases. In addition to the objective eye movement information in Experiment 1, Experiment 2 was conducted to verify the effect on subjective complicacy perception of crossings in Chinese characters and English letters. In addition to Chinese characters, English letters were simultaneously evaluated to confirm whether crossings affect subjective complicacy perception in the characters of different languages.

Method

Participants. This experiment involved 34 participants aged 18–25 years. All of the participants were native Chinese speakers. Stimuli were presented using picture cards. The evaluation was conducted on English letters and Chinese characters separately. Each participant participated in all parts of the experiment, including English and Chinese sections. The participants were asked to measure character complicacy directly from the characters shown in the picture cards. Complicacy was scored from 1 to 5, from simple to complex.

Materials. A total of 60 representative Chinese characters with crossings were used (

Table 2. English letters and Chinese characters in experiment 2.

), including 30 with more crossings and 30 with fewer crossings. The stroke count of all of the Chinese characters was between 7 and 15 strokes. All of the Chinese characters were displayed in MingLiU font. For the English letters, six letters with crossings and six letters without crossings were used and displayed in Arial font. The size of each experimental character was 400 × 400 pixels presented on a 5 × 5 cm picture card. The experiment was manipulated with the structure characteristics (comparison between the number of crossings) as the independent variable and complicacy (1–5) as the dependent variables.

Results

English letter crossings and complicacy ANOVA: The effect of structure characteristics on complicacy (F_1,340 = 136.320, p < 0.01, η² = .805) was significant, as shown in Figure 8. This result suggested that the number of crossings in English letters affected the subjective complicacy of the participants, and identifying English letters consisting of a high number of crossings was more complicated for the participants.

Chinese character crossings and complicacy ANOVA: The effect of structure characteristics on complicacy (F_1,1971 = 172.829, p < 0.01, η² = .822) was significant, suggesting that the number of crossings in Chinese characters affected the subjective complicacy of the participants, and identifying Chinese characters comprising more crossings was more complicated for the participants.

Discussion

Through analysis of eye movement, this study examined the effect on form complicacy and legibility of Chinese characters, indirectly inferred the intellectual representation and process of the participants, investigated the effect of module type on visual feature legibility of Chinese characters, and examined the association between human subjective perception and eye movement information. Chinese character identification is an automatic top-down process (Cheng & Wu, 1994), and form and structure significantly affects perception of Chinese characters. Several studies have indicated that form and structure are the main factors affecting character legibility (Ai, 1965; Cheng, 1981; Tsai et al., 2001; Yu et al., 1996). In Experiment 1, we determined that character complicacy affected eye movement because of module type difference. The effect of module type on total number of fixations, total saccadic amplitude, and average saccadic amplitude was significant.

Other studies on recurrence showed that recurrence frequency and recurrence duration can increase significantly when the readers had weaker reading ability, when articles were less coherent, or when reading content was more difficult (Chen et al., 2010). In addition, form and structure complicacy can also affect recurrence rate and fixation duration (Yang & McConkie, 1999). In Experiment 1, we discovered that the effect of module type on recurrence was significant, particularly for unitary and enclosed characters, for which recurrence was significant and intensive. This result suggested that these two types of Chinese character were more difficult to identify than other types and required repeated reading. Furthermore, we determined that left-to-right type characters were easier to identify than other types. This is consistent with previous studies where left-to-right characters were more readily legible, at a higher identification rate and with shorter identification lead-time, whereas enclosed and unitary characters were the most difficult to identify (Chen & Huang, 1999; Cheng & Wu, 1994; Zhang & Sheng, 1999). These findings confirm that the visual characteristics of form and structure are great influencers in the identification of Chinese characters, and suggest that the form and structure of Chinese characters comprises visual memory characteristics because form and structure have different organizations and characteristics.

The eye movement data in this study showed that total saccadic amplitude and average saccadic amplitude of enclosed and L-shaped module types were the lowest of the module types. Saccadic amplitude is known to be correlated to information density, as higher information density can yield shorter saccadic amplitude (Chen et al., 2010). In addition, when a character is unfamiliar or characteristically complex, fixation duration extended and saccade decreased (Kliegl, Nuthmann, & Engbert, 2006; Rayner & McConkie, 1976; Reichle, Pollatsek, Fisher, & Rayner, 1998). Therefore, regarding object eye movement information, Chinese characters with enclosed and L-shaped modules were evidently more difficult to identify, and the elements of complicacy of both types of module require further investigation.

Based on feature integration theory, the form identification of Chinese characters is a process from feature identification to form cognition. Although this has been widely accepted in the Chinese character cognition domain, little information on the feature identification process and related rules are identified in the absence of effective research methods. Therefore, this study investigated the effect of stroke crossings on character legibility by employing eye tracking techniques to examine the correlations between the microgenetic cognitive process and eye movement information in Chinese character identification. In Experiment 1, we discovered that the effect of the number of crossings in different module types on vision was significant. The total fixation duration and average fixation duration increased when the characters consisted of more crossings, and total fixation duration and average fixation duration reduced when the characters consisted of fewer crossings. This result matches the finding that structure complicacy affects total fixation duration and average fixation duration reported by Yang and McConkie (1999) and Rayner and Pollatsek (1989).

Experiment 2 was conducted to confirm the association between stroke crossings and legibility in both English letters and Chinese characters. The experiment outcomes showed that the number of crossings in both English letters and Chinese characters affected the subjective complicacy of the participants. A high number of crossings caused increased subjective complicacy. Research on the form features of Chinese characters has indicated that stroke crossings are identified first in Chinese characters formed using “十” and that readers tend to focus on stroke crossings (Hukuda & Watanabe, 1996; Peng & Yu, 1983; Shen, 1992). Through eye movement information, the current study proved that the effect of stroke crossings on legibility was significant in both Chinese and English, suggesting that stroke crossings comprise additional visual foci and information, compounding legibility and causing irradiation. Therefore, we suggest that, during typographical design, strokes should be constricted, or reduced in thickness, at crossings to minimize irradiation. In addition, if crossings and seams can be used to form a visual attention-grabber and to create points of intrigue, they would capture the visual focus of viewers and enable them to better outline the characters. In doing this, the “visual tempo” of various Chinese character design can be improved (He & Liao, 2011).

Conclusion

Form and structure are a crucial principle of learning Chinese. Font use, legibility and intrigue are the focus of character design for signs, titles of printed materials, and signboards. Typographical designers should pay attention to the size, stroke, and structure of characters and be meticulous in the appearance of characters, which is the most critical rule of type design.

This study confirms from its eye tracking analysis that character form and structure and stroke crossings significantly affect legibility, and the more complex these factors are, the more subjective complicacy is felt. Furthermore, this study suggests that when the typography incorporates structural rearrangement to reduce stroke crossings, as shown in Figure 1, this can be beneficial for the improvement of legibility.

This study is limited to the topics of form and structure complicacy and the identification and legibility of single Chinese characters, and does not discuss the written stroke order, lexical understanding, or the readability of sentences and paragraphs. Future studies are recommended to incorporate English letters and Japanese characters in comparison studies, correlation between stroke numbers and letter complexity, stroke order, and complicacy and legibility comparison between enclosed characters and non-enclosed characters.