Rhythmic Synchrony with Artificial Agents and Its Effects on Frequency of Visual Illusions Seen in White Noise

Abstract

Rhythmic synchrony among different individuals has often been observed in various religious rituals and it has been known to bring various psychological effects in human minds. This study investigated the effects of induced rhythmic synchrony with artificial agents in drumming on participants’ visual illusions. The participants completed a task with three cartoon agents on a computer screen beating drums taking turns. We then investigated whether participants were tended to find more meaningful shapes in displayed random dots (pareidolia) when rhythms of intervals between each agents’ drumbeats were in-sync rather than out-of-sync. We simultaneously compared an active condition, in which participants took the role as one of three agents to beat a drum, with a passive condition, in which they only observed three agents beating the drums. The results showed that pareidolia appeared strongly in participants where the drum rhythm was in sync, regardless of active and passive conditions.

1. Introduction

People often experience rhythmic synchrony, including in their own body movements, musical performances, and dances in various rituals. This sort of interpersonal rhythmic synchrony, in which a person’s actions are synchronized with those of another, such as two people drumming in time with each other, is known to produce various psychological effects within individuals. For example, it has been reported that, depending on the participants’ experiences, synchronized rhythm of body movement and action can evoke various changes in their cognitive and emotional systems—such as causing arousal and capturing attention—and even changes in the reward system [1,2,3,4]. Furthermore, the effects of interpersonal rhythmic synchrony have been discussed in connection to social cognition, including the formation of bonds between individuals. For example, it has been reported that one feels a strong sense of familiarity and empathy with that person and is likely to adopt altruistic behavior toward that person and his or her surroundings if the rhythm of one’s body movement is synchronized with that of another person [2,5,6,7,8,9,10,11]. In such ways, the effects of rhythmic synchrony on individuals have been widely researched in areas ranging from basic cognitive functions through social relationships.

A religious ritual are one typical examples of rhythmic activity among different individuals. In some religious rituals involving intense rhythmic synchrony between the participants’ body movements and the playing of musical instruments, it has been reported that participants frequently feel some kind of supernatural presence, enter a trance state, and sometimes experience visual illusions or auditory hallucinations [12]. These illusory experiences are considered to serve an important purpose in terms of strengthening participants’ feelings of elation and faith and might be a foundation of human religious activities [13]. Hence, it is important to understand these illusionary experiences that are evoked by rhythmic synchrony from the viewpoints of psychology and neuroscience. However, although these phenomena are empirically known, few experimental studies have been conducted on these sorts of illusory effects in ritual-like situations because of replicability issues in experimental settings. For example, it is quite difficult to strictly control and replicate the degree of rhythmic synchrony that is made by different human participants.

Our claim is that the technique of human-robot or human-agent interaction would enable researchers to overcome the replicability issue and shed light on the cognitive process of illusionary experience evoked by rhythmic synchrony in ritual-like situations. It is known that some of the supernatural experiences that are evoked by religious rituals are kinds of perceptual illusions and they are driven by specific patterns of physical stimuli. Hence, the cognitive processes of supernatural experiences have received attention in recent psychological studies [14,15]. Some researchers have tried to evoke supernatural experiences artificially by using robotic systems or artificial agents because these systems can easily reproduce the stimulus pattern based on quantified parameters. For example, Blanke [16] and his colleagues succeeded in making participants sense an apparition by using a robotic arm that physically stimulated participants’ backs.

Our hypothesis is that, even in an interaction with virtual agents, rhythmic synchrony induces illusionary experiences. To investigate this hypothesis, we manipulated the degree of rhythmic synchrony as an independent variable in a virtual drumming session with cartoon agents on a computer display (not actual humans) and investigated whether strong synchrony could also strengthen the intensity of illusions in a ritual-like experimental situation. In this experiment, the word “synchrony” can be defined as sharing the same rhythm among different agents. Three cartoon agents hit each drum one-by-one at variable intervals, and the degree of synchrony was quantified as the variance of the drumming intervals of the three agents. Low variance meant that the rhythm of the three agents was in sync. Furthermore, in addition to rhythmic synchrony, we also manipulated whether participants actively participated in the drumming or not as an independent variable. Previous studies have reported that the effects of synchrony were affected by the involvement of the participant’s muscles [7]. Therefore, we hypothesized that the direct participation could also influence the results of the present study.

We focused on pareidolia as an index of whether a visual illusion was present or absent [17,18,19,20]. Pareidolia is the general name given to the visual illusion of finding a meaningful shape, such as that of an animal, in a perceptually meaningless physical pattern. Neuroscience research has shown that such illusions involve the same brain-activity patterns in the visual cortex and other parts of the brain as if the participant was actually seeing the illusory object [18]. This type of visual illusion is strongly seen in patients with psychiatric disorders, such as dementia with Lewy bodies [19]. Even in healthy participants, it has been reported that pareidolia often manifests in random-dot patterns by attenuating cognitive control—for example, by having the participants solve a difficult problem with high cognitive load [20]. In previous studies, the presence of pareidolia was measured by a binary choice (yes or no). In contrast, in this study, we measured the degree of illusion by using a Likert multi-scale questionnaire to ask participants to rate the strength of the pareidolia. Furthermore, the participants were asked to choose an object category and draw what they saw in the random-dot images in order to investigate the content of the illusion.

In this study, we investigated to what degree rhythmic synchrony and active versus passive participation affected the strength of pareidolia, even in a virtual drumming session with agents when the rhythm could be strictly controlled.

2. Methods

2.1. Participants

The participants were 20 healthy adults without mental illness (average age: 23.6 years SD = 1.9; 13 women). The participants were recruited from undergraduate and graduate students of Doshisha University and we also checked that they did not have strong religious beliefs or psychiatric disorders at the interview. The content of the experiment was planned in accordance with the Declaration of Helsinki and received the approval of the ethics committees of the Graduate School of Engineering, Osaka University, and the Faculty of Psychology, Doshisha University. The content of the experiment was explained to participants before the experiment, and they voluntarily participated after signing a consent form.

2.2. Task and Experimental Conditions

Figure 1 shows the sequence of trials and an example of white noise image used in the experiment task. The task involved three drumming agents (cartoon images of robot arms drawn by the author, H.T.) beating drums in succession on a computer screen, at variable intervals. We investigated the strength of pareidolia within an image of random dots displayed in the center of the screen under two by two conditions. One condition was is-sync or out-of-sync conditions (in-sync condition—the variance of the interval of the agents beating the drum was small; out-of-sync condition—the variance of the interval was large) and the other condition was active or passive conditions (active condition—the participants operated one drumming agent for rhythmic synchrony; passive condition—only virtual agents operated the three agents beating the drums). All of the conditions were prepared as within-subject design. During the task, the participants subjectively evaluated the strength of the illusion on a four-point Likert-scale and selected the shape they saw from nine given categories by pressing the corresponding button. Furthermore, the participants were asked to report the precise image of the visual illusion through drawings when they saw noteworthy objects in random dots.

2.3. Experimental Procedure

The experiment was conducted in a dimly lit room. The participants were seated on chairs and performed the task on laptops (13-inch display) that were placed on a desk in front of them. The participants listened to the drumming beats through headphones.

First, the instructions were displayed on the screen for two seconds for both the active condition (in which the participants pressed a key to beat one of the three drums) and the passive condition (in which the computer beat the three drums and the participants did not press any key). Next, the gaze point was displayed in the middle of the screen, and images of the three drums and the three robot arm agents were shown (please refer to Figure 1 for the positions). In the active condition, the participants pressed a key to beat the drum; they were first instructed to beat the drum when the gaze point was displayed. In the passive condition, after the gaze point was presented, the robot arm below it began to beat the drum. Three drums were sequentially beaten with intervals starting from the bottom drum, followed by the left, and then the right drum; this order was then repeated. In the active condition, the participants were instructed to try to beat the drum as far as possible in order to match the rhythm of the other drums being beaten by the virtual agents. The other two virtual agents would beat the drums following a timing determined stochastically in accordance with the normal distribution based on the previously beaten drum. In the in-sync condition, the mean was 0.5 s, and the standard deviation was 0.05 s; in the out-of-sync condition, the mean was 0.5 s, and the standard deviation was 0.25 s. After the gaze point had been displayed for three seconds, a square image of random dots was generated from white noise (see Figure 1) and it displayed in the same position as the gaze point for 20 s. The three drums continued to beat while the image of random dots was displayed. Twelve specific patterns of random dots were prepared based on white noise and each was randomly presented at each trial. Each specific random dot pattern was only presented once in each experimental condition. In the active condition, the participants continued to beat the drum. In addition, they were instructed to continuously look at the gaze point and then look at the image of random dots. After it had been displayed for 20 s, the participants evaluated the degree to which they had seen something meaningful in the image of random dots on a four-point Likert scale: (1) I saw absolutely nothing, (2) I saw hardly anything, (3) I saw something a little, (4) I clearly saw something. After that, from among nine categories, the participants selected the category to which the shape that they had seen in the image of random dots belonged: (1) the face of an actual living person, (2) the face of an imaginary person, (3) the entire body of an actual living person, (4) the entire body of an imaginary person, (5) an actual animal, (6) an imaginary animal, (7) a plant, (8) other, and (9) I did not see anything. The participants pressed a key to evaluate and choose categories. In addition, for cases where the participants saw something remarkable in the random dots, a pen and paper were placed by the side of the computer, and the participants were instructed to draw the precise image that they saw.

In a random order (experimental conditions were randomly arranged for each trial), the participants completed the trials a total of 48 times, 12 times for each of the four conditions of the active and in-sync, passive and in-sync, active and out-of-sync, and passive and out-of-sync conditions.

3. Results

3.1. Analysis of Rhythmic Synchrony

The mean beat intervals of the three drums were fixed at 0.5 s, regardless of the experimental condition. The variance in the intervals between the drumbeats was calculated to quantify the in-sync and out-of-sync conditions as well as the extent to which the interval between the three drumbeats were in sync. In the in-sync and active and the in-sync and passive conditions, the variance was 0.0129 s² and 0.0013 s², respectively; in the out-of-sync and active and the out-of-sync and passive conditions, the variance was 0.382 s² and 0.294 s², respectively. Following the analysis of variance (ANOVA) of the two factors, the variance of the interval between the drumbeats was significantly greater in the out-of-sync condition than in the in-sync condition (F(1,19) = 204.570, p < 0.0001). This confirmed that there was a difference between the in-sync and out-of-sync conditions in the rate of synchrony of the interval between the drumbeats. In addition, the variance of the interval between the drumbeats was significantly greater in the active than in the passive condition (F(1,19) = 7.221, p < 0.05). This can be attributed to fluctuations, including the rhythm of the participants themselves, which increased the variance in the interval between drumbeats in the active condition.

3.2. Analysis of Pareidolia Induced by Rhythmic Synchrony

Next, the strength of pareidolia experienced by participants within an image of random dots displayed in the center of the screen was evaluated while using a four-point Likert scale. The mean number of times each option was selected for each condition are shown in

Table 1. The mean number of times each option was selected for each condition.

	Sync-Action	Sync-Passive	Insync-Active	Insync-Passive
(1) I saw absolutely nothing	6.00 (SD 3.15)	6.75 (SD 2.50)	7.20 (SD 3.40)	6.90 (SD 3.30)
(2) I saw hardly anything	2.90 (SD 2.95)	2.25 (SD 2.23)	2.35 (SD 2.22)	2.40 (SD 2.35)
(3) I saw something a little	2.85 (SD 2.11)	2.80 (SD 2.39)	2.3 (SD 2.95)	2.65 (SD 2.76)
(4) I clearly saw something	0.25 (SD 0.55)	0.20 (SD 0.41)	0.20 (SD 0.52)	0.10 (SD 0.31)

. The mean scores were 1.78, 1.70, 1.62, and 1.66 in the in-sync and active, in-sync and passive, out-of-sync and active, and out-of-sync and passive conditions, respectively (left of Figure 2). We performed a two-way ANOVA to investigate how the difference of conditions affected the degree of pareidolia (dependent variable) and found a significant main effect in the in-sync vs. out-of-sync conditions (F(1,19) = 4.720, p < 0.05). However, there was no significant main effect in the active/passive conditions (F(1,19) = 0.098, n.s.) and in the interaction between the two independent variables (F(1,19) = 1.350, n.s.). Furthermore, within the categories of the shapes that the participants saw in the image of random dots, the mean of the ratio of the selection of categories in which the participants saw something meaningful (all the categories other than “I didn’t see anything”) was 45.0, 40.8, 35.5, and 36.0% in the in-sync and active, in-sync and passive, out-of-sync and active, and out-of-sync and passive conditions, respectively (right side of Figure 2). We performed a two-way ANOVA to investigate how the difference of conditions affected this ratio (depending variable) and found a main effect in the in-sync vs. out-of-sync conditions (F(1,19) = 6.819 p < 0.05). However, there was no significant main effect in the active/passive conditions (F(1,19) = 0.461, n.s.) and in the interaction between the two independent variables (F(1,19) = 0.533, n.s.).

We also condensed the nine categories into four: animals, plants, other, and none (namely, “I did not see anything”). Figure 3 shows the ratios of the participants’ selections for each category. The participants tended to see shapes of living creatures rather than non-living objects in the patterns of random dots. Based on a chi-square test, no significant differences were found in the ratios of categories of shapes seen when participants reported that they had “seen something”.

When the participants saw a noteworthy image during the experiment, they were asked to draw the illusion that they saw as best they could. Figure 4 shows some typical examples of pareidolia images drawn by participants. In this way, the participants reported seeing reasonably specific shapes as their pareidolia illusions, including shapes of people, faces of animals, and flowers in a vase.

3.3. Analysis of Consistency of Participants’ Responses

Twelve specific patterns of random dots were each presented to participants four times during the experiment. In order to investigate the extent to which they selected consistent categories for the same patterns of random dots, the consistency rate for the selection of categories by each participant for the same patterns of random dots (what percentage of the same category was selected in each random dot image among the participant’s choices) and the overall consistency rate, regardless of whether the patterns of random dot images were compared.

The results of these comparisons showed a strongly significant trend for participants to select similar categories for the same patterns of random dots (for the same patterns, the mean of the specific image category consistency rate was 51.3%, while the mean of the overall category consistency rate was 45.8%; paired t-test, t(19) = 3.46, p < 0.01). Conversely, on comparing each participant’s consistency rates on category selection for the same patterns of random dots with regard to all of the other participants, as well as the category consistency rate, regardless of random dot images, no significant differences were observed (for the same pattern, the mean category consistency rate was 39.1% while, for all the patterns, the mean category consistency rate was 38.9%; paired t-test, t(19) = 1.318, n.s.). This suggests that each image of random dots did not contain a universal feature or information that would make it easy for participants to see an illusion of a specific category.

4. Discussion

In this study, we investigated to what degree rhythmic synchrony affected the strength of pareidolia in a virtual drumming session with agents. The results showed that the degree to which they experienced pareidolia was relatively stronger in the condition in which the interval of the drumbeats was in-sync (equal intervals) than in the condition in which they were out-of-sync. Conversely, there were no major differences in terms of the types of shapes they saw, regardless of the presence or absence of rhythm synchrony. Overall, there was a strong tendency for participants to see animals. In addition, the same participants tended to see illusions of the same category within the same images of random dots. This means that, rather than ignoring the visual features of the random dots and selecting a category based on their imagination, the participants actually saw an illusion corresponding to the visual features of random dots. Conversely, between the participants—for example, even for the same image of random dots—they tended to see illusions of shapes from different categories. This signifies that there was no universal feature contained within the random-dot patterns that would cause participants to see illusions in specific categories; in other words, the specific content of the illusions seen by the participants was not shared between them. This finding suggests that a factor not directly related to vision—rhythm synchrony—affected the participants’ visual experience of pareidolia. Although there have been several studies on the effects of synchronized (in-sync) rhythm on human psychology, to our knowledge, this study presents a new finding that participants experienced a visual illusion more strongly when a rhythm stimulus was in-sync than when it was out-of-sync.

Why was it, then, that the participants experienced pareidolia more strongly when presented with an in-sync rhythm than when presented with an out-of-sync rhythm? One interpretation is the difference between the conditions in terms of the cognitive resources the participants allocated to the images of random dots (the degree of difficulty of the task). In other words, it was easier for the participants to adapt to the simple rhythm in the in-sync condition than to the irregular rhythm in the out-of-sync condition; as a result, they allocated more of their cognitive resources in the in-sync condition to the image of random dots, thereby experiencing the illusion more strongly. However, there were no differences in the degrees to which participants experienced pareidolia between the active condition—in which they themselves created the rhythm and hence experienced a high cognitive load—and the passive condition—in which they did not have to create the rhythm and, hence, experienced a low cognitive load. Therefore, it is difficult to explain the changes due to rhythmic synchrony in the degree to which they experienced pareidolia simply from differences in the allocation of cognitive resources.

We believe that the relationship between rhythm synchrony and pareidolia may be explained from the perspective of prediction. It is believed that our brains are constantly forecasting changes based on bottom-up input from the environment, whether we are conscious of this or not, and our brains are always adjusting top-down processing to reduce prediction errors [21]. For example, it has been reported that, when listening to a musical performance, our brains detect when the synchrony of the rhythm collapses, which produces specific brain activity [22]. It is considered that this sort of adapting mechanism for prediction errors maintains cognitive consistency by activating top-down processing, such as in the frontal lobe. In this sort of state, it is possible that perceptual experiences that deviate from the context are more likely to be suppressed. Conversely, as prediction errors do not occur in a condition in which the drum rhythm is in sync, top-down processing, such as in the frontal lobe, is weakened, and the sensitivity of the visual cortex to bottom-up feature quantities is increased. Consequently, it is possible that the brain is more likely to remember as illusions the various shapes recalled from partial feature quantities.

5. Limitation and Future Work

In this study, we only focused on young university students without strong religious beliefs. The results may imply that the illusionary experience induced by rhythmic synchrony is a kind of perceptual illusion that occurs, regardless of religious belief. However, various individual traits may have affected the results of our experiment. Hence, future experiments should strictly control various individual traits (e.g., age, the strength and content of religious belief) and investigate how these traits would affect the strength of pareidolia induced by rhythmic synchrony. Furthermore, the issue of reproducibility in psychological experiments has been recently discussed [23]. It is also necessary to evaluate the reproducibility of the results reported in this study by conducting similar experiments with various participants in various situations in order to evaluate the universality of our findings.

What factors were integral to the experimental results in this study are still debatable. In this study, a cartoon agent was used as the experimental stimulus, and the movements of this agent consisted of both visual and auditory modalities. One might argue that the amount of information that was given by the agent was too rich and distinctive. A simpler physical stimulus, such as beep sounds presented in a synchronized rhythm, might have the same effect as reported in this paper. In this case, the present results can be interpreted in terms of perceptual psychology. For example, a synchronized (patterned) auditory beep may be effective as a priming stimulus to facilitate pattern search in random dot images (a similar phenomenon was reported in [20]). Therefore, the extent to which the agency (anthropomorphic nature) of beings with synchronized rhythms contributes to the results needs to be carefully examined in the future works.

Furthermore, it will be necessary to build a computational model to explain the phenomenon in more detail and verify it experimentally. One good reference material to use when creating this model is the deep neural network (DNN) widely researched in recent years [24]. Although the DNN is known to have high image recognition capability, it has been reported that integration of the perceived partial feature quantity and the memories accumulated in the network can produce image transformations, such as pareidolia. In the future, an in-depth investigation of the relationship between DNN pattern changes and image transformations may explain the psychological factors that strengthen pareidolia. In addition, using techniques, like fMRI, to measure brain functions to analyze details of the brain activity level (such as in the visual and prefrontal cortex when performing this task) and the functional linkages between the brain regions might clarify the neural basis for this phenomenon.

Going forward, the current findings will provide clues toward a quantitative understanding of our psychological states during rituals that involve synchronized rhythm in which participants experience visual illusions, and toward discovering what specific functions rituals have played in human societies. In the current experiment, the shapes seen as illusions differed among participants, even for the same image of random dots. However, if, in an actual ritual, the experience of visual illusions is limited only to individuals, this will not induce feelings of unity within the group and it seems necessary for the participants to experience some sort of shared illusion. Sherif showed that, when a visual illusion is induced in more than one person, the illusions that were inconsistent at first gradually become consistent within the group [25]. "Shared reality" is strongly associated with the formation of bonds between different individuals [26] and it has also been reported that the feeling of sharing the same reality as the robot strengthens the bond between humans and artificial robots [27]. In the future, regarding the aspect of group rhythmic synchrony considered in this experiment, it is hoped that, by clarifying how individual visual illusions come to be shared by an entire group, it will be possible to elucidate in detail the psychological processes that participants in rituals undergo.

Finally, in the current study, the task was performed virtually on a laptop computer. However, in the future, we would like to make a similar attempt in a real situation and present participants with a strong illusionary experience by manipulating factors that are unique to the real situation (e.g., brightness and temperature of the room). For this purpose, we have developed a drumming robot with a robot arm (Figure 5) [28], through which we hope to replicate the current experiment in a real situation. We expect that the replicated experiment will provide deeper insight into supernatural phenomena experienced during religious rituals.

6. Conclusions

Although there are still many considerable points, this study is the first to report the relationship between the synchronization of rhythms between different agents and the pareidolia illusion. We believe that our findings provide good insight into the illusion of synchronized group rhythms, a phenomenon commonly reported in religious rituals, from the perspective of cognitive psychology and neuroscience.