Next Article in Journal
Hox Gene Collinearity May Be Related to Noether Theory on Symmetry and Its Linked Conserved Quantity
Previous Article in Journal
Analysis of Dynamical Heat Conductivity of the Reservoir and Fluid Evacuation Zone on the Gas Condensate Well Flow Rate
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Elderly Body Movement Alteration at 2nd Experience of Digital Art Installation with Cognitive and Motivation Scores

1
The Graduate School of Sciences and Technology for Innovation, Yamaguchi University, Yamaguchi 755-8611, Japan
2
Pediatrics, Saitama Medical University, Saitama 350-0495, Japan
3
Suncoop, Yamaguchi 753-0872, Japan
4
Wisconsin National Primate Research Center, University of Wisconsin, Madison, WI 53715, USA
5
Department of Cell and Regenerative Biology, University of Wisconsin, Madison, WI 53705, USA
6
Graduate School of Information Sciences, Tohoku University, Sendai 980-8579, Japan
*
Authors to whom correspondence should be addressed.
J 2020, 3(2), 138-150; https://doi.org/10.3390/j3020012
Submission received: 18 December 2019 / Revised: 3 April 2020 / Accepted: 6 April 2020 / Published: 14 April 2020

Abstract

:
The prevalence of advanced medical treatment has led to global population aging, resulting in increased numbers of dementia patients. One of the most intractable symptoms of dementia is apathy, or lack of interest and enthusiasm, which can accompany memory and cognitive deterioration. Development of a novel method to ameliorate apathy is desirable. In this feasibility trial, we propose a series of digital art installations as a candidate dementia intervention approach. Seven, three-minute scenes of digital images and sounds were presented to visitors either passively or in response to their reactions (motion and sound). We evaluated the potential of this application as an intervention against apathy in an elderly living home. We collected the dementia global standard Mini-Mental State Examination (MMSE) and questionnaire scores of sensory pleasure and motivation along with behavioral motion data in twenty participants. We further compared responses between the first and second experiences in the thirteen participants that were present for both days. Overall, we found a significant increase in participants’ motivation. In these subjects, head and right hand motion increased over different scenes and MMSE degrees, but was most significant during passive scenes. Despite a small number of subjects and limited evaluations, this new digital art technology holds promise as an apathy intervention in the elderly and can be improved with use of wearable motion sensors.

1. Introduction

The world population is aging and with this, there is a concomitant increase in the number of dementia patients. Apathy, or the persistent lack of motivation to do things, is known to occur with aging, and recent evidence suggests that it may be a behavioral marker for more rapidly progressing forms of dementia. Therefore, tools to measure “motive” and methods to support positive emotions in older people are of great importance for human mental health. Clinically useful diagnostic and intervention tools for apathy must be more powerful, but must also be able to detect the early, preclinical condition. By designing home-use tools for this using information technology, there is the potential to promote novel cognition in ways that may be particularly important for targeting decreased motivation associated with aging in this population. Sensor controls used in digital art, such as depth image sensors with three dimensional (3D) reconstructing applications and software libraries like Processing [1] (https://processing.org/) and openFrameworks [2] (https://openFrameworks.cc/) have improved dramatically over the last ten years, becoming more available and easier to use. Using such tools, five students developed a digital art installation designed to diminish apathy in the elderly population in the creative design workshop at our university. This installation was then exhibited at our regional city illumination event, ‘Tokiwa fantasia’ (https://www.tokiwapark.jp/event/tokiwa2018.html) for one month around the New Year’s holidays. Through this exhibition, we confirmed that the installation could successfully entertain visitors of varied ages, including older people, throughout a one-month period. In this paper, we describe the development of this innovative aging mental support tool and report on the results of a verification trial of this tool performed at a retirement home.
The elderly is often not comfortable with novel technology and the apathy associated with age-related neuronal deterioration can exacerbate this discomfort [3]. Social interaction and communication between younger and older people can help alleviate the apathy associated with aging [4]. The digital installation developed by the students highlights the benefits of psychological promotion and social facilitation for people of all ages, including the elderly. To evaluate this, we assessed the ability of the students’ digital art installation to ameliorate aging apathy in twenty elderly subjects living in a retirement group home. Our intention was to develop technology appropriate for use in the home, as opposed to the clinic. In order to accomplish this, our goal was to determine if elderly people would willingly participate and interact with the installation in a relaxed, stress-free environment, similar to their everyday life, as opposed to a more structured randomized clinical trial-type design.
The experiment consisted of having the subjects experience a series of seven animation scenes. Four of the scenes were human-interactive using a 3D depth camera sensor with reactive animation/sound controlled by Processing [1] and included background healing music (Figure 1, m1–m4). Another scene was also interactive, but utilized a microphone sensor controlled by openFrameworks [2] (Figure 1 (s1)). The final two animation scenes were presented passively to the subjects (Figure 1 (p1, p2)). Each animation was designed to stimulate the subjects brain using sight, sound, motivation and motor function [5]. An important unique feature of our hardware was the 3D effect we achieved by using multiple layered half-transparent screens to stimulate spatial cognitive function in the elderly. In order to enter this space, the subjects needed to shake the screen.
In addition to the presentation part of our aging intervention system, we included a second part designed to quantitatively analyze the psychological and learning effects of interacting with the system to provide feedback for future improvement. In the current study, we explored the possibility of meaningfully detecting joint motion by manually tracking video images every second during the most active 15 s period. If this works well, there is the future potential to automate this analysis with the use of wearable or remote sensors. Elderly with dementia may present with behavioral hypo- or hyper-responsiveness [6,7]. To assess this, motion of the upper body including the back, neck, both hands and shoulders were acquired with different installation scenes. After each testing day, subjects completed questionnaires of sensory pleasure and motivation. The detected motion patterns were analyzed taking the psychological questionnaire and cognitive impairment as measured by the Mini-Mental State Examination (MMSE) [8] into account. We chose to use the MMSE for several reasons including its ease of administration, minimal exam and analysis time for both the subjects and the caregivers, and familiarity of the caregivers with this exam as it is the current standard examination used at the retirement home. The questionnaires of sensory pleasure and motivation were also carefully chosen for simplicity and to avoid any discomfort for the participants and caregivers. This preliminary study was designed to assess psychological aspects of the evaluation as opposed to deeper clinical questions. Specifically, we were interested in determining if older people would be willing to participate in the digital art entertainment and to understand whether this type of entertainment could play a role in normal elderly life.

2. Materials and Methods

2.1. Study Fundamental Protocols

The Institute Review Board of Yamaguchi University, and the Department of mechanical engineering reviewed and approved this study (2016004). All the study members have complied with the approved contents. The retirement home was randomly chosen and approached about participation in this study. Retirement home caregivers invited all residents of the retirement home to participate in this randomized trial, regardless of clinical background. Letters of intent to participate were signed by the participants or their responsible family members and were managed and controlled by the retirement home administration. Each participant underwent the same evaluation twice, Day 1 and Day 2, separated by one week. Evaluations were administered by the home caregivers at the participants’ retirement home in January, 2017. One to four subjects participated in the digital art entertainment installment at a time. Individual subjects selected who they would undergo the experience with, resulting in a random selection. During the entertainment, familiar caregivers interacted with participants with normal daily conversation to keep their minds relaxed. The students’ interactions with the participants were limited to a quick greeting, short responses to questions, or supporting the participant’s hand so they wouldn’t fall.

2.2. Participant Age, Sex, and MMSE Scores

The caregivers were also responsible for protecting the subjects’ personal information and assigning participant identification numbers which were then shared with the study personnel. Cognitive function for each subject was determined by MMSE scoring performed in the retirement home by caregivers within one month of the experiment. Participants were scored on MMSE from a maximum of 30 to a minimum of 9. Thirteen females (age: 83.8 y, 4.0 sd) (MMSE: 24.2, 6.2 sd) and seven males (age: 84.4 y, 5.6 sd) (MMSE: 25.0, 4.8 sd) participated in the first day of testing. A total of nine females (age: 85.4 y, 2.0 sd) (MMSE: 23.4, 6.9 sd) and four males (age: 86.9 y, 6.9 sd) (MMSE: 22.0, 4.2 sd) attended both the first and second test days. The average score per total and standard deviation (Ave/total, SD) of MMSE qualitative assessment were “orientation time” (3.1/5, 1.9), “orientation place” (3.8/5, 1.5), “registration” (2.8/3, 0.8), “attention and calculation” (2.8/5, 1.6), “recall” (1.8/3, 1.2), “language” (7.8/8, 0.6) and “copying” (0.92/1, 0.27). Subjects dropped from the study due to scheduling reasons, not because they found the experience unpleasant. No special steps were taken to introduce the intervention to the participants.

2.3. Digital Art Works Aapparatus and Presentation

Five university students developed the apparatus hardware including the 3D-effect screen space made by multiple parallel transparent plastic sheets (length 150, width 60, thick 0.03 mm), a Kinect sensor (Xbox360), PC, cameras and microphones (Figure 1a). The same students assisted the caregivers in guiding participants into and out of the entertainment area, and remained in the area as assistants to the caregivers.
Seven scenes (Figure 1b) were operated by either Processing [1] (Figure 1, m1–m4) or OpenFrameworks [2] (Figure 1, s1), or were prepared by animation maker applications (Adobe Creative Cloud) (Figure 1, p1,p2). Processing was accomplished using the website source codes (Kinect4W 2.0 library for Processing [9]) and were designed to respond to the participant’s motion using a Kinect 3D whole body sensor. OpenFrameworks referred to the website source codes (Particle system 20189713 OpenFrameworks [10]) and used a microphone sensor to respond to the participant’s voice and handclap. Adobe Creative Cloud was used to create animations for passive presentation to the participants. The digital art default output consisted of a simple symbol mark (circles, lines, leaves, gates) in motion in each regular speed coefficient [9,10] on a two dimensional background. The interactive function (Figure 1, m1–m4, s1) depended on sensor detection of the subjects’ motion and sound in the marks’ direction of movement and alteration in the two dimensional speed of output.
Each of the seven scenes took approximately three minutes, and were presented serially in random order for a total experience of approximately twenty-five minutes.

2.4. Questions of Sensory Pleasure and Motivation and Analysis

Immediately following their experience, a student asked each participant three types of Lickert five-point scale questions [11] to determine their subjective impression of the digital installation; Q1. Visual pleasure: “Was it visually pleasant?”, Q2. Auditory pleasure: “Was it auditorily pleasant?” and Q3. Motive: “Do you currently feel motivated?”. Linear regression was used for correlation analysis between MMSE and sensory pleasure and motivation scores using Excel (Microsoft) and R software.

2.5. Data Acquisition and Analysis

Participant reactions during each of the seven scenes were recorded by video camera (Sony Handycam) without utilizing any wearable sensors to avoid causing any stress to the participants. To analyze each subjects’ representative motion per scene, observers manually focused on the most active fifteen seconds during the scene and obtained the video coordinates of the seven body joints per second by visual identification. This manual tracking method was considered appropriate given that in qualitative pre-observation of the overall video data, each scene was found to have uniform function over the duration. We first checked the validity of the motion data automatically acquired by the Kinect sensors against the data acquired manually. We found that the sitting elderly people move much less than other generation’s, one causing multiple acquisition failures by the Kinect sensors compared to the much more precise manually acquired data. Consequently, we adopted the manual method. To validate the reproducibility of the manual tracking over all the observers, we undertook preliminarily training with the same video data and feedback for each observer’s own difference, to acquire the same coordinate as well as time. The video data were converted into jpeg files and coordinates were measured using TMPGEnc and ImageJ (NIH) software. Next, each joint velocity per second was calculated (Excel, Microsoft). Several sets of video data were used to confirm acceptable inter-rater reliability for the manual measurements. These motion data were normalized by each participant’s sitting height measured from the same video data. Means and standard deviations were screened and data including MMSE and questionnaire scores (explained in Section 2.4) were explored by single or multiple regression model correlation coefficients (R square value) [12], principal component analysis (PCA) by correlation matrix [13] and multivariate analysis of variance (MANOVA) and Wilks’s lambda [14] in Excel (Microsoft) and R.

3. Results

3.1. Pscyhological Facilitation Effect on Motive

Question collected scores of visual (Figure 2b) and auditory pleasure (Figure 2c) were very positive with no difference between Days 1 and 2, while the motivation score indicated higher motivation on Day 2 than Day 1 (Figure 2d). No significant correlations were found between the question scores to MMSE (Figure 2a).

3.2. Behavioral Alteration with Repeated Experience Taking Cognitive Function into Account

Next, we analyzed subjects’ behavioral alteration in response to experiencing the scenes (Figure 3a) twice (Day 1 and Day 2) in relation to cognitive status, as measured by MMSE. Behavioral alteration was assessed by tracking motion of the seven body joints indicated in Figure 3b. Figure 3c is a representative graph of the comparison between Day 1 (blue) and Day 2 (red). Participants’ motion was suppressed on Day 1, regardless of MMSE scores, but there was a direct correlation between motion and MMSE score on Day 2 at Scene p2.
To search for statistical significance of this relationship, we created a heat map of linear regression probabilities for each body joint per scene and day (Table 1, pink = present (p value closer to 0), green = present (p value closer to 1)). In Figure 3c, certain alterations of the head motion were recognized between different scenes, however, it could be viewed as a common response such as restricted motion on Day 1. There were more positive regression lines on Day 2, regardless of scene. To characterize details of motion specificity per scene in samples with lower variability, we attempted to explore each variance by linear regression probability comparison with a visual heat map strategy. While none of the probabilities were significant, all body joints in p2 Day 2 were pink, suggesting some potential correlation between upper body motion and MMSE.
We further analyzed the motion-MMSE plots (e.g., Figure 3c) for differences between Day 1 and Day 2 (Table 2). We confirmed that the participants’ head (PHead = 0.02), neck (PNeck = 0.03) and hip motion (PHip = 0.04) was significantly greater on Day 2 than on Day 1 during Scene p2, as was head motion (PHead = 0.03) during Scene p1. We further found less movement during Scene m3 (PLhand = 0.96) and Scene s1 (PLhand = 0.98). Gray arrows show the representative examples further analyzed in Figure 4.

3.3. Featuring Scene-Specific Factor (Motion, MMSE, Sensory Pleasure, and Motive) Contribution by PCA

Since our regression models suggested some difference in response to scenes based on classification (motion (m1–m4), sound (s1) or passive (p1, p2)), we further explored the contribution of scene-specific factors utilizing PCA. To limit the factors considered for the reliability optimization, we chose to focus on representative factors ‘Head’ and ’Rhand’ because they were the most powerful contributors to differences between Day 1 and Day 2. PCA was calculated using these motion factors and age, MMSE scores, and visual/auditory pleasure and motive ratings (Figure 4). Because the cumulative contribution ratio from the 1st to 3rd reached 73%, we compared the data distribution between Day 1 and Day 2 on the 1st–2nd (Figure 4a–h) and 1st–3rd (Figure 4e–h) component dimensions with the positive factor loading vectors projecting from the mean center (Figure 4d,h). Each scene in Figure 4 is representative of the minimal probability estimated by MANOVA of Day 1 to Day 2 difference in either of scene categories motion (Figure 4a,e), sound (Figure 4b,f) and passive (Figure 4c,g).
The gray arrows in the PCA for Scene p2 (Figure 4) show significant shifting in the 1st–2nd components (Figure 4c), and 1st–3rd components (Figure 4g). These distributions and shifting patterns can also be seen in Scene m4 (Figure 4a,e), though they are not significant, while the patterns for Scene s1 were less diverse during the Day 2 experience (Figure 4b,f; blue plots and covariance ellipse).
Figure 4d indicates contributing factors to the pattern shift from Day 1 to Day 2. Two major vectors (gray) were independently discriminated in the 1st–2nd components; increase in visual and auditory pleasure during all scenes (a–c) and increased Head and Rhand motion (a, c but not b). In the 1st–3rd components (Figure 4h), shift vectors generally indicate co-elevation of motion, sensory pleasure and positive motivation (e–g). However, factors MMSE and age contributed less in both the 1st–2nd and 1st–3rd components, as seen by vectors that are too short or vertical (h) to the gray shift vectors’ direction.

4. Discussion

The digital art installation was originally presented as an evening (17:00–21:00) illumination amusement in the regional play land around the New Year’s holiday. Many visitors of varied ages who had never experienced such an installation were able to spontaneously enjoy both the active and passive functions included. Given this success, we attempted to recreate the entertainment at the retirement home for the elderly who were not able to visit the original installment because of cold and dark winter nights. To adapt this installation to a potential in-home intervention to decrease apathy in the elderly, we designed the examination to be easy and stress free to perform for both the elderly participants and their caregivers. A comparison to clinical definitions for terms like “intervention” or “dementia” was undertaken. Then, all the older participants’ emotional responses were recorded in their own voice, showing that they were comfortable with video data. This was used to analyse the first reaction to the experience and a subsequent experience one week later to find comparative differences in behaviours between active and passive scenes in their usual home. The brief and simple interview was designed to enable each participant to immediately share their acute experiences after the unfamiliar experience. The participant’s visual and auditory impression of the installation revealed that it was a highly positive experience, but scores did not change between Day 1 and Day 2 (Figure 2b,c). In contrast, the question scores indicated that the participant’s motivation was significantly elevated on Day 2 compared to Day 1 (Figure 2d), comparing to visual or auditory pleasure (Figure 2b,c), generally with lower correlation with MMSE (Figure 2a).
Apathy, not clinically determined, but a simply decreased willingness to act, that was confirmed by the interview scores, is a serious negative characteristic observed in older people. Our students chose to target apathy in their next generation art work installation. In the elderly, spontaneous and active approaches to objects tend to decline due to motor, sensory, cognitive or emotional neuro-dysfunction. To prevent or delay age-associated apathy, we developed a task that the elderly could participate in, and from which they could gain a sense of accomplishment through motivation leading to self-directed participation. Originally, we assumed that the major impediment to motivation would be lack of familiarity with the art installation, and we learned that motivation did increase on Day 2 compared to Day 1. Furthermore, we found a certain learning-like quantitative effect in response to this novel prototype, shown in differences in head, neck and hip motion on Day 2 that were significantly greater compared to Day 1, especially in passive scenes (Table 2). This difference was not dependent on cognitive condition which suggests that this motivation facilitating program would be valuable for elderly regardless of cognitive condition (Table 1, Figure 3 and Figure 4). Based on our general observations at the winter festival, range of motion in the older people was qualitatively lower than that in younger people. This consistently low level of motion made it difficult to detect their motion in the dark experiment space. Therefore, we chose to track the participants’ motion manually in the video data. Because the stimulation contents of the installation were generally random and constant, it was difficult to distinguish what and when the participant reacted. Preliminary evaluation in our lab suggested a plan to detect the most representative behaviors. We found that the duration of the participants’ most representative behaviors could be determined by evaluating the most active 15 s during each 3 min scene.
Body motion analysis has been previously used to understand psycho-cognitive reactions in seated people in several contexts, including head movements in autistic children in social contexts [15], adult body posture variation during Stroop test performance [16], and use of a computerized tutor that simulates discourse patterns and pedagogical strategies of a typical human tutor [17].
Subjects’ motion was activated more by passive than interactive scenes on Day 2 (Table 2), possibly reflecting the additional time elderly may need to acclimate to unfamiliar situations particularly those requiring interaction. Reports indicate that several weeks of exposure to conventional art, such as sculpture [18], confers benefits for dementia patients. The complexity, precision and balance of artwork created was significantly associated with their visuospatial function and severity of cognitive impairment [19]. Our new approach, using one week of exposure to a digital art installation, was well accepted by older adults, though this approach did not allow for participants to create their own artwork. Our combined active and passive methods hold promise in that they may be varied to personalize the therapy for individual patients with different conditions. Based on studies using conventional art, further development of our virtual digital art methodology to allow patients to create their own 2D or 3D drawings and models and interact with the system over longer periods of time has therapeutic potential.
Regarding the similarity in response between days to the sound scenes, it could be because these did not require any body motion. In fact, we observed several participants singing their favorite songs during Scene s1, possibly enjoying a synchronized image reaction. These findings suggest other parameters that may be required to more fully quantify behavioral output, such as vocal discrimination, for full understanding of psychological expression.
In the multivariate analysis by PCA in Figure 4, we found no less than three kinds of shifting directions from Day 1 to Day 2: 1. Head/Rhand, 2. VisPleas/AudPlease in the 1st–2nd components and 3. motive and VisPlease/AudPleas/Head/Rhand in the 1st–3rd components. The plots, variance ellipses and shift vectors allowed us to generalize responses by feature per scene. Our results suggest the importance of intervention technology being designed that utilizes diverse testing to elicit individual variation, as well as the importance of repeat testing to allow understanding of time and conditional shift directions [15,20,21,22,23,24,25,26].
This program was designed and implemented with the thought that older people would benefit psychologically from interacting with university students completing their engineering education. This is a group that does not normally communicate frequently with older people, at least not in Japan that has the largest population of older people in the world. The students specifically challenged themselves to apply their digital works in a novel way to the question of mental activation in older people. Before our evaluation, we were concerned that cultural differences between the students and older adults might cause discomfort for the older individuals and, as the worst possible case, accentuation of dementia symptoms. Contrary to this fear, the subjects had a pleasant response to the students’ digital works (Figure 2). This might be interpreted as the ability to encourage crosstalk across generations. As the next stage in this research, basic functional responses to the digital work should be considered, such as the effect of a second exposure on brain neuronal activities as measured by electroencephalogram (EEG) [27]. Increased interactions between individuals across the age spectrum is widely believed to be a key factor in improving neuropsychiatric disease outcomes, however, spontaneous communication between young and old is often restrained by differences in neuronal activities between the generations. It is possible that the technology we have developed could play the role of an intermediate in these interactions by attracting both younger and older generations. From our preliminary information, we learned that the participants appreciated daily communication with the younger generation, though this was seldom realized in their living environment. Our design, involving communicative interplay between the generations likely had a strong positive affect on subjects who reported sensory pleasure and motivation [28]. We suggest that such inter-generational interactive programs show great potential for innovative technological approaches to improving apathy associated with the aging process [29]. A limitation of this report is the lack of data on social interactions between the participants and the students or the participants and their caregivers. It is possible that these social interactions may have affected the participants behavior. It is important to note, however, that throughout the experiment the students attempted to communicate with the participants in a calm and consistent manner. Our study paradigm should be furthermore validated considering behavioral interactions between the participants and caregivers and students [15,20,21,22,23,24,25,26].
This study did not directly address the relevant brain locations activated during such a digital entertainment experience. It is likely that cerebral cortex regions including the inferior frontal gyrus, cingulate, Para hippocampal gyrus, parietal lobe, and thalamus, as well as cerebellum are involved [30,31], though further studies are required to clarify this point and to further the development of digital art therapy against aging-associated apathy. A technological limitation of the current study is the manual detection of body motion used. Preferably, an automatic detection method, such as a small wearable sensor like an accelerograph [32,33], would be used.
The individual sensor techniques used in this study were not novel, but taken together they suggest a novel design concept. The series of multimodal sensors used along with the 3D digital art projection output may induce users to interact both actively and/or passively. This has the potential to overcome apathy, one of the most difficult challenges associated with elderly care. The multivariate method we used to correlate changes in individual subjects’ responses between the two weeks of testing offers insight into the slow but progressive learning that the elderly achieved. This may represent learning plasticity and represents an area for potential intervention.
In this study, we chose to limit our neurocognitive and neuropsychiatric assessments to the MMSE and minimal questions to give priority to reducing the stress of the examination process for both the participants and caregivers. The next step in determining the clinical utility of this intervention is to perform more in-depth diagnostic assessments both before and after the intervention using tools such as the functional Clinical Dementia Rating (CDR), Assessment Staging Tool (FAST) [34] and Apathy Evaluation Scale [35] according to the international diagnostic criteria for a multidimensional construct [36].

5. Conclusions

This small study explored the feasibility of using a digital art experience in the elderly to promote their motivation. By their second experience with this digital art installation, we found increasing physical motion reflecting participants’ motivation, parallel to their MMSE and visual/auditory pleasure higher scores. Admittedly, this is a small study. Additional subjects are required to fully evaluate the potential of this intervention for age-related apathy.

Supplementary Materials

The following are available online at https://www.mdpi.com/2571-8800/3/2/12/s1, Video S1: Sound example.

Author Contributions

Study conceptualization, M.K., R.C.; methodology and digital art works, R.S., Y.M., J.T., F.K. (Fujun Kim), Y.D.; validation analysis, T.T., F.K. (Fumio Kinoshita), K.F., K.H., M.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by MICT SCOPE (172308006) Japan and JSPS KAKENHI Grant Number JP17K18648 and P16307. The development of the digital art works was supported by Ube city and Yamaguchi University, Yamaguchi, Japan.

Acknowledgments

We acknowledge the participants and their elderly home supporter, Junko Toyota. We appreciate Mayer Kimiko Kubota, Koji Yasuhira, Daisuke Kihara and Tokiwa park staff of Ube city. We thank Kazuhiko Mitsui, Tadao Fujita, Yasuo Nakano, Mitsuo Sakata, Masao Fujii, Kengo Yahara, Kazuo Higashiya, Ikuo Uemura, Hideharu Tamura and Keiji Shinosaki for their support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Fry, B.; Reas, C. Processing. 2001. Available online: https://processing.org/ (accessed on 18 December 2019).
  2. Lieberman, Z.; Watson, T.; Castro, A. Openframeworks. 2007. Available online: https://openframeworks.cc (accessed on 18 December 2019).
  3. Lanctôt, K.L.; Agüera-Ortiz, L.; Brodaty, H.; Francis, P.; Geda, Y.E.; Ismail, Z.; Marshall, G.A.; Mortby, M.; Onyike, C.U.; Padala, P.R.; et al. Apathy associated with neurocognitive disorders: Recent progress and future directions. Alzheimer’s Dement. 2016, 13, 84–100. [Google Scholar] [CrossRef] [PubMed]
  4. Tullo, E.; Young, T.J.; Lee, R. Medical students’ views about person-centred communication in dementia care. Dementia 2016, 17, 573–584. [Google Scholar] [CrossRef] [Green Version]
  5. Sauvaget, C.; Yamada, M.; Fujiwara, S.; Sasaki, H.; Mimori, Y. Dementia as a Predictor of Functional Disability: A Four-Year Follow-Up Study. Gerontology 2002, 48, 226–233. [Google Scholar] [CrossRef] [PubMed]
  6. De Oliveira, A.M.; Radanovic, M.; De Mello, P.C.H.; Buchain, P.C.; Vizzotto, A.D.B.; Celestino, D.L.; Stella, F.; Piersol, C.V.; Forlenza, O.V. Nonpharmacological Interventions to Reduce Behavioral and Psychological Symptoms of Dementia: A Systematic Review. BioMed. Res. Int. 2015, 2015, 1–9. [Google Scholar] [CrossRef]
  7. Eggermont, L.H.P.; Swaab, D.F.; Hol, E.; Scherder, E.J. Observation of Hand Movements by Older Persons with Dementia: Effects on Cognition. Dement. Geriatr. Cogn. Disord. 2009, 27, 366–374. [Google Scholar] [CrossRef] [PubMed]
  8. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. Mini-mental state. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef]
  9. Thomas Lengeling/KinectPV2. Available online: https://github.com/ThomasLengeling/KinectPV2 (accessed on 18 December 2019).
  10. Nakauchi, J. Programming DE graffiti. Available online: https://junkiyoshi.com/openframeworks20180713/ (accessed on 18 December 2019).
  11. Likert, R. A Technique for the Measurement of Attitudes. Arch. Psychol. 1932, 140, 1–55. [Google Scholar]
  12. Srj Chatterjee, S.; Hadi, A.S.; Price, B. Regression Analysis by Example. J. Am. Stat. Assoc. 2000, 95, 1381. [Google Scholar]
  13. Richards, L.E.; Jolliffe, I.T. Principal Component Analysis. J. Mark. Res. 1988, 25, 410. [Google Scholar] [CrossRef]
  14. Mardia, K.; Kent, J.T.; Bibby, J. Multivariate Analysis; Academic Press: Cambridge, MA, USA, 1979; ISBN 0-12-471250-9. [Google Scholar]
  15. Koshiba, M.; Senoo, A.; Mimura, K.; Shirakawa, Y.; Karino, G.; Obara, S.; Ozawa, S.; Sekihara, H.; Fukushima, Y.; Ueda, T.; et al. A cross-species socio-emotional behaviour development revealed by a multivariate analysis. Sci. Rep. 2013, 3, 2630. [Google Scholar] [CrossRef]
  16. Bibbo, D.; Carli, M.; Conforto, S.; Battisti, F. A Sitting Posture Monitoring Instrument to Assess Different Levels of Cognitive Engagement. Sensors 2019, 19, 455. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Westlund, J.K.; D’Mello, S.K.; Olney, A.M. Motion Tracker: Camera-Based Monitoring of Bodily Movements Using Motion Silhouettes. PLoS ONE 2015, 10, e0138636. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Seifert, K.; Spottke, A.; Fliessbach, K. Effects of sculpture based art therapy in dementia patients—A pilot study. Heliyon 2017, 3, e00460. [Google Scholar] [CrossRef] [PubMed]
  19. Witkoski, S.A.; Chaves, M.L.F. Evaluation of artwork produced by Alzheimer’s disease outpatients in a pilot art therapy program. Dement. Neuropsychol. 2007, 1, 217–221. [Google Scholar] [CrossRef] [PubMed]
  20. Senoo, A.; Okuya, T.; Sugiura, Y.; Mimura, K.; Honda, Y.; Tanaka, I.; Kodama, T.; Tokuno, H.; Yui, K.; Nakamura, S.; et al. Effects of constant daylight exposure during early development on marmoset psychosocial behavior. Prog. Neuro-Psychopharmacol. Boil. Psychiatry 2011, 35, 1493–1498. [Google Scholar] [CrossRef] [PubMed]
  21. Koshiba, M.; Mimura, K.; Sugiura, Y.; Okuya, T.; Senoo, A.; Ishibashi, H.; Nakamura, S. Reading marmoset behavior ‘semantics’ under particular social context by multi-parameters correlation analysis. Prog. Neuro-Psychopharmacol. Boil. Psychiatry 2011, 35, 1499–1504. [Google Scholar] [CrossRef]
  22. Koshiba, M.; Shirakawa, Y.; Mimura, K.; Senoo, A.; Karino, G.; Nakamura, S. Familiarity Perception Call Elicited under Restricted Sensory Cues in Peer-Social Interactions of the Domestic Chick. PLoS ONE 2013, 8, e58847. [Google Scholar] [CrossRef] [Green Version]
  23. Koshiba, M.; Karino, G.; Senoo, A.; Mimura, K.; Shirakawa, Y.; Fukushima, Y.; Obara, S.; Sekihara, H.; Ozawa, S.; Ikegami, K.; et al. Peer attachment formation by systemic redox regulation with social training after a sensitive period. Sci. Rep. 2013, 3, 2503. [Google Scholar] [CrossRef]
  24. Mimura, K.; Nakamura, S.; Koshiba, M. A flexion period for attachment formation in isolated chicks to unfamiliar peers visualized in a developmental trajectory space through behavioral multivariate correlation analysis. Neurosci. Lett. 2013, 547, 70–75. [Google Scholar] [CrossRef]
  25. Koshiba, M.; Senoo, A.; Karino, G.; Ozawa, S.; Tanaka, I.; Honda, Y.; Usui, S.; Kodama, T.; Mimura, K.; Nakamura, S.; et al. Susceptible period of socio-emotional development affected by constant exposure to daylight. Neurosci. Res. 2015, 93, 91–98. [Google Scholar] [CrossRef] [Green Version]
  26. Koshiba, M.; Karino, G.; Mimura, K.; Nakamura, S.; Yui, K.; Kunikata, T.; Yamanouchi, H. Psycho-Cognitive Intervention for ASD from Cross-Species Behavioral Analyses of Infants, Chicks and Common Marmosets. CNS Neurol. Disord.-Drug Targets 2016, 15, 1. [Google Scholar] [CrossRef] [PubMed]
  27. Cid-Fernández, S.; Lindín, M.; Díaz, F. The importance of age in the search for ERP biomarkers of aMCI. Boil. Psychol. 2019, 142, 108–115. [Google Scholar] [CrossRef] [PubMed]
  28. Ballhausen, N.; Schnitzspahn, K.M.; Horn, S.S.; Kliegel, M. The interplay of intention maintenance and cue monitoring in younger and older adults’ prospective memory. Mem. Cogn. 2017, 45, 1113–1125. [Google Scholar] [CrossRef] [PubMed]
  29. Kodadek, L.M.; A Bettendorf, B.; Uihlein, J.A.; Derse, A.R. The Memory Art Project: Medical students and older adults. WMJ Off. Publ. State Med Soc. Wis. 2010, 109, 311–316. [Google Scholar]
  30. Han, U.H.; Bolo, N.; Daniels, M.A.; Arenella, L.; Lyoo, I.K.; Renshaw, P. Brain activity and desire for Internet video game play. Compr. Psychiatry 2011, 52, 88–95. [Google Scholar] [CrossRef] [Green Version]
  31. Dong, G.-H.; DeVito, E.E.; Du, X.; Cui, Z. Impaired inhibitory control in ’internet addiction disorder’: A functional magnetic resonance imaging study. Psychiatry Res. 2012, 203, 153–158. [Google Scholar] [CrossRef] [Green Version]
  32. Hartman, Y.A.; Karssemeijer, E.G.; Van Diepen, L.A.; Rikkert, M.G.O.; Thijssen, D.H.J. Dementia Patients Are More Sedentary and Less Physically Active than Age- and Sex-Matched Cognitively Healthy Older Adults. Dement. Geriatr. Cogn. Disord. 2018, 46, 81–89. [Google Scholar] [CrossRef]
  33. Mc Ardle, R.; Morris, R.; Hickey, A.; Del Din, S.; Koychev, I.; Gunn, R.N.; Lawson, J.; Zamboni, G.; Ridha, B.; Sahakian, B.; et al. Gait in Mild Alzheimer’s Disease: Feasibility of Multi-Center Measurement in the Clinic and Home with Body-Worn Sensors: A Pilot Study. J. Alzheimer’s Dis. 2018, 63, 331–341. [Google Scholar] [CrossRef]
  34. Reisberg, B. Global measures: Utility in defining and measuring treatment response in dementia. Int. Psychogeriatr. 2007, 19, 421–456. [Google Scholar] [CrossRef]
  35. Marin, R.S.; Biedrzycki, R.C.; Firinciogullari, S. Reliability and validity of the apathy evaluation scale. Psychiatry Res. 1991, 38, 143–162. [Google Scholar] [CrossRef]
  36. Robert, P.; Lanctôt, K.; Agüera-Ortiz, L.; Aalten, P.; Bremond, F.; DeFrancesco, M.; Hanon, C.; David, R.; Dubois, B.; Dujardin, K.; et al. Is it time to revise the diagnostic criteria for apathy in brain disorders? The 2018 international consensus group. Eur. Psychiatry 2018, 54, 71–76. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The set structure and digital art installation scenes: (a) One to four subjects participated in the digital art entertainment installment at a time. During participation, motion of each subject was recorded by depth camera placed behind the subjects, and sound was recorded by a microphone placed in front of the subjects. The interactive art projection in the 3D-effect screen space was controlled by CPU. (b) Nine digital image/sound sub-works were categorized into three scene types with associated subject activity sensing: ‘motion’ tracked by a depth camera, ‘sound’ assessed by microphone, both of which function to induce the participants to move their body and speak to alter the digital art work image and sound (a supplemental file), as ‘active’ responses. ‘Passive’ scenes were presented without any subject sensing and no activity-dependent image/sound alteration. Projected output example photos include human motion-dependent images (m1–m4), sound-dependent images (s1) and original images without sensor function (p1, p2).
Figure 1. The set structure and digital art installation scenes: (a) One to four subjects participated in the digital art entertainment installment at a time. During participation, motion of each subject was recorded by depth camera placed behind the subjects, and sound was recorded by a microphone placed in front of the subjects. The interactive art projection in the 3D-effect screen space was controlled by CPU. (b) Nine digital image/sound sub-works were categorized into three scene types with associated subject activity sensing: ‘motion’ tracked by a depth camera, ‘sound’ assessed by microphone, both of which function to induce the participants to move their body and speak to alter the digital art work image and sound (a supplemental file), as ‘active’ responses. ‘Passive’ scenes were presented without any subject sensing and no activity-dependent image/sound alteration. Projected output example photos include human motion-dependent images (m1–m4), sound-dependent images (s1) and original images without sensor function (p1, p2).
J 03 00012 g001
Figure 2. Confirmation of the participant psychological reaction scores from one to five. (a) After digital art entertainment experience, three questions were asked and recorded by experimenters. Each linear regression analysis result is denoted by a blue (Day 1) or red (Day 2) formula and the square of correlation coefficient (R2). (b–d) The results of reactive scores (shown in a) between Day 1 and Day 2 were compared by one-way ANOVA (p: probability).
Figure 2. Confirmation of the participant psychological reaction scores from one to five. (a) After digital art entertainment experience, three questions were asked and recorded by experimenters. Each linear regression analysis result is denoted by a blue (Day 1) or red (Day 2) formula and the square of correlation coefficient (R2). (b–d) The results of reactive scores (shown in a) between Day 1 and Day 2 were compared by one-way ANOVA (p: probability).
J 03 00012 g002aJ 03 00012 g002b
Figure 3. Behavioral alteration from Day 1 to Day 2 considering cognitive scores. (a) An example of three subjects at the digital art installation with a Kinect sensor to detect participants’ motion (a black rectangle on a tripod). (b) Identification of the seven body joints assessed. (c) An example of motion (head motion for the thirteen participants) correlation with MMSE by linear regression analysis on Day 1 (red) and Day 2 (blue). Formulas and R2 for each regression line are presented (red = Day 1; blue = Day 2).
Figure 3. Behavioral alteration from Day 1 to Day 2 considering cognitive scores. (a) An example of three subjects at the digital art installation with a Kinect sensor to detect participants’ motion (a black rectangle on a tripod). (b) Identification of the seven body joints assessed. (c) An example of motion (head motion for the thirteen participants) correlation with MMSE by linear regression analysis on Day 1 (red) and Day 2 (blue). Formulas and R2 for each regression line are presented (red = Day 1; blue = Day 2).
J 03 00012 g003
Figure 4. PCA by correlation matrix revealed distribution pattern shifts from Day 1 to Day 2. Gray arrowed vectors display shifts from Day 1 (red) to Day 2 (blue). Ellipses represent the group mean center with covariance axes calculated by PCA variance-covariance matrix in the 1st–2nd component dimensions (a–d), or 1st–3rd component dimensions (e–h). Three scenes comparison; (a,e) Scene m4: Motion reactive scene, (b,f) Scene s1: Sound reactive scene, (c,g) Scene p2: Not reactive, (d,h). Synthesized factor loading vectors project from the mean center. An asterisk denotes MANOVA (Wilks’ lambda) probability comparison between Day 1 and Day 2 is significant (p < 0.05).
Figure 4. PCA by correlation matrix revealed distribution pattern shifts from Day 1 to Day 2. Gray arrowed vectors display shifts from Day 1 (red) to Day 2 (blue). Ellipses represent the group mean center with covariance axes calculated by PCA variance-covariance matrix in the 1st–2nd component dimensions (a–d), or 1st–3rd component dimensions (e–h). Three scenes comparison; (a,e) Scene m4: Motion reactive scene, (b,f) Scene s1: Sound reactive scene, (c,g) Scene p2: Not reactive, (d,h). Synthesized factor loading vectors project from the mean center. An asterisk denotes MANOVA (Wilks’ lambda) probability comparison between Day 1 and Day 2 is significant (p < 0.05).
J 03 00012 g004
Table 1. Linear regression probability of each targeted joint motion per scene with MMSE score.
Table 1. Linear regression probability of each targeted joint motion per scene with MMSE score.
1
m1m2m3m4s1p1p20.9
PDay1Day2Day1Day2Day1Day2Day1Day2Day1Day2Day1Day2Day1Day20.8
Head0.290.840.750.670.20.560.490.650.430.940.280.60.590.250.7
Neck0.480.80.750.580.380.520.790.690.440.870.480.670.850.260.6
Hip0.310.80.690.650.470.520.620.510.360.570.760.750.40.260.5
Rhand0.240.570.570.560.90.290.740.470.630.890.560.710.590.250.4
Lhand0.320.690.630.630.980.290.870.510.510.880.840.540.820.260.3
Rshldr0.690.70.790.730.390.490.580.570.570.840.560.620.70.180.2
Lshldr0.260.910.880.630.390.50.80.570.280.750.320.710.80.260.1
J 03 00012 i001 J 03 00012 i001 J 03 00012 i001
Table 2. The difference probabilities between Day 1 and Day 2 in the plots of MMSE versus body joint motion by MANOVA. Note * indicates statistical significance (p < 0.05).
Table 2. The difference probabilities between Day 1 and Day 2 in the plots of MMSE versus body joint motion by MANOVA. Note * indicates statistical significance (p < 0.05).
Pm1m2m3m4s1p1p21
Head0.2170.2110.5760.1330.3560.025 *0.024 *0.9
Neck0.4380.4670.4350.1910.5590.1680.034 *0.8
Hip0.4620.3710.3920.3890.6300.1000.042 *0.7
Rhand0.1020.2670.5720.1080.5750.0950.1090.6
Lhand0.2830.6240.9550.5910.9770.1540.1260.5
Rshldr0.7770.3700.5610.3540.7470.2030.0780.4
Lshldr0.4380.5960.7060.5400.8200.1630.0790.3
J 03 00012 i001 J 03 00012 i001 J 03 00012 i0010.2
0.1

Share and Cite

MDPI and ACS Style

Tao, T.; Sato, R.; Matsuda, Y.; Takata, J.; Kim, F.; Daikubara, Y.; Fujita, K.; Hanamoto, K.; Kinoshita, F.; Colman, R.; et al. Elderly Body Movement Alteration at 2nd Experience of Digital Art Installation with Cognitive and Motivation Scores. J 2020, 3, 138-150. https://doi.org/10.3390/j3020012

AMA Style

Tao T, Sato R, Matsuda Y, Takata J, Kim F, Daikubara Y, Fujita K, Hanamoto K, Kinoshita F, Colman R, et al. Elderly Body Movement Alteration at 2nd Experience of Digital Art Installation with Cognitive and Motivation Scores. J. 2020; 3(2):138-150. https://doi.org/10.3390/j3020012

Chicago/Turabian Style

Tao, Ting, Ryota Sato, Yusuke Matsuda, Jumpei Takata, Fijun Kim, Yukio Daikubara, Koji Fujita, Kotaro Hanamoto, Fumio Kinoshita, Ricki Colman, and et al. 2020. "Elderly Body Movement Alteration at 2nd Experience of Digital Art Installation with Cognitive and Motivation Scores" J 3, no. 2: 138-150. https://doi.org/10.3390/j3020012

APA Style

Tao, T., Sato, R., Matsuda, Y., Takata, J., Kim, F., Daikubara, Y., Fujita, K., Hanamoto, K., Kinoshita, F., Colman, R., & Koshiba, M. (2020). Elderly Body Movement Alteration at 2nd Experience of Digital Art Installation with Cognitive and Motivation Scores. J, 3(2), 138-150. https://doi.org/10.3390/j3020012

Article Metrics

Back to TopTop