Next Article in Journal
Prevalence of Positive Effects on Body Fat Percentage, Cardiovascular Parameters, and Cardiorespiratory Fitness after 10-Week High-Intensity Interval Training in Adolescents
Next Article in Special Issue
Multigenerational Social Housing and Group-Rearing Enhance Female Reproductive Success in Captive Rhesus Macaques (Macaca mulatta)
Previous Article in Journal
Prescription Patterns, Recurrence, and Toxicity Rates of Adjuvant Treatment for Stage III/IV Melanoma—A Real World Single-Center Analysis
Previous Article in Special Issue
Quantitative Rapid Test for Detection and Monitoring of Active Pulmonary Tuberculosis in Nonhuman Primates
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Biology
Volume 11
Issue 3
10.3390/biology11030423
biology-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Behavioral Management as a Coping Strategy for Managing Stressors in Primates: The Influence of Temperament and Species

by
Sierra Palmer
1,2,†,
Scott Hunter Oppler
1,2,† and
Melanie L. Graham
1,2,*
1
Department of Surgery, University of Minnesota, Minneapolis, MN 55108, USA
2
Department of Veterinary Population Medicine, University of Minnesota, St. Paul, MN 55108, USA
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Biology 2022, 11(3), 423; https://doi.org/10.3390/biology11030423
Submission received: 22 February 2022 / Revised: 5 March 2022 / Accepted: 9 March 2022 / Published: 10 March 2022
(This article belongs to the Special Issue Husbandry of Non-Human Primates in Breeding and Research: Latest Insights and Developments)
Browse Figures
Versions Notes

Abstract

:

Simple Summary

Species, motivation, and temperament are characteristics that influence environmental perception and learning in animals, and consequently, welfare. We investigated the relationship between these individual traits and training success in primates to acquire skills for cooperation and participation with medical care including sample collection, drug administration, vitals monitoring, and examination. Despite behavioral differences related to temperament, all animals successfully completed the training program without significant differences in time required to acquire target skills. Training time was significantly different between rhesus and cynomolgus macaques, likely reflecting species differences in memory, motivation, reasoning, and learning. However, with the perspective of typical study duration and long lifespan in primates, this difference in time to completion was clinically irrelevant. A well-designed training program that is properly applied can establish positive coping skills in primates across temperament and other traits to strengthen psychological resilience, improving welfare and reducing stress confounding for more accurate scientific translation.

Abstract

Primates involved in biomedical research experience stressors related to captivity, close contact with caregivers, and may be exposed to various medical procedures while modeling clinical disease or interventions under study. Behavioral management is used to promote behavioral flexibility in less complex captive environments and train coping skills to reduce stress. How animals perceive their environment and interactions is the basis of subjective experience and has a major impact on welfare. Certain traits, such as temperament and species, can affect behavioral plasticity and learning. This study investigated the relationship between these traits and acquisition of coping skills in 83 macaques trained for cooperation with potentially aversive medical procedures using a mixed-reinforcement training paradigm. All primates successfully completed training with no significant differences between inhibited and exploratory animals, suggesting that while temperament profoundly influences behavior, training serves as an important equalizer. Species-specific differences in learning and motivation manifested in statistically significant faster skill acquisition in rhesus compared with cynomolgus macaques, but this difference was not clinically relevant. Despite unique traits, primates were equally successful in learning complex tasks and displayed effective coping. When animals engage in coping behaviors, their distress decreases, improving welfare and reducing inter- and intra- subject variability to enhance scientific validity.

1. Introduction

The ability to effectively cope with stressful situations has a major impact on physical and psychological wellbeing in both animals and humans. Coping consists of strategies and behaviors used to manage situations that are perceived as stressful, divided into two types, positive (e.g., creation of a favorable association or reappraisal) or negative (e.g., avoidance and escape) [1]. Chronic, uncontrolled stress in the absence of effective coping has been associated with an increased risk of developing anxiety, depression, and a range of other disorders [2,3,4]. In the healthcare setting, anxiety or depression, as well as overall patient dissatisfaction, can increase the likelihood of poor compliance with drug or treatment regimens, negatively impacting a variety of patient health outcomes [5,6]. In contrast, positive coping has the potential to reduce distress associated with illness and aversive medical interventions to the extent that effective coping strategies have been shown to improve patient quality of life (QOL) as well as decrease morbidity and mortality [7,8,9,10,11]. A combination of appropriate cognitive and behavioral responses is necessary in order to reinterpret, or blunt, aversive events or demands imposed by stressors and effectively cope [12,13]. Animals involved in biomedical research are intended to closely model diseases and therapies under investigation, which not only exposes them to similar stressors that affect patient quality of life (i.e., inherently imposed by a specific disease state and its intensive medical management) but also those related to the introduction of frequent research interventions in a captive environment. Even with the most skilled care and appropriate pain management, if animals are not adequately prepared these interventions can be perceived as unpredictable and uncontrollable. As a result, interactions with an aversive stimulus can have a variety of behavioral consequences including conditioned anxiety, attempts to avoid or escape treatment, or direct aggression towards caregivers. The frequency of medical intervention can intensify distress as animals do not necessarily habituate to aversive procedures simply through repetition alone. Ultimately, the ability to cope with these stressors impacts animal welfare and scientific outcome parameters [14,15,16,17,18].
In the captive setting, behavioral management is a comprehensive approach combining enrichment, sociality, and training to enhance welfare and foster positive coping [19]. Operant conditioning is an important and frequently used tool in learning and behavioral modification for both routine situations and stressful situations where avoidance learning can lead to maladaptive coping. In the research setting, a number of benefits have been observed in animal models trained to cooperate with their own care, including safer animal-caregiver interactions, improved model validity from reduced levels of outcome-confounding stress, and enhanced welfare [20,21,22,23,24]. Likewise, in pediatric patients, the use of behavioral management techniques that incorporate the opportunity to express choice successfully fosters beneficial cooperative behavior during aversive treatments, highlighting the translational relevance of these paradigms [25,26].
The success of any training paradigm is dependent on the acquisition of specific skills and the speed, or efficiency, of learning. Positive reinforcement training (PRT), defined as ‘adding’ a rewarding stimulus (e.g., preference food item, toys, positive social interaction) upon performance of a desired behavior, is generally considered the ideal training approach [21]. However, PRT alone has been shown to be less effective for training behaviors that have mildly aversive outcomes (e.g., pain from a blood collection or drug injection) and require animals to make a value assessment of whether a reward is ‘worth it’ [27,28]. Though the connection is not well understood, prior research has suggested that temperament is correlated with successful training of simple tasks and can influence an individual’s relative predilection for appetitive versus defensive (e.g., avoidance) motivators, affecting eventual cost–benefit decisions and the overall efficacy of PRT-alone training paradigms [29,30]. Primates with more inhibited temperaments, with behavioral tendencies towards withdrawal or apprehension of the unfamiliar [31], have been shown to have a more difficult time learning simple tasks under a PRT-only paradigm compared to primates with exploratory temperaments, suggesting that inhibited animals may place a greater value on avoidance over reward [32]. When PRT alone is ineffective in suppressing avoidance motivated behavior, a concurrent negative reinforcement training (NRT) component, usually a mild unwanted condition (e.g., reduced working space, elimination of additional sessions when animals accomplish behaviors to promote productivity), can be used intentionally and selectively (as opposed to accidentally) because of the inherent aversive stimuli occurring in most medical situations [33]. Giving animals the opportunity to rehearse the behavior necessary for cooperation or coping with procedures in stepwise in a highly controlled safe environment alters experience from an undesirable one to a tolerable or even a sought after experience when effectively converted to reward seeking. This strategy, desensitization, increases familiarity with a task, while reducing fear, to help decrease overall stress related to the task and can foster the expression of choice and cooperative behavior, indicators of successful coping, in future instances of the task [21,34,35].
Coping has also been linked with temperament, or personality [36,37]. Temperament encompasses the consistent emotional and behavioral traits of an individual that are a major factor influencing the subjective environmental experience [38,39,40,41,42]. The dimensions of temperament and personality are remarkably similar across species [43,44]. Rodent temperament has been used to predict anxiety traits and evaluate physiological mechanisms related to psychopathology [45,46,47,48], while nonhuman primate (NHP) behavior, social constructs, cognitive function, and temperament-defining traits are each closely related to those of humans [49,50,51,52,53]. Temperament influences reactivity to acute and chronic stress, manifesting as changes in physiological parameters such as heart rate, blood pressure, and endocrine response [54,55,56], and has been associated with overall response to stressful clinical situation [38,57]. Since temperament and personality influence how individuals perceive their environment and cope with its stressors, there is likely a similar influence on individual responses to targeted interventions aimed at improving coping and fostering resilience [29,36,37,58,59,60], including various behavioral training paradigms [36]. Temperament has been shown to indirectly influence learning by affecting motivation and preference in learning styles [61,62,63,64]. As such, mixed reinforcement paradigms may have the considerable advantage of motivation across temperament to support wider-spread acquisition of positive coping across all temperament types [65,66,67].
While training animals for cooperation in biomedical research is becoming more common, few studies have examined factors that might influence the success of such behavioral management programs. Using our well-established training program designed to foster NHP cooperation with a variety of research and medical interventions, we evaluated the influence of individual characteristics such as temperament and species on training outcomes and the acquisition of important coping skills [20]. This retrospective cohort study assessed training success in male and female rhesus (Macaca mulatta) and cynomolgus macaques (Macaca fascicularis) with inhibited and exploratory temperaments. Animals were trained using a standardized mixed reinforcement paradigm [68] to cooperatively present a limb to a caregiver in their familiar home enclosure for the performance of a variety of medically relevant manipulations. While holding environmental and social conditions constant, we probed the influence of temperament, species, sex, and age on the time required for animals to acquire the targeted skills. We defined success as full cooperation with caregivers with the target task, suggestive of a motivational preference towards appetitive, rewarding stimuli relative to avoidance. In human patients, behavioral engagement, active and voluntary participation, and the need for restraint (or lack thereof) have similarly been used to assess coping in the context of medical-related stimuli in human patients [66,69,70]. Similarly, we considered engagement and voluntary participation with direct human contact and medical stimuli indicative of positive coping. There is important translational relevance in this work, especially in pediatrics, where reactivity in children is particularly closely tied to temperament due to developmentally primitive coping mechanisms [71,72]. This study evaluated the relationship between temperament, species, and other demographic variables and the acquisition of coping skills with the aim to both improving captive animal management practices and also reveal novel insights into the interplay between temperament and coping that can inform strategies in the clinic [73,74].

2. Materials and Methods

2.1. Animal Subjects

All animal use was approved by the University of Minnesota Institutional Animal Care and Use Committee, was in compliance with the Animal Welfare Act, and adhered to the principles stated in the Guide for Care and Use of Laboratory Animals.
A total of 51 healthy rhesus (female = 24, male = 27) and 32 healthy cynomolgus (female = 11, male = 21) macaques, ages ranging from 1.5 to 13 years (median = 4.0 years), were enrolled in this study (Table S1). All animals were purpose-bred and acquired from institutionally approved commercial vendors. Animals were concurrently enrolled in separate metabolic studies as part of a preclinical research program. All animals were housed in same-sex pairs or groups, except in rare cases of demonstrated social incompatibility. Water was available ad libitum, and primate biscuits (2055c or 7195 Envigo Harlan Teklad Nonhuman Primate Diet or 5048 LabDiet Certified Primate Diet) were provided twice daily based on body weight and supplemented with additional food enrichment consisting of fresh fruits, vegetables, grains, beans, and nuts. All animals participated in an environmental enrichment program that included novel toys, music, social play and regularly scheduled access to a large play and exercise area with swimming access. Room temperature was maintained at 20–26.7 °C, humidity was maintained at 30–70%, and lights were programmed to a 12 h-on (5:30 a.m.–5:30 p.m.), 12 h-off circadian light cycle with 30 min dawn/dusk intervals. Animals were observed at least twice daily for general appearance, behavior, and body condition as a part of routine health monitoring. Semi-annual veterinary physical examinations were performed and included assessments of weight, body condition, heart rate, temperature, lymph nodes, abdomen, oral cavity, dermis, ears, and nose, as well as the performance of a complete blood count and chemistry panel. Weights were taken at least monthly, and veterinary rounds for routine evaluation were performed weekly.

2.2. Primate Temperament Assessment

Primate behavioral traits and temperament were assessed from each animal’s home enclosure at the time of admission to the facility. Each animal’s behavior was documented by experienced trainers over a duration of approximately 10 min. Additionally, each primate was offered high-value food items by hand from these unfamiliar trainers in order to assess their willingness to take high-value appetitive items from a stranger. Study directors used the observed behaviors to evaluate and classify each animal’s temperament as either “inhibited” or “exploratory”. To do this, observed behaviors were classified as either inhibited-type, exploratory-type, or neutral-type; behaviors that are accepted as highly characteristic of either exploratory or inhibited animals were given greater weight during this evaluation process. A sample list of typical behaviors, and their respective temperament associations, is provided in Table 1. Generally, animals that exhibited behavioral tendencies of being bold, curious, and willing to explore novel items were classified as exploratory, while animals with behavioral tendencies indicative of being fearful, avoidant, and easily disturbed by the unfamiliar were classified as inhibited [75]. The temperament classifications of each animal were validated through subsequent blind scoring by the principal investigator of the laboratory.

2.3. Primate Training

Primates were trained to accept oral fluids and foods, shift on cue, and voluntarily present a limb to facilitate the performance of medically relevant manipulation. The specific target behavior was defined as cued lower limb presentation by the animal via an opening in the home enclosure and remaining stationary in hold for cooperation with a range of commonly performed clinical manipulations such as sample collection (heel stick, intravenous catheter placement, vascular access port access), drug administration (subcutaneous (SC) or intramuscular (IM) injection), vitals monitoring, and examination. For each animal, training was performed by multiple trainers using a standardized, mixed reinforcement paradigm. The training program was divided into three distinct phases, referred to as P(re)-Phase, Phase-1, and Phase-2, each consisting of multiple sequential subphases; animals performed training sessions within a given subphase until successfully completing the subphase’s predefined objectives. Early subphases of training incorporated a combination of both positive reinforcement (PR) and negative reinforcement (NR), while later subphases transitioned to PR-only. Application of NR was strategically used to reduce and extinguish avoidance and escape-type behaviors associated with neophobia. In these early stages of training, a panel that animals had previously been desensitized to was used to limit enclosure space and bring the animals into closer proximity to the trainer; once animals were in close proximity, were relaxed and began to accept treats, the panel was removed (NR) and an additional jackpot (highly preferred item, e.g., cupcake or popsicle) reward was given (PR). In this context, the application of NR along with conventional PR facilitated all animals being able to receive and experience appetitive rewards in a highly controlled manner, as opposed to having the opportunity to continuously avoid both close proximity to the trainer and the reception of appetitive rewards [33].
The objective of the first phase of training, P-phase, was to desensitize the animal to both close contact with trainers and to the enclosure’s squeeze-back panel, and to associate trainers with rewards. Desensitizing the animal to the squeeze-back panel allowed for its future use as a neutral tool (related to space manipulation versus its conventional purpose for restraint) in subsequent training phases without risk of inducing fear, and to encourage acceptance of preferred treats by hand from trainers in order to strengthen the reward value of treat-giving as a form of PR in subsequent training phases. P-Phase consisted of 5 subphases, described below, each designed to gradually build on the skills acquired in the previous subphase to achieve these objectives.
Each P-Phase training session occurred with the animal positioned in an area of the home enclosure planned for training and working. Each of the P-Phase subphases followed the same basic structure with slight variations, described below. At the start of each session, the animal’s affective state and attitude (e.g., engaged, neutral/calm, curious, anxious) were evaluated; if an animal was identified as being highly anxious or aggressive, the session would be stopped or not initiated for the evaluation of potential simple environmental factors that could be considered to better accommodate the individual animal’s engagement with the session. In these situations, some examples of possible adaptations include adjusting to the animal’s preference towards male or female trainer, time of day, or performance order of training in the room. Following attitude assessment, a variety of highly palatable treats, such as fresh or dried fruit, fruit snacks, assorted nuts, or assorted cereals were offered to the animal. Treats were initially offered by hand at the front of the enclosure, close to the location where the animal would be positioned for limb presentation in subsequent training phases. If the animal rejected taking any treats by hand, other offering methods were attempted: first, a treat was placed at the front of the enclosure with the trainer present; if the treat was still not accepted, it was finally offered at the front of the enclosure with the trainer out of the animal’s sight. Following initial treat offering, regardless of treat acceptance, the main part of the session began: treats were offered to the animal at least once every 20–30 s, with increased frequency if the animal accepted the treats and remained in the desired target location in its enclosure. A single session continued until either the animal had taken treats by hand 9 different times, or 4 min had elapsed since the start of the session, whichever occurred first. At the end of the session, the animal’s affective state and attitude were again evaluated, and a final treat was offered to the animal by hand.
At the end of each P-Phase training session, a score was assigned to determine whether the animal adequately met the objectives of the session to be able to move on to the next subphase during the next training session. When an animal did not meet the required score to advance, the same subphase was repeated for the next training session. In the first subphase of P-Phase (P-1), up to 2 points could be assigned: one point was given if the animal took a treat from the trainer’s hand at the end of the session, and one point was given if the animal demonstrated a neutral or positive attitude (e.g., engaged, calm, or curious) at the end of the session; each of the two points were required to advance to the next subphase. Training sessions for subphases P-2, P-3, and P-4 followed the same attitude assessment and treat-offering procedures described above with a gradual increased incorporation of the enclosure’s squeeze-back panel. After the same start-of-session attitude assessment and treat offering, subphase P-2 was performed with the panel taking up 30% of the enclosure’s space, subphase P-3 was performed with the panel taking up 50% of the enclosure’s space, and subphase P-4 was performed with the panel taking up 90% of the enclosure’s space. Scoring and passing criteria for subphases P-2 through P-4 were similar to that used for P-1, but score was based on attitude and willingness to take offered treats by hand throughout the session, rather than the end of the session (as was carried out for P-1).
The final subphase of P-Phase, P-5, was designed to introduce the animal to contact with the trainer. Each P-5 session began with a start-of-session attitude assessment and treat offering, followed by incorporation of the squeeze-back panel to take up 90% of the enclosure (the same position as in P-4). Once the squeeze-back was engaged, the trainer intermittently touched the animal’s toes within the enclosure or attempted to gently hold their leg for a maximum of 10 consecutive seconds per intermittent attempt. Treats were offered over the duration of the session. Each P-5 training session continued until either the animal had a non-reactive response to toe-touching and/or limb holding three different times, or 4 min had elapsed since the start of the session, whichever occurred first. At the end of the session, the animal’s affective state and attitude were evaluated. Similarly to other subphases, each P-5 session was scored based on attitude and treat taking during the session. With respect to attitude, an animal received a score of 2 points if it was engaged, neutral/calm, or curious during the session, 1 point if it was submissive or tentative, and 0 points where anxious or aggressive attitude/behavior was observed. An additional point was assigned if the animal took at least one treat by hand during the session. Two of the three possible points were required to complete P-Phase training and advance to Phase 1 of the training paradigm. Upon completion of the entire P-Phase, an animal had demonstrated the ability to both calmly take treats from the trainer with the squeeze-back panel engaged and allow the trainer to touch their toes and legs. A complete summary of P-Phase training steps, including a breakdown of session-end criteria and session passing-criteria is provided in Table 2.
The objective of Phase-1 of the training paradigm was to familiarize each animal with voluntary limb presentation through an opening in their home enclosure for handling while remaining calm and stationary. Counterconditioning was used to associate typical stimuli animals encounter during intensive medical management with positive events (highly preferred food reward and control over the environment and space) and to associate these encounters as ‘safe’ with trainers. In Phase-1, the amount of space restricted by the squeeze-back panel was gradually decreased through subsequent subphases. In the latter subphases of Phase-1, each animal was also acclimated to handling techniques required for routine clinical evaluation, including ‘skill sets’, consisting of light toe hold, handler hand switch, foot squeeze, startle, knee flex/extend, leg movement on axis, skin pinch (for evaluating skin turgor/hydration), heel tap and squeeze (simulating heel stick), and needle sticks. Following the successful completion of Phase-1, animals advanced to Phase-2, with the objective of transitioning to voluntarily present their limb on cue with different trainers. At the end of each Phase-1 and Phase-2 training session, the animal was presented with a jackpot to mark the achievement of a complicated behavior. Throughout all Phase-1 and Phase-2 training sessions, total session time, hold time, and number of animal escape attempts or aggression were recorded. Training was designed to introduce new behaviors and skills at pace with the development of coping, evident in relaxed and cooperative task performance; animals that displayed ineffective coping, including signs of anxiety such as avoidance, escape attempt, alarm reaction, or aggressive behavior required repetition of a session in the current subphase. This ensured that animals were not exposed to high stress or overwhelmed and only given a progressively more difficult task as coping developed during a simpler and perceptively less-threatening task. Complete details of Phase-1 and Phase-2 training methodology as used in this study have previously been described by Graham et al. [68].

2.4. Data Analysis

The relationship between primate temperament, species, age, and sex with various training outcome measures was assessed using independent samples t-tests, logistic regression, and Kaplan–Meier time-to-event analysis. Logistic regression was used to assess the predictive value of age, species, and sex in association with temperament on training success and willingness to take treats. For logistic regression analysis, age and training time were categorized as binary variables: for age, animals were classified as “young” or “mature” based on whether the animal was above or below the mean age of the entire cohort (4.3 years); for training time, animals were classified as “slower” or “faster” based on whether time to complete training was above or below the average total training time for the cohort. Kaplan–Meier time-to-event analysis was used to assess differences in time-to-training completion for the entire training paradigm and individual training phases, stratified by species and temperament. Overall differences in training times by species were further investigated using independent sample t-tests.

3. Results

Of the 83 primates assessed for temperament, 29% were classified as inhibited, and 71% were classified as exploratory (Table S1). Traits including sex, age, species, and temperament were analyzed to determine if there were differences in skill acquisition or coping. All primates (100%) in this study successfully completed all components of the training paradigm. At the conclusion of training animals consistently demonstrated the ability to voluntarily present a lower limb through an opening in their home enclosure, cooperate with handling, and remain relaxed during a potentially mildly aversive medical manipulation, consistent with positive coping.
We analyzed the willingness to take treats from unfamiliar trainers between exploratory and inhibited animals during their initial temperament assessments to investigate the relationship between temperament and novelty seeking. Cynomolgus macaques that were categorized as inhibited were less likely than exploratory animals to take treats from an unfamiliar trainer at time of admission to the facility (OR = 0.08, 95% CI = (0.01, 0.44), p = 0.01) (Table S2). Similar comparison was not performed for rhesus macaques due to insufficient data on treat acceptance to adequately power the analysis.
We compared both individual phase and total training time across rhesus and cynomolgus macaques to assess the relationship between species and training success. Cynomolgus macaques took significantly longer, on average, to complete the entire training paradigm (median = 5.20 h) compared to rhesus macaques (median = 2.98 h) (Table 3), supported by both Kaplan–Meier time-to-event analysis (Log-rank χ2 = 31.63 (df = 1), p < 0.0001) (Figure 1a) and an independent samples t-test comparing average completion time by group (p < 0.0001). Additionally, cynomolgus macaques took significantly longer to complete P-phase training (Figure 1b), Phase-1 training (Figure 1c), and Phase-2 training (Figure 1d) compared to rhesus macaques. Median training times by species for each training phase are presented in Table 3.
Because significant differences were found in training time between rhesus and cynomolgus macaques, data was stratified by species for analysis of the effects of temperament, sex, and age on training. In each species, there was no difference between inhibited and exploratory animals in the number of total training sessions or total training time required to complete the entire training paradigm, after controlling for sex and age (Table 4 and Table S2). Additionally, there was no difference by temperament in either species in number of training sessions or training time required to complete any of the three individual training phases (Table 4 and Table S2). These results were supported by the Kaplan–Meier analyses comparing time required to complete overall training and individual training phases by temperament in each species (Figure 2 and Figure 3). Age and sex were controlled for in all multivariate logistic regression analyses, and sex was determined to not be an independent predictor of training success for the entire training paradigm or any of its individual phases for either species. Median training time by sex for each species are presented in Table 5. In rhesus macaques, age was determined to not be an independent predictor of training success for the entire training paradigm or any of its individual phases. However, in cynomolgus macaques, logistic regression analysis showed that age was a significant independent predictor in time to complete the P-Phase of training, with older animals completing P-Phase training in less time than younger animals (OR = 0.16, 95% CI = (0.02–0.84), p = 0.04). This was supported by independent samples t-test comparing average completion time by age (p = 0.03). In contrast, there was no difference in Phase-1, Phase-2, or total training time by age in cynomolgus macaque. Median training time by age for each species are presented in Table 6.

4. Discussion

This study explored the influence of various traits on acquisition of successful coping in primates. All animals that were enrolled successfully completed training using a mixed reinforcement paradigm, resulting in animals performing behaviors used for complex medical care in cooperation with caregivers using PRT to maintain the behavioral repertoire.

4.1. Training Differences by Species

Interestingly, we observed significantly different time requirements to complete the same training paradigm between cynomolgus and rhesus macaques suggests there are inherent learning differences and motivational preferences between species [76,77], although the basis for this has yet to be defined. However, this difference of 3 h versus 5 h average total training time in rhesus and cynomolgus macaques, respectively, has little clinical relevance; this is a modest difference in investment in time to train behaviors that will be used in the majority of animals for years. Notwithstanding, understanding the average time commitment required for training a certain animal could have practical implications for study planning related to time management and budgeting. Knowledge of these behavioral differences could prove to be valuable in guiding future model selection, in addition to shaping effective behavioral management practices. From the scientific perspective, deeper understanding of differences in learning behavior influences preclinical model selection to ensure members of the selected species are able to demonstrate the capacity to cooperate, cope, and comply with study procedures in ways that more closely represent the clinical situation. Overall, advancements in our understanding of species-specific differences in training ability can help improve accuracy of study timeline planning, and improve animal welfare by increasing understanding of what is necessary in order to prepare animals to acquire the coping skills necessary to flourish in the research environment.

4.2. Temperament and Behavioral Motivations

We also focused on the interaction between temperament and skills acquisition since behavioral inhibition is associated with an increased risk for stress, and subsequently, anxiety and depression [2,8,9,41,42]. Behaviorally inhibited primates were significantly less likely to accept treats from novel handlers, consistent with vigilance, neophobia, and avoidant tendencies during unfamiliar situations. These results agree with previous research showing that inhibited primates are less likely to directly take treats, and further support the use of the human intruder test to accurately assess temperament [20]. Training can support cognitive processes capable of moderating negative reactivity or modulate fearful reactions to allow instead for engagement with frequent, rewarding, and successful interactions, an adaptation consistent with positive coping that decreases stress. In this study, behaviorally inhibited NHPs successfully performed target behaviors at the same rate as those that were more exploratory, suggesting the early-phase mixed reinforcement used in our program is useful to support skill acquisition and positive coping mechanisms in animals that display high levels of reactivity and distress to novel or unfamiliar objects or people.
Temperament-associated differences in willingness to accept highly palatable treats or other appetitive rewards has important implications related to motivational preferences, which can help inform the design of the most suitable training paradigm for a given animal (e.g., PRT only vs. mixed reinforcement). Previous studies have shown that inhibited primates are significantly less successful in training with a PRT-only based approach, which might be explained by lower motivation to accept positive reinforcers, such as high-valued treats, by behaviorally inhibited animals relative to their more exploratory counterparts [20]. For these inhibited animals that exhibit more fearful, avoidant, hesitant, and relatively neophobic behaviors, the appeal of removing (NR) something unwanted may initially be more motivational than appetitive (PR) rewards [75]. When an unwanted stimulus or condition is subtracted (NR), these animals experience relief, a powerful reinforcer, and trust can be built on the understanding that the trained interaction is safe. When combined sparingly with PRT, NRT can be used to improve training efficacy to reinforce that all interactions with trainers end in a favorable outcome for an animal [68,78,79]. We show that although inhibited animals were less likely to take treats at the initiation of training, they were accepting treats comparably to exploratory animals by the end of the P-Phase of training. Using P-phase as an easy learning phase to modulate reactive tendencies and instead engage animals with frequent rewarding appeared to effectively diminish the effects of temperament in future phases, evident by the absence of significant difference in the time required to complete subsequent training phases. It should be highlighted that during Phase-1 and -2 training, primates were offered a high valued “jackpot” at the end of each training session to offset potential training-related anxiety, foster a positive association with training. Affective state-based cognitive biases can occur when there is no way to buffer or counter-condition a stressful event, and post-training high value rewards can help provide this buffer [80]. It is important to note that even the more inhibited animals were able to benefit from the “jackpot” reward strategy owing to the coping skills and willingness to accept appetitive reinforcers developed during the P-Phase of training and the mixed reinforcement-based training paradigm. These findings suggest that the mixed reinforcement training model, with the incorporation of appropriate familiarization and desensitization steps (in the form of the P-Phase here), supports the learning and development of improved coping strategies required for training success in both exploratory animals, and notably, inhibited animals. The combination of push–pull motivators utilized by mixed-reinforcement may be more effective than either push or pull motivators used in isolation, and should be considered in the design of training programs, especially for use in animals with inhibited temperaments who are less likely to have high success with appetitive rewards alone [79].
We did not explore the role of social learning in this analysis. Animals have demonstrated the ability to learn at a faster rate after observing conspecifics performing a task, and animals in this study were housed among animals with varying experience with cooperative tasks [81]. Others have shown the role observational learning has in positively shaping animal perceptions of handlers/trainers [82]. We generally find that animals who observe conspecifics having positive interactions with trainers have reduced overall anxiety behaviors related to close human contact. As such, future studies could investigate the effects of the incorporation of social learning on similarly designed training paradigms. As macaques are highly social animals with hierarchical societies, social rank should also be considered as a variable that may have a potential impact on training efficiency during the learning of complex tasks; rank was not analyzed in this study. Such differences have previously been described in rhesus macaques during simple task training [83,84,85].

4.3. Implications for Welfare and Scientific Validity

Despite individual differences in time to complete training, it is important to note that all animals in our study, regardless of temperament, age, sex, or other demographic characteristics, were able to successfully complete the entire training paradigm and acquire the coping skills necessary for appropriately dealing with exposure to potentially adverse medical situations. In biomedical research, ideal sample choice not only includes selecting the best species for a specific question of interest, but also requires the use of a set of subjects that are as representative of the target population as possible. This has most notably led to a recent push for sex-balancing in scientific studies, but also supports the inclusion of other demographic characteristics, such as temperament, to improve generalizability. Temperament-related selection bias can lead to favoring exploratory animals when candidates are chosen based on preconceived perceptions of their trainability, willingness to cooperate, lack of aggression, and other indicators of affective state [86]. Considering temperament is an interaction between individual biological factors and contextual factors, excluding more inhibited animals in favor of those that are more exploratory has the potential to bias results [7,8,9,10,11]. Despite preconceived notions, the present findings show that inhibited animals can be equally successful at learning a complex training task, and should not be excluded from studies based on perceived study suitability. Using behavioral management as a great equalizer, behaviorally inhibited animals can develop equivalent coping skills to protect their welfare and limit stress bias, so that study populations are more generalizable, improving validity and translatability [36].
Welfare is largely affected by the ways that an animal perceives both its environment and its ability to exert control over it, both of which are closely intertwined with temperament, coping skills, and training. This study shows that well-designed cooperative training has the ability to enable animals of any temperament to develop the coping abilities required to adequately deal with routinely experienced medically necessary aversive situations and reduce consequent stress, demonstrating cooperative training’s critical role in fostering overall welfare in captive animals [68]. Further, training often serves as a form of enrichment, giving animals the chance to perform novel physical and cognitive activities, in addition to providing the opportunity to bond with familiar trainers. Captive animals benefit from the opportunity to learn and perform novel behaviors, and training provides this opportunity [21,87].
An important benefit of successful training is the opportunity for animals to express choice and control over their own care, a key component of welfare. Without proper training, animals often develop a learned helplessness to deal with potentially aversive experiences. Learned helplessness, associated with the inability to escape a stressful situation, can induce uncontrollable, unmediated stress and anxiety [88]. Beyond its ethical implications, the welfare benefits and potential reduction in stress introduced by training can help improve the validity of scientific results and the translational value of the research model. Excess stress has been shown to affect a range of research-related physiologic outcome measures such as heart rate, blood pressure, cortisol, and functional immune state in humans [89,90,91,92], and its reduction through training may lead to a more representative model with less stress-related confounding of results. We have demonstrated previously that the ability to make choices to cooperate with a task, learned during training, fosters a sense of control that is protective psychologically and physically in NHPs [68].

4.4. Translational Relevance

Beyond its significance for animals in captivity, this research has implications for the management of clinical patients who experience potentially fear-inducing medical procedures for the treatment of a variety of acute and chronic medical conditions. Coping skills training in patients can effectively reduce anxiety and increase compliance with treatment to improve outcomes [93,94,95], but there are gaps in understanding of conditions necessary to support practical treatment options. Primates have become increasingly prominent in understanding biological mechanisms underlying neuropsychiatric disorders, e.g., autism spectrum disorder, emphasizing their relevance in biobehavioral research towards successful interventions [96,97,98]. Behavioral management techniques that foster coping in animals undergoing intensive medical management have strong potential to similarly benefit clinical patients. For example, certain adjustments in management technique for behaviorally inhibited patients, such as reducing the ability to perform avoidance-type behaviors and increasing the opportunity to experience reward while in a controlled setting, could more successfully address the needs of these patients in order to improve coping, foster higher levels of cooperation with medical procedures, and reduce overall stress.

5. Conclusions

Effective coping is essential for animals involved in transplant, metabolic, and infectious disease studies, among others, where they are routinely exposed to medical interventions such as physical examinations, blood sampling, and drug administration. Recognizing and acknowledging individual and species-related differences supports the proper application of training paradigms for successful cooperation and fosters resilience in animal subjects which improves the accuracy of the model to enhance translation and increase animal welfare.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/biology11030423/s1, Table S1: Primate Demographics; Table S2: Temperament and Training Outcomes.

Author Contributions

Conceptualization, M.L.G., S.P., S.H.O.; Analysis, M.L.G., S.P., S.H.O.; Writing—Original Draft Preparation, S.P., S.H.O.; Writing—Review and Editing, M.L.G., S.P., S.H.O.; Visualization, S.P., S.H.O.; Supervision, M.L.G.; Project Administration, M.L.G.; Funding Acquisition, M.L.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health Grant/Award Numbers: U19AI067151, U01AI12013050, R42DK109853, U01AI102463, Juvenile Diabetes Research Foundation Grant/Award Number: 3-SRA-2016-259, 3-SRA-2015-39-Q-R, 2-SRA-2016-260-S-B, the University of Minnesota Stem Cell Institute-Adjacent Possible Grants and AHC FRD grant program #15.04, Richard M. Schulze Family Foundation individual philanthropy through the University of Minnesota Foundation.

Institutional Review Board Statement

All animal use was approved by the University of Minnesota Institutional Animal Care and Use Committee (protocol code 1603-33558A, date of approval 4 June 2016; protocol code 1903-36845A and date of approval 4 February 2019), and was in compliance with the Animal Welfare Act, and adhered to the principles stated in the Guide for Care and Use of Laboratory Animals.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

We gratefully acknowledge the excellent and expert contributions of Carlen Hill, Jody Janecek, Ruby Klish, Rachael Lee, Christian Moses, Brenna Mulhollam, Lucas Mutch, Melanie Niewinski, Jordan Truell, Lisa Yang for clinical care, husbandry, and training of our animals, Margret Tavai-Tuisalo’o, and Jade Wilder for administrative support at the University of Minnesota’s Preclinical Research Center, and Laura Hocum Stone for support with manuscript preparation and editing.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Folkman, S.; Moskowitz, J.T. Coping: Pitfalls and promise. Annu. Rev. Psychol. 2004, 55, 745–774. [Google Scholar] [CrossRef] [PubMed]
  2. Leandro, P.G.; Castillo, M.D. Coping with stress and its relationship with personality dimensions, anxiety, and depression. Procedia-Soc. Behav. Sci. 2010, 5, 1562–1573. [Google Scholar] [CrossRef] [Green Version]
  3. Lambert, K.G.; Hyer, M.M.; Rzucidlo, A.A.; Bergeron, T.; Landis, T.; Bardi, M. Contingency-based emotional resilience: Effort-based reward training and flexible coping lead to adaptive responses to uncertainty in male rats. Front. Behav. Neurosci. 2014, 8, 124. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Guidi, J.; Lucente, M.; Sonino, N.; Fava, G.A. Allostatic load and its impact on health: A systematic review. Psychother. Psychosom. 2021, 90, 11–27. [Google Scholar] [CrossRef] [PubMed]
  5. DiMatteo, M.R.; Lepper, H.S.; Croghan, T.W. Depression is a risk factor for noncompliance with medical treatment: Meta-analysis of the effects of anxiety and depression on patient adherence. Arch. Intern. Med. 2000, 160, 2101–2107. [Google Scholar] [CrossRef]
  6. Demyttenaere, K. Risk factors and predictors of compliance in depression. Eur. Neuropsychopharmacol. 2003, 13, 69–75. [Google Scholar] [CrossRef]
  7. Koolhaas, J.M.; Korte, S.M.; De Boer, S.F.; Van Der Vegt, B.J.; Van Reenen, C.G.; Hopster, H.; De Jong, I.C.; Ruis, M.A.W.; Blokhuis, H.J. Coping styles in animals: Current status in behavior and stress-physiology. Neurosci. Biobehav. Rev. 1999, 23, 925–935. [Google Scholar] [CrossRef]
  8. Graves, P.L.; Mead, L.A.; Wang, N.Y.; Liang, K.Y.; Klag, M.J. Temperament as a potential predictor of mortality: Evidence from a 41-year prospective study. J. Behav. Med. 1994, 17, 111–126. [Google Scholar] [CrossRef]
  9. Janowski, K.; Steuden, S. The Temperament Risk Factor, Disease Severity, and Quality of Life in Patients with Psoriasis. Ann. Dermatol. 2020, 32, 452–459. [Google Scholar] [CrossRef]
  10. Sherwood, A.; Blumenthal, J.A.; Koch, G.G.; Hoffman, B.M.; Watkins, L.L.; Smith, P.J.; O’Connor, C.M.; Adams, K.F., Jr.; Rogers, J.G.; Sueta, C.; et al. Effects of Coping Skills Training on Quality of Life, Disease Biomarkers, and Clinical Outcomes in Patients with Heart Failure: A Randomized Clinical Trial. Circ. Heart Fail. 2017, 10, e003410. [Google Scholar] [CrossRef] [Green Version]
  11. Wald, R.L.; Dowling, G.C.; Temoshok, L.R. Coping styles predict immune system parameters and clinical outcomes in patients with HIV. Retrovirology 2006, 3, P65. [Google Scholar] [CrossRef]
  12. Folkman, S. Personal control and stress and coping processes: A theoretical analysis. J. Pers. Soc. Psychol. 1984, 46, 839–852. [Google Scholar] [CrossRef] [PubMed]
  13. D’Onofrio, G.; Simeoni, M.; Rizza, P.; Caroleo, M.; Capria, M.; Mazzitello, G.; Sacco, T.; Mazzuca, E.; Panzino, M.T.; Cerantonio, A.; et al. Quality of life, clinical outcome, personality and coping in chronic hemodialysis patients. Ren. Fail. 2017, 39, 45–53. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Broom, D.M.; Kirkden, R.D. Welfare, stress, behaviour and pathophysiology. Vet. Pathophysiol. 2004, Vol. 1, 337–369. [Google Scholar]
  15. Veissier, I.; Boissy, A. Stress and welfare: Two complementary concepts that are intrinsically related to the animal’s point of view. Physiol. Behav. 2007, 92, 429–433. [Google Scholar] [CrossRef] [PubMed]
  16. Poole, T. Happy animals make good science. Lab. Anim. 1997, 31, 116–124. [Google Scholar] [CrossRef]
  17. Prescott, M.J.; Lidster, K. Improving quality of science through better animal welfare: The NC3Rs strategy. Lab. Anim. 2017, 46, 152–156. [Google Scholar] [CrossRef]
  18. Koolhaas, J.; Van Reenen, C. Animal behavior and well-being symposium: Interaction between coping style/personality, stress, and welfare: Relevance for domestic farm animals. J. Anim. Sci. 2016, 94, 2284–2296. [Google Scholar] [CrossRef]
  19. Bloomsmith, M.A.; Perlman, J.E.; Hutchinson, E.; Sharpless, M. Behavioral management programs to promote laboratory animal welfare. In Management of Animal Care and Use Programs in Research, Education, and Testing; CRC Press: Boca Raton, FL, USA, 2017; pp. 63–82. [Google Scholar]
  20. Coleman, K.; Tully, L.A.; McMillan, J.L. Temperament correlates with training success in adult rhesus macaques. Am. J. Primatol. Off. J. Am. Soc. Primatol. 2005, 65, 63–71. [Google Scholar] [CrossRef]
  21. Laule, G.E.; Bloomsmith, M.A.; Schapiro, S.J. The use of positive reinforcement training techniques to enhance the care, management, and welfare of primates in the laboratory. J. Appl. Anim. Welf. Sci. 2003, 6, 163–173. [Google Scholar] [CrossRef]
  22. Graham, M.L.; Prescott, M.J. The multifactorial role of the 3Rs in shifting the harm-benefit analysis in animal models of disease. Eur. J. Pharmacol. 2015, 759, 19–29. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Graham, M.L.; Schuurman, H.-J. Validity of animal models of type 1 diabetes, and strategies to enhance their utility in translational research. Eur. J. Pharmacol. 2015, 759, 221–230. [Google Scholar] [CrossRef] [PubMed]
  24. Graham, M.L.; Schuurman, H.-J. Pancreatic islet xenotransplantation. Drug Discov. Today Dis. Models 2017, 23, 43–50. [Google Scholar] [CrossRef]
  25. Ruhe, K.M.; Badarau, D.O.; Brazzola, P.; Hengartner, H.; Elger, B.S.; Wangmo, T. Participation in pediatric oncology: Views of child and adolescent patients. Psychooncology 2016, 25, 1036–1042. [Google Scholar] [CrossRef]
  26. Olszewski, A.E.; Goldkind, S.F. The Default Position: Optimizing Pediatric Participation in Medical Decision Making. Am. J. Bioeth. 2018, 18, 4–9. [Google Scholar] [CrossRef]
  27. Fernström, A.L.; Fredlund, H.; Spångberg, M.; Westlund, K. Positive reinforcement training in rhesus macaques—Training progress as a result of training frequency. Am. J. Primatol. Off. J. Am. Soc. Primatol. 2009, 71, 373–379. [Google Scholar] [CrossRef]
  28. Schapiro, S.J.; Perlman, J.E.; Thiele, E.; Lambeth, S. Training nonhuman primates to perform behaviors useful in biomedical research. Lab. Anim. 2005, 34, 37–42. [Google Scholar] [CrossRef]
  29. Coleman, K. Individual differences in temperament and behavioral management practices for nonhuman primates. Appl. Anim. Behav. Sci. 2012, 137, 106–113. [Google Scholar] [CrossRef] [Green Version]
  30. Balconi, M.; Falbo, L.; Conte, V.A. BIS and BAS correlates with psychophysiological and cortical response systems during aversive and appetitive emotional stimuli processing. Motiv. Emot. 2012, 36, 218–231. [Google Scholar] [CrossRef]
  31. Kagan, J.; Reznick, J.S.; Snidman, N. Biological bases of childhood shyness. Science 1988, 240, 167–171. [Google Scholar] [CrossRef]
  32. Coleman, K.; Pierre, P.J. Assessing anxiety in nonhuman primates. ILAR J. 2014, 55, 333–346. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Graham, M.L. Positive Reinforcement Training and Research; CRC Press: Boca Raton, FL, USA; Taylor & Francis Group: Boca Raton, FL, USA, 2017; Volume 2017. [Google Scholar]
  34. Prescott, M.J.; Buchanan-Smith, H.M. Training Nonhuman Primates Using Positive Reinforcement Techniques: A Special Issue of the Journal of Applied Animal Welfare Science; Psychology Press: London, UK, 2016. [Google Scholar]
  35. Roter, D.L. Patient participation in the patient-provider interaction: The effects of patient question asking on the quality of interaction, satisfaction and compliance. Health Educ. Monogr. 1977, 5, 281–315. [Google Scholar] [CrossRef] [PubMed]
  36. Rudolph, K.D.; Dennig, M.D.; Weisz, J.R. Determinants and consequences of children’s coping in the medical setting: Conceptualization, review, and critique. Psychol. Bull. 1995, 118, 328. [Google Scholar] [CrossRef] [PubMed]
  37. Miller, K.S.; Vannatta, K.; Compas, B.E.; Vasey, M.; McGoron, K.D.; Salley, C.G.; Gerhardt, C.A. The role of coping and temperament in the adjustment of children with cancer. J. Pediatr. Psychol. 2009, 34, 1135–1143. [Google Scholar] [CrossRef] [PubMed]
  38. Rothbart, M.K. Temperament, development, and personality. Curr. Dir. Psychol. Sci. 2007, 16, 207–212. [Google Scholar] [CrossRef]
  39. Strelau, J. Temperament risk factor: The contribution of temperament to the consequences of the state of stress. In Extreme Stress and Communities: Impact and Intervention; Springer: Berlin/Heidelberg, Germany, 1995; pp. 63–81. [Google Scholar]
  40. De Pauw, S.S.; Mervielde, I. Temperament, personality and developmental psychopathology: A review based on the conceptual dimensions underlying childhood traits. Child Psychiatry Hum. Dev. 2010, 41, 313–329. [Google Scholar] [CrossRef]
  41. Williams, P.G.; Suchy, Y.; Rau, H.K. Individual differences in executive functioning: Implications for stress regulation. Ann. Behav. Med. 2009, 37, 126–140. [Google Scholar] [CrossRef]
  42. Keltikangas-Järvinen, L.; Kettunen, J.; Ravaja, N.; Näätänen, P. Inhibited and disinhibited temperament and autonomic stress reactivity. Int. J. Psychophysiol. 1999, 33, 185–196. [Google Scholar] [CrossRef]
  43. Bolhuis, J.E.; Schouten, W.G.; de Leeuw, J.A.; Schrama, J.W.; Wiegant, V.M. Individual coping characteristics, rearing conditions and behavioural flexibility in pigs. Behav. Brain Res. 2004, 152, 351–360. [Google Scholar] [CrossRef]
  44. Rueda, M.R.; Rothbart, M.K. The influence of temperament on the development of coping: The role of maturation and experience. New Dir. Child Adolesc. Dev. 2009, 2009, 19–31. [Google Scholar] [CrossRef]
  45. Turner, C.A.; Flagel, S.B.; Blandino, P.; Watson, S.J.; Akil, H. Utilizing a unique animal model to better understand human temperament. Curr. Opin. Behav. Sci. 2017, 14, 108–114. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Sheynin, J.; Beck, K.D.; Pang, K.C.H.; Servatius, R.J.; Shikari, S.; Ostovich, J.; Myers, C.E. Behaviourally inhibited temperament and female sex, two vulnerability factors for anxiety disorders, facilitate conditioned avoidance (also) in humans. Behav. Process. 2014, 103, 228–235. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Cavigelli, S.A. Behavioral inhibition in rodents: A model to study causes and health consequences of temperament. In Behavioral Inhibition; Springer: Berlin/Heidelberg, Germany, 2018; pp. 35–58. [Google Scholar]
  48. Hazari, A.; Salberg, S.; Griep, Y.; Yamakawa, G.R.; Mychasiuk, R. Examining changes in rodent temperament following repetitive mild traumatic brain injury in adolescence. Behav. Neurosci. 2020, 134, 384. [Google Scholar] [CrossRef] [PubMed]
  49. Phillips, K.A.; Bales, K.L.; Capitanio, J.P.; Conley, A.; Czoty, P.W.; ‘t Hart, B.A.; Hopkins, W.D.; Hu, S.L.; Miller, L.A.; Nader, M.A. Why primate models matter. Am. J. Primatol. 2014, 76, 801–827. [Google Scholar] [CrossRef] [Green Version]
  50. Zhao, H.; Jiang, Y.H.; Zhang, Y.Q. Modeling autism in non-human primates: Opportunities and challenges. Autism Res. 2018, 11, 686–694. [Google Scholar] [CrossRef]
  51. Camus, S.; Ko, W.K.D.; Pioli, E.; Bezard, E. Why bother using non-human primate models of cognitive disorders in translational research? Neurobiol. Learn. Mem. 2015, 124, 123–129. [Google Scholar] [CrossRef]
  52. Miller, C.; Bard, K.A.; Juno, C.J.; Nadler, R.D. Behavioral responsiveness of young chimpanzees (Pan troglodytes) to a novel environment. Folia Primatol. 1986, 47, 128–142. [Google Scholar] [CrossRef]
  53. Rouff, J.H.; Sussman, R.W.; Strube, M.J. Personality traits in captive lion-tailed macaques (Macaca silenus). Am. J. Primatol. Off. J. Am. Soc. Primatol. 2005, 67, 177–198. [Google Scholar] [CrossRef]
  54. Kagan, J. Temperament and the reactions to unfamiliarity. Child Dev. 1997, 68, 139–143. [Google Scholar] [CrossRef]
  55. Wilson, D.S.; Clark, A.B.; Coleman, K.; Dearstyne, T. Shyness and boldness in humans and other animals. Trends Ecol. Evol. 1994, 9, 442–446. [Google Scholar] [CrossRef]
  56. Kagan, J.; Snidman, N. Early childhood predictors of adult anxiety disorders. Biol. Psychiatry 1999, 46, 1536–1541. [Google Scholar] [CrossRef]
  57. Harper, F.W.; Goodlett, B.D.; Trentacosta, C.J.; Albrecht, T.L.; Taub, J.W.; Phipps, S.; Penner, L.A. Temperament, personality, and quality of life in pediatric cancer patients. J. Pediatr. Psychol. 2014, 39, 459–468. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  58. Archard, G.A.; Braithwaite, V. The importance of wild populations in studies of animal temperament. J. Zool. 2010, 281, 149–160. [Google Scholar] [CrossRef]
  59. DeMaso, D.R.; Snell, C. Promoting coping in children facing pediatric surgery. Semin. Pediatr. Surg. 2013, 22, 134–138. [Google Scholar] [CrossRef] [PubMed]
  60. De Boer, S.F.; Buwalda, B.; Koolhaas, J.M. Untangling the neurobiology of coping styles in rodents: Towards neural mechanisms underlying individual differences in disease susceptibility. Neurosci. Biobehav. Rev. 2017, 74, 401–422. [Google Scholar] [CrossRef]
  61. Zarrabi, F. Investigating the relationship between learning style and metacognitive listening awareness. Int. J. Listening 2020, 34, 21–33. [Google Scholar] [CrossRef]
  62. Davis, S.E. Effects of Motivation, Preferred Learning Styles, and Perceptions of Classroom Climate on Achievement in Ninth and Tenth Grade Math Students. Ph.D. Thesis, University of Florida, Gainesville, FL, USA, 2007. [Google Scholar]
  63. Webb, L.E.; van Reenen, C.G.; Jensen, M.B.; Schmitt, O.; Bokkers, E.A. Does temperament affect learning in calves? Appl. Anim. Behav. Sci. 2015, 165, 33–39. [Google Scholar] [CrossRef]
  64. Choi, N.; Cho, H.-J. Temperament and home environment characteristics as predictors of young children’s learning motivation. Early Child. Educ. J. 2020, 48, 607–620. [Google Scholar] [CrossRef]
  65. Zimmer-Gembeck, M.J.; Skinner, E.A. The development of coping: Implications for psychopathology and resilience. Dev. Psychopathol. 2016, 1–61. [Google Scholar] [CrossRef]
  66. Blount, R.L.; Simons, L.E.; Devine, K.A.; Jaaniste, T.; Cohen, L.L.; Chambers, C.T.; Hayutin, L.G. Evidence-based assessment of coping and stress in pediatric psychology. J. Pediatr. Psychol. 2008, 33, 1021–1045. [Google Scholar] [CrossRef] [Green Version]
  67. Lyons, D.M.; Parker, K.J.; Schatzberg, A.F. Animal models of early life stress: Implications for understanding resilience. Dev. Psychobiol. 2010, 52, 402–410. [Google Scholar] [CrossRef]
  68. Graham, M.L.; Rieke, E.F.; Mutch, L.A.; Zolondek, E.K.; Faig, A.W.; Dufour, T.A.; Munson, J.W.; Kittredge, J.A.; Schuurman, H.J. Successful implementation of cooperative handling eliminates the need for restraint in a complex non-human primate disease model. J. Med. Primatol. 2012, 41, 89–106. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  69. Manimala, M.R.; Blount, R.L.; Cohen, L.L. The Effects of Parental Reassurance versus Distraction on Child Distress and Coping During Immunizations. Children’s Health Care 2000, 29, 161–177. [Google Scholar] [CrossRef]
  70. Ayers, S.; Baum, A.; McManus, C.; Newman, S.; Wallston, K.; Weinman, J.; West, R. Cambridge Handbook of Psychology, Health and Medicine; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
  71. Connor-Smith, J.K.; Flachsbart, C. Relations between personality and coping: A meta-analysis. J. Pers. Soc. Psychol. 2007, 93, 1080–1107. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  72. Compas, B.E.; Connor-Smith, J.; Jaser, S.S. Temperament, stress reactivity, and coping:implications for depression in childhood and adolescence. J. Clin. Child Adolesc. Psychol. 2004, 33, 21–31. [Google Scholar] [CrossRef] [PubMed]
  73. Freeman, H.D.; Brosnan, S.F.; Hopper, L.M.; Lambeth, S.P.; Schapiro, S.J.; Gosling, S.D. Developing a comprehensive and comparative questionnaire for measuring personality in chimpanzees using a simultaneous top-down/bottom-up design. Am. J. Primatol. 2013, 75, 1042–1053. [Google Scholar] [CrossRef] [Green Version]
  74. Bert, A.; Abbott, D.H.; Nakamura, K.; Fuchs, E. The marmoset monkey: A multi-purpose preclinical and translational model of human biology and disease. Drug Discov. Today 2012, 17, 1160–1165. [Google Scholar]
  75. Box, H. Studies of temperament in simian primates with implications for socially mediated learning. Int. J. Comp. Psychol. 1999, 12, 203–218. [Google Scholar]
  76. Warren, J.M. Possibly unique characteristics of learning by primates. J. Hum. Evol. 1974, 3, 445–454. [Google Scholar] [CrossRef]
  77. Rumbaugh, D.M. Evidence of qualitative differences in learning processes among primates. J. Comp. Physiol. Psychol. 1971, 76, 250. [Google Scholar] [CrossRef]
  78. Perone, M. Negative effects of positive reinforcement. Behav. Anal. 2003, 26, 1–14. [Google Scholar] [CrossRef]
  79. Westlund, K. Training laboratory primates—Benefits and techniques. Primate Biol. 2015, 2, 119–132. [Google Scholar] [CrossRef]
  80. Bethell, E.; Holmes, A.; MacLarnon, A.; Semple, S. Cognitive bias in a non-human primate: Husbandry procedures influence cognitive indicators of psychological well-being in captive rhesus macaques. Anim. Welf. 2012, 21, 185–195. [Google Scholar] [CrossRef]
  81. Hopper, L.M. Leveraging Social Learning to Enhance Captive Animal Care and Welfare. J. Zool. Bot. Gard. 2021, 2, 3. [Google Scholar] [CrossRef]
  82. Luna, D.; González, C.; Byrd, C.J.; Palomo, R.; Huenul, E.; Figueroa, J. Do Domestic Pigs Acquire a Positive Perception of Humans through Observational Social Learning? Animals 2021, 11, 127. [Google Scholar] [CrossRef]
  83. Kemp, C.; Thatcher, H.; Farningham, D.; Witham, C.; MacLarnon, A.; Holmes, A.; Semple, S.; Bethell, E.J. A protocol for training group-housed rhesus macaques (Macaca mulatta) to cooperate with husbandry and research procedures using positive reinforcement. Appl. Anim. Behav. Sci. 2017, 197, 90–100. [Google Scholar] [CrossRef]
  84. Wooddell, L.J.; Kaburu, S.S.; Dettmer, A.M. Dominance rank predicts social network position across developmental stages in rhesus monkeys. Am. J. Primatol. 2020, 82, e23024. [Google Scholar] [CrossRef]
  85. Drea, C.M.; Wallen, K. Low-status monkeys “play dumb” when learning in mixed social groups. Proc. Natl. Acad. Sci. USA 1999, 96, 12965–12969. [Google Scholar] [CrossRef] [Green Version]
  86. McKinley, J.; Buchanan-Smith, H.M.; Bassett, L.; Morris, K. Training common marmosets (Callithrix jacchus) to cooperate during routine laboratory procedures: Ease of training and time investment. J. Appl. Anim. Welf. Sci. 2003, 6, 209–220. [Google Scholar] [CrossRef]
  87. Coleman, K.; Maier, A. The use of positive reinforcement training to reduce stereotypic behavior in rhesus macaques. Appl. Anim. Behav. Sci. 2010, 124, 142–148. [Google Scholar] [CrossRef] [Green Version]
  88. Vollmayr, B.; Gass, P. Learned helplessness: Unique features and translational value of a cognitive depression model. Cell Tissue Res. 2013, 354, 171–178. [Google Scholar] [CrossRef] [PubMed]
  89. Gasperin, D.; Netuveli, G.; Dias-da-Costa, J.S.; Pattussi, M.P. Effect of psychological stress on blood pressure increase: A meta-analysis of cohort studies. Cad. Saude Publica 2009, 25, 715–726. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  90. Michaud, K.; Matheson, K.; Kelly, O.; Anisman, H. Impact of stressors in a natural context on release of cortisol in healthy adult humans: A meta-analysis. Stress 2008, 11, 177–197. [Google Scholar] [CrossRef] [PubMed]
  91. Schubert, C.; Lambertz, M.; Nelesen, R.A.; Bardwell, W.; Choi, J.B.; Dimsdale, J.E. Effects of stress on heart rate complexity—A comparison between short-term and chronic stress. Biol. Psychol. 2009, 80, 325–332. [Google Scholar] [CrossRef] [Green Version]
  92. Segerstrom, S.C.; Miller, G.E. Psychological stress and the human immune system: A meta-analytic study of 30 years of inquiry. Psychol. Bull. 2004, 130, 601–630. [Google Scholar] [CrossRef] [Green Version]
  93. Pierce, J.S.; Kozikowski, C.; Lee, J.M.; Wysocki, T. Type 1 diabetes in very young children: A model of parent and child influences on management and outcomes. Pediatr. Diabetes 2017, 18, 17–25. [Google Scholar] [CrossRef] [Green Version]
  94. Powers, S.W.; Bloont, R.L.; Bachanas, P.J.; Cotter, M.W.; Swan, S.C. Helping preschool leukemia patients and their parents cope during injections. J. Pediatr. Psychol. 1993, 18, 681–695. [Google Scholar] [CrossRef]
  95. Grey, M.; Whittemore, R.; Jaser, S.; Ambrosino, J.; Lindemann, E.; Liberti, L.; Northrup, V.; Dziura, J. Effects of coping skills training in school-age children with type 1 diabetes. Res. Nurs. Health 2009, 32, 405–418. [Google Scholar] [CrossRef] [Green Version]
  96. Watson, K.K.; Platt, M.L. Of mice and monkeys: Using non-human primate models to bridge mouse-and human-based investigations of autism spectrum disorders. J. Neurodev. Disord. 2012, 4, 21. [Google Scholar] [CrossRef] [Green Version]
  97. Bauman, M.D.; Schumann, C. Advances in nonhuman primate models of autism: Integrating neuroscience and behavior. Exp. Neurol. 2018, 299, 252–265. [Google Scholar] [CrossRef]
  98. Feczko, E.J.; Bliss-Moreau, E.; Walum, H.; Pruett, J.R., Jr.; Parr, L.A. The macaque social responsiveness scale (mSRS): A rapid screening tool for assessing variability in the social responsiveness of rhesus monkeys (Macaca mulatta). PLoS ONE 2016, 11, e0145956. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Biology 11 00423 g001 550
Figure 1. Training Completion Time by Primate Species. Kaplan–Meier time-to-event analysis comparing the time required to successfully complete all phases and individual phases of the training paradigm between rhesus and cynomolgus macaques. There was a significant difference between species for the completion of (a) all phases (Log-rank χ2 = 31.6 (df = 1), p = <0.0001), (b) P-Phase (Log-rank χ2 = 4.2 (df = 1), p = 0.04), (c) Phase-1 (Log-rank χ2 = 22.3 (df = 1), p = <0.0001), and (d) Phase-2 (Log-rank χ2 = 18.7 (df = 1), p = <0.0001).
Figure 1. Training Completion Time by Primate Species. Kaplan–Meier time-to-event analysis comparing the time required to successfully complete all phases and individual phases of the training paradigm between rhesus and cynomolgus macaques. There was a significant difference between species for the completion of (a) all phases (Log-rank χ2 = 31.6 (df = 1), p = <0.0001), (b) P-Phase (Log-rank χ2 = 4.2 (df = 1), p = 0.04), (c) Phase-1 (Log-rank χ2 = 22.3 (df = 1), p = <0.0001), and (d) Phase-2 (Log-rank χ2 = 18.7 (df = 1), p = <0.0001).
Biology 11 00423 g001
Biology 11 00423 g002 550
Figure 2. Training Completion Time by Temperament in Rhesus Macaques. Kaplan–Meier time-to-event analysis comparing the time required to successfully complete all phases and individual phases of the training paradigm between inhibited and exploratory rhesus macaques. There were no significant differences between groups for completing (a) all phases (Log-rank χ2 = 0.03 (df = 1), p = 0.86), (b) P-Phase (Log-rank χ2 = 1.15 (df = 1), p = 0.30), (c) Phase-1 (Log-rank χ2 = 0.50 (df = 1), p = 0.50), or (d) Phase-2 (Log-rank χ2 = 0.26 (df = 1), p = 0.61).
Figure 2. Training Completion Time by Temperament in Rhesus Macaques. Kaplan–Meier time-to-event analysis comparing the time required to successfully complete all phases and individual phases of the training paradigm between inhibited and exploratory rhesus macaques. There were no significant differences between groups for completing (a) all phases (Log-rank χ2 = 0.03 (df = 1), p = 0.86), (b) P-Phase (Log-rank χ2 = 1.15 (df = 1), p = 0.30), (c) Phase-1 (Log-rank χ2 = 0.50 (df = 1), p = 0.50), or (d) Phase-2 (Log-rank χ2 = 0.26 (df = 1), p = 0.61).
Biology 11 00423 g002
Biology 11 00423 g003 550
Figure 3. Training Completion Time by Temperament in Cynomolgus Macaques. Kaplan–Meier time-to-event analysis comparing the time required to successfully complete all phases and individual phases of the training paradigm between inhibited and exploratory cynomolgus macaques. There were no significant differences between groups for completing (a) all phases (Log-rank χ2 = 0.41 (df = 1), p = 0.52), (b) P-Phase (Log-rank χ2 = 0.60 (df = 1), p = 0.45), (c) Phase-1 (Log-rank χ2 = 0.02; (df = 1), p = 0.88), or (d) Phase-2 (Log-rank χ2 = 0.63 (df = 1), p = 0.43).
Figure 3. Training Completion Time by Temperament in Cynomolgus Macaques. Kaplan–Meier time-to-event analysis comparing the time required to successfully complete all phases and individual phases of the training paradigm between inhibited and exploratory cynomolgus macaques. There were no significant differences between groups for completing (a) all phases (Log-rank χ2 = 0.41 (df = 1), p = 0.52), (b) P-Phase (Log-rank χ2 = 0.60 (df = 1), p = 0.45), (c) Phase-1 (Log-rank χ2 = 0.02; (df = 1), p = 0.88), or (d) Phase-2 (Log-rank χ2 = 0.63 (df = 1), p = 0.43).
Biology 11 00423 g003
Table
Table 1. Primate Behavioral Ethogram Used for the Classification of Primate Temperament.
Table 1. Primate Behavioral Ethogram Used for the Classification of Primate Temperament.
BehaviorDescriptionAssociation
ActiveMoving about, walking, running, climbing, jumping; not lethargic.E 1, I 2
AggressiveHigh frequency displays; threats.I *
AlarmedFearful; alarm calling; maximizes space to technician.I *
BoldFearless; not restrained or tentative; not timid, shy, or coy.E *
CalmReacts in an even, calm way; is not easily disturbed; not agitated; restful; peaceful.E *
ConfidentReadily explores, investigates novel items.E *
DepressedIsolated, withdrawn, sullen, brooding, and has reduced activity.I *
CuriousReadily explores, eagerE *
EngagedInterested, interactive, amiableE
GentleResponds to technicians in an easy-going, kind, and considerate manner; not rough or threatening.E
PlayfulEngages in play behavior.E
SubmissiveDisplays lower hierarchical behavior; presenting, fear grimace, eye-averting, avoiding, cowering E, I
TentativeTimid, shy, hesitant.I
VigilantAlert; ready, attentive, watchful.I
1 E = Exploratory-Type behavior, 2 I = Inhibited-Type behavior, * Weighted behavior.
Table
Table 2. Training Details for P-Phase.
Table 2. Training Details for P-Phase.
Sub-PhaseSBP 1 Engagement %Food Reward OfferedToe-TouchingTrial End CriteriaSub-Phase Passing Criteria
P-10%YesNoAnimal takes at least 9 treats by hand OR trial time reaches 4 minScore of 2. Animal takes food by hand (1 point) post trial AND has an attitude that is engaged, neutral/calm, or curious (1 point) post trial.
P-230%YesNoAnimal takes at least 9 treats by hand OR trial time reaches 4 minScore of 2. Animal takes food by hand (1 point) mid-session AND has an attitude that is engaged, neutral/calm, or curious (1 point) mid-session.
P-350%YesNoAnimal takes at least 9 treats by hand OR trial time reaches 4 minScore of 2. Animal takes food by hand (1 point) mid-session AND has an attitude that is engaged, neutral/calm, or curious (1 point) mid-session.
P-490%YesNoAnimal takes at least 9 treats by hand OR trial time reaches 4 minScore of 2. Animal takes food by hand (1 point) mid-session AND has an attitude that is engaged, neutral/calm, or curious (1 point) mid-session.
P-590%YesYesAnimal is non-reactive to toe touching/holding limbs for 10 s (×3) OR trial time reaches 4 minScore of at least 2. Animal takes food by hand (1 point) mid-session, has a mid-session attitude that is engaged, neutral/calm, or curious (2 points), or has a mid-session attitude that is submissive or tentative (1 point).
1 SBP = Squeeze-back panel.
Table
Table 3. Training Time Summary by Species.
Table 3. Training Time Summary by Species.
SpeciesAll Phases
Median (IQR)
(Hrs)		P(re) Phase
Median (IQR)
(Hrs)		Phase 1
Median (IQR) (Hrs)		Phase 2
Median (IQR)
(Hrs)
Rhesus
(n = 51)		2.98
(2.45–3.93)		0.23
(0.12–0.42)		2.03
(1.77–2.53)		0.59
(0.23–0.97)
Cynomolgus
(n = 32)		5.20
(3.99–6.02)		0.36
(0.16–0.71)		3.06
(2.57–3.89)		1.53
(0.82–2.03)
p-Value
(Rhesus v. Cynomolgus)		<0.0001 *0.0213 *<0.0001 *<0.0001 *
* p < 0.05.
Table
Table 4. Training Time Summary by Temperament.
Table 4. Training Time Summary by Temperament.
Species All Phases
Median (IQR) (Hrs)		P(re) Phase
Median (IQR)
(Hrs)		Phase 1
Median (IQR) (Hrs)		Phase 2
Median (IQR)
(Hrs)
Rhesus
(n = 51)		Inhibited
(n = 8)		2.83
(2.39–4.83)		0.42
(0.08–0.81)		1.93
(1.61–2.51)		0.52
(0.26–0.96)
Exploratory
(n = 43)		2.98
(2.45–3.93)		0.20
(0.13–0.38)		2.10
(1.77–2.53)		0.59
(0.23–1.03)
p-Value
(I v. E)		0.90210.29160.54100.7460
Cynomolgus
(n = 32)		Inhibited
(n = 16)		5.00
(3.85–6.40)		0.43
(0.28–0.79)		2.83
(2.50–4.26)		1.20
(0.82–2.28)
Exploratory
(n = 16)		5.30
(4.15–6.02)		0.26
(0.13–0.67)		3.37
(2.66–3.74)		1.59
(0.88–1.81)
p-Value
(I v. E)		0.45540.57130.99120.5534
Table
Table 5. Training Time Summary by Sex.
Table 5. Training Time Summary by Sex.
Species All Phases
Median (IQR) (Hrs)		P(re) Phase
Median (IQR)
(Hrs)		Phase 1
Median (IQR) (Hrs)		Phase 2
Median (IQR)
(Hrs)
Rhesus
(n = 51) 		Female
(n = 24)		3.05
(2.43–3.68)		0.25
(0.17–0.47)		2.04
(1.68–2.52)		0.63
(0.34–1.02)
Male
(n = 27)		2.97
(2.55–4.15)		0.18
(0.12–0.42)		2.03
(1.83–2.62)		0.58
(0.22–0.93)
p-Value
(Female v. Male)		0.70600.41400.42770.9366
Cynomolgus
(n = 32) 		Female
(n = 11)		4.53
(3.82–6.17)		0.27
(0.13–0.53)		2.78
(2.08–3.70)		1.57
(0.95–2.28)
Male
(n = 21)		5.47
(4.15–5.95)		0.42
(0.19–0.72)		3.25
(2.64–4.09)		1.48
(0.73–1.94)
p-Value
(Female v. Male)		0.48410.22930.11700.7677
Table
Table 6. Training Time Summary by Age.
Table 6. Training Time Summary by Age.
Species All Phases
Median (IQR) (Hrs)		P(re) Phase
Median (IQR)
(Hrs)		Phase 1
Median (IQR) (Hrs)		Phase 2
Median (IQR)
(Hrs)
Rhesus
(n = 51)		Young
(n = 26)		2.93
(2.44–4.20)		0.26
(0.13–0.44)		1.99
(1.81–2.47)		0.60
(0.23–1.11)
Mature
(n = 25)		3.11
(2.48–3.63)		0.20
(0.12–0.33)		2.14
(1.72–2.67)		0.59
(0.28–0.87)
p-Value
(Young v. Mature)		0.86830.41410.26670.2054
Cynomolgus
(n = 32)		Young
(n = 22)		5.34
(4.15–5.88)		0.47
(0.28–0.84)		3.09
(2.47–4.04)		1.56
(0.95–2.08)
Mature
(n = 10)		4.74
(3.76–6.20)		0.18
(0.13–0.33)		3.02
(2.58–3.73)		1.38
(0.70–1.96)
p-Value
(Young v. Mature)		0.40040.0296 *0.47940.5810
* p < 0.05.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Palmer, S.; Oppler, S.H.; Graham, M.L. Behavioral Management as a Coping Strategy for Managing Stressors in Primates: The Influence of Temperament and Species. Biology 2022, 11, 423. https://doi.org/10.3390/biology11030423

AMA Style

Palmer S, Oppler SH, Graham ML. Behavioral Management as a Coping Strategy for Managing Stressors in Primates: The Influence of Temperament and Species. Biology. 2022; 11(3):423. https://doi.org/10.3390/biology11030423

Chicago/Turabian Style

Palmer, Sierra, Scott Hunter Oppler, and Melanie L. Graham. 2022. "Behavioral Management as a Coping Strategy for Managing Stressors in Primates: The Influence of Temperament and Species" Biology 11, no. 3: 423. https://doi.org/10.3390/biology11030423

APA Style

Palmer, S., Oppler, S. H., & Graham, M. L. (2022). Behavioral Management as a Coping Strategy for Managing Stressors in Primates: The Influence of Temperament and Species. Biology, 11(3), 423. https://doi.org/10.3390/biology11030423

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop