# Improving Ovine Behavioral Pain Diagnosis by Implementing Statistical Weightings Based on Logistic Regression and Random Forest Algorithms

^{1}

^{2}

^{*}

## Abstract

**:**

## Simple Summary

## Abstract

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Dataset

^{®}, Zoetis, São Paulo, SP, Brazil) was administered intramuscularly (IM). Then, dissociative anesthesia was performed by applying 0.5 mg/kg of diazepam (Compaz

^{®}, Cristália, Itapira, SP, Brazil) and 5 mg/kg of ketamine (Cetamin

^{®}, Syntec; Santana de Parnaíba, SP, Brazil) intravenously (IV). For intraoperative analgesia, lumbosacral epidural anesthesia was performed with 0.1 mL/kg of 1% lidocaine without a vasoconstrictor (Xylestesin

^{®}, Cristália, Itapira, SP, Brazil) and anesthetic infiltration with 2% lidocaine without vasoconstrictor (Xylestesin

^{®}, Cristália, Itapira, SP, Brazil) at the incision site after the introduction of the trocar. The same experienced surgeon performed all laparoscopies for follicular aspiration and follicular cell replacement using three 5 mm trocars introduced in the retro-umbilical region. Dissociative anesthesia was supplemented during the procedure with 5 mg/kg IV ketamine for those sheep that demonstrated head or limb movement or a 20% increase in heart rate compared to the pre-procedure rate. All sheep received postoperative analgesia 3–4 h after anesthetic recovery with 0.5 mg/kg 2% meloxicam (Maxicam

^{®}, Ourofino, Cravinhos, SP, Brazil) and 0.2 mg/kg morphine (Dimorf

^{®}, Cristália, Itapira, SP, Brazil) IV, separately.

#### 2.2. Statistical Description

#### 2.2.1. Creation of the Multilevel Binomial Logistic Regression Algorithm

^{2}, and Akaike (AIC) and Bayesian (BIC) information criterion (jtools::summ; stats::logLik; and lmtest::lrtest). To illustrate the importance of each USAPS behavior, the Wald statistic of each explanatory variable of the fixed effects was calculated and presented in a bar plot (ggplot2::ggplot). The Wald statistic is presented by dividing the slope by its standard error. Subsequently, based on the fixed effects coefficients estimated by the algorithm, the probability of each sheep in the database (100% of the sheep) needing analgesia (suffering pain) (stats::predict) was calculated. This probability was used to verify the quality of the pain diagnosis described below.

#### 2.2.2. Creation of the Random Forest Algorithm

#### 2.2.3. Quality of Pain Diagnosis

## 3. Results

#### 3.1. Creation of the Multilevel Binomial Logistic Regression Algorithm

^{2}> 0.75 of the full model indicates that two-thirds of the data variation was explained by the proposed model. There was low variation in random effects, indicating adequate data consistency. These results suggest an adequate fit of the multilevel binomial logistic regression algorithm.

^{2}of the fixed effects, the same value of total pseudo-R

^{2}, and as non-significant for the difference in log-likelihood in relation to the full model, demonstrating that there was no substantial improvement in the fit of the model when the non-significant slope coefficients of the full model were disregarded.

#### 3.2. Creation of the Random Forest Algorithm

#### 3.3. Quality of Pain Diagnosis

## 4. Discussion

## 5. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Webster, J. Animal welfare: Freedoms, dominions and “A life worth living”. Animals
**2016**, 6, 35. [Google Scholar] [CrossRef] [Green Version] - Steagall, P.V.; Bustamante, H.; Johnson, C.B.; Turner, P.V. Pain management in farm animals: Focus on cattle, sheep and pigs. Animals
**2021**, 11, 1483. [Google Scholar] [CrossRef] - Mclennan, K.M. Why pain is still a welfare issue for farm animals, and how facial expression could be the answer. Agriculture
**2018**, 8, 127. [Google Scholar] [CrossRef] [Green Version] - Leung, V.; Rousseau-Blass, F.; Beauchamp, G.; Pang, D.S.J. ARRIVE Has Not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in Vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLoS ONE
**2018**, 13, e0197882. [Google Scholar] [CrossRef] [Green Version] - Fisher, A.D. Addressing pain caused by mulesing in sheep. Appl. Anim. Behav. Sci.
**2011**, 135, 232–240. [Google Scholar] [CrossRef] - Orihuela, A.; Ungerfeld, R. Tail Docking in Sheep (Ovis Aries): A review on the arguments for and against the procedure, advantages/disadvantages, methods, and new evidence to revisit the topic. Livest. Sci.
**2019**, 230, 103837. [Google Scholar] [CrossRef] - Small, A.; Fisher, A.D.; Lee, C.; Colditz, I. Analgesia for sheep in commercial production: Where to next? Animals
**2021**, 11, 1127. [Google Scholar] [CrossRef] - Taylor, K.; Rego, L.; Weber, T. Recommendations to improve the EU non-technical summaries of animal experiments. ALTEX Altern. To Anim. Exp.
**2018**, 35, 193–210. [Google Scholar] [CrossRef] - Gigliuto, C.; De Gregori, M.; Malafoglia, V.; Raffaeli, W.; Compagnone, C.; Visai, L.; Petrini, P.; Avanzini, M.A.; Muscoli, C.; Viganò, J.; et al. Pain assessment in animal models: Do we need further studies? J. Pain Res.
**2014**, 7, 227–236. [Google Scholar] - Gregory, M.H.; Capito, N.; Kuroki, K.; Stoker, A.M.; Cook, J.L.; Sherman, S.L. A review of translational animal models for knee osteoarthritis. Arthritis
**2012**, 2012, 14. [Google Scholar] [CrossRef] - Piel, M.J.; Kroin, J.S.; Van Wijnen, A.J.; Kc, R.; Im, H.J. Pain assessment in animal models of osteoarthritis. Gene
**2014**, 537, 184–188. [Google Scholar] [CrossRef] [PubMed] - Guatteo, R.; Guémené, D. Sources of known and/or potential pain in farm animals. Adv. Anim. Biosci.
**2014**, 5, 319–332. [Google Scholar] [CrossRef] - Zufferey, R.; Minnig, A.; Thomann, B.; Zwygart, S.; Keil, N.; Schüpbach, G.; Miserez, R.; Zanolari, P.; Stucki, D. Animal-based indicators for on-farm welfare assessment in sheep. Animals
**2021**, 11, 2973. [Google Scholar] [CrossRef] [PubMed] - Price, D.D. Psychological and neural mechanisms of the affective dimension of pain. Science
**2000**, 288, 1769–1772. [Google Scholar] [CrossRef] [Green Version] - Fell, L.R.; Shutt, D.A. Behavioural and hormonal responses to acute surgical stress in sheep. Appl. Anim. Behav. Sci.
**1989**, 22, 283–294. [Google Scholar] [CrossRef] - Mellor, D.J.; Murray, L. Effects of tail docking and castration on behaviour and plasma cortisol concentrations in young lambs. Res. Vet. Sci.
**1989**, 46, 387–391. [Google Scholar] [CrossRef] - Lester, S.J.; Mellor, D.J.; Holmes, R.J.; Ward, R.N.; Stafford, K.J. Behavioural and cortisol responses of lambs to castration and tailing using different methods. N. Z. Vet. J.
**1996**, 44, 45–54. [Google Scholar] [CrossRef] [PubMed] - Paull, D.R.; Lee, C.; Atkinson, S.J.; Fisher, A.D. Effects of meloxicam or tolfenamic acid administration on the pain and stress responses of Merino lambs to mulesing. Aust. Vet. J.
**2008**, 86, 303–311. [Google Scholar] [CrossRef] - Molony, V.; Kent, J.E.; Robertson, I.S. Behavioural responses of lambs of three ages in the first three hours after three methods of castration and tail docking. Res. Vet. Sci.
**1993**, 55, 236–245. [Google Scholar] [CrossRef] - Molony, V.; Kent, J.E. Assessment of acute pain in farm animals using behavioral and physiological measurements. J. Anim. Sci.
**1997**, 75, 266–272. [Google Scholar] [CrossRef] - Small, A.H.; Marini, D.; Dyall, T.; Paull, D.; Lee, C. A randomised field study evaluating the effectiveness of buccal meloxicam and topical local anaesthetic formulations administered singly or in combination at improving welfare of female Merino lambs undergoing surgical mulesing and hot knife tail docking. Res. Vet. Sci.
**2018**, 118, 305–311. [Google Scholar] [CrossRef] [PubMed] - Futro, A.; Masłowska, K.; Dwyer, C.M. Ewes direct most maternal attention towards lambs that show the greatest pain-related behavioural responses. PLoS ONE
**2015**, 10, e0134024. [Google Scholar] [CrossRef] - Molony, V.; Kent, J.E.; Viñuela-Fernández, I.; Anderson, C.; Dwyer, C.M. Pain in lambs castrated at 2 days using novel smaller and tighter rubber rings without and with local anaesthetic. Vet. J.
**2012**, 193, 81–86. [Google Scholar] [CrossRef] - Thornton, P.; Waterman-Pearson, A. Behavioural responses to castration in lambs. Anim. Welf.
**2002**, 11, 203–212. [Google Scholar] - Kent, J.E.; Molony, V.; Graham, M.J. Comparison of methods for the reduction of acute painproduced by rubber ring castration or tail docking of week-old lambs. Vet. J.
**1998**, 155, 39–51. [Google Scholar] [CrossRef] - Dinniss, A.S.; Stafford, K.J.; Mellort, D.J.; Bruce, R.A.; Ward, R.N. The behaviour pattern of lambs after castration using a rubber ring and/or castrating clamp with or without local anaesthetic. N. Z. Vet. J.
**1999**, 47, 198–203. [Google Scholar] [CrossRef] - Molony, V.; Kent, J.E.; McKendrick, I.J. Validation of a method for assessment of an acute pain in lambs. Appl. Anim. Behav. Sci.
**2002**, 76, 215–238. [Google Scholar] [CrossRef] - Kent, J.E.; Molony, V.; Robertson, I.S. Comparison of the Burdizzo and rubber ring methods for castrating and tail docking lambs. Vet. Rec.
**1995**, 136, 192–196. [Google Scholar] [CrossRef] - Grant, C. Behavioural responses of lambs to common painful husbandry procedures. Appl. Anim. Behav. Sci.
**2004**, 87, 255–273. [Google Scholar] [CrossRef] - Lomax, S.; Dickson, H.; Sheil, M.; Windsor, P.A. Topical anaesthesia alleviates short-term pain of castration and tail docking in lambs. Aust. Vet. J.
**2010**, 88, 67–74. [Google Scholar] [CrossRef] [PubMed] - Melches, S.; Mellema, S.C.; Doherr, M.G.; Wechsler, B.; Steiner, A. Castration of lambs: A welfare comparison of different castration techniques in lambs over 10 weeks of age. Vet. J.
**2007**, 173, 554–563. [Google Scholar] [CrossRef] [PubMed] - Small, A.H.; Jongman, E.C.; Niemeyer, D.; Lee, C.; Colditz, I.G. Efficacy of precisely injected single local bolus of lignocaine for alleviation of behavioural responses to pain during tail docking and castration of lambs with rubber rings. Res. Vet. Sci.
**2020**, 133, 210–218. [Google Scholar] [CrossRef] [PubMed] - Izer, J.M.; LaFleur, R.A.; Weiss, W.J.; Wilson, R.P. Development of a pain scoring system for use in sheep surgically implanted with ventricular assist devices. 2018, 32, 706–715. J. Investig. Surg.
**2018**, 32, 706–715. [Google Scholar] [CrossRef] - Fitzpatrick, J.; Scott, M.; Nolan, A. Assessment of pain and welfare in sheep. Small Rumin. Res.
**2006**, 62, 55–61. [Google Scholar] [CrossRef] - Silva, N.E.O.F.; Trindade, P.H.E.; Oliveira, A.R.; Taffarel, M.O.; Moreira, M.A.P.; Denadai, R.; Rocha, P.B.; Luna, S.P.L. Correction: Validation of the Unesp-Botucatu composite scale to assess acute postoperative abdominal pain in sheep (USAPS). PLoS ONE
**2022**, 17, e0268305. [Google Scholar] [CrossRef] - Streiner, D.L.; Norman, G.R.; Cairney, J. Health Measurement Scales—A Practical Guide to Their Development and Use; Oxford University Press: Oxford, UK, 2015; Volume 60. [Google Scholar]
- Mokkink, L.B.; Prinsen, C.A.C.; Patrick, D.L.; Alonso, J.; Bouter, L.M.; de Vet, H.C.W.; Terwee, C.B. COSMIN Manual for Systematic Reviews of PROMs COSMIN Methodology for Systematic Reviews of Patient-Reported Outcome Measures (PROMs) User Manual. Available online: https://cosmin.nl/wp-content/uploads/COSMIN-syst-review-for-PROMs-manual_version-1_feb-2018.pdf (accessed on 14 August 2022).
- Lötsch, J.; Ultsch, A. Machine learning in pain research. Pain
**2018**, 159, 623. [Google Scholar] [CrossRef] - Kapos, F.P.; Look, J.O.; Zhang, L.; Hodges, J.S.; Schiffman, E.L. Predictors of long-term TMD pain intensity: An 8-year cohort study. J. Oral Facial Pain Headache
**2018**, 32, 113. [Google Scholar] [CrossRef] - Gugliotta, M.; Da Costa, B.R.; Dabis, E.; Theiler, R.; Jüni, P.; Reichenbach, S.; Landolt, H.; Hasler, P. Surgical versus conservative treatment for lumbar disc herniation: A prospective cohort study. BMJ Open
**2016**, 6, e012938. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Turner, D.; Griffiths, A.M.; Steinhart, A.H.; Otley, A.R.; Beaton, D.E. Mathematical weighting of a clinimetric index (Pediatric Ulcerative Colitis Activity Index) was superior to the judgmental approach. J. Clin. Epidemiol.
**2009**, 62, 738–744. [Google Scholar] [CrossRef] [PubMed] - Turner, D.; Griffiths, A.M.; Walters, T.D.; Seah, T.; Markowitz, J.; Pfefferkorn, M.; Keljo, D.; Waxman, J.; Otley, A.; Leleiko, N.S.; et al. Mathematical weighting of the pediatric Crohn’s disease activity index (PCDAI) and comparison with its other short versions. Inflamm. Bowel Dis.
**2012**, 18, 55–62. [Google Scholar] [CrossRef] - Nikaido, T.; Sumitani, M.; Sekiguchi, M.; Konno, S. The Spine PainDETECT questionnaire: Development and validation of a screening tool for neuropathic pain caused by spinal disorders. PLoS ONE
**2018**, 13, e0193987. [Google Scholar] [CrossRef] [Green Version] - Lötsch, J.; Geisslinger, G.; Heinemann, S.; Lerch, F.; Oertel, B.G.; Ultsch, A. Quantitative sensory testing response patterns to capsaicin- and ultraviolet-B–induced local skin hypersensitization in healthy subjects: A machine-learned analysis. Pain
**2018**, 159, 11. [Google Scholar] [CrossRef] - Sutton, G.A.; Dahan, R.; Turner, D.; Paltiel, O. A behaviour-based pain scale for horses with acute colic: Scale construction. Vet. J.
**2013**, 196, 394–401. [Google Scholar] [CrossRef] - Kilkenny, C.; Browne, W.; Cuthill, I.C.; Emerson, M.; Altman, D.G. Animal research: Reporting in vivo experiments: The ARRIVE guidelines. Br. J. Pharmacol.
**2010**, 160, 1577. [Google Scholar] [CrossRef] - Russell, W.M.S.; Burch, R.L. The Principles of Humane Experimental Technique; Methuen Publishing Ltd.: London, UK, 1959. [Google Scholar]
- Banks, R.E. The 4th R of research. Contemp. Top. Lab. Anim. Sci.
**1995**, 34, 50–51. [Google Scholar] - Michael, K.; Nachtsheim, C.; Neter, J.; Li, W. Applied Linear Statistical Models, 5th ed.; McGraw-Hill/Irwin: New York, NY, USA, 2004. [Google Scholar]
- James, G.; Witten, D.; Hastie, T.; Tibshirani, R. An Introduction to Statistical Learning with Applications in R; Springer: New York, NY, USA, 2014. [Google Scholar]
- Heiberg, T.; Nordby, T.; Kvien, T.K.; Buanes, T. Development and Preliminary validation of the pancreatic cancer disease impact score. Support. Care Cancer
**2013**, 21, 1677–1684. [Google Scholar] [CrossRef] [Green Version] - Bashir, N.S.; Walters, T.D.; Griffiths, A.M.; Ungar, W.J. An assessment of the validity and reliability of the pediatric child health utility 9d in children with inflammatory bowel disease. Children
**2021**, 8, 343. [Google Scholar] [CrossRef] - Sun, H.; Papadopoulos, E.J.; Hyams, J.S.; Griebel, D.; Lee, J.J.; Tomaino, J.; Mulberg, A.E. Well-defined and reliable clinical outcome assessments for pediatric Crohn disease: A critical need for drug development. J. Pediatr. Gastroenterol. Nutr.
**2015**, 60, 729–736. [Google Scholar] [CrossRef] - Shaoul, R.; Day, A.S. An overview of tools to score severity in pediatric inflammatory bowel disease. Front. Pediatr.
**2021**, 9, 271. [Google Scholar] [CrossRef] [PubMed] - Carlsen, K.; Frederiksen, N.W.; Wewer, V. Integration of EHealth into pediatric inflammatory bowel disease care is safe: 3 years of follow-up of daily care. J. Pediatr. Gastroenterol. Nutr.
**2021**, 72, 723–727. [Google Scholar] [CrossRef] [PubMed] - Brondani, J.T.; Luna, S.P.L.; Padovani, C.R. Refinement and initial validation of a multidimensional composite scale for use in assessing acute postoperative pain in cats. Am. J. Vet. Res.
**2011**, 72, 174–183. [Google Scholar] [CrossRef] [PubMed] - Brondani, J.T.; Mama, K.R.; Luna, S.P.L.; Wright, B.D.; Niyom, S.; Ambrosio, J.; Vogel, P.R.; Padovani, C.R. Validation of the english version of the UNESP-Botucatu multidimensional composite pain scale for assessing postoperative pain in cats. BMC Vet. Res.
**2013**, 9, 1–15. [Google Scholar] [CrossRef] [PubMed] [Green Version] - de Oliveira, M.G.C.; Luna, S.P.L.; Nunes, T.L.; Firmino, P.R.; de Lima, A.G.A.; Ferreira, J.; Trindade, P.H.E.; Júnior, R.A.B.; de Paula, V.V. Post-operative pain behaviour associated with surgical castration in donkeys (Equus Asinus). Equine Vet. J.
**2021**, 53, 261–266. [Google Scholar] [CrossRef] [PubMed] - Pinho, R.H.; Leach, M.C.; Minto, B.W.; Rocha, F.D.L.; Luna, S.P.L. Postoperative pain behaviours in rabbits following orthopaedic surgery and effect of observer presence. PLoS ONE
**2020**, 15, e0240605. [Google Scholar] [CrossRef] - Pinho, R.I.H.; Luna, S.I.P.L.; Trindade, P.H.E.; Justo, A.A.; Cima, D.S.; Fonseca, M.W.; Minto, B.W.; Rocha, F.D.L.; Miller, A.; Flecknell, P.; et al. Validation of the Rabbit Pain Behaviour Scale (RPBS) to assess acute postoperative pain in rabbits (Oryctolagus Cuniculus). PLoS ONE
**2022**, 17, e0268973. [Google Scholar] [CrossRef] - Belli, M.; de Oliveira, A.R.; de Lima, M.T.; Trindade, P.H.E.; Steagall, P.V.; Luna, S.P.L. Clinical validation of the short and long UNESP-Botucatu scales for feline pain assessment. PeerJ
**2021**, 9, e11225. [Google Scholar] [CrossRef] [PubMed] - Luna, S.P.L.; Trindade, P.H.E.; Monteiro, B.P.; Crosignani, N.; della Rocca, G.; Ruel, H.L.M.; Yamashita, K.; Kronen, P.; Te Tseng, C.; Teixeira, L.; et al. Multilingual validation of the short form of the Unesp-Botucatu feline pain scale (UFEPS-SF). PeerJ
**2022**, 10, e13134. [Google Scholar] [CrossRef] - de Oliveira, F.A.; Luna, S.P.L.; do Amaral, J.B.; Rodrigues, K.A.; Sant’Anna, A.C.; Daolio, M.; Brondani, J.T. Validation of the UNESP-Botucatu unidimensional composite pain scale for assessing postoperative pain in cattle. BMC Vet. Res.
**2014**, 10, 1–14. [Google Scholar] [CrossRef] [Green Version] - Gleerup, K.B.; Andersen, P.H.; Munksgaard, L.; Forkman, B. Pain evaluation in dairy cattle. Appl. Anim. Behav. Sci.
**2015**, 171, 25–32. [Google Scholar] [CrossRef] [Green Version] - Luna, S.P.L.; de Araújo, A.L.; da Nóbrega Neto, P.I.; Brondani, J.T.; de Oliveira, F.A.; Azerêdo, L.M.D.S.; Telles, F.G.; Trindade, P.H.E. Validation of the UNESP-Botucatu pig composite acute pain scale (UPAPS). PLoS ONE
**2020**, 15, e0233552. [Google Scholar] [CrossRef] - da Rocha, P.B.; Driessen, B.; McDonnell, S.M.; Hopster, K.; Zarucco, L.; Gozalo-Marcilla, M.; Hopster-Iversen, C.; Trindade, P.H.E.; da Rocha, T.K.G.; Taffarel, M.O.; et al. A critical evaluation for validation of composite and unidimensional postoperative pain scales in horses. PLoS ONE
**2021**, 16, e0255618. [Google Scholar] [CrossRef] - Taffarel, M.O.; Luna, S.P.L.; de Oliveira, F.A.; Cardoso, G.S.; de Moura Alonso, J.; Pantoja, J.C.; Brondani, J.T.; Love, E.; Taylor, P.; White, K.; et al. Refinement and partial validation of the UNESP-Botucatu Multidimensional composite pain scale for assessing postoperative pain in horses. BMC Vet. Res.
**2015**, 11, 1–12. [Google Scholar] [CrossRef] [PubMed] [Green Version] - de Oliveira, M.G.C.; de Paula, V.V.; Mouta, A.N.; Lima, I.D.O.; Macêdo, L.B.D.; Nunes, T.L.; Trindade, P.H.E.; Luna, S.P.L. Validation of the Donkey Pain Scale (DOPS) for assessing postoperative pain in donkeys. Front. Vet. Sci.
**2021**, 8, 532. [Google Scholar] [CrossRef] [PubMed] - Trindade, P.H.E.; Taffarel, M.O.; Luna, S.P.L. Spontaneous behaviors of post-orchiectomy pain in horses regardless of the effects of time of day, anesthesia, and analgesia. Animals
**2021**, 11, 1629. [Google Scholar] [CrossRef] [PubMed] - Box, G.E.; Draper, N.R. Empirical Model-Building and Response Surfaces; John Wiley & Sons: New York, NY, USA, 1987. [Google Scholar]
- Mead, B.E.; Karp, J.M. All models are wrong, but some organoids may be useful. Genome Biol.
**2019**, 20, 1–3. [Google Scholar] [CrossRef] [PubMed] - McLennan, K.; Mahmoud, M. Development of an automated pain facial expression detection system for sheep (Ovis Aries). Animals
**2019**, 9, 196. [Google Scholar] [CrossRef] [Green Version] - Noor, A.; Zhao, Y.; Koubaa, A.; Wu, L.; Khan, R.; Abdalla, F.Y.O. Automated sheep facial expression classification using deep transfer learning. Comput. Electron. Agric.
**2020**, 175, 105528. [Google Scholar] [CrossRef] - Evangelista, M.C.; Watanabe, R.; Leung, V.S.Y.; Monteiro, B.P.; O’Toole, E.; Pang, D.S.J.; Steagall, P.V. Facial Expressions of pain in cats: The development and validation of a feline grimace scale. Sci. Rep.
**2019**, 9, 19128. [Google Scholar] [CrossRef] [Green Version] - Viscardi, A.V.; Hunniford, M.; Lawlis, P.; Leach, M.; Turner, P.V. Development of a piglet grimace scale to evaluate piglet pain using facial expressions following castration and tail docking: A pilot study. Front. Vet. Sci.
**2017**, 4, 51. [Google Scholar] [CrossRef]

**Figure 1.**Importance of each behavior from USAPS based on the Wald statistic of the fixed effects in the multilevel binomial logistic regression algorithm using the expert opinion (no-event = no indication of analgesia; event = indication of analgesia) as the predictor variable.

**Figure 2.**Importance of each behavior from USAPS based on the random forest algorithm using the expert opinion (no event = no indication of analgesia; event = indication of analgesia) as the predictor variable.

**Figure 3.**Two-dimensional perceptual map of the multiple correspondence analysis showing the dispersion of expert opinion of no-indication or indication to apply analgesia and probability of sheep needing analgesia according to the multilevel binomial logistic regression model (Circles and squares indicate each evaluation; the greater the number of dark squares and the greater the number of light circles, the greater the accuracy of the algorithm).

**Figure 4.**Two-dimensional perceptual map of the multiple correspondence analysis showing the dispersion of expert opinion of no-indication or indication to apply analgesia and probability of sheep needing analgesia according to the random forest algorithm (Circles and squares indicate each evaluation; the greater the number of dark squares and the greater the number of light circles, the greater the accuracy of the algorithm).

**Figure 5.**Two-dimensional perceptual map of the multiple correspondence analysis showing the dispersion of the time points (

**A**), behavioral items of the USAPS (

**B**), evaluators (

**C**), and phases (

**D**) based on 100% of the sheep dataset [Time points: before laparoscopy (M1), 3–4h after anesthetic recovery, and before postoperative analgesia (M2), 1 h after the administration of postoperative analgesia (M3) and 24 h after laparoscopy (M4); Smaller circles indicate each evaluation and larger circles indicate the centroid)].

**Table 1.**Findings based on a multilevel binomial logistic regression algorithm using the expert opinion (no-event = no indication of analgesia; event = indication of analgesia) as the predictor variable.

N = 1536 | Slope Coefficient (β) | |||
---|---|---|---|---|

Fixed Effects | Estimate | SE | Z-Value | p-Value |

Linear coefficient (α) | −4.0592 | 0.3949 | −10.2799 | 8.69^{−25} **** |

Interaction score | ||||

(0) Active, attentive to the environment, interacts and/or follows other animals | ||||

(1) Apathetic: may remain close to other animals, but interacts little | 1.4327 | 0.2411 | 5.9415 | 2.83^{−9} **** |

(2) Very apathetic: isolated or not interacting with other animals, not interested in the environment | 2.6128 | 0.4838 | 5.4004 | 6.65^{−8} **** |

Locomotion score | ||||

(0) Moves about freely, without altered locomotion; when stopped, the pelvic limbs are parallel to the thoracic limbs | ||||

(1) Moves about with restriction and/or short steps and/or pauses and/or lameness; when stopped, the thoracic or pelvic limbs may be more open and further back than normal | 2.2143 | 0.2555 | 8.6650 | 4.51^{−18} **** |

(2) Difficulty and/or reluctant to stand up and/or not moving and/or walking abnormally and/or limping; may lean against a surface | 2.8235 | 0.3096 | 9.1207 | 7.46^{−20} **** |

Head position score | ||||

(0) Head above the withers or eating | ||||

(1) Head at the height of withers | 0.1648 | 0.2469 | 0.6677 | 0.5043 |

(2) Head below the withers (except when eating) | 0.1400 | 0.3513 | 0.3984 | 0.6903 |

Appetite score | ||||

(0) Normorexia and/or rumination present | ||||

(1) Hyporexia | 1.5283 | 0.4657 | 3.2821 | 0.0010 *** |

(2) Anorexia | 0.4517 | 0.3172 | 1.4239 | 0.1545 |

Activity score | ||||

(0) Moves normally | ||||

(1) Restless, moves more than normal, or lies down and stands up frequently | 1.9310 | 0.3507 | 5.5059 | 3.67^{−8} **** |

(2) Moves less frequently or only when stimulated using a stick or does not move | 1.6895 | 0.2530 | 6.6781 | 2.42^{−11} **** |

Posture score | ||||

Arched back | 0.5527 | 0.4081 | 1.3543 | 0.1756 |

Extends the head and neck | 0.6284 | 0.3031 | 2.0729 | 0.0382 ** |

Lying down with head resting on the ground or close to the ground | 0.7387 | 0.4271 | 1.7295 | 0.0837 * |

Moves the tail quickly (except when breastfeeding) and repeatedly and/or keeps the tail straight (except to defecate/urinate) | 2.0622 | 0.7126 | 2.8940 | 0.0038 *** |

Random effects | Variance | SD | Groups | |

Sheep (intercept) | 8.6422^{−9} | 9.2963^{−5} | 48 | |

Observers (intercept) | 5.3755^{−2} | 2.3185^{−1} | 4 | |

Moments (intercept) | 3.0738^{−1} | 5.5442^{−1} | 4 | |

Phases (intercept) | 9.1439^{−8} | 3.0239^{−4} | 2 | |

Model parameters | Full model | Null model | Short model | |

Log-Likelihood (df) | −325.48 (19) | −683.97 (5) | −329.22 (14) | |

P-value Log-Likelihood vs. Full Model | - | <2.2^{−16} **** | 0.1874 | |

AIC | 688.96 | 1377.96 | 686.44 | |

BIC | 790.36 | 1404.64 | 761.15 | |

Pseudo-R² (fixed effects) | 0.76 | - | 0.75 | |

Pseudo-R² (total) | 0.78 | 0.60 | 0.78 |

**Table 2.**Optimal cut-off, specificity, sensitivity, and area under the curve from receiver operating characteristic curve of the original USAPS and weighted USAPS from multilevel binomial logistic regression algorithm (no event = no indication of analgesia; event = indication of analgesia).

Dataset | Parameters | Original USAPS (Total Sum) | Weighted USAPS (Probability to Need Analgesia Based on Logistic Regression) | p-Value |
---|---|---|---|---|

100% | Optimal cut-off | 03.50 (03.50–04.50) | 43.88 (29.61–61.38) | - |

Specificity | 87.67 (85.50–93.49) | 91.21 (87.21–94.43) | - | |

Sensitivity | 91.97 (85.61–93.94) | 92.88 (87.88–96.06) | - | |

AUC | 95.32 (94.30–96.35) | 96.83 (95.98–97.68) | 1.381^{−9} | |

Test data (30%) | Optimal cut-off | 03.50 (03.50–04.50) | 59.63 (43.14–66.67) | - |

Specificity | 86.07 (80.74–93.03) | 92.21 (86.89–95.90) | - | |

Sensitivity | 92.16 (83.82–96.08) | 92.16 (87.25–96.57) | - | |

AUC | 94.87 (92.94–96.80) | 96.59 (95.02–98.15) | 4.891^{−4} |

**Table 3.**Optimal cut-off, specificity, sensitivity, and area under the curve from the receiver operating characteristic curve of the original USAPS and weighted USAPS from random forest algorithm (no event = no indication of analgesia; event = indication of analgesia).

Dataset | Parameters | Original USAPS (Total Sum) | Weighted USAPS (Probability to Need Analgesia Based on Random Forest) | p-Value |
---|---|---|---|---|

Training data (70%) | Optimal cut-off | 03.50 (03.50–04.50) | 42.21 (20.68–64.09) | - |

Specificity | 88.92 (86.55–94.94) | 94.62 (90.82–96.99) | - | |

Sensitivity | 91.45 (85.09–94.08) | 93.42 (90.13–96.49) | - | |

AUC | 95.47 (94.25–96.69) | 97.50 (96.56–98.45) | 1.822^{−9} | |

Test data (30%) | Optimal cut-off | 03.50 (03.50–04.50) | 35.41 (35.26–65.13) | - |

Specificity | 86.07 (80.74–93.03) | 89.34 (85.25–93.85) | - | |

Sensitivity | 92.16 (83.82–96.08) | 95.10 (90.69–98.04) | - | |

AUC | 94.87 (92.94–96.80) | 96.28 (94.17–97.85) | 0.0067 |

Behavioral Items | Ranking | ||
---|---|---|---|

Multiple Binomial Logistic Regression | Random Forest | Delta | |

‘Activity 2′ | 1st | 3rd | 2 |

‘Locomotion 2′ | 2nd | 1st | −1 |

‘Interaction 2′ | 3rd | 6th | 3 |

‘Interaction 1′ | 4th | 4th | 0 |

‘Locomotion 1′ | 5th | 2nd | −3 |

‘Extend Head’ | 6th | 9th | 3 |

‘Head Position 2′ | 7th | 14th | 7 |

‘Appetite 1′ | 8th | 7th | −1 |

‘Head Position 1′ | 9th | 13th | 4 |

‘Activity 1′ | 10th | 5th | −5 |

‘Lying Down’ | 11th | 10th | −1 |

‘Moves Tail’ | 12th | 8th | −4 |

‘Appetite 2′ | 13th | 11th | −2 |

‘Arched Back’ | 14th | 12th | −2 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Trindade, P.H.E.; Mello, J.F.S.R.d.; Silva, N.E.O.F.; Luna, S.P.L.
Improving Ovine Behavioral Pain Diagnosis by Implementing Statistical Weightings Based on Logistic Regression and Random Forest Algorithms. *Animals* **2022**, *12*, 2940.
https://doi.org/10.3390/ani12212940

**AMA Style**

Trindade PHE, Mello JFSRd, Silva NEOF, Luna SPL.
Improving Ovine Behavioral Pain Diagnosis by Implementing Statistical Weightings Based on Logistic Regression and Random Forest Algorithms. *Animals*. 2022; 12(21):2940.
https://doi.org/10.3390/ani12212940

**Chicago/Turabian Style**

Trindade, Pedro Henrique Esteves, João Fernando Serrajordia Rocha de Mello, Nuno Emanuel Oliveira Figueiredo Silva, and Stelio Pacca Loureiro Luna.
2022. "Improving Ovine Behavioral Pain Diagnosis by Implementing Statistical Weightings Based on Logistic Regression and Random Forest Algorithms" *Animals* 12, no. 21: 2940.
https://doi.org/10.3390/ani12212940