Comparison of Hamstrings and Quadriceps Muscle Activation in Male and Female Professional Soccer Players
Abstract
Featured Application
Abstract
1. Introduction
2. Materials and Methods
2.1. Ethical Considerations
2.2. Participants
2.3. Study Design
2.4. Procedure
2.5. Data Processing
2.6. Statistical Analysis
3. Results
4. Discussion
4.1. H:Q Ratio
4.2. VM:VL Ratio
4.3. Intragroup Muscular Pattern
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Ekstrand, J.; Waldén, M.; Hägglund, M. Hamstring injuries have increased by 4% annually in men’s professional football, since 2001: A 13-year longitudinal analysis of the UEFA Elite Club injury study. Br. J. Sports Med. 2016, 50, 731–737. [Google Scholar] [CrossRef]
- Nouni-Garcia, R.; Asensio-Garcia, M.R.; Orozco-Beltran, D.; Lopez-Pineda, A.; Gil-Guillen, V.F.; Quesada, J.A.; Bernabeu Casas, R.C.; Carratala-Munuera, C. The FIFA 11 programme reduces the costs associated with ankle and hamstring injuries in amateur Spanish football players: A retrospective cohort study. Eur. J. Sport Sci. 2019, 19, 1150–1156. [Google Scholar] [CrossRef]
- Askling, C.; Karlsson, J.; Thorstensson, A. Hamstring injury occurrence in elite soccer players after preseason strength training with eccentric overload. Scand. J. Med. Sci. Sports 2003, 13, 244–250. [Google Scholar] [CrossRef]
- Brockett, C.; Morgan, D.; Proske, U. Predicting hamstring strain injury in elite athletes. Med. Sci. Sports Exerc. 2004, 36, 379–387. [Google Scholar] [CrossRef]
- Croisier, J.L.; Ganteaume, S.; Binet, J.; Genty, M.; Ferret, J.-M. Strength imbalances and prevention of hamstring injury in professional soccer players: A prospective study. Am. J. Sports Med. 2008, 36, 1469–1475. [Google Scholar] [CrossRef] [PubMed]
- Ekstrand, J.; Hagglund, W. Injury incidence and injury patterns in professional football: The UEFA injury study. Br. J. Sports Med. 2011, 45, 553–558. [Google Scholar] [CrossRef] [PubMed]
- Engebretsen, A.H.; Myklebust, G.; Holme, I.; Engebretsen, L.; Bahr, R. Intrinsic risk factors for hamstring injuries among male soccer players: A prospective cohort study. Am. J. Sports Med. 2010, 38, 1147–1153. [Google Scholar] [CrossRef] [PubMed]
- McCall, A.; Carling, C.; Davison, M.; Nedelec, M.; Le Gall, F.; Berthoin, S.; Dupont, G. Injury risk factors, screening tests and preventative strategies: A systematic review of the evidence that underpins the perceptions and practices of 44 football (soccer) teams from various premier leagues. Br. J. Sports Med. 2015, 49, 583–589. [Google Scholar] [CrossRef]
- Navarro, E.; Chorro, D.; Torres, G.; García, C.; Navandar, A.; Veiga, S. A review of risk factors for hamstring injury in soccer: A biomechanical approach. Eur. J. Hum. Mov. 2015, 34, 52–74. [Google Scholar]
- Fousekis, K.; Tsepis, E.; Poulmedis, P.; Athanasopoulos, S.; Vagenas, G. Intrinsic risk factors of non-contact quadriceps and hamstring strains in soccer: A prospective study of 100 professional players. Br. J. Sports Med. 2011, 45, 709–714. [Google Scholar] [CrossRef]
- Cameron, M.; Adams, R.; Maher, C. Motor control and strength as predictors of hamstring injury in elite players of Australian football. Phys. Ther. Sport 2003, 4, 159–166. [Google Scholar] [CrossRef]
- Greco, C.; Silva, W.L.; Camarda, S.R.; Denadai, B.S. Fatigue and rapid hamstring/quadriceps force capacity in professional soccer players. Clin. Physiol. Funct. Imaging 2013, 33, 18–23. [Google Scholar] [CrossRef] [PubMed]
- Greig, M.; Siegler, J.C. Soccer-specific fatigue and eccentric hamstrings muscle strength. J. Athl. Train. 2009, 44, 180–184. [Google Scholar] [CrossRef]
- Small, K.; McNaughton, L.; Greig, M.; Lovell, R. Effect of timing of eccentric hamstring strengthening exercises during soccer training: Implications for muscle fatigability. J. Strength Condit. Res. 2009, 23, 1077–1083. [Google Scholar] [CrossRef] [PubMed]
- Larruskain, J.; Lekue, J.A.; Diaz, N.; Odriozola, A.; Gil, S.M. A comparison of injuries in elite male and female football players: A 5-Season prospective study. Scand. J. Med. Sci. Sports 2017, 28, 237–245. [Google Scholar] [CrossRef]
- Arendt, E.A.; Agel, J.; Dick, R.J. Anterior cruciate ligament injury patterns among collegiate men and women. J. Athl. Train. 1999, 34, 86. [Google Scholar]
- Waldén, M.; Hägglund, M.; Werner, J.; Ekstrand, J. The epidemiology of anterior cruciate ligament injury in football (soccer): A review of the literature from a gender-related perspective. Knee Surg. Sports Traumatol. Arthrosc. 2011, 19, 3–10. [Google Scholar] [CrossRef]
- Alentorn-Geli, E.; Myer, G.D.; Silvers, H.J.; Samitier, G.; Romero, D.; Lázaro-Haro, C.; Cugat, R. Prevention of non-contact anterior cruciate ligament injuries in soccer players. Part 1: Mechanisms of injury and underlying risk factors. Knee Surg. Sports Traumatol. Arthrosc. 2009, 17, 705–729. [Google Scholar] [CrossRef]
- Boling, M.C.; Padua, D.A.; Marshall, S.W.; Guskiewicz, K.; Pyne, S.; Beutler, A. A Prospective Investigation of Biomechanical Risk Factors for Patellofemoral Pain Syndrome:The Joint Undertaking to Monitor and Prevent ACL Injury (JUMP-ACL) Cohort. Am. J. Sports Med. 2009, 37, 2108–2116. [Google Scholar] [CrossRef]
- Kaeding, C.C.; Pedroza, A.D.; Reinke, E.K.; Huston, L.J.; Spindler, K.P. Risk Factors and Predictors of Subsequent ACL Injury in Either Knee After ACL Reconstruction:Prospective Analysis of 2488 Primary ACL Reconstructions From the MOON Cohort. Am. J. Sports Med. 2015, 43, 1583–1590. [Google Scholar] [CrossRef]
- Wojtys, E.M.; Huston, L.J. Neuromuscular performance in normal and anterior cruciate ligament-deficient lower extremities. Am. J. Sports Med. 1994, 22, 89–104. [Google Scholar] [CrossRef] [PubMed]
- Cowling, E.J.; Steele, J.R. Is lower limb muscle synchrony during landing affected by gender? Implications for variations in ACL injury rates. J. Electromyogr. Kinesiol. 2001, 11, 263–268. [Google Scholar] [CrossRef]
- Letafatkar, A.; Rajabi, R.; Tekamejani, E.E.; Minoonejad, H.J.T.k. Effects of perturbation training on knee flexion angle and quadriceps to hamstring cocontraction of female athletes with quadriceps dominance deficit: Pre–post intervention study. Knee 2015, 22, 230–236. [Google Scholar] [CrossRef] [PubMed]
- Cross, K.M.; Gurka, K.K.; Saliba, S.; Conaway, M.; Hertel, J. Comparison of hamstring strain injury rates between male and female intercollegiate soccer athletes. Am. J. Sports Med. 2013, 41, 742–748. [Google Scholar] [CrossRef] [PubMed]
- Markolf, K.L.; O’Neill, G.; Jackson, S.R.; McAllister, D.R. Effects of Applied Quadriceps and Hamstrings Muscle Loads on Forces in the Anterior and Posterior Cruciate Ligaments. Am. J. Sports Med. 2004, 32, 1144–1149. [Google Scholar] [CrossRef] [PubMed]
- Landry, S.C.; McKean, K.A.; Hubley-Kozey, C.L.; Stanish, W.D.; Deluzio, K.J. Neuromuscular and Lower Limb Biomechanical Differences Exist between Male and Female Elite Adolescent Soccer Players during an Unanticipated Side-cut Maneuver. Am. J. Sports Med. 2007, 35, 1888–1900. [Google Scholar] [CrossRef]
- Van Dyk, N.; Bahr, R.; Whiteley, R.; Tol, J.L.; Kumar, B.D.; Hamilton, B.; Farooq, A.; Witvrouw, E. Hamstring and quadriceps isokinetic strength deficits are weak risk factors for hamstring strain injuries: A 4-year cohort study. Am. J. Sports Med. 2016, 44, 1789–1795. [Google Scholar] [CrossRef]
- Evangelidis, P.E.; Pain, M.T.; Folland, J. Angle-specific hamstring-to-quadriceps ratio: A comparison of football players and recreationally active males. J. Sports Sci. 2015, 33, 309–319. [Google Scholar] [CrossRef]
- Kannus, P. Isokinetic evaluation of muscular performance. Int. J. Sports Med. 1994, 15, S11–S18. [Google Scholar] [CrossRef]
- Caterisano, A.; Moss, R.E.; Pellinger, T.K.; Woodruff, K.; Lewis, V.C.; Booth, W.; Khadra, T. The effect of back squat depth on the EMG activity of 4 superficial hip and thigh muscles. J. Strength Condit. Res. 2002, 16, 428–432. [Google Scholar]
- Navarro, E.; Chorro, D.; Torres, G.; Navandar, A.; Rueda, J.; Veiga, S. Electromyographic activity of quadriceps and hamstrings of a professional football team during Bulgarian Squat and Lunge exercises. J. Hum. Sport Exerc. 2020, 1. [Google Scholar] [CrossRef]
- Torres, G.; Chorro, D.; Navandar, A.; Rueda, J.; Fernández, L.; Navarro, E. Assessment of Hamstring: Quadriceps Coactivation without the Use of Maximum Voluntary Isometric Contraction. Appl. Sci. 2020, 10, 1615. [Google Scholar] [CrossRef]
- Begalle, R.L.; DiStefano, L.J.; Blackburn, T.; Padua, D.A. Quadriceps and Hamstrings Coactivation During Common Therapeutic Exercises. J. Athl. Train. (Allen Press) 2012, 47, 396–405. [Google Scholar] [CrossRef]
- Pincivero, D.M.; Aldworth, C.; Dickerson, T.; Petry, C.; Shultz, T. Quadriceps-hamstring EMG activity during functional, closed kinetic chain exercise to fatigue. Eur. J. Appl. Physiol. 2000, 81, 504–509. [Google Scholar] [CrossRef] [PubMed]
- Santana, J. Single-leg training for 2-legged sports: Efficacy of strength development in athletic performance. Strength Condit. J. 2001, 23, 35. [Google Scholar] [CrossRef]
- Ruas, C.V.; Pinto, R.S.; Haff, G.G.; Lima, C.; Pinto, M.D.; Brown, L.E. Alternative Methods of Determining Hamstrings-to-Quadriceps Ratios: A Comprehensive Review. Sports Med. 2019, 5, 11. [Google Scholar] [CrossRef]
- Nimphius, S.; McBride, J.M.; Rice, P.E.; Goodman-Capps, C.L.; Capps, C.R. Comparison of Quadriceps and Hamstring Muscle Activity during an Isometric Squat between Strength-Matched Men and Women. J. Sports Sci. Med. 2019, 18, 101–108. [Google Scholar]
- Youdas, J.W.; Hollman, J.H.; Hitchcock, J.R.; Hoyme, G.J.; Johnsen, J.J. Comparison of hamstring and quadriceps femoris electromyographic activity between men and women during a single-limb squat on both a stable and labile surface. J. Strength Condit. Res. 2007, 21, 105–111. [Google Scholar] [CrossRef] [PubMed]
- Hermens, H.J.; Freriks, B.; Disselhorst-Klug, C.; Rau, G. Development of recommendations for SEMG sensors and sensor placement procedures. J. Electromyogr. Kinesiol. 2000, 10, 361–374. [Google Scholar] [CrossRef]
- Robertson, R.; Dowling, J. Design and responses of Butterworth and critically damped digital filters. J. Electromyogr. Kinesiol. 2003, 13, 569–573. [Google Scholar] [CrossRef]
- De Luca, C.; Gilmore, L.D.; Kuznetsov, M.; Roy, S.H. Filtering the surface EMG signal: Movement artifact and baseline noise contamination. J. Biomech. 2010, 43, 1573–1579. [Google Scholar] [CrossRef]
- Fukuda, T.Y.; Echeimberg, J.O.; Pompeu, J.E.; Lucareli, P.R.G.; Garbelotti, S.; Gimenes, R.O.; Apolinário, A. Root mean square value of the electromyographic signal in the isometric torque of the quadriceps, hamstrings and brachial biceps muscles in female subjects. J. Appl. Res. 2010, 10, 32–39. [Google Scholar]
- Cohen, J. A power primer. Psychol. Bull. 1992, 112, 155. [Google Scholar] [CrossRef] [PubMed]
- McCurdy, K.; O’Kelley, E.; Kutz, M.; Langford, G.; Ernest, J.; Torres, M. Comparison of lower extremity EMG between the 2-leg squat and modified single-leg squat in female athletes. J. Sport Rehab. 2010, 19, 57–70. [Google Scholar] [CrossRef] [PubMed]
- Liporaci, R.F.; Saad, M.C.; Bevilaqua-Grossi, D.; Riberto, M.J.B.O.S.; Medicine, E. Preseason intrinsic risk factors—associated odds estimate the exposure to proximal lower limb injury throughout the season among professional football players. BMJ Open Sport Exerc. Med. 2018, 4, e000334. [Google Scholar] [CrossRef] [PubMed]
- Marras, W.S.; Davis, K.G. A non-MVC EMG normalization technique for the trunk musculature: Part 1. Method development. J. Electromyogr. Kinesiol. 2001, 11, 1–9. [Google Scholar] [CrossRef]
- Suydam, S.M.; Manal, K.; Buchanan, T.S. The advantages of normalizing electromyography to ballistic rather than isometric or isokinetic tasks. J. Appl. Biomech. 2017, 33, 189–196. [Google Scholar] [CrossRef]
- Robertson, G.E.; Caldwell, G.E.; Hamill, J.; Kamen, G.; Whittlesey, S. Research Methods in Biomechanics; Human Kinetics: Champaign, IL, USA, 2013. [Google Scholar]
- Wright, J.; Delong, T.; Gehlsen, G. Electromyographic Activity of the Hamstrings During Performance of the Leg Curl, Stiff-Leg Deadlift, and Back Squat Movements. J. Strength Condit. Res. 1999, 13, 168–174. [Google Scholar]
- El-Ashker, S.; Carson, B.; Ayala, F.; De Ste Croix, M. Sex-related differences in joint-angle-specific functional hamstring-to-quadriceps strength ratios. Knee Surg. Sports Traumatol. Arthrosc. 2017, 25, 949–957. [Google Scholar] [CrossRef]
- Andrade, M.D.S.; De Lira, C.A.B.; Koffes, F.D.C.; Mascarin, N.C.; Benedito-Silva, A.A.; Da Silva, A.C. Isokinetic hamstrings-to-quadriceps peak torque ratio: The influence of sport modality, gender, and angular velocity. J. Sports Sci. 2012, 30, 547–553. [Google Scholar] [CrossRef]
- Kong, P.W.; Burns, S.F. Bilateral difference in hamstrings to quadriceps ratio in healthy males and females. Phys. Ther. Sport 2010, 11, 12–17. [Google Scholar] [CrossRef]
- Gobbi, A.; Domzalski, M.; Pascual, J. Sports Traumatology, Arthroscopy. Comparison of anterior cruciate ligament reconstruction in male and female athletes using the patellar tendon and hamstring autografts. Knee Surg. Sports Traumatol. Arthrosc. 2004, 12, 534–539. [Google Scholar] [CrossRef] [PubMed]
- Myer, G.D.; Ford, K.R.; Hewett, T.E. The effects of gender on quadriceps muscle activation strategies during a maneuver that mimics a high ACL injury risk position. J. Electromyogr. Kinesiol. 2005, 15, 181–189. [Google Scholar] [CrossRef] [PubMed]
- Monajati, A.; Larumbe-Zabala, E.; Goss-Sampson, M.; Naclerio, F. Surface electromyography analysis of three squat exercises. J. Hum. Kinet. 2019, 67, 73–83. [Google Scholar] [CrossRef] [PubMed]
- Krishnan, C.; Huston, K.; Amendola, A.; Williams, G.N. Quadriceps and hamstrings muscle control in athletic males and females. J. Orthop. Res. 2008, 26, 800–808. [Google Scholar] [CrossRef]
- Jaberzadeh, S.; Yeo, D.; Zoghi, M. The Effect of Altering Knee Position and Squat Depth on VMO: VL EMG Ratio During Squat Exercises. Physiother. Res. Int. 2016, 21, 164–173. [Google Scholar] [CrossRef]
- Panagiotopoulos, E.; Strzelczyk, P.; Herrmann, M.; Scuderi, G.J. Sports Traumatology, Arthroscopy. Cadaveric study on static medial patellar stabilizers: The dynamizing role of the vastus medialis obliquus on medial patellofemoral ligament. Knee Surg. Sports Traumatol. Arthrosc. 2006, 14, 7–12. [Google Scholar] [CrossRef]
- Mostamand, J.; Bader, D.L.; Hudson, Z. Reliability testing of vasti activity measurements in taped and untaped patellofemoral conditions during single leg squatting in healthy subjects: A pilot study. J. Bodyw. Mov. Ther. 2013, 17, 271–277. [Google Scholar] [CrossRef]
- Souza, D.R.; Gross, M.T. Comparison of vastus medialis obliquus: Vastus lateralis muscle integrated electromyographic ratios between healthy subjects and patients with patellofemoral pain. Phys. Ther. 1991, 71, 310–316. [Google Scholar] [CrossRef]
- Araújo, S.R.S.; Medeiros, F.B.; Zaidan, A.D.; Pimenta, E.M.; Abreu, E.A.d.C.; Ferreira, J.C. Comparison of two classification criteria of lateral strength asymmetry of the lower limbs in professional soccer players. Rev. Brasil. Cineantropometria Desempenho Hum. 2017, 19, 644–651. [Google Scholar]
- Menzel, H.-J.; Chagas, M.H.; Szmuchrowski, L.A.; Araujo, S.R.; de Andrade, A.G.; de Jesus-Moraleida, F.R. Analysis of lower limb asymmetries by isokinetic and vertical jump tests in soccer players. J. Strength Condit. Res. 2013, 27, 1370–1377. [Google Scholar] [CrossRef] [PubMed]
- Crossley, K.; Bennell, K.; Green, S.; McConnell, J. A systematic review of physical interventions for patellofemoral pain syndrome. Clin. J. Sport Med. 2001, 11, 103–110. [Google Scholar] [CrossRef] [PubMed]
- Dimitrova, N.; Arabadzhiev, T.; Hogrel, J.-Y.; Dimitrov, G.V. Fatigue analysis of interference EMG signals obtained from biceps brachii during isometric voluntary contraction at various force levels. J. Electromyogr. Kinesiol. 2009, 19, 252–258. [Google Scholar] [CrossRef] [PubMed]
- Fukuda, T.Y.; Alvarez, A.S.; Nassri, L.F.G.; Godoy, C.M.G. Quantitative electromyographic assessment of facial muscles in cross-bite female children. Rev. Bras. Eng. Biomed. 2008, 2008, 121–129. [Google Scholar] [CrossRef]
- Bilodeau, M.; Schindler-Ivens, S.; Williams, D.; Chandran, R.; Sharma, S.S. EMG frequency content changes with increasing force and during fatigue in the quadriceps femoris muscle of men and women. J. Electromyogr. Kinesiol. 2003, 13, 83–92. [Google Scholar] [CrossRef]
- Urabe, Y.; Kobayashi, R.; Sumida, S.; Tanaka, K.; Yoshida, N.; Nishiwaki, G.A.; Tsutsumi, E.; Ochi, M. Electromyographic analysis of the knee during jump landing in male and female athletes. Knee 2005, 12, 129–134. [Google Scholar] [CrossRef]
- Krishnan, C.; Williamns, N. Sex Differences in Quadriceps and Hamstrings EMG–Moment Relationships. Med. Sci. Sports Exerc. 2009, 41, 1652–1660. [Google Scholar] [CrossRef]
- Ninos, J.C.; Irrgang, J.J.; Burdett, R.; Weiss, J.R. Electromyographic analysis of the squat performed in self-selected lower extremity neutral rotation and 30 of lower extremity turn-out from the self-selected neutral position. J. Orthop. Sports Phys. Ther. 1997, 25, 307–315. [Google Scholar] [CrossRef][Green Version]
- Thelen, D.G.; Chumanov, E.S.; Sherry, M.A.; Heiderscheit, B.C. Neuromusculoskeletal models provide insights into the mechanisms and rehabilitation of hamstring strains. Exerc. Sport Sci. Rev. 2006, 34, 135–141. [Google Scholar] [CrossRef]
Female Players | Male Players | Significance | Effect Size (d) | |||
---|---|---|---|---|---|---|
Mean | SD | Mean | SD | |||
H:Q ratio in the lunge exercise | 0.25 | 0.18 | 0.18 | 0.10 | p > 0.05 | - |
H:Q ratio in the Bulgarian squat exercise | 0.24 | 0.16 | 0.18 | 0.06 | p > 0.05 | - |
Female Players | Male Players | Significance | Effect Size (d) | |||
---|---|---|---|---|---|---|
Mean | SD | Mean | SD | |||
VM:VL ratio in the lunge exercise | 1.12 | 0.36 | 2.64 | 1.88 | p = 0.001 | 1.42 |
VM:VL ratio in the Bulgarian squat exercise | 1.10 | 1.88 | 2.04 | 0.72 | p < 0.001 | 1.76 |
Exercise Muscle | Female Players | Male Players | Significance | Effect Size (d) | |||
---|---|---|---|---|---|---|---|
Mean (%) | SD (%) | Mean (%) | SD (%) | ||||
Lunge | RF | 19.52 | 10.69 | 15.54 | 6.47 | p > 0.05 | - |
VL | 38.42 | 5.29 | 26.53 | 7.33 | p < 0.001 | 1.75 | |
VM | 42.07 | 9.75 | 57.93 | 9.82 | p = 0.001 | 1.62 | |
BF | 46.87 | 12.92 | 44.64 | 9.08 | p > 0.05 | - | |
ST | 53.13 | 12.92 | 55.36 | 9.08 | p > 0.05 | - | |
Bulgarian squat | RF | 21.70 | 10.16 | 15.90 | 6.32 | p > 0.05 | - |
VL | 36.99 | 5.57 | 28.87 | 6.57 | p = 0.006 | 1.29 | |
VM | 41.31 | 8.91 | 55.23 | 8.08 | p = 0.001 | 1.67 | |
BF | 49.92 | 1.57 | 44.64 | 9.08 | p > 0.05 | - | |
ST | 50.08 | 13.57 | 55.36 | 9.08 | p > 0.05 | - |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Torres, G.; Armada-Cortés, E.; Rueda, J.; San Juan, A.F.; Navarro, E. Comparison of Hamstrings and Quadriceps Muscle Activation in Male and Female Professional Soccer Players. Appl. Sci. 2021, 11, 738. https://doi.org/10.3390/app11020738
Torres G, Armada-Cortés E, Rueda J, San Juan AF, Navarro E. Comparison of Hamstrings and Quadriceps Muscle Activation in Male and Female Professional Soccer Players. Applied Sciences. 2021; 11(2):738. https://doi.org/10.3390/app11020738
Chicago/Turabian StyleTorres, Gonzalo, Estrella Armada-Cortés, Javier Rueda, Alejandro F. San Juan, and Enrique Navarro. 2021. "Comparison of Hamstrings and Quadriceps Muscle Activation in Male and Female Professional Soccer Players" Applied Sciences 11, no. 2: 738. https://doi.org/10.3390/app11020738
APA StyleTorres, G., Armada-Cortés, E., Rueda, J., San Juan, A. F., & Navarro, E. (2021). Comparison of Hamstrings and Quadriceps Muscle Activation in Male and Female Professional Soccer Players. Applied Sciences, 11(2), 738. https://doi.org/10.3390/app11020738