Next Article in Journal
Performing Advanced Trauma Life Support (ATLS) across Borders: Midterm Follow-Up of the Aeromedical Evacuation after Civilian Bus Accident at Madeira
Previous Article in Journal
The 10 “Cardinal Sins” in the Clinical Diagnosis and Treatment of Endometriosis: A Bayesian Approach
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 12
Issue 13
10.3390/jcm12134555
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Brief Report

Inter-Rater Reliability of the Polish Version of the Alberta Infant Motor Scale in Children with Heart Disease

by
Maria Ferenstein
,
Katarzyna Ostrzyżek-Przeździecka
,
Jakub S. Gąsior
* and
Bożena Werner
*
Department of Pediatric Cardiology and General Pediatrics, Medical University of Warsaw, 02-091 Warsaw, Poland
*
Authors to whom correspondence should be addressed.
J. Clin. Med. 2023, 12(13), 4555; https://doi.org/10.3390/jcm12134555
Submission received: 9 June 2023 / Revised: 4 July 2023 / Accepted: 5 July 2023 / Published: 7 July 2023
(This article belongs to the Section Clinical Rehabilitation)
Browse Figures
Versions Notes

Abstract

:
There is an urgent need for the systematic monitoring of motor and cognitive neurodevelopment and the evaluation of motor skill development in infants and children with heart disease. Familiarizing students and early graduates with the developmental care needed by these patients may help in the system-wide implementation of early motor screening in this population. The purpose of this study was to investigate the agreement between a last-year physiotherapy student and an experienced pediatric physiotherapist when applying the Polish version of the Alberta Infant Motor Scale (AIMS) to a heterogenous group of children with congenital heart defects. Agreement between raters was verified based on the observation of 80 (38 females) patients with heart disease aged 1–18 months using a Bland–Altman plot with limits of agreement and an intraclass correlation coefficient. The bias between raters for the total score for four age groups (0–3 months, 4–7 months, 8–11 months and 12–18 months) was between −0.17 and 0.22 (range: −0.54–0.78), and the ICC was between 0.875 and 1.000. Thus, a reliable assessment of motor development or motor skills using the Polish version of the AIMS can be performed in pediatric patients with heart defects by clinically inexperienced last-year physiotherapy students who are familiarized with the AIMS manual.

1. Introduction

An increasing number of studies have reported delayed motor milestone achievements, the impaired quality of motor behavior and general motor problems in patients with complex congenital heart disease (ConHD) [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]. Indeed, children with ConHD have an 11-fold risk of developing severe motor problems compared to their peers without any known heart failure [18]. It has been shown that the poorer motor development outcomes from infancy to adolescence in this population are related to various modifiable and non-modifiable factors: the presence of genetic anomalies [19,20,21,22,23]; alterations in brain development due to brain injury [7,11,24,25]; a low birth weight [21,26]; single-ventricle physiology/anatomy [8,14,16,27,28,29,30]; surgery in early infancy [31]; a higher number of cardiac surgeries [20,29,32,33,34]; palliative surgery intervention(s) [34,35]; pre-, peri- and post-operative complications (e.g., preoperative hypoxemia [36,37,38]; perioperative white matter injury [39]; a long intubation period [40]; a long hospital stay [5,20,29,40,41,42,43,44,45,46]; and others like the need for tube feeding [22,29] or being of the female sex [43]).
Authors of recent articles on motor development delay and lower basic motor skills in patients with ConHD stressed the importance of early motor screening in infants, but also indicated the need for the systematic longitudinal evaluation and monitoring of motor skill development through the elementary school years [3,6,8,9,13,14,16]. Very recently (2023), significant gaps in the research on developmental care for hospitalized infants with complex ConHD were identified by the American Heart Association Science Advisory Group—one of them was to understand how to measure and promote gross motor development in infants with complex ConHD [9].
Valid, reliable, convenient and relatively inexpensive scales for assessing motor development and motor competence are necessary to allow wide group of researchers and practitioners to identify motor development deficiencies in pediatric cardiac patients and to determine the effectiveness of selected interventions. The scales for evaluating motor development such as, e.g., the Bayley Scales of Infant Development, require specific training and usually have a high cost [47]. The Alberta Infant Motor Scale (AIMS) is an easy and fast-to-apply standardized observational assessment scale providing information on achieved developmental milestones, currently developing motor skills/activities and motor performance that is widely used in term and preterm infants from birth until they achieve independent walking [48]. The construction of the scale makes it approachable for both novice and experienced researchers [49]. High agreement between groups of rater/specialists with the same (experienced and/or trained pediatric physical therapists, early intervention specialists with a minimum of three years of experience, and physiotherapists, neurologists and physiatrists) or a different experience in the evaluation of pediatric patients (trained students vs. experienced examiners or physical therapists with different levels of experience) using the AIMS based on live or video observations of typically developing and preterm infants, as well as those at risk for motor delay, was confirmed [48,49,50,51,52,53,54,55]. Satisfactory agreement between experienced, trained specialists was also found with different versions of the AIMS other than the English language version [56,57,58,59,60,61]. In 2022, cultural adaptation and validation of the Polish version of the AIMS were assessed. The reliability of the observation of the developmental assessment of apparently healthy infants between two experienced (minimum 7 years) pediatric physiotherapists was 0.99 [62]. Very recently (2023), the authors established the norms of the Polish AIMS version [63].
In infants with heart disease, there is an urgent need to monitor motor development systematically and, in cases of developmental delay, to promote early intervention as soon as possible to harness neuroplasticity [8]. Motor development in children with heart defects was previously assessed according to the AIMS; however, it was only carried out by experienced/trained specialists [6,11,64]. The observation of motor development using standardized measures is a core competence in pediatric physical therapy education. Familiarizing students and early graduates with the developmental care needs of hospitalized infants with complex heart disease may help in the dissemination and implementation of early motor development monitoring in this population. However, early-stage practitioners may under- or over-report the number of children with motor developmental delay when using selected tools or scales; therefore, agreement between more experienced clinicians and students when performing the assessment should be primarily verified. The purpose of this study was to investigate agreement between two raters—a last-year physiotherapy student and pediatric physiotherapist with 10 years of clinical experience—with the Polish version of the AIMS in a heterogenous group of children with congenital heart defects.

2. Materials and Methods

2.1. Population

The study was conducted at the Department of Pediatric Cardiology and General Pediatrics of the Medical University of Warsaw. Infants aged 0 to 18 months with a diagnosed heart disease qualified for the study. The exclusion criteria were: less than six weeks since last sternotomy surgery, central venous cannula, and non-invasive ventilation. Demographic and clinical data about infants were collected in terms of: sex, gestational age, birth weight, mode of delivery, Apgar score, and type of heart anomaly. This study is a part of the research project on the assessment of determinants of motor development and physical activity levels in pediatric patients with heart disease.

2.2. Raters and Assessment

Two physiotherapists examined the patients simultaneously: the first one has a higher master’s degree in physiotherapy and 10 years of experience in working with toddlers and the AIMS (experienced assessor—K.O-P.), and the second is a last-year (5th-year) physiotherapy student interested in pediatric physical therapy (inexperienced assessor—M.F.). The student did not undergo professional AIMS training, but she was encouraged to be familiar with the AIMS manual (English language) before the start of the examinations [65]. During the examinations, both observers performed a motor assessment without consulting with each other’s observations during the course of the study.
The assessment was performed according to the manual [65]. The parent/caregiver was informed that the motor assessment was to be conducted during the second day of hospitalization and was asked about the time of day when the child is awake, with a time chosen at least 30 min after their last meal and when they would be active. The parent/caregiver was present during the assessment and instructed to interact with the child to achieve and maintain an optimal state. If the child was anxious, both examiners observed their motor abilities from a distance and guided the parent/caregiver on how to interact with the child.

2.3. Assessment Tool

The assessment tool used was the Polish language version of the Albert Infant Motor Scale [62,63,66]. The Polish version of the AIMS was recently validated and adapted for infants aged 0–18 months and can be applied to this population for clinical and scientific purposes [62]. The Alberta Infant Motor Scale is a norm-referenced, observational scale that evaluates the gross motor development of children from birth to 18 months, until the achievement of independent walking. The scale contains 58 items organized into four subscales describing the development of spontaneous motor skills in four positions: prone (21 items), supine (9 items), sitting (12 items) and standing (16 items) [48,65,66]. The examiner observes the movement of the infant in each one of the positions with minimal handling, taking into account weight bearing, posture and antigravity movements. Gross motor development is defined by the AIMS total score, which is expressed in percentiles based on age. Cut-off points at the 10th centile at 4 months and the 5th centile at 8 months based upon the normative data have been shown to be predictive of abnormal development [48,65,66]. Depending on the patient’s age, the assessment was performed in a crib or on a mat. Each infant assessment took between 10 to 20 min depending on infant age [48,65,66].

2.4. Statistical Analysis

Agreement on the AIMS score between assessors was verified using a Bland–Altman plot with limits of agreement (LoA) [67] and an intraclass correlation coefficient (ICC) [68], and interpretation was carried out using that proposed by Hopkins et al.: 0 to 0.30, small; 0.31 to 0.49, moderate; 0.50 to 0.69, large; 0.70 to 0.89, very large; and 0.90 to 1.00, nearly perfect [69]. All calculations were performed using STATISTICA 13 (StatSoft Inc., Tulsa, OK, USA) and MedCalc software version 19.4.1 (MedCalc Software, Ostend, Belgium). The Bland–Altman plots were created using PQStat version 1.8.4 (Poznań, Poland). GraphPad Prism 5 (GraphPad Software Inc., San Diego, CA, USA, 2005) was used to create a correlation plot.

3. Results

Of the 80 (n = 38 females) pediatric patients with heart disease (ConHD, n = 75; cardiac arrhythmia, n = 3; and cardiomyopathy, n = 2) who participated in the study, 38 were born vaginally and 42 were born by cesarean delivery. Participants’ median age, median gestational age, median birth weight and median Apgar score were 7 months (range: 1–18), 39 weeks (range: 27–41), 3200 g (range: 840–4490) and 10 points (range: 5–10). Patients’ characteristics by four age groups (0–3 months, 4–7 months, 8–11 months, and 12–18 months) are presented in Table 1.
The agreement statistics results are presented in Table 2. The lowest ICC values were observed for the youngest participants (0–3 months and 4–7 months) in a standing position. The highest bias for the AIMS total score was observed for participants aged 8–11 months (0.22), 4–7 months (−0.17) and 0–3 months (0.11). The Bland–Altman plots are shown in Figure 1, representing the agreement between experienced and inexperienced assessors in the AIMS total score for selected age groups and the total sample. There was no association between the number of examinations and differences between assessors in the AIMS total score for the total sample (Figure 2).

4. Discussion

The aim of this study was to analyze the inter-rater reliability of the Polish version of the AIMS in the pediatric population with heart disease based on two raters with different clinical experience. The results support high degrees of agreement between the raters for the total scores in each age group of patients and underscore that the assessment of motor development and/or motor skills using the AIMS is not dependent on the clinical experience of the observers. Motor development assessment using the AIMS can be performed in pediatric patients with heart defects aged 0–18 months by clinically inexperienced last-year physiotherapy students who are familiarized with the AIMS manual. Providing practical instructions (with resources) and guidance on how to measure and promote gross motor development in infants with complex heart defects to physiotherapy students and early-stage physiotherapists may help with filling in the gap in the research on developmental care for pediatric cardiac patients.
The results of the reliability and agreement statistics presented here are consistent with those from a healthy population from Canada [48] and Poland [62] (total ICC values of inter-rater reliability in both studies were also set at 0.99). Nevertheless, it is crucial to assess the inter-rater reliability of motor development scales in a diverse group of pediatric patients. Authors from China [61], Serbia [58] and Spain [59] have evaluated the inter-rater reliability of the AIMS among children at risk for delayed motor development (low birth weight, prematurity, low Apgar score, intraventricular hemorrhage, and neurological symptoms). In all of these studies, the ICC value for the total score was above 0.9 which proves the excellent reliability of the tool [58,59,61]. Taking into account these results presented here for children with heart defects, it can be concluded that the AIMS has high reliability not only among healthy children and can be used in infants with various types of disorders.
A tool that is minimally invasive, easy to use and culturally adaptable is needed to assess the motor development of children with heart defects. Albuquerque et al. examined the concurrent validity of the AIMS in relation to the gross motor subtest of the Bayley Scale III [47]. Delayed gross motor development was observed in 21% of patients according to the Bayley III, and in 12% for the 5th percentile and 21% for the 10th percentile of the AIMS. Adopting the AIMS 10th percentile as a cutoff point yielded the best combination of values for sensitivity and specificity for detecting motor delays [47]. The AIMS, unlike the Bayley III, allows for verification of weight bearing, posture and antigravity movements, which is helpful in the early detection of possible abnormalities. Some of the biggest advantages of the AIMS are that the examination requires minimal handling by the therapist and takes a maximum of 20 min, minimizing the risk of stress among high-risk children like those with heart defects. Despite its advantages, the AIMS has a couple of inconsistencies. One of them is the situation when the item “hands to feet” is observed, but the item “hands to knees” belonging to the “motor window” is not observed [51]. Raters are aware that if an infant is able to touch their feet, they should also be able to touch their knees; however, the latter item cannot be credited if it was not observed. This problem was described by Blanchard et al. [51], who decided to directly contact the co-author of the AIMS. Dr. Darrah recommended “not starting the window too far back and believes that experienced clinicians may have an advantage over less experienced clinicians in making this decision. When suspecting that an infant is capable of higher items, she recommends giving him or her time to “warm up” a little first before scoring.” [51]. However, this information is not included in the scale manual. According to the results and Dr. Darrah’s commentary, Blanchard et al. suggested that for infants younger than seven months of age, the evaluation in the supine position is more challenging to score reliably between raters than the other positions [51]. In our study, the ICC for the supine position among children aged 4–7 months was 0.986, which is one of the lower values compared to the other results. However, this value is still excellent, which emphasizes that the experience did not affect the motor evaluation of children with heart defects, and the author’s comment does not solve the problem.
When the child has mastered the ability to roll from a supine to a prone position with rotation, all items on the AIMS supine subscale can be considered as observed. Blanchard et al. pointed out that this item is dependent on the examiner’s ability to detect rotation in motion [51]. Misdiagnosis of rotation can falsify the supine subscale score in infants aged 4–7 months. In our study, both examiners detected rotation during rolling from supine to prone, and the ICC score for supine in this age group was 0.986. The lowest value of the ICC (0.875) was recorded for standing positions in the 0–3 month age group, which is consistent with studies conducted by the authors from Taiwan [53] and the Republic of Korea [57]. This correlation can be attributed to the low variety in standing positions (subjective assessment of active trunk control while standing) described (narrow range of results) for children in the first months of life. The examiner, while holding the child, has to independently decide on how to score the child’s movements in the standing position, which can cause discrepancies. In addition, in the first months of life, infants learn to control and lift their heads against gravity. The AIMS positions assume that the child can only lift their head in a pronated position to angles of 45 or 90 degrees, which significantly narrows the assessment by the examiner. According to Wang et al., the lower inter-rater reliability of the results for the 0–3 and 4–7 month groups is due to the fact that younger children are able to perform fewer items than older children, which significantly weakens the ICC value [61].
Our study, similarly to Snyder et al. [49], assessed the agreement in an assessment of motor development between an experienced physiotherapist and a student who received a short theoretical overview. In both studies, similar results for both raters were shown, obtaining a total ICC of 0.99 (our study) and 0.98 (Snyder et al. [49]). Therefore, it can be concluded that the AIMS manual and assessment rules allow for adequate assessment, even when performed by an inexperienced assessor. The pictures and simple descriptions use in the AIMS led students to report improvements in their skills in the area of assessing infant motor development, which underscores that the tool can be used effectively in the classroom.
Assessment of motor development using the AIMS in pediatric patients who change body position frequently and those who are highly sensitive may be a challenge for inexperienced therapists. Some positions in the first months of life, such as standing, may not be achieved by young patients without additional support. It could be difficult for an inexperienced therapist to assess trunk control just by observation without manual assessment. In our study, the greatest difficulty for the inexperienced assessor was assessing lumbar lordosis in two four-point kneeling positions.
It should be noted that a more detailed explanation of the “motor window” used in the AIMS should help in motor development assessment. Also, it is worth expanding the number of items in the standing subscale in the first months of life. We agree with other authors who have recommended further studies comparing the reliability of the AIMS in a group of patients with various disorders.
The main limitations are that this study was conducted in only one center and it involved only one experienced assessor and only one student for whom there was no test–retest assessment analysis.

5. Conclusions

The agreement between two raters with different experiences—a last-year physiotherapy student and a pediatric physiotherapist with 10 years of clinical experience in pediatric physiotherapy—in assessing the motor development and motor skills of patients with heart defects aged 0–18 months using the Polish version of the AIMS is satisfactory for both the total and subgroup scores in each age group. Therefore, motor development assessment using the AIMS can be performed on these patients by clinically inexperienced last-year physiotherapy students who are familiarized with the AIMS manual. However, the assessment of test–retest reliability for the AIMS examination performed by clinically inexperienced last-year medical students and/or early graduates on a cohort of pediatric cardiac patients should be performed in future studies. There is a need to update the tool in the supine and prone positions due to doubts about the assessment methodology by clinicians.

Author Contributions

Conceptualization, J.S.G.; methodology, J.S.G. and K.O.-P.; formal analysis, J.S.G.; investigation, M.F. and K.O.-P.; data curation, M.F. and K.O.-P.; writing—original draft preparation, M.F., K.O.-P. and J.S.G.; writing—review and editing, M.F., K.O.-P., J.S.G. and B.W.; supervision, J.S.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partly supported by a statutory grant to the Department of Pediatric Cardiology and General Pediatrics, Medical University of Warsaw, Poland (grant number: 10/Z/GW/N/21).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the Medical University of Warsaw (AKBE/95/2021; 14 June 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data can be provided by the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Baunsgaard, M.M.; Henriksen, T.B.; Gilberg, C.K.; Wibroe, D.B.; Haugsted, T.; Østergaard, J.R.; Hjortdal, V.E.; Lauridsen, M.H. Isolated CHDs and neurodevelopmental follow-up using the Bayley Scales of Infant and Toddler Development and the Ages and Stages Questionnaire at 18 and 36 months. Cardiol. Young 2022, 32, 390–397. [Google Scholar] [CrossRef] [PubMed]
  2. Bjarnason-Wehrens, B.; Dordel, S.; Schickendantz, S.; Krumm, C.; Bott, D.; Sreeram, N.; Brockmeier, K. Motor development in children with congenital cardiac diseases compared to their healthy peers. Cardiol. Young 2007, 17, 487–498. [Google Scholar] [CrossRef] [PubMed]
  3. Bolduc, M.E.; Dionne, E.; Gagnon, I.; Rennick, J.E.; Majnemer, A.; Brossard-Racine, M. Motor Impairment in Children with Congenital Heart Defects: A Systematic Review. Pediatrics 2020, 146, e20200083. [Google Scholar] [CrossRef] [PubMed]
  4. Cheatham, S.L.; Carey, H.; Chisolm, J.L.; Heathcock, J.C.; Steward, D. Early results of neurodevelopment following hybrid stage I for hypoplastic left heart syndrome. Pediatr. Cardiol. 2015, 36, 685–691. [Google Scholar] [CrossRef]
  5. Chien, C.H.; Lee, T.Y.; Lin, M.T. Factors affecting motor development of toddlers who received cardiac corrective procedures during infancy. Early Hum. Dev. 2021, 158, 105392. [Google Scholar] [CrossRef] [PubMed]
  6. Fourdain, S.; Simard, M.N.; Dagenais, L.; Materassi, M.; Doussau, A.; Goulet, J.; Gagnon, K.; Prud’Homme, J.; Vinay, M.C.; Dehaes, M.; et al. Gross Motor Development of Children with Congenital Heart Disease Receiving Early Systematic Surveillance and Individualized Intervention: Brief Report. Dev. Neurorehabil. 2021, 24, 56–62. [Google Scholar] [CrossRef]
  7. Huisenga, D.; la Bastide-Van Gemert, S.; Van Bergen, A.; Sweeney, J.; Hadders-Algra, M. Developmental outcomes after early surgery for complex congenital heart disease: A systematic review and meta-analysis. Dev. Med. Child Neurol. 2021, 63, 29–46. [Google Scholar] [CrossRef] [Green Version]
  8. Huisenga, D.; la Bastide-van Gemert, S.; Van Bergen, A.H.; Sweeney, J.K.; Hadders-Algra, M. Motor development in infants with complex congenital heart disease: A longitudinal study. Dev. Med. Child Neurol. 2023, 65, 117–125. [Google Scholar] [CrossRef]
  9. Lisanti, A.J.; Uzark, K.C.; Harrison, T.M.; Peterson, J.K.; Butler, S.C.; Miller, T.A.; Allen, K.Y.; Miller, S.P.; Jones, C.E.; American Heart Association Pediatric Cardiovascular Nursing Committee of the Council on Cardiovascular and Stroke Nursing. Developmental Care for Hospitalized Infants with Complex Congenital Heart Disease: A Science Advisory from the American Heart Association. J. Am. Heart Assoc. 2023, 12, e028489. [Google Scholar] [CrossRef]
  10. Longmuir, P.E.; Banks, L.; McCrindle, B.W. Cross-sectional study of motor development among children after the Fontan procedure. Cardiol. Young 2012, 22, 443–450. [Google Scholar] [CrossRef]
  11. Mäenpää, H.; Häkkinen, A.; Sarajuuri, A. Changes in Motor Development During a 4-Year Follow-up on Children with Univentricular Heart Defects. Pediatr. Phys. Ther. 2016, 28, 446–451. [Google Scholar] [CrossRef]
  12. Ricci, M.F.; Fung, A.; Moddemann, D.; Micek, V.; Bond, G.Y.; Guerra, G.G.; Day, C.; Robertson, C.M.T.; Western Canadian Complex Pediatric Therapies Follow-up Program. Comparison of motor outcomes between preschool children with univentricular and biventricular critical heart disease not diagnosed with cerebral palsy or acquired brain injury. Cardiol. Young 2021, 31, 1788–1795. [Google Scholar] [CrossRef]
  13. Sprong, M.C.A.; Broeders, W.; van der Net, J.; Breur, J.M.P.J.; de Vries, L.S.; Slieker, M.G.; van Brussel, M. Motor Developmental Delay After Cardiac Surgery in Children with a Critical Congenital Heart Defect: A Systematic Literature Review and Meta-analysis. Pediatr. Phys. Ther. 2021, 33, 186–197. [Google Scholar] [CrossRef]
  14. Sprong, M.C.A.; van Brussel, M.; de Vries, L.S.; van der Net, J.; Nijman, J.; Breur, J.M.P.J.; Slieker, M.G. Longitudinal Motor-Developmental Outcomes in Infants with a Critical Congenital Heart Defect. Children 2022, 9, 570. [Google Scholar] [CrossRef]
  15. Teixeira, J.; Caflisch, J.; Chaouch, A.; Beck, I.; Feldmann, M.; Polentarutti, S.; Balmer, C.; Latal, B. Motor and visuomotor function in 10-year-old children with congenital heart disease: Association with behaviour. Cardiol. Young 2022, 32, 1310–1315. [Google Scholar] [CrossRef]
  16. Tripathi, T.; Harrison, T.M.; Simsic, J.M.; Cabral, T.I.; Heathcock, J.C. Screening and Evaluation of Neurodevelopmental Impairments in Infants Under 6 Months of Age with Congenital Heart Disease. Pediatr. Cardiol. 2022, 43, 489–496. [Google Scholar] [CrossRef]
  17. Wehrle, F.M.; Bartal, T.; Adams, M.; Bassler, D.; Hagmann, C.F.; Kretschmar, O.; Natalucci, G.; Latal, B. Similarities and Differences in the Neurodevelopmental Outcome of Children with Congenital Heart Disease and Children Born Very Preterm at School Entry. J. Pediatr. 2022, 250, 29–37.e21. [Google Scholar] [CrossRef]
  18. Holm, I.; Fredriksen, P.M.; Fosdahl, M.A.; Olstad, M.; Vøllestad, N. Impaired motor competence in school-aged children with complex congenital heart disease. Arch. Pediatr. Adolesc. Med. 2007, 161, 945–950. [Google Scholar] [CrossRef] [Green Version]
  19. Brosig, C.L.; Bear, L.; Allen, S.; Hoffmann, R.G.; Pan, A.; Frommelt, M.; Mussatto, K.A. Preschool Neurodevelopmental Outcomes in Children with Congenital Heart Disease. J. Pediatr. 2017, 183, 80–86.e81. [Google Scholar] [CrossRef] [Green Version]
  20. Fuller, S.; Nord, A.S.; Gerdes, M.; Wernovsky, G.; Jarvik, G.P.; Bernbaum, J.; Zackai, E.; Gaynor, J.W. Predictors of impaired neurodevelopmental outcomes at one year of age after infant cardiac surgery. Eur. J. Cardiothorac. Surg. 2009, 36, 40–47. [Google Scholar] [CrossRef]
  21. Gaynor, J.W.; Stopp, C.; Wypij, D.; Andropoulos, D.B.; Atallah, J.; Atz, A.M.; Beca, J.; Donofrio, M.T.; Duncan, K.; Ghanayem, N.S.; et al. Neurodevelopmental outcomes after cardiac surgery in infancy. Pediatrics 2015, 135, 816–825. [Google Scholar] [CrossRef] [Green Version]
  22. Mussatto, K.A.; Hoffmann, R.G.; Hoffman, G.M.; Tweddell, J.S.; Bear, L.; Cao, Y.; Brosig, C. Risk and prevalence of developmental delay in young children with congenital heart disease. Pediatrics 2014, 133, e570–e577. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Zeltser, I.; Jarvik, G.P.; Bernbaum, J.; Wernovsky, G.; Nord, A.S.; Gerdes, M.; Zackai, E.; Clancy, R.; Nicolson, S.C.; Spray, T.L.; et al. Genetic factors are important determinants of neurodevelopmental outcome after repair of tetralogy of Fallot. J. Thorac. Cardiovasc. Surg. 2008, 135, 91–97. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Sarajuuri, A.; Jokinen, E.; Mildh, L.; Tujulin, A.M.; Mattila, I.; Valanne, L.; Lönnqvist, T. Neurodevelopmental burden at age 5 years in patients with univentricular heart. Pediatrics 2012, 130, e1636–e1646. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Stegeman, R.; Sprong, M.C.A.; Breur, J.; Groenendaal, F.; de Vries, L.S.; Haas, F.; van der Net, J.; Jansen, N.J.G.; Benders, M.J.N.L.; Claessens, N.H.P. Early motor outcomes in infants with critical congenital heart disease are related to neonatal brain development and brain injury. Dev. Med. Child Neurol. 2022, 64, 192–199. [Google Scholar] [CrossRef]
  26. Newburger, J.W.; Sleeper, L.A.; Bellinger, D.C.; Goldberg, C.S.; Tabbutt, S.; Lu, M.; Mussatto, K.A.; Williams, I.A.; Gustafson, K.E.; Mital, S.; et al. Early developmental outcome in children with hypoplastic left heart syndrome and related anomalies: The single ventricle reconstruction trial. Circulation 2012, 125, 2081–2091. [Google Scholar] [CrossRef] [Green Version]
  27. Brandlistuen, R.E.; Stene-Larsen, K.; Holmstrom, H.; Landolt, M.A.; Eskedal, L.T.; Vollrath, M.E. Motor and social development in 6-month-old children with congenital heart defects. J. Pediatr. 2010, 156, 265–269.e1. [Google Scholar] [CrossRef] [Green Version]
  28. Hoskoppal, A.; Roberts, H.; Kugler, J.; Duncan, K.; Needelman, H. Neurodevelopmental outcomes in infants after surgery for congenital heart disease: A comparison of single-ventricle vs. two-ventricle physiology. Congenit. Heart Dis. 2010, 5, 90–95. [Google Scholar] [CrossRef]
  29. Mussatto, K.A.; Hoffmann, R.; Hoffman, G.; Tweddell, J.S.; Bear, L.; Cao, Y.; Tanem, J.; Brosig, C. Risk Factors for Abnormal Developmental Trajectories in Young Children with Congenital Heart Disease. Circulation 2015, 132, 755–761. [Google Scholar] [CrossRef] [Green Version]
  30. Yoshida, T.; Hiraiwa, A.; Ibuki, K.; Makimoto, M.; Inomata, S.; Tamura, K.; Kawasaki, Y.; Ozawa, S.; Hirono, K.; Ichida, F. Neurodevelopmental outcomes at 3 years for infants with congenital heart disease and very-low birthweight. Pediatr. Int. 2020, 62, 797–803. [Google Scholar] [CrossRef]
  31. Hülser, K.; Dubowy, K.O.; Knobl, H.; Meyer, H.; Schölmerich, A. Developmental outcome and psychosocial adjustment in children after surgery for congenital heart disease during infancy. J. Reprod. Infant Psychol. 2007, 25, 139–151. [Google Scholar] [CrossRef]
  32. Goldsworthy, M.; Franich-Ray, C.; Kinney, S.; Shekerdemian, L.; Beca, J.; Gunn, J. Relationship between Social-Emotional and Neurodevelopment of 2-Year-Old Children with Congenital Heart Disease. Congenit. Heart Dis. 2016, 11, 378–385. [Google Scholar] [CrossRef] [PubMed]
  33. Gunn, J.K.; Beca, J.; Hunt, R.W.; Goldsworthy, M.; Brizard, C.P.; Finucane, K.; Donath, S.; Shekerdemian, L.S. Perioperative risk factors for impaired neurodevelopment after cardiac surgery in early infancy. Arch. Dis. Child. 2016, 101, 1010–1016. [Google Scholar] [CrossRef]
  34. Majnemer, A.; Limperopoulos, C.; Shevell, M.; Rosenblatt, B.; Rohlicek, C.; Tchervenkov, C. Long-term neuromotor outcome at school entry of infants with congenital heart defects requiring open-heart surgery. J. Pediatr. 2006, 148, 72–77. [Google Scholar] [CrossRef] [PubMed]
  35. Dittrich, H.; Bührer, C.; Grimmer, I.; Dittrich, S.; Abdul-Khaliq, H.; Lange, P.E. Neurodevelopment at 1 year of age in infants with congenital heart disease. Heart 2003, 89, 436–441. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Aisenberg, R.B.; Rosenthal, A.; Nadas, A.S.; Wolff, P.H. Developmental delay in infants with congenital heart disease. Correlation with hypoxemia and congestive heart failure. Pediatr. Cardiol. 1982, 3, 133–137. [Google Scholar] [CrossRef]
  37. Hövels-Gürich, H.H.; Konrad, K.; Skorzenski, D.; Nacken, C.; Minkenberg, R.; Messmer, B.J.; Seghaye, M.C. Long-term neurodevelopmental outcome and exercise capacity after corrective surgery for tetralogy of Fallot or ventricular septal defect in infancy. Ann. Thorac. Surg. 2006, 81, 958–966. [Google Scholar] [CrossRef] [PubMed]
  38. Stieh, J.; Kramer, H.H.; Harding, P.; Fischer, G. Gross and fine motor development is impaired in children with cyanotic congenital heart disease. Neuropediatrics 1999, 30, 77–82. [Google Scholar] [CrossRef]
  39. Claessens, N.H.P.; Algra, S.O.; Ouwehand, T.L.; Jansen, N.J.G.; Schappin, R.; Haas, F.; Eijsermans, M.J.C.; de Vries, L.S.; Benders, M.J.N.L.; CHD Lifespan Study Group Utrecht. Perioperative neonatal brain injury is associated with worse school-age neurodevelopment in children with critical congenital heart disease. Dev. Med. Child Neurol. 2018, 60, 1052–1058. [Google Scholar] [CrossRef] [Green Version]
  40. Uzark, K.; Smith, C.; Donohue, J.; Yu, S.; Romano, J.C. Infant Motor Skills After a Cardiac Operation: The Need for Developmental Monitoring and Care. Ann. Thorac. Surg. 2017, 104, 681–686. [Google Scholar] [CrossRef] [Green Version]
  41. Campbell, M.J.; Ziviani, J.M.; Stocker, C.F.; Khan, A.; Sakzewski, L. Neuromotor performance in infants before and after early open-heart surgery and risk factors for delayed development at 6 months of age. Cardiol. Young 2019, 29, 100–109. [Google Scholar] [CrossRef] [PubMed]
  42. Khalid, O.M.; Harrison, T.M. Early Neurodevelopmental Outcomes in Children with Hypoplastic Left Heart Syndrome and Related Anomalies After Hybrid Procedure. Pediatr. Cardiol. 2019, 40, 1591–1598. [Google Scholar] [CrossRef]
  43. Knirsch, W.; Liamlahi, R.; Hug, M.I.; Hoop, R.; von Rhein, M.; Prêtre, R.; Kretschmar, O.; Latal, B. Mortality and neurodevelopmental outcome at 1 year of age comparing hybrid and Norwood procedures. Eur. J. Cardiothorac. Surg. 2012, 42, 33–39. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Limperopoulos, C.; Majnemer, A.; Shevell, M.I.; Rohlicek, C.; Rosenblatt, B.; Tchervenkov, C.; Darwish, H.Z. Predictors of developmental disabilities after open heart surgery in young children with congenital heart defects. J. Pediatr. 2002, 141, 51–58. [Google Scholar] [CrossRef]
  45. Long, S.H.; Eldridge, B.J.; Harris, S.R.; Cheung, M.M. Motor skills of 5-year-old children who underwent early cardiac surgery. Cardiol. Young 2016, 26, 650–657. [Google Scholar] [CrossRef] [PubMed]
  46. Long, S.H.; Harris, S.R.; Eldridge, B.J.; Galea, M.P. Gross motor development is delayed following early cardiac surgery. Cardiol. Young 2012, 22, 574–582. [Google Scholar] [CrossRef] [PubMed]
  47. de Albuquerque, P.L.; Lemos, A.; Guerra, M.Q.; Eickmann, S.H. Accuracy of the Alberta Infant Motor Scale (AIMS) to detect developmental delay of gross motor skills in preterm infants: A systematic review. Dev. Neurorehabil. 2015, 18, 15–21. [Google Scholar] [CrossRef]
  48. Piper, M.C.; Pinnell, L.E.; Darrah, J.; Maguire, T.; Byrne, P.J. Construction and validation of the Alberta Infant Motor Scale (AIMS). Can. J. Public Health 1992, 83 (Suppl. S2), S46–S50. [Google Scholar]
  49. Snyder, P.; Eason, J.M.; Philibert, D.; Ridgway, A.; McCaughey, T. Concurrent validity and reliability of the Alberta Infant Motor Scale in infants at dual risk for motor delays. Phys. Occup. Ther. Pediatr. 2008, 28, 267–282. [Google Scholar] [CrossRef]
  50. Almeida, K.M.; Dutra, M.V.; Mello, R.R.; Reis, A.B.; Martins, P.S. Concurrent validity and reliability of the Alberta Infant Motor Scale in premature infants. J. Pediatr. 2008, 84, 442–448. [Google Scholar] [CrossRef]
  51. Blanchard, Y.; Neilan, E.; Busanich, J.; Garavuso, L.; Klimas, D. Interrater reliability of early intervention providers scoring the alberta infant motor scale. Pediatr. Phys. Ther. 2004, 16, 13–18. [Google Scholar] [CrossRef] [PubMed]
  52. Boonzaaijer, M.; van Dam, E.; van Haastert, I.C.; Nuysink, J. Concurrent Validity Between Live and Home Video Observations Using the Alberta Infant Motor Scale. Pediatr. Phys. Ther. 2017, 29, 146–151. [Google Scholar] [CrossRef] [PubMed]
  53. Jeng, S.F.; Yau, K.I.; Chen, L.C.; Hsiao, S.F. Alberta infant motor scale: Reliability and validity when used on preterm infants in Taiwan. Phys. Ther. 2000, 80, 168–178. [Google Scholar] [CrossRef] [Green Version]
  54. Syrengelas, D.; Siahanidou, T.; Kourlaba, G.; Kleisiouni, P.; Bakoula, C.; Chrousos, G.P. Standardization of the Alberta infant motor scale in full-term Greek infants: Preliminary results. Early Hum. Dev. 2010, 86, 245–249. [Google Scholar] [CrossRef] [PubMed]
  55. Uesugi, M.; Tokuhisa, K.; Shimada, T. The Reliability and Validity of the Alberta Infant Motor Scale in Japan. J. Phys. Ther. Sci. 2008, 20, 169–175. [Google Scholar] [CrossRef] [Green Version]
  56. Aimsamrarn, P.; Janyachareon, T.; Rattanathanthong, K.; Emasithi, A.; Siritaratiwat, W. Cultural translation and adaptation of the Alberta Infant Motor Scale Thai version. Early Hum. Dev. 2019, 130, 65–70. [Google Scholar] [CrossRef]
  57. Ko, J.; Lim, H.K. Reliability Study of the Items of the Alberta Infant Motor Scale (AIMS) Using Kappa Analysis. Int. J. Environ. Res. Public Health 2022, 19, 1767. [Google Scholar] [CrossRef]
  58. Lackovic, M.; Nikolic, D.; Filimonovic, D.; Petronic, I.; Mihajlovic, S.; Golubovic, Z.; Pavicevic, P.; Cirovic, D. Reliability, Consistency and Temporal Stability of Alberta Infant Motor Scale in Serbian Infants. Children 2020, 7, 16. [Google Scholar] [CrossRef] [Green Version]
  59. Morales-Monforte, E.; Bagur-Calafat, C.; Suc-Lerin, N.; Fornaguera-Martí, M.; Cazorla-Sánchez, E.; Girabent-Farrés, M. The Spanish version of the Alberta Infant Motor Scale: Validity and reliability analysis. Dev. Neurorehabilit. 2017, 20, 76–82. [Google Scholar] [CrossRef]
  60. Valentini, N.C.; Saccani, R. Brazilian validation of the Alberta Infant Motor Scale. Phys. Ther. 2012, 92, 440–447. [Google Scholar] [CrossRef]
  61. Wang, H.; Li, H.; Wang, J.; Jin, H. Reliability and Concurrent Validity of a Chinese Version of the Alberta Infant Motor Scale Administered to High-Risk Infants in China. BioMed Res. Int. 2018, 2018, 2197163. [Google Scholar] [CrossRef] [Green Version]
  62. Eliks, M.; Sowińska, A.; Gajewska, E. The Polish Version of the Alberta Infant Motor Scale: Cultural Adaptation and Validation. Front. Neurol. 2022, 13, 949720. [Google Scholar] [CrossRef] [PubMed]
  63. Eliks, M.; Sowińska, A.; Steinborn, B.; Gajewska, E. The standardization of the Polish version of the Alberta Infant Motor Scale. BMC Pediatr. 2023, 23, 236. [Google Scholar] [CrossRef] [PubMed]
  64. Rajantie, I.; Laurila, M.; Pollari, K.; Lönnqvist, T.; Sarajuuri, A.; Jokinen, E.; Mälkiä, E. Motor development of infants with univentricular heart at the ages of 16 and 52 weeks. Pediatr. Phys. Ther. 2013, 25, 444–450. [Google Scholar] [CrossRef] [PubMed]
  65. Piper, M.; Darrah, J. Motor Assessment of the Developing Infant. In Alberta Infant Motor Scale (AIMS), 2nd ed.; Elsevier: Amsterdam, The Netherlands, 2021. [Google Scholar]
  66. Darrah, J.; Piper, M.; Watt, M.J. Assessment of gross motor skills of at-risk infants: Predictive validity of the Alberta Infant Motor Scale. Dev. Med. Child Neurol. 1998, 40, 485–491. [Google Scholar] [CrossRef] [PubMed]
  67. Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986, 1, 307–310. [Google Scholar] [CrossRef]
  68. Ranganathan, P.; Pramesh, C.S.; Aggarwal, R. Common pitfalls in statistical analysis: Measures of agreement. Perspect. Clin. Res. 2017, 8, 187–191. [Google Scholar] [CrossRef]
  69. Hopkins, W.G.; Marshall, S.W.; Batterham, A.M.; Hanin, J. Progressive statistics for studies in sports medicine and exercise science. Med. Sci. Sports Exerc. 2009, 41, 3–13. [Google Scholar] [CrossRef] [Green Version]
Jcm 12 04555 g001 550
Figure 1. Bland–Altman plots representing the agreement between experienced and inexperienced assessors in the AIMS total score for selected age groups and total sample. The solid blue line indicates the bias, solid red lines are the 95% LoA (±1.96 SD) and dotted lines are confidence interval LoA.
Figure 1. Bland–Altman plots representing the agreement between experienced and inexperienced assessors in the AIMS total score for selected age groups and total sample. The solid blue line indicates the bias, solid red lines are the 95% LoA (±1.96 SD) and dotted lines are confidence interval LoA.
Jcm 12 04555 g001
Jcm 12 04555 g002 550
Figure 2. Correlation between difference in total score for the total sample between assessors and number of examinations. R—Spearman correlation coefficient.
Figure 2. Correlation between difference in total score for the total sample between assessors and number of examinations. R—Spearman correlation coefficient.
Jcm 12 04555 g002
Table
Table 1. Patients’ characteristics.
Table 1. Patients’ characteristics.
0–3 Months4–7 Months8–11 Months12–18 Months
N19231820
Sex (♀/♂)7/1212/1110/89/11
Diagnosis (ConHD/cardiac arrhythmia/cardiomyopathy)19/0/021/1/116/1/119/1/0
Delivery (vaginally/
cesarean)		9/1011/125/1313/7
Gestational age (weeks)38 (34–41)39 (35–41)38 (27–40)39 (31–41)
Birth weight (g)3190 (1630–4380)3230 (2100–4490)3115 (840–4120)3380 (1180–4070)
Apgar score (points)10 (5–10)10 (8–10)10 (5–10)10 (5–10)
N—number of participants; ♀—females; ♂—males; ConHD—congenital heart disease.
Table
Table 2. Inter-rater agreement statistics.
Table 2. Inter-rater agreement statistics.
Experienced Assessor
Mean ± SD		Inexperienced
Assessor
Mean ± SD		ICC (95% CI)Bias (95% CI)−1.96 SD (95% CI)+1.96 SD (95% CI)
0–3 months
(n = 19)		Prone1.74 ± 1.101.68 ± 1.060.965 (0.910–0.986)0.05 (−0.14–0.25)−0.74 (−1.08–−0.40)0.85 (0.51–1.19)
Supine2.42 ± 0.842.42 ± 0.841.0000.0 (0.0–0.0)0.0 (0.0–0.0)0.0 (0.0–0.0)
Sitting0.63 ± 0.680.63 ± 0.681.0000.0 (0.0–0.0)0.0 (0.0–0.0)0.0 (0.0–0.0)
Standing1.26 ± 0.651.21 ± 0.540.875 (0.682–0.952)0.05 (−0.14–0.25)−0.74 (−1.08–−0.40)0.85 (0.51–1.19)
Total6.05 ± 2.745.95 ± 2.950.989 (0.971–0.996)0.11 (−0.17–0.38)−1.01 (−1.48–−0.53)1.22 (0.74–1.69)
4–7 months
(n = 23)		Prone5.43 ± 3.105.35 ± 3.020.995 (0.989–0.998)0.09 (−0.09–0.27)−0.73 (−1.04–−0.42)0.90 (0.59–1.22)
Supine6.09 ± 2.136.26 ± 2.220.986 (0.968–0.994)−0.17 (−0.39–0.04)−1.14 (−1.51–−0.77)0.79 (0.42–1.16)
Sitting1.78 ± 0.901.83 ± 0.980.988 (0.971–0.995)−0.04 (−0.13–0.05)−0.45 (−0.61–−0.30)0.37 (0.21–0.52)
Standing2.09 ± 0.602.13 ± 0.630.904 (0.777–0.959)−0.04 (−0.20–0.12)−0.76 (−1.04–−0.49)0.68 (0.40–0.95)
Total15.39 ± 5.9615.57 ± 6.080.995 (0.989–0.998)−0.17 (−0.54–0.19)−1.81 (−2.43–−1.18)1.46 (0.84–2.09)
8–11 months
(n = 18)		Prone14.39 ± 5.4414.06 ± 5.340.994 (0.985–0.998)0.33 (−0.05–0.72)−1.17 (−1.83–−0.51)1.84 (1.17–2.50)
Supine9.17 ± 3.079.11 ± 3.100.998 (0.996–0.999)0.06 (−0.06–0.17)−0.41 (−0.61–−0.20)0.52 (0.31–0.72)
Sitting9.06 ± 3.619.33 ± 3.430.992 (0.979–0.997)−0.28 (−0.56–0.01)−1.40 (−1.90–−0.91)0.85 (0.35–1.35)
Standing5.56 ± 2.965.44 ± 2.870.997 (0.991–0.999)0.11 (−0.05–0.27)−0.52 (−0.80–−0.24)0.75 (0.47–1.03)
Total38.17 ± 11.5937.94 ± 11.490.998 (0.994–0.999)0.22 (−0.33–0.78)−1.96 (−2.93–−0.99)2.41 (1.44–3.37)
12–18 months
(n = 20)		Prone14.05 ± 6.1614.05 ± 6.000.997 (0.992–0.999)0.00 (−0.34–0.34)−1.42 (−2.01–−0.83)1.42 (0.83–2.01)
Supine12.40 ± 5.8112.35 ± 5.860.999 (0.999–0.999)0.05 (−0.06–0.16)−0.39 (−0.57–−0.21)0.49 (0.31–0.67)
Sitting10.95 ± 2.6110.95 ± 2.611.0000.0 (0.0–0.0)0.0 (0.0–0.0)0.0 (0.0–0.0)
Standing9.30 ± 5.129.35 ± 5.110.999 (0.999–0.999)−0.05 (−0.16–0.06)−0.49 (−0.67–−0.31)0.39 (0.21–0.57)
Total46.70 ± 12.7046.70 ± 12.610.999 (0.997–0.999)0.00 (−0.40–0.40)−1.68 (−2.38–−0.98)1.68 (0.98–2.38)
Total sample
(n = 80)		Prone8.73 ± 6.928.61 ± 6.820.998 (0.997–0.999)0.11 (−0.02–0.25)−1.05 (−1.28–−0.83)1.28 (1.05–1.51)
Supine7.49 ± 5.017.51 ± 5.010.999 (0.998–0.999)−0.03 (−0.10–0.05)−0.65 (−0.77–−0.53)0.60 (0.48–0.72)
Sitting5.44 ± 4.995.51 ± 5.000.999 (0.998–0.999)−0.07 (−0.14–−0.01)−0.68 (−0.80–−0.56)0.53 (0.41–0.65)
Standing4.48 ± 4.324.46 ± 4.320.999 (0.998–0.999)0.01 (−0.06–0.09)−0.65 (−0.78–−0.52)0.67 (0.55–0.80)
Total26.13 ± 18.7126.10 ± 18.650.999 (0.999–0.999)0.03 (−0.17–0.22)−1.65 (−1.98–−1.33)1.70 (1.38–2.03)
SD—standard deviation; ICC—intraclass correlation coefficient; CI—confidence interval.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ferenstein, M.; Ostrzyżek-Przeździecka, K.; Gąsior, J.S.; Werner, B. Inter-Rater Reliability of the Polish Version of the Alberta Infant Motor Scale in Children with Heart Disease. J. Clin. Med. 2023, 12, 4555. https://doi.org/10.3390/jcm12134555

AMA Style

Ferenstein M, Ostrzyżek-Przeździecka K, Gąsior JS, Werner B. Inter-Rater Reliability of the Polish Version of the Alberta Infant Motor Scale in Children with Heart Disease. Journal of Clinical Medicine. 2023; 12(13):4555. https://doi.org/10.3390/jcm12134555

Chicago/Turabian Style

Ferenstein, Maria, Katarzyna Ostrzyżek-Przeździecka, Jakub S. Gąsior, and Bożena Werner. 2023. "Inter-Rater Reliability of the Polish Version of the Alberta Infant Motor Scale in Children with Heart Disease" Journal of Clinical Medicine 12, no. 13: 4555. https://doi.org/10.3390/jcm12134555

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop