Inter-Professional and Methodological Agreement in Using the Cutting Movement Assessment Score (CMAS)

: Background: The cutting movement assessment score (CMAS) provides a qualitative assessment of the side-step cutting (S-SC) technique. Previous research has been undertaken primarily by biomechanists experienced with S-SC evaluations. Little is known about the agreement between various sports science and medicine practitioners to ascertain whether the tool can be used effectively by different practitioners in the ﬁeld. Currently, the CMAS uses three camera views (CVS) to undertake the evaluation, and it would be worthwhile to know whether the CMAS can be effectively conducted with fewer camera views to improve clinical utility. Therefore, the aim of the study was to examine the inter-rater agreement between different sports science and medicine practitioners and agreement between using different CVS to evaluate the S-SC technique using the CMAS. Methods: Video data were collected from 12 male rugby union players performing a 45 ◦ S-SC manoeuvre toward both the left and right directions. Five different sports science and medicine practitioners evaluated footage from three cameras of one left and one right trial from each player using the CMAS. Twelve different trials were also evaluated by the sports rehabilitator using single and multiple CVS. Agreements (percentage; Kappa coefﬁcients (K)) between different practitioners and conﬁgurations of the CVS were explored. Results: Good to excellent inter-rater agreements were found between all practitioners for total score (K = 0.63–0.84), with moderate to excellent inter-rater agreements observed across all items of the CMAS (K = 0.5–1.0). Excellent agreement was found between using three CVS vs. two CVS that included at least a sagittal view (K = 0.96–0.97). Lower agreement (K = 0.83) was found between angle-frontal views with three CVS. Conclusions: The CMAS can be used effectively by various practitioners to evaluate the movement quality of S-SC. The use of two CVS that include at least a sagittal plane view would sufﬁce to evaluate the S-SC technique against the CMAS.


Introduction
Side-step cutting is a common action associated with the incidence of non-contact anterior cruciate ligament (ACL) injuries in soccer [1][2][3], handball [4], rugby [5], and American football [6]. The side-step cut is an evasive manoeuvre which from a forward approach (e.g., run) involves a lateral foot placement ('plant') opposite to the intended direction of travel to generate a medially directed impulse to initiate a change of direction in this opposite direction. Due to the lateral positioning of the plant leg relative to the centre of mass, high multiplanar knee joint loads (anterior tibial shear force, knee abduction moment, and knee internal rotation moments) are experienced during such actions [7,8], which could increase ACL strain [9,10] and subsequent injury.
Whilst movement screening of landing and change of direction actions to predict noncontact ACL injuries has so far and is likely to remain elusive [11], movement evaluations Biomechanics 2023, 3 182 of change of direction actions such as side-step cutting remain important in order to help identify movement quality deficits associated with potentially greater knee joint loads in athletes, such as knee valgus and lateral trunk flexion [12,13]. Movement quality may be defined as "the ability to control the limbs and achieve sufficient balance and kinematic alignment during functional activities, not displaying movement asymmetries or risk factors linked to ACL injuries" [13]. Screening movement quality allows practitioners to 'profile' athletes to offer avenues for intervention to help reduce the relative risk of injury. For instance, technique modification training has been shown to be effective for reducing knee joint loads during side-step cutting [14,15] and pivoting actions [16]. However, such programmes are far more effective if an initial pre-evaluation of technique is performed to be able to pinpoint undesired postures and movement characteristics to help individualise technique modification training [12].
One of the difficulties of implementing a pre-evaluation of movement quality is how accessible sophisticated biomechanical analysis is for every athlete from grass roots to the elite level both in terms of resources and available time to implement [17,18]. Thus, field-based tools to evaluate side-step cutting maybe useful to overcome such limitations and bring biomechanics to the athlete/player.
Field-based techniques have involved objective measurement of variables from a 2D evaluation. Weir Alderson, Smailes, Elliott, and Donnelly [19] developed an innovative 2D analysis tool that could estimate knee joint loads during 45 • side-step cutting manoeuvres based on 2D measurements of several postural and technical parameters in the frontal and sagittal planes. A practical issue of the tool is the quantification of 2D variables (e.g., lateral trunk flexion, dynamic knee valgus, and thigh abduction angle) during a multiplanar movement such as side-step cutting, which may have limited accuracy in relation to gold-standard 3D motion analysis due to parallax error. Furthermore, logistical issues in conducting 2D digitising (time-consuming analysis) and the feasibility for practitioners from a non-biomechanics background to conduct such an analysis may limit the clinical utility of the tool. However, the emergence of automated tracking systems could speed up the rather laborious digitisation process and the process of feeding back to the athlete. However, to the best of our knowledge an automatic tracking system to evaluate cutting biomechanics has yet to be established or validated. More recently, Della Villa et al. [20] investigated the efficacy of a scoring system to identify athletes with high peak knee abduction moments during 90 • side-step cutting based on 2D video analysis of frontal and sagittal plane joint kinematics combined with force plate analysis (video vector). The tool involved five-item scoring criteria based on limb stability (frontal plane knee alignment), pelvis stability (frontal plane pelvis alignment), trunk stability (frontal plant trunk alignment), shock absorption (amount of knee flexion), and movement strategy (hip and knee flexion) taken at the point of maximum knee flexion. The authors found that the tool was able to discriminate between athletes exhibiting high and low peak knee abduction moments from 'gold standard' 3D motion analysis. However, to apply this tool in the field requires the use of a force platform to superimpose the ground reaction force vector over the video images (video vector) to partially determine limb stability, which is unlikely to be readily available for most practitioners working in the field and likely requires a level of biomechanics expertise. Additionally, the tool fails to consider other high-risk technical deficits that have been associated with high-risk cutting [8,21], such as penultimate foot contact braking, foot progression angle, and sagittal plane trunk position.
The cutting movement assessment score (CMAS) [22,23] is a qualitative tool that evaluates movement quality using two to three video cameras capturing side-step cutting manoeuvres based on a nine-item evaluation of various postural or technical aspects associated with the presence of high knee abduction moments ( Table 1). The tool (total point score) has been found to be associated with the magnitude of peak knee abduction moments determined from 'gold standard' 3D motion analysis [22,23] and found to discriminate between individuals that possess safer compared to hazardous cutting mechanics [23] and effectively evaluate the efficacy of technique modification programmes [15]. How-ever, research using the CMAS thus far has been conducted by biomechanists, and little is known about the inter-rater agreement amongst different sports science and medicine practitioners, which could help inform its clinical utility. This is important because poor to moderate inter-rater reliability has been observed between different practitioners when evaluating jump-landing movement quality using tuck jump assessment [24]. Possessing good inter-rater agreement between practitioners would ascertain that consistent evaluations of movement quality of side-step cutting using the CMAS can be made between different sports science and medicine practitioners irrespective of educational background and practitioner experiences. Furthermore, to help evaluate some of the frontal plane CMAS items (Table 1), Dos'Santos et al. [23] added a camera at 45 • to cameras placed in the sagittal and frontal planes ( Figure 1). The inclusion of three cameras to evaluate side-step cutting mechanics may further impact clinical utility of the CMAS. Exploring the effect of using solely one, two, or three cameras for screening movement quality using the CMAS would be worthwhile to establish whether fewer camera views are required, to reduce resource implications and speed up the data collection and analysis process for practitioners. Therefore, the aims of this study were twofold: (1) examine the inter-rater agreement between different sports science and medicine practitioners; and (2) examine the agreement between using different camera views to evaluate side-step cutting techniques of male academy rugby union players using the CMAS. Male rugby union players were considered in this study due to the association of side-step cutting manoeuvres in the incidence of non-contact ACL injuries in professional male rugby union [5]. It was hypothesised that good inter-rater agreement would be observed between each sports science and medicine practitioner to ascertain its utility in the field. Furthermore, it was hypothesised that good agreement between evaluations using all three camera views with those using two camera views only would be observed, whilst using single-camera views would show poor agreement compared to using three camera views for evaluation.  hypothesised that good inter-rater agreement would be observed between each sports science and medicine practitioner to ascertain its utility in the field. Furthermore, it was hypothesised that good agreement between evaluations using all three camera views with those using two camera views only would be observed, whilst using single-camera views would show poor agreement compared to using three camera views for evaluation.

Participants
Twelve male academy rugby union players (mean (SD) age, height, and mass were 17.1 (0.7) years, 177.5 (6.0) cm, and 79.0 (7.4) kg, respectively) participated in this study. All participants were playing their corresponding sport for a minimum of five years, participating in 3-5 skill sessions, 2 resistance training sessions, and 1 competitive match a week. All participants were free from any injuries and had not suffered a prior traumatic knee injury. While testing was conducted, all players were in season (competition phase). The University's ethics committee provided approval for the study (HST1920-223), and each player signed an institutionally approved consent form before data collection. Parental consent was attained from players under the age of 18.

Research Design
Five different professionals including a sports rehabilitator, biomechanist, physiotherapist, a strength and conditioning coach, and a sports coach specialising in sprinting actions participated in this study. Each practitioner had at least 5 years of applied experience in each of their professions and were familiar with side-cutting from a coaching or rehabilitation perspective. The lead author (biomechanist) had the most experience with conducting side-step cutting evaluations using both quantitative and qualitative analysis methods. Furthermore, the agreement between different camera configurations was examined. This study utilised a cross-sectional strategy to ascertain the agreements between different sports science and medicine practitioners and camera combinations, while examining the movement quality of 12 rugby union academy players performing a 45 • side-step cutting manoeuvre. Each sports science and medicine practitioner were provided with a manual of how to screen the videos against the CMAS and were given example videos prior to screening to familiarise themselves with the operational definitions [17]. This information was provided at least one month prior to undertaking the evaluations.

Experimental Procedures (Video Data Collection)
Testing was conducted during one training session before sport-specific training. All participants performed a 15 min standardised progressive warm-up consisting of activation, mobilisation exercises, dynamic stretches, and bodyweight exercises, then running and change of direction drills and practice trials of a 45 • cut.
Each participant performed four trials (two trials planting with each leg) of a 45 • side-step cut outdoors on the rugby field whilst holding a ball, as the ball carrier is the most injured scenario in non-contact ACL injuries [5]. The side-step cut task consisted of a 5 m approach, 45 • cut, and 5 m exit towards the finish. Timing cells were set up at the start and the finish of the task. The completion time (total 10 m distance) was determined using single-beam Brower timing gates (Draper, UT, USA). Time was noted to the nearest 0.001 s. Timing gates were set at an estimated hip height of all players, to be sure that only one body part broke the beam. Participants started 0.5 m behind the first gate, to avoid any early triggering of the start gate, from a two-point split stance. Participants were given the instructions to sprint as quickly as possible, then cut (lateral foot plant), then reaccelerate as quickly as possible through a 2nd set of timing gates. To allow subsequent qualitative screening of side-step cutting trials, three Panasonic Lumix FZ-200 high-speed cameras (Osaka, Japan) sampling at 100 Hz were used to record each trial. The cameras were placed on tripods 3.5 m away from the cutting check mark and at a height of 0.60 m. They were positioned in the frontal and sagittal planes, with a camera also at 20 • relative to the frontal plane ( Figure 1).

Data and Statistical Analysis for Exploring Agreement between Different Professionals
Trials were screened against the 9-item CMAS screening tool (Table 1) [17,22,23]. The participants who exhibited any of the characteristics/deficits within the CMAS were awarded a score, with a higher score representing lower movement quality and hypothetically greater peak knee abduction moments [22,23]. All video files were viewed in Quintic Biomechanics v31 (Solihull, UK). The software was used to play videos at varying speeds as well as frame-by-frame along with drawing tools to help judge items of the CMAS in line with recent recommendations [17]. Two randomly selected trials (one left cut, one right cut) from each participant were used for further analysis by each professional. Each sports science and medicine professional scored the same left cut trial and the same right cut trial of each participant based on viewing each camera view in each plane (all three camera views).
Statistical Analysis was conducted in SPSS v26 for Windows (Champaign, IL, USA) and MS Excel. Normality for total scores for each cutting direction from each practitioner was confirmed using a Shapiro-Wilk test. Subsequently, total scores for each cutting direction from each practitioner were compared using a repeated measures ANOVA with location of any significant differences located using a Bonferroni post hoc test. Significance was set as p < 0.05.
To determine inter-rater reliability, a two-way mixed effects model Intra-Class Correlation Coefficient (ICC 3,k ) was utilised. ICCs were interpretated based on the guidelines by Koo and Li [25]: poor (<0.5), moderate (0.75), good (0.75-0.90), and excellent (>0.90) reliability. Inter-rater agreement between each sports science and medicine practitioner for the total score and each item of the CMAS was assessed using percentage agreements and Kappa coefficient. Percentage agreements were determined using the formula = (agreements/[agreements + disagreements] × 100). Percentage agreements were interpreted in line with previous research [26,27] and the scale was as follows: excellent (>80%), moderate (51-79%), and poor (<50%). The Kappa coefficient was calculated in MS Excel using the formula K = Pr (a) − Pr (e) /1 − Pr (e), where Pr (a) = relative observed agreement between raters; Pr (e) = hypothetic probability of chance agreement, using the observed data to calculate the probabilities of each observer randomly saying each category. The Kappa coefficient was evaluated based on Landis and Koch's [28]

Data and Statistical Analysis for Exploring Agreement between Different Camera Combinations
One randomly selected trial from each participant was used to examine the agreement between different camera combinations (12 trials in total). This randomly selected trial was different from those used for the assessment of inter-professional agreement, so no prior knowledge influenced the scoring. One (AR) investigator scored each trial based on viewing each camera view, a combination of two cameras, and all three cameras. Each trial was evaluated with single-camera views initially. This was followed by considering two camera views and then finally all three camera views. The following camera combinations were compared:

1.
All three cameras vs. front and side.
All three vs. angle and side. 4.
All three vs. side.
All three vs. front.
Percentage agreement and Kappa coefficients were used to explore agreement between different camera combinations using the abovementioned formulas and evaluated using the same criteria.

Inter-Professional Agreement
• Significant (p ≤ 0.001) differences in total CMAS score were observed between the biomechanist and all four practitioners for the right cut, whilst significant differences were also observed between the S&C and sprint coaches and the sports rehabilitator (p = 0.016; p = 0.047, respectively) and the S&C coach and physiotherapist (p = 0.005) ( Table 2). • For the left cut, the sprint and S&C coaches revealed significantly (p < 0.001; p = 0.005 between physiotherapist and S&C coach) greater total scores than the three other practitioners ( Table 2). In addition, the biomechanist showed a significantly (p = 0.046) lower total score than the physiotherapist (Table 2). • Tables 3 and 4 show the inter-rater agreements (percentage agreement and Kappa coefficients) between different practitioners for the right and left cut, respectively.    • For the right cut task, moderate to excellent agreements (Table 3) were observed for total score and all items of the CMAS. Excellent (ICC 3,k = 0.941) inter-rater reliability for total score was also observed. • For the left cut task, moderate to excellent agreements were observed for the left cut (Table 4), whilst good (ICC 3,k = 0.896) inter-rater reliability for the total score was observed.

•
The best agreement tended to occur between the sports rehabilitator and biomechanist (moderate to excellent) (Tables 3 and 4). • Table 3 shows that items 3 (initial hip rotation), 6 (lateral trunk flexion toward plant leg side), and 7 (trunk upright or leaning back) of the CMAS for the right cut, whilst Table 4 shows that items 3 and 6 for the left cut revealed at least four inter-rater agreements that fell below moderate (e.g., <0.6).

Agreement between Different Camera Combinations
• Table 5 shows the agreement between different camera combinations for the total score and each item of the CMAS.  (3), initial hip rotation, (4) initial knee valgus, (5) initial foot rotation, • Excellent agreement was observed for the total score between using all three camera views for evaluation vs. using different combinations of two camera views (Table 5). • However, only fair to moderate agreement in the total score was observed between using all three camera views for evaluation vs. evaluation using a single-camera view ( Table 5). • Table 5 shows that only items 1 (clear penultimate foot contact braking strategy) and 3 (initial hip rotation) fell below moderate (K < 0.6) in exploring the agreement between three camera views and two camera views from front and angle only. • All other agreements for individual items of the CMAS between three camera views and two camera views revealed moderate to excellent K or moderate to excellent percentage agreements (Table 5).

•
Inter-rater agreements between three camera views and the sagittal view only revealed excellent agreement for items 1 (clear penultimate foot contact braking strategy), 7 (trunk upright or leaning back), and 8 (limited knee flexion) of the CMAS, with all other items revealing slight (K) or poor (%) agreement (Table 5). • Conversely, frontal-and angle-only views revealed slight (K) or poor (%) agreement for items 1, 7, and 8 of the CMAS, with all other items revealing moderate to excellent agreement (Table 5).

Discussion
The findings of the present study revealed moderate to excellent inter-rater agreement (K) for total score and moderate to excellent agreement across all items of the CMAS between different sports science and medicine practitioners in using the CMAS to evaluate the side-step cutting technique in academy rugby union players. This finding suggests that the CMAS is a tool that can be used effectively by various practitioners regardless of biomechanical background to evaluate the side-step cutting technique for injury risk profiling. Furthermore, excellent agreement (total score K) was found between using three camera views for evaluation versus using only two camera views, with better agreements observed when at least one of the views was of the sagittal plane. This suggests that the use of only two cameras could suffice to qualitatively evaluate the side-step cutting technique against the CMAS, providing one of the views involves the sagittal plane. Single-camera views had much lower (fair to moderate) agreement against a three-camera approach (total score K), and thus single-camera views are not advocated to evaluate the side-step cutting technique against the CMAS.

Inter-Professional Agreement
Good to excellent agreements (total score K = 0.63-0.81) between all sports science and medicine practitioners were revealed. This suggests that the CMAS can be used between a variety of practitioners for evaluating the side-step cutting technique and substantiates previously reported intra-and inter-rater agreement and reliability for the CMAS [13]. Interestingly, greater agreements (left and right cut task) were revealed between the biomechanist, sports rehabilitator, and physiotherapist (total score K = 0.81-0.68), whereas lower agreements were evident involving the sprint and strength and conditioning (S&C) coaches (total score K = 0.67-0.62). This perhaps reflects the greater familiarity of these professions with injury risk screening/profiling compared to S&C and sports coaches. These findings partly agree with Dudley et al. [24] using the tuck jump assessment tool as to the influence of educational background on agreement using qualitative screening tools. However, these authors found only poor to moderate intra-and inter-rater agreement between clinicians of varying educational backgrounds. Conversely, Mayhew et al. [29] using the same qualitative tool explored inter-rater agreement between two experienced physiotherapists and two S&C coaches with more varied experience. They reported good inter-rater agreement when the physiotherapists (K = 0.65) and S&C coaches (K = 0.62) were compared, but slightly higher inter-rater agreements (K = 0.67-0.8) between physiotherapists and S&C coaches. Despite the slightly lower agreements observed between S&C and sprint coaches used in the present study, the agreements remained good and suggest that the tool can be effectively used to evaluate the movement quality of side-step cutting by practitioners with diverse backgrounds in biomechanics and sports injury aetiology.
Despite the good inter-rater agreement and reliability observed, significant differences in the total score were observed between practitioners, substantiating previous research [24] that found significant differences between practitioners of differing clinical and educational experience in the total tuck jump assessment score. Unlike the work of Dudley et al. [24], the present study did find a consistent trend in (total) scoring. For instance, the comparison of total score in each cutting direction between practitioners tended to reveal lower (significant for all comparisons except the sports rehabilitator for the left cut) scores by the biomechanist, whereas sprint and S&C coaches revealed significantly higher scores in both cutting directions (significantly greater than the sports rehabilitator, physiotherapist, and biomechanist in each cutting direction). This perhaps suggests that the sprint and S&C coaches with potentially less familiarity with injury risk screening/profiling tend to 'over-score' some items of the CMAS, whereas the biomechanist perhaps tended to 'under-score' compared to a sports rehabilitator and physiotherapist who generally revealed similar scores in each direction. The biomechanist in the present study perhaps had the greatest experience of using the CMAS than all practitioners involved in the study, which suggests that prior to using the tool practitioners should undertake extended verification (beyond what was undertaken in this study) with more experienced practitioners to aid the implementation of the CMAS, particularly for items of the CMAS that tend to have lower agreements.
Items of the CMAS that tended to result in lower agreements were 3 (initial hip rotation), 6 (lateral trunk flexion toward plant leg), and 7 (right cut only) (trunk upright or leaning back during final foot contact), with four or more agreements falling below 0.6 (K). The lateral trunk flexion in particular had kappa coefficients falling to 0.39-0.41 in some cases. These observations are similar to previous research involving the CMAS [22,23]. Jones et al. [22] reported lower intra-and inter-rater percentage agreements and K for the frontal plane trunk position (intra-rater = 75%, K = 0.62; inter-rater = 62.5%, K = 0.40) between an experienced biomechanist and graduate sports science student. Dos'Santos et al. [23] also reported a moderate K (0.431-0.599) for initial knee valgus, initial foot position, frontal and sagittal plane trunk position, and limited knee flexion during FFC, but only slight for initial hip rotation (K = 0.194), between the lead researcher (biomechanist/S&C) and an experienced biomechanist. Furthermore, the authors reported slight agreement (K = 0.067) for initial hip rotation for inter-rater reliability between the lead researcher (biomechanist/S&C) and a recent sports science graduate. Thus, it would appear to further improve inter-rater agreement between different practitioners further training and/or guidelines should be provided for these three criteria. The interested reader should refer to the recent published guidelines for more information [17].

Agreement between Different Camera Combinations
The use of two camera views to evaluate side-step cutting against the CMAS revealed good to excellent agreement with the use of three camera views (total score K = 0.83-0.97). Originally, the CMAS used two camera views (frontal and sagittal planes) to evaluate the side-step cutting technique [22]. However, the authors reported lower inter-rater agreement for frontal plane variables due to some element of 'pre-rotation' by some participants prior to the plant step. They suggested that this may be overcome with the inclusion of a third camera at an angle (45 • ) to the frontal and sagittal planes. In a follow-up study by Dos'Santos et al. [23], this approach was adopted and led to improved inter-and intrarater reliability and a higher association between CMAS score and peak knee abduction moments as determined from 3D motion analysis. However, including further cameras has resource implications and may prolong analysis against the CMAS. The results of this study suggest that the use of two cameras may suffice to evaluate the side-step cutting technique against the CMAS. It is advisable that one of those camera views involves the sagittal plane, as higher agreements were observed for angle-sagittal (total score K = 0.97) and front-sagittal (total score K = 0.96) than angle-frontal (total score K = 0.83). Indeed, of the single-camera views, only the sagittal plane had sufficient agreement (K = 1.0) for item 1 of the CMAS (clear penultimate foot contact braking strategy), whereas all other combinations of agreements revealed poor agreement (K = 0) with three camera views. Interestingly, front-angle camera view combinations had lower agreement with three camera views for clear penultimate foot contact braking strategy (1), initial hip rotation (3), initial knee valgus (4), initial foot rotation (5), plant side lateral flexion (6), and limited knee flexion during final foot contact (8) compared to frontal-sagittal and angle-sagittal camera combinations (Table 5).
All single-camera views had low agreement with evaluations involving three camera views (total score K = 0.34-0.44), thus, evaluating the side-step cutting technique from one camera view against the CMAS criteria cannot be recommended. These data do suggest that only the sagittal view provides perfect agreement (K = 1) for identifying (1) clear penultimate foot contact braking strategy, (7) upright or leaning back during final foot contact, and (8) limited knee flexion during final foot contact (Table 5), which is unsurprising as these characteristics are performed in the sagittal plane. However, a similar magnitude of agreements (items 2, 3, 4, 5, 6, and 9 of the CMAS) can be seen between evaluations involving front and angle camera views (Table 5), thus suggesting that either a frontal or angle camera view is needed along with a sagittal view specifically to determine the three previously mentioned items of the CMAS.
The present study only examined agreement using the CMAS with healthy male rugby union payers. Thus, the findings of the present study are limited to evaluations involving injury-free athletes. It is unknown how CMAS scores may be impacted with previously injured players or players exhibiting knee pain. Further research is required exploring the use of the CMAS with such players. Furthermore, it is currently unknown whether a cut-off exists for using the CMAS to identify athletes with increased risk of injury, and this provides an avenue for future research.

Conclusions
To conclude, moderate to excellent inter-rater agreements were observed between different sports science and medicine practitioners, which suggests that the CMAS can be used effectively to evaluate the side-step cutting technique in the field by a variety of applied practitioners. The use of two camera views to evaluate the side-step cutting technique against the CMAS criteria can be used rather than the previously recommended three camera views. However, one of those camera views must be of the sagittal plane to specifically evaluate the presence of a penultimate foot contact braking strategy, knee flexion, and sagittal plane trunk position during final foot contact.