Reliability and Validity of DPA-1 Testing After Anterior Cruciate Ligament Reconstruction

Summary. There is a lack of equipment and methods for the reliable and valid measurements of human neuromuscular control. To overcome this limitation, an analyzer of dynamic parameters (DPA-1) of human hand and leg movements was constructed by Kaunas University of Technology and “Katra” engineers in collaboration with the Lithuanian Academy of Physical Education. The aim of the study was to determine the reliability and validity of the tests performed on the DPA-1 in healthy and injured subjects after the anterior cruciate ligament (ACL) reconstruction surgery. Material and Methods. The men who had undergone a unilateral ACL reconstruction (n=17, on the average 3.8 months [SD, 2.1] after the surgery) and healthy untrained men (n=17) performed the research protocol twice within 24 hours in between. Average reaction time, mean and maximal movement speed, time to reach maximal speed, and movement distance of the right and left feet for the patients and of the dominant foot for the healthy subjects using the DPA-1 as well as the scores of isokinetic muscle strength and self-assessment tests were registered. Results. There was a significantly reduced concentric peak torque on the injured knee compared with the uninjured knee during knee extension, and the mean score of the Lysholm scale for the injured knee was 69.1 (SD, 13.7) (P<0.05, compared between legs). The test-retest reliability for all the DPA-1 tests varied from 0.68 to 0.94 (P<0.05). However, there were no significant differences in most variables measured by the DPA-1 between injured knee, uninjured knee, and control knee. Conclusions. The results revealed low validity of the DPA-1 tests for the evaluation of patients following ACL surgery, despite the reliability of these tests varied from moderate to very high.


Introduction
Anterior cruciate ligament (ACL) injuries frequently occur in sports and exercise and have been reported to be 4 to 8 times more frequent among women than men (1).Most ACL injuries happen in situations like landing, side cutting, or deceleration, which involve eccentric quadriceps muscle forces (2,3).Consequently, such injuries require ACL reconstruction surgery, and rehabilitation takes from 6 to 12 months on the average (4).The ability of the patients to return to sports after ACL reconstruction is governed by various factors, which include the postoperative knee function, social reasons, and psychological hindrances such as fear of reinjury (5).Strength training and neuromuscular training programs are used in clinical practice to enhance muscle strength and dynamic stability during activities by inducing compensatory biomechanical and neuromuscular responses (6,7).The restoration of neuromuscular control of the lower extremity has been recognized as one of the key factors to re-establish dynamic joint stability and the knee function (8,9).
It is relevant that everyone would return to athletic activities in rational time.Rapid return, however, very often leads to discomfort, pain, or reinjury.Walden et al. (10) have previously reported that many elite football players suffer from synovitis and other overuse injuries shortly after their comeback to football, possibly indicating a premature return.Therefore, reliable information of the current state of the injured site and its improvement during rehabilitation course is necessary.Different methods have been used to assess capabilities of the knee function after ACL injury, including isokinetic dynamometry (8,11), motion analyses (8,12), electromyography (12), self-report of knee function survey (5,13), and clinical examination (5,14).
There is a lack of equipment and methods for reliable and valid measurements of human neuromuscular control.To overcome this limitation, an ana-lyzer of dynamic parameters of human hand and leg movements (DPA-1, Patent No. 5251; 08/25/2005) was constructed by Kaunas University of Technology and "Katra" engineers in collaboration with the Lithuanian Academy of Physical Education.The DPA-1 has been found to be sensitive enough to show differences in movement accuracy between dominant and nondominant hands and identify gender differences in the speed of hand movements in healthy men and women (15).Furthermore, it was established that a quick and accurate hand movement was performed more slowly than a simple quick movement (16).However, no studies have examined changes in the neuromuscular control of the lower extremity using the DPA-1.The assessment of neuromuscular control of the knee joint is of particular importance as bilateral deficits in knee joint proprioception have been documented after a unilateral ACL injury (17).Therefore, the aim of this study was to determine the reliability and validity of the tests performed on the DPA-1 in healthy and injured subjects after the ACL reconstruction surgery.

Material and Methods
Subjects.The patient group comprised 17 male subjects (age, 25.8 years [SD, 6.7]; body weight, 87.8 kg [SD, 14.3]; height, 181.0 cm [SD, 7.6]) who had undergone the unilateral ACL reconstruction with a semitendinosus/gracilis (STG) graft in some cases combined with a meniscal injury (n=12) in the Department of Orthopedics and Traumatology, Hospital of Lithuanian University of Health Sciences, or Kaunas Red Cross Clinical Hospital.The average time from surgery to experiment was 3.8 months (SD, 2.1).The Lachman test was performed by a surgeon before sending a patient to the laboratory for testing.The individuals were eligible for the inclusion in the experiment if they had a negative Lachman test score and no previous ACL surgery as well as a normal contralateral hip and ankle joint function (Lysholm score, >95).
The healthy male subjects (n=17; age, 31.3 years [SD, 7.8]; body weight, 81.7 kg [SD, 12.2]; height, 179.6 cm [SD, 6.5]) were physically active at a frequency of 1 to 3 times per week, but did not participate professionally in any sporting program.There was no significant difference in weight and height between patients and healthy subjects, although healthy subjects were younger (P<0.05).The dominant leg in all the healthy subjects was the right one.The dominant leg was established subjectively by asking each participant which leg they preferred to kick a ball as far as possible (18).For the healthy subjects, the inclusion criteria were no history of ACL surgery and a normal hip and ankle joint function for both legs.Each subject read and signed a written informed consent form consistent with the principles outlined in the Declaration of Helsinki.All the subjects gave informed consent according to the requirements of the Ethics Committee of Lithuanian Academy of Physical Education.
Research Protocol.Testing according to the research protocol was performed twice: first under control conditions (test 1) and then after 24 hours (test 2).On arrival of the patients to the laboratory, self-assessment questionnaires (Lysholm Scale and Lower Extremity Functional Scale, LEFS) were filled out on the first testing day.Then, the patients underwent a neuromuscular control assessment on the DPA-1 followed by isokinetic muscle strength tests.Only the DPA-1 testing was repeated on the second testing day.The healthy subjects performed only the DPA-1 tests on the first testing day and then repeated procedures after 24 hours.The patients' contralateral foot served as an internal control, while the healthy subjects' dominant foot served an external control.The experiment was performed within 2 months starting from the first week of January with the subjects from both groups executing the tests randomly.
Assessment of Neuromuscular Control.Knee neuromuscular control was evaluated by examining foot movement accuracy, speed, and reaction time using the DPA-1.During the test, the subject was seated in a special chair above the table with the DPA-1 fastened to it (Fig. 1).The subject's back was straight and leant against the backrest; both legs were bent 90° at the pelvic and knee joints.The position of the DPA-1 chair was regulated so that the subject could sit comfortably and take a standard position.The distance between the computer screen and the subject's eyes was approximately 90 cm.The subject had to react to the target on the computer screen (a red circle) and push the handle of the device with their foot to reach the target as quickly as possible and in the most accurate trajectory.The trajectory of the foot movement was identically repeated on the computer screen.During each movement, the subject set the handle symbol of 3.5 mm in diameter to the start zone, i.e., the center of a green circle, the diameter of which was 10 mm.The target was generated intermittently in 1-3 s after the handle stopped in the start zone.The end point of the movement was fixed when the center of the handle symbol stopped in the target circle and remained there for no shorter than 0.02 s.On the screen, the height of the task performance field was 200 mm; and the width, 270 mm.The target -a red circle measuring 3.5 mm in diameter -appeared in the middle of the field of task performance (height, 170 mm; width, 135 mm).The distance between the start zone and the target center was 160 mm.The target appeared on the computer screen in the same place in front of the start zone (standard task) or randomly on the left, right, and in front of the start zone (variable task).The standard and variable tasks included 15 repetitions and were performed with one foot without stopping; then, after a 2-min break, the same tasks were performed with the other foot.
The average reaction time, mean and maximal movement speed, time to reach maximal speed, and movement distance of the right and left foot for the patients and of the dominant foot for the healthy subjects were registered.The patients in all the cases started the tests with the uninjured knee, while healthy subjects performed the measurements in a random order.The subjects were familiarized with the test procedure completing one test before starting the experiment.Before each task, subjects were allowed to perform 3 repetitions, and their results were not recorded.
Assessment of Isokinetic Strength.The maximum isokinetic concentric strength of knee extensor and flexor muscles was assessed using an isokinetic dynamometer (System 3; Biodex Medical Systems, Shirley, New York) at an angle velocity of 60° and 300° per second.Peak torque measurements were made on both sides for the injured persons.The subjects sat upright in the dynamometer chair and were tied up with chest, waist, and thigh straps.The axis of rotation of the dynamometer was visually aligned with the axis of rotation of the subject's right knee joint.The ankle pads were placed just above the subject's lateral malleoli.The subjects were instructed to keep their hands crossed in front of their chest during all the testing sessions and were encouraged to execute maximal voluntary contraction with maximal exertion.
Lower Extremity Functional Scale.The LEFS is a measure of activity limitation developed for musculoskeletal conditions of the lower extremity (19,20).On this scale, participants rate the difficulty in performing 20 activities of the lower extremity on a 5-point scale (0 refers to extreme difficulty or inability to perform activity, and 4 refers to no difficulty).The responses are summed to give a score ranging from 0 to 80, with 0 indicating high levels of activity limitation and 80 indicating low levels of activity limitation.In a heterogeneous population with lower limb conditions, the LEFS was found to have a high internal consistency (α=0.96) and high testretest reliability (r=0.86) and correlated well with the physical function subscale and the physical component summary scores of the Medical Outcomes Study 36-Item Short-Form Health Survey (r=0.80 and r=0.64, respectively) (19,21).The LEFS was converted to a 100-point scale by dividing the scores by 80 and then multiplying by 100 in order to facilitate comparison with the Lysholm scale.
Lysholm Scale.The modified Lysholm scale, as described by Tegner and Lysholm (22), is an 8-item questionnaire that was originally designed to evaluate the activity of patients following the knee ligament surgery.It is scored on a 100-point scale with 25 points for knee stability, 25 points for pain, 15 points for locking, 10 points each for swelling and stair climbing, and 5 points each for limb, use of a support, and squatting.This scale has been used extensively in clinical studies.
Statistical Analysis.The descriptive data are presented as mean (SD).The test-retest reliability was analyzed using the coefficient of variation (CV) and intraclass correlation coefficients (ICC).SPSS (SPSS Inc., Version 10.0, Chicago, IL) was used to calculate the ICC.One-way analyses of variance were used to examine the hypotheses about the validity that specified that there would be a difference in the injured and uninjured knees and between the patients and the healthy subjects.The level of significance for this study was set at P<0.05.Pearson correlation coefficients were calculated to examine the relationship between the DPA-1 and functional variables.The correlation was considered strong if r was more than 0.5; moderate, if r was between 0.5-0.3;weak, if r was between 0.3-0.1;insubstantial or trivial, if r was less than 0.1 (23).

Results
There was a significantly reduced concentric peak torque of the injured knee compared with the uninjured knee during knee extension at a velocity of 60°/s (P<0.05) (Fig. 2).Peak torque decreased as the knee extension velocity increased, but the difference between legs was smaller at a velocity of 300°/s.Similarly, there was a significantly reduced injured knee capacity as the mean scores were 69.1 (SD, 13.7) for the Lysholm scale and 82.1 (SD, 16.6) for the LEFS (P<0.05, as compared with the healthy leg) (Fig. 3).The reliability of all the indices measured by the DPA-1 is displayed in Table 1.There was no significant difference in absolute values between the injured and uninjured knees, though both sides of the injured subjects demonstrated lower values than the control group in average speed and time to maximal speed during both standard and variable tasks (P<0.05).No significant difference was found between the test and retest values for all the measures.The CV ranged from 4.8 to 22.6 with the lowest values registered for the movement distance as compared with other variables.The test-retest re-  liability for all the tests was from moderate to very high with ICCs from 0.68 to 0.94 (P<0.05 in all cases).There were no significant differences in the CVs and ICCs between the injured, uninjured, and control knees.In addition, there was no significant difference in the CV and ICCs between the standard and variable tasks.The magnitude of correlations was high between the Lysholm and LEFS scores (r=0.67,P<0.05, data not shown) and moderate between peak torque and both clinical tests (r ranged from 0.32 to 0.49, P<0.05).Knee function indices measured by the DPA-1 correlated better with knee extension peak torque than they did with Lysholm or LEFS scores (Table 2).Moreover, the correlations between the DPA-1 indices and peak torque at a velocity of 300°/s were stronger than at a velocity of 60°/s with the exception for the average speed.The correlations between most of the DPA-1 indices and peak torque during standard tasks were stronger than during variable tasks.

Discussion
The main finding of this study was that the assessment of knee function performed by the DPA-1 was not able to distinguish between the uninjured and injured knees after the ACL reconstruction, while strength and clinical tests pointed out to an obvious disorder of the reconstructed knee.The results reveal low validity of the DPA-1 tests for the evaluation of patients following the ACL surgery, despite the reliability of these tests ranged from moderate to very high.
The ICC greater than 0.75 represents high reliability of a test in a clinical trial (24).All the variables, except the average speed for an injured limb during a variable task, showed ICC values greater than 0.75 in our study.It is worth noting that the ICC values for some variables while testing the in-jured leg were above 0.90, representing excellent reliability.Only small differences in the reaction, speed, and accuracy variables were detected between standard and variable tasks.This strongly suggests that the basic DPA-1 tests are reliable for the assessment of knee function regardless of task complexity.Furthermore, the ICCs and CV were similar for the injured, uninjured, and control knees indicating that tests performed by the DPA-1 were equally reliable for the patients after the ACL surgery and for the healthy persons.
Unexpectedly, the results revealed no differences in the mean values of all the DPA-1 variables between the injured and uninjured legs.These findings strongly suggest that the DPA-1 tests cannot differentiate between the severity of the knee function impairment after the ACL reconstruction surgery, while bilateral strength deficits were confirmed by isokinetic and clinical testing.Therefore, this technique does not allow a quantitative evaluation of a neuromuscular system condition and improvement during the rehabilitation period.On the other hand, the values of some variables (average speed, time to peak speed, and maximum speed) were lower for both the injured and uninjured knees of the patients compared with the healthy controls.While this result advocates that muscle contraction speed is generally reduced overall due to surgery and impaired physical activity, it is unlikely that these sources would have a great impact on the knee neuromuscular control as there were no significant changes in other variables of the DPA-1 test.
For validity assessment, the present study also correlated the values of the DPA-1 tests with isokinetic peak torque and self-assessment questionnaire scores.It is well known that validity is present when the instrument performs as expected in relation to another measurement (25).It is excellent when there is a gold standard to which the results can be compared (26).The tests used in our study might not fulfill all the obligations for a gold standard, but have been shown to be reliable, valid, and responsive to changes with time (19,25,27).The results, however, evoked some discrepancies regarding data interpretation.No correlation was found between the scores of clinical tests and any of the DPA-1 variables for the patients after the ACL reconstruction.However, the correlation between isokinetic peak torque and most of the DPA-1 variables ranged from weak to moderate with stronger relationships found at a velocity of 300°/s.The discrepancies arise possibly due to the complexity of mechanisms involved in knee performance.After the ACL reconstruction, the level of the knee function depends largely on a wide range of dissimilar factors, including muscle coordination and control through afferent and efferent information processing, resistance magnitude, speed of exercise, visible input, one-or two-legged performance, etc. (4,8,28).The results of strength, clinical, and motor control tests were independent from one another in the present study, and this supports the contention that the separate determinants of knee performance have little in common.
In the present study, strength deficits between the injured and uninjured knees were significantly smaller in the high-speed isokinetic movements as compared with low-speed ones.It is very doubtful that this finding contains any physiological relevance.We believe that it might be related to the fear of re-injury since testing at low speed requires maximum force exertion.Consequently, the torque deficit between injured and uninjured knee extensions might be exacerbated when tested at low speed.Fear of re-injury is an important psychologi-cal factor during the whole rehabilitation period following surgery, although patients already can tolerate resistance and pliometric training or even are able to perform sport-specific exercises (5).
In general, the DPA-1 test was found to be a poor diagnostic tool to evaluate an improvement following the ACL reconstruction.The design of the tests might be viewed as a possible cause for this.We have chosen typical tests used for the evaluation of hand motor control, but the distance between the start zone and the target center, target size, or movement direction might not be appropriate for the assessment of knee neuromuscular control.Further studies are needed to determine if modification of the tests would increase their sensitivity to the degree of being able to distinguish between the injured and uninjured sides of patients after the ACL surgery.Moreover, it is important to note that the results of the present study should not be considered as valid to a wide population of patients with neuromuscular disorders since the study focused exclusively on the patients with anterior cruciate ligament deficiency.

Conclusions
DPA-1 tests were shown to be reliable as testretest reliability varied from moderate to very high.The results did not support validity of DPA-1 measurements for the evaluation of recovery following the ACL reconstruction.Therefore, the DPA-1 does not appear to provide significant information about neuromuscular system control for clinicians and therapists.

Statement of Conflicts of Interest
The authors state no conflicts of interest.

Fig. 3 .Fig. 2 .
Fig. 3. Mean values and standard deviations of Lysholm and Lower Extremity Functional Scale scores for the injured knee *P<0.05,compared with the uninjured knee (a score of 100); †P<0.05,compared with the Lysholm scale score.Lower Extremity Functional Scale

Table 1 .
CV, coefficient of variation; ICC, intraclass correlation coefficient.*P<0.05;†P<0.05,compared with a standard task, ‡P<0.05,compared with the uninjured knee, §P<0.05,compared with the injured knee.Mean Values and Standard Deviations as well as Mean Coefficient of Variation and Intraclass Correlation Coefficients for DPA-1 Variables of Knee Function during Standard and Variable Tasks

Table 2 .
Coefficient of Correlation for DPA-1 Variables with Peak Torque and Lysholm and Lower Extremity Functional Scale (LEFS) Scores