A Pathway to Hallux Valgus Correction: Intra- and Interexaminer Reliability of Hallux Alignment

Abstract

Hallux Valgus is an orthopedic deformity of the forefoot region characterized by a lateral deviation of the first toe by more than 15 degrees with a medial deviation of the first metatarsal, often associated with a rotational deformity of the hallux phalanges. The work presented here is part of a broader study. To assess the (mis)alignment of the hallux, computerized photogrammetry was performed with Kinovea^® software. Scientific articles about the reliability of photogrammetry for the (mis)alignment of a hallux evaluation are unknown. The main objective of this work is to verify the reliability of intra- and interexaminer evaluations in the assessment of a hallux (mis)alignment using computerized photogrammetry. For the intrarater evaluation, one examiner analyzed an aleatory sample of 40 feet to measure the (mis)alignment of the hallux, repeating the analysis five or fourteen days later. For the interrater evaluation, two examiners analyzed an aleatory sample of 20 feet. An intraclass correlation coefficient (ICC) and paired samples t-test were applied with a significance level of 0.05. Both inter- and intraexaminer-reliability analyses were rated as excellent (ICCs > 0.7), indicating yet another way to assess hallux (mis)alignment by nonradiological means, avoiding the radiation exposure associated with radiographs and the cost associated with the equipment acquisition.

1. Introduction

Hallux Valgus is one of the most common forefoot pathologies encountered in orthopedic practice [1]. Hallux Valgus is an orthopedic pathology defined as a lateral deviation in the first metatarsophalangeal joint [2] of more than fifteen degrees with gradual evolution [3,4]. The alteration can directly influence the quality of life of the affected individual, as it largely causes functional alterations and pain in the affected region or in other places dependent on the good functioning of the feet [5]. There is a higher prevalence in women (30%) than in men (13%) [6], affecting 20% of patients aged between 18 and 65 years and more than 35% of patients aged ≥65 years. Additionally, it can also modify gait in this population, especially when walking on uneven ground [7]. There are surgical and conservative treatments; they are determined according to the evaluation criteria through the degree of (mis)alignment of the first toe represented by a lateral deviation of the hallux in relation to the first metatarsal by more than 15° [3]. The diagnosis can be performed using a radiographic examination [8], with clinical and imaging observation in the evaluation determined in four parameters such as none, mild, moderate, and severe [9] or the measurement of the hallux deviation angle with the use of a goniometer [10] in the forefoot region. The application of methods such as photogrammetry, which uses image software to calculate measurements through previously marked points in anatomical regions, may be an option for recording and monitoring the evolution of the pathology or for research with means to evaluate changes after applying techniques or the use of orthoses in cases of Hallux Valgus. Kinovea^® software is used for the analysis, comparison, and evaluation of sports and training, especially suitable for physical-education teachers and coaches. Some advantages of this program are observation, measurement, and video comparison [11].

Treatment with surgical intervention tends to significantly improve the alignment of the first toe and symptoms caused by Hallux Valgus [12,13]; however, the various correction surgeries must be considered with caution as they also present a high number of complications such as arthrosis, the limitation of movement of the metatarsophalangeal joints and ankle, transfer injuries (calluses), and varus deformities [14]. Conservative treatment with a static orthosis at night also demonstrated efficiency in the evolution and prevention of Hallux Valgus progression [15,16].

Another form of conservative treatment is the use of a dynamic orthosis which applies a corrective force to the hallux during walking, allowing free movement of the metatarsophalangeal joint, being considered even more effective than a static orthosis [16]. Most conservative treatments for Hallux Valgus involve the use of an orthosis that fixes the limb associated with mechanical traction to favor alignment. However, these devices reduce adherence to use because they do not favor the gait as a static orthosis does and increase the volume of the foot, making them difficult to use with shoes [17].

As a result, a Hallux Valgus correction orthosis does not meet all the usability criteria, especially in terms of the perceived ease of use of the product, reducing the attitude toward use [18]. A device called a foot piece is being developed. Foot pieces are defined as small reliefs used in insoles placed at specific points to correct the positioning of the foot and body [19]. The device will be used to cause a small elevation in the anterior region of the first ray of the foot in order to slightly modify function in a static and dynamic way, distributing the plantar pressure on the first toe and on the first metatarsophalangeal joint, which is commonly greater in individuals with Hallux Valgus dysfunction [20]. The device can provide more comfort and safety, occupying a small area in the shoe, with easy adaptation as it can be attached to any type of open or closed shoe, positioned on the insole or surface of the shoe, and can be an excellent resource for the conservative treatment of Hallux Valgus.

The work presented here is part of a broader study that intends to evaluate the results of using a new orthosis for the conservative treatment of Hallux Valgus. The implementation of methods that can inform the intraexaminer error margin and make the results with this orthosis more reliable is essential to prove the effectiveness of this new proposal for the conservative treatment of this pathology. Therefore, the main objective of this work (named prestudy) is to verify the reliability of intra- and interexaminer evaluations in the assessment of hallux alignment, based on computerized photogrammetry using Kinovea® software, as a first step of the broader study. According to the objectives of this prestudy, the hypotheses under study are H0—“the coefficient of agreement is equal to zero” versus H1—“the coefficient of agreement is greater than zero”. Therefore, if we obtain a p-value < 0.05, then the null hypothesis is rejected, which means that the ICC > 0. If so, the ICC interpretation is made according to the criteria defined in Section 2.5. To be excellent, the ICC should be higher than 0.7.

2. Materials and Methods

2.1. Participants and Inclusion Criteria

To participate in the broader study, participants were invited and recruited through the Clinical School of Physiotherapy at the UniRV—the University of Rio Verde—in Rio Verde, Goiás, Brazil; after an explanation of the procedures, an informed written consent was obtained from all participants prior to the study.

Both the Institutional Review Board—the Research Ethics Committee (CEP) of UniRV—and the Ethics Council of the FMH (CEIFMH)—University of Lisbon, Portugal—approved the study (CEP approval ID: 4.039.312 and CEIFMH approval ID: 20/2021).

The inclusion criteria defined for admission to the research were as follows: participants aged ≥ 18 years, presenting a lateral deviation of the first toe with a metatarsophalangeal angle ≥15 degrees, with an absence of orthopedic disorders or surgical procedures that would influence the lower limb, not undergoing treatment/physical therapy in the lower limbs, who do not use a plantar orthosis or other devices for the treatment of Hallux Valgus or who stopped using them during the study period and reported no pain or immediate discomfort when they tried the device under study.

2.2. Stage of the Study, Procedures, and Tools

The study comprised 3 fundamental stages, as shown in Figure 1.

The first step was to confirm the lateral deviation of the big toe over the first metatarsal bone by fifteen degrees or more. With the participant in the standing position, barefoot and with feet parallel, a large Carci^® goniometer was positioned in the medial region of the foot with the metatarsophalangeal joint as a reference for the axis of the goniometer. Members were elected respecting the inclusion criteria of the study. All assessments were performed by the same researcher.

Each participant legible for the study was integrated into the 2nd stage. In this stage, the participant should fulfill the Brazilian version of the Foot Functional Index questionnaire (FFI) to quantitatively measure the index of foot dysfunction caused by Hallux Valgus. The FFI questionnaire is a tool developed in English by Bddiman-Mak et al. [21] and adapted and validated for the Brazilian Portuguese language [22]. The FFI questionnaire integrates 23 items divided into functional and pain assessment scales. Since it is not the objective of this manuscript, it will not be developed here. Martinez et al. [22] present a complete explanation of the FFI method. To compare the results, the instrument was applied before and after using the device under study.

2.3. Hallux (Mis)Alignment Evaluation

To carry out the evaluation of the hallux (mis)alignment, computerized photogrammetry was used. Despite the growing number of studies that use computerized photogrammetry [23,24,25,26,27], there are no reference values on its use in the evaluation of Hallux Valgus; likewise, few studies were found that verified the reliability of these measurements. One this study found [28] refers to the evaluation of Hallux Valgus through the use of self-photography. Thus, verifying the repeatability of the hallux measurement becomes essential to control the margins of error allowing a reliable measurement. Applying a mixed approach adapted from the methods used by Nix et al. [28] and Yamaguchi et al. [29], four areas on the feet of the participants were marked with circular stickers (3 mm) to determine and standardize the verification of the angle between the first toe and the first metatarsal in the anterior region of the first ray (head and base of the proximal phalanx of the hallux and head and base of the first metatarsal), as explained by Godoy [30] (Figure 2). For placing adhesives on anatomical points, techniques and methods of palpation were carried out, using a digital caliper with a level device attached. Thus, it was possible to position the points in the upper region of the anatomical structures in the transverse plane and reduce the margin of error in the reassessment.

A wooden platform measuring 40 × 40 × 10 cm (W × L × H) was used to position the participants in the standing position with the lower limbs spread out in the usual way. To standardize the recording of images, a Webcam Logitech^® 920 was used, fixed on a tripod 45 cm away from the anterior region of the foot (Figure 3), with a leveling control in relation to the ground to allow the perpendicular capture of images analyzed in a transverse plane, reducing the margins of error in the reproduction of evaluations.

Kinovea^® software was used to analyze the collected images (Figure 4), which is recognized for having good reliability with results over 90% in the biomechanical analysis [26].

2.4. Intra- and Interexaminer Reliability of Hallux (Mis)Alignment Assessment Procedures

For intraexaminer reliability, one examiner performed all measures explained in Section 2.3. Two analyzes were carried out on the same random subsample of 20 feet, with an interval of 5 days to determine the margin of error in verifying the angle of (mis)alignment of the hallux, in the fixation of the adhesives in the anatomical structures, and in the projection of the angles in the Kinovea^® software (which will be identified as Situation A). Additionally, the assessment of the (mis)alignment of the hallux through the Kinovea^® software based on the images recorded was performed twice in a random subsample of 20 images, with an interval of 14 days (which will be identified as Situation B). Considering both situations, A and B, used in this investigation, we intend to verify the repeatability of the hallux measurement considering all procedures and only the analysis of the images, respectively.

For interexaminer reliability, two examiners performed all measures, explained before, in a subsample of 20 feet. Both situations (A and B) were performed as well. Both examiners have more than 10 years of experience.

2.5. Data Analysis

For data processing, version 28 of the Statistical Package for the Social Sciences (SPSS^©) was used and descriptive analyses were performed using measures of dispersion (amplitude and standard deviation) and location (frequency, mean, median, or percentage) to summarize the results of both the sample characterization and the (mis)alignment of the segment.

To verify intraexaminer reliability, an intraclass correlation coefficient (ICC) and paired t-test were applied with a significance level of p ≤ 0.05. Additionally, a Bland–Altman analysis was performed to assess the agreement between the two intrarater assessments for hallux alignment, to visualize bias and error, in addition to the presence of outliers. The ICC interpretation adopted the following assumptions [31]: <0.4, poor reliability; 0.4–0.59, reasonable reliability; 0.6–0.74, good reliability; and ≥0.7, excellent reliability.

One of the greatest challenges for studies that want to assess behavioral changes or angular changes as an influence of something is differentiating between the real changes that occurred and random measurement error. While the traditional p-value-based analyses of difference (e.g., t-tests, ANOVAs) provide information on the statistical significance of a reported change in scores obtained, they do not inform as to the likely cause or origin of that change, that is, the contribution of both real modifications and random measurement error to the reported change. The differentiation between real change and random measurement error was carried out through the utilization of the statistics of the standard error of measurement (SEM) and minimal detectable change (MDC), both of which are considered best practices in the clinical domain and therefore have been widely employed in the clinical literature, as reported by [32]. The SEM was estimated from the standard deviation of a sample of scores at baseline (${Sd}_{baseline}$) and a test–retest-reliability index (ICC) of the measurement instrument or test used. The SEM was computed according to Equation (1):

$$\mathrm{S}\mathrm{E}\mathrm{M}={Sd}_{baseline}\times \left(\sqrt{1-ICC}\right)$$

(1)

Confidence intervals (CIs) were also calculated for 95% based on Equation (2):

$${\mathrm{C}\mathrm{I}}_{95\%}=1.96\times \mathrm{S}\mathrm{E}\mathrm{M}$$

(2)

The MDC was estimated from the SEM and a degree of confidence, usually 95%, as reported by [32]. The MDC was computed according to Equation (3):

$${\mathrm{M}\mathrm{D}\mathrm{C}}_{95\mathrm{\%}}=\mathrm{S}\mathrm{E}\mathrm{M}\times 1.96\times \sqrt{2}$$

(3)

In addition to calculating the MDC, the use of SEM also allows the results to be expressed in the original measurement of the test (in this study, degrees), thus facilitating its interpretation [33]. The MDC value can be considered as the minimum value of change that needs to be observed to consider that a change is real and not due to chance. Knowing this value is necessary to understanding the meaning of the differences found in the alignment after using the foot piece.

2.6. Samples and Participants

For study purposes, the sample consists of the number of feet with confirmed Hallux Valgus orthopedic dysfunction. A subsample of 20 feet was included in each situation (A and B). For Situation A, 10 participants were invited to repeat the procedures after 5 days. The choice of participants in Situation A was random and depended on personal convenience in terms of the availability of time to dislocate to the physiotherapy clinic and repeat all procedures. The only criteria used to invite participants was that they should have deformities in both feet. For Situation B, a complete random subsample of 20 feet was obtained from databases. The only criteria used was to select an equal number of left and right feet.

A total of 30 participants was part of this study; 10 participants were randomly selected to participate in Situation A of the study and another 20 to participate in Situation B.

Table 1. The main characteristics of the sample by the situation analyzed (A and B) for intra- and interrater analyses.

	Situation	Sex (n)		Feet (n)		Age (n)		IMC
		F	M	Right	Left	Mean	Sd	Mean	Sd
Intrarater analysis	A	6	4	10	10	48.4	13.34	21.48	1.73
	B	13	7	10	10	39.15	15.37	24.24	3.15
Interrater analysis	A	10	10	10	10	47.6	13.12	23.85	2.75
	B	13	7	10	10	39.15	15.37	24.24	3.15

shows the main characteristics of the sample (sex, feet, age, and BMI) by the situation analyzed.

3. Results

Intra- and Interexaminer Reliability Results

As reported before, a random subsample of 20 feet was used to assess intra- and interexaminer-reliability results. Both, the intra- and interexaminer-reliability assessments were performed for the following two situations:

• Situation A—after repeating all the procedures from fixing the adhesives to analyzing the images using Kinovea^® software (with an interval of 5 up to 7 days);
• Situation B—assessment of hallux (mis)alignment using Kinovea^® software based on previously saved images (with an interval of 14 days).

Table 2. Mean value and standard deviation (Sd) of the data collected in both evaluated moments—initial and final—by the same examiner.

Test	Situation	Initial Evaluation		Final Evaluation		p-Value
		Mean	Sd	Mean	Sd
Hallux (mis)alignment (degrees)	A	21.92	4.55	22.04	4.91	0.249
	B	22.44	4.72	22.48	4.74	0.795

presents the mean value and standard deviation of the data collected in the first moment (initial evaluation) and in the second moment (final evaluation), considering the two situations analyzed (Situations A and B), by the same examiner.

Table 3. Mean value and standard deviation (Sd) of the data obtained by each examiner.

Test	Situation	Examiner 1		Examiner 2		p-Value
		Mean	Sd	Mean	Sd
Hallux (mis)alignment (degrees)	A	21.92	4.55	22.49	3.91	0.387
	B	22.44	4.72	22.52	4.32	0.841

presents the mean value and standard deviation of the data collected by each examiner (Examiner 1 and Examiner 2), considering the two situations analyzed (Situations A and B).

Table 4. Results of the intraclass correlation coefficient (ICCs) and confidence intervals (CI_95%) for both situations analyzed—Situations A and B—for intra- and interexaminer reliability.

Test		Situation	ICC	CI95%
Hallux (mis)alignment (degrees)	Intraexaminer reliability	A	0.998	(0.994–0.999)
		B	0.995	(0.988–0.998)
	Interexaminer reliability	A	0.871	(0.680–0.949)
		B	0.967	(0.917–0.987)

shows the intraclass correlation coefficient results and confidence intervals for both situations analyzed (Situations A and B).

In Situation A, the intra- and interexaminer ICCs, repeating all procedures for location, fixing the adhesives, and analyzing the images using Kinovea^® software, were 0.998 and 0.871, respectively; therefore, the level of intra- and interexaminer reliability in the evaluation and reassessment was classified as excellent [24,31]. The paired sample t-test revealed that there were no significant statistical differences between the hallux (mis)alignment when assessed by the same examiner at different moments (t(19) = 1.189; p = 0.249) or by a different examiner (t(19) = −0.886; p = 0.387). The Bland–Altman analyses demonstrate the difference and the mean between the first and second assessments made by the same examiner (intraexaminer evaluation) and between both examiners (interexaminer evaluation); most measurements were distributed within acceptable limits of variation, indicating that the two evaluations of angular measurement made by the same evaluator or by different evaluators tend to produce similar results (Figure 5 and Figure 6, respectively).

In Situation B, the intra- and interexaminer ICCs, repeating only the analysis of the 20 images randomly selected and assessed, with an interval of 14 days, using Kinovea^® software, were 0.995 and 0.967, respectively, which means excellent intra- and interexaminer reliability [24,31]. The paired sample t-test revealed that there were no significant statistical differences between the hallux (mis)alignment when assessed by the same examiner at different moments (t(19) = 0.264; p = 0.795) or by a different examiner (t(19) = 0.203; p = 0.841). The Bland–Altman analyses demonstrate the difference and the mean between the first and second assessments made by the same examiner (intraexaminer evaluation) and between both examiners (interexaminer evaluation); most measurements were distributed within acceptable limits of variation, indicating that the two evaluations of angular measurements made by the same evaluator or by different evaluators tend to produce similar results (Figure 7 and Figure 8, respectively).

Table 5. Statistics of the SEM, CI_95%, and MDC_95% for both situations—A and B—analyzed for intra- and interexaminer reliability.

		Situation	SEM	CI95%	MDC95%
Hallux (mis)alignment (degrees)	Intraexaminer reliability	A	0.20	±0.392	0.55°
		B	0.33	±0.65	0.91°
	Interexaminer reliability	A	1.63	±3.19	4.51°
		B	0.86	±1.68	2.38°

shows the results obtained for the standard error of measurement (SEM), respective confidence intervals (CI_95%), and the minimal detectable change (MDC) for both situations analyzed (Situations A and B) for intra- and interexaminer reliability.

As we can see in Table 4 and Table 5, the ICCs values are quite high, and the SEM values are quite low, respectively, for intraexaminer reliability. These values are indicative of excellent intraobserver reliability [31,34], revealing a low probability of random and systematic error [35]. For interexaminer reliability, the results were not so favorable, with SEM values between 1.63 and 0.86 for Situations A and B, respectively.

The last column in Table 5 shows the MDC of the test which does not occur due to error. Despite this value not being relevant for verifying the reliability of the test, it is nevertheless crucial to the clinical approach [34]. Thus, and according to the data obtained, a value of change observed in a user in a posttreatment situation that is lower than the values obtained is not distinguishable from the measurement error, that is, there was no change in the evaluated parameter. Likewise, if the value obtained is equal to or above the values referred to in the last column of the table, there was a true change in (mis)alignment assessed by the test and not a measurement error. As we can see, the MDC value was higher for Situation B in the intraexaminer assessment and for Situation A in the interexaminer assessment. Considering that Situation A represents a more complex situation, due to having to repeat all the procedures from the placement of the adhesives through to the positioning of the feet and the respective image collection, we consider that this situation presents more potential for the presence of errors. Even so, it is curious how it showed a better ICC than Situation B and better SEM and MDC results considering the intraexaminer assessment.

4. Discussion

High intra- and interexaminer reliability was observed in the measurements of hallux (mis)alignment with computerized photogrammetry using Kinovea^® software. Concerning the results obtained, it can be concluded that photogrammetry is reliable when performed by the same examiner or by different examiners. Considering the results obtained by the intraexaminer-reliability assessment, in Situation A (the most complex situation), we can conclude that the use of the foot piece, used in our broader study, will improve the (mis)alignment of the hallux if the difference obtained between both measures (before and after using the foot piece) is higher than 0.55°. The results obtained in our study were similar to the results reported by Yamaguchi et al. [29] in terms of intrarater reliability (ICC = 0.998) but better considering the SEM and MDC_95% (0.6° and 1.7°, respectively). Additionally, our results were better than the results reported by Nix et al. [28], where the intrarater reliability for digital photographs was ICC = 0.97.

Considering the results obtained by the interexaminer-reliability assessment for Situation B, the results were similar to those reported by Yamaguchi et al. [29] (ICC = 0.99) but worst considering the SEM and MDC_95% (0.6 and 1.7°, respectively); for Situation A, the results obtained in our study were globally the worst. Additionally, our results were quite similar to the results reported by Nix et al. [28], where the interrater reliability for digital photographs was ICC = 0.96.

The good results obtained, particularly in intraexaminer reliability, may be related to the high experience of the researchers (more than 10 years), which was also presented in research carried out by Jones and Curran [10]; in their research, the authors compared the intrarater and interrater evaluations of experienced examiners (who showed good reliability (ICCs > 0.953) in both situations) and inexperienced examiners (who showed fair (ICC= 0.322) to good reliability (ICC = 0.597)) between the evaluations, respectively, when they evaluated the dorsiflexion movement of the first metatarsophalangeal joint at the first ray of the foot using goniometry.

A high intertester reliability was also obtained using radiography examination methods in the hallux abductus angle, intermetatarsal angle, and interphalangeal abductus angle with ICCs equal to 0.96, 0.87, and 0.77, respectively [36]. Since our results were better than those reported in this research, the use of a method that does not expose the patient to radiation and that can be performed in an office without the need of a large structure for diagnosis seems to be important, even for monitoring the evolution of the pathology.

The degree of Hallux Valgus can also be assessed using the visual estimation method, which despite bringing satisfactory results in some intraexaminer comparisons, as presented by Jones and Curran [10], the results are still below the rates presented in this research (ICC < 0.8).

5. Conclusions

The main objective of this work was to verify the reliability of intra- and interexaminer evaluations in the assessment of hallux (mis)alignment using computerized photogrammetry. Two examiners analyzed an aleatory sample of 20 feet to measure the (mis)alignment of the hallux repeating the analysis 5 to 7 or 14 days later (Situations A and B, respectively). Computerized photogrammetry using Kinovea^® software showed the inter- and intrarater reliability for measuring the hallux alignment angle, thus being a tool reliable for use in the clinical practice of our study. The good results obtained may be related to the high experience of the researchers (more than 10 years). From a clinical point of view, the results of this study indicate that the results equal to or above 0.51° for the degree of (mis)alignment of the hallux, after using the foot piece, are due to a true improvement and not to a measurement error; this is valid when the assessment of the hallux (mis)alignment is carried out by the same examiner.

The use of photogrammetry with Kinovea^® software to assess Hallux Valgus (mis)alignment proved to be efficient as a resource for research involving the application of methods that require a comparison of results, as it presented an excellent ICCs in the intraexaminer-reliability assessment (Situation A: 0.998 and Situation B: 0.995) and interexaminer-reliability assessment (Situation A: 0.871 and Situation B: 0.967); in addition, it seems to be a good alternative because it is not invasive and does not expose individuals to the radiation in radiological examinations, especially in treatments that require follow-up, and can be applied in the care space itself or in places that do not require the accommodation of large equipment.