A Postoperative Displacement Measurement Method for Femoral Neck Fracture Internal Fixation Implants Based on Femoral Segmentation and Multi-Resolution Frame Registration

Abstract

Femoral neck fractures have a high incidence in the geriatric population and are associated with high mortality and disability rates. With the minimally invasive nature, internal fixation is widely used as a treatment option to stabilize femoral neck fractures. The fixation effectiveness and stability of the implant is an essential guide for the surgeon. However, there is no long-term reliable evaluation method to quantify the implant’s fixation effect without affecting the patient’s behavior and synthesizing long-term treatment data. For the femur’s symmetrical structure, this study used 3D convolutional networks for biomedical image segmentation (3D-UNet) to segment the injured femur as a mask, aligned computerized tomography (CT) scans of the patient at different times after surgery and quantified the displacement in the specified direction using the generated 3D point cloud. In the experimental part, we used 10 groups containing two CT images scanned at the one-year interval after surgery. By comparing manual segmentation of femur and segmentation of femur as a mask using neural network, the mask obtained by segmentation using the 3D-UNet network with symmetric structure fully meets the requirements of image registration. The data obtained from the 3D point cloud calculation is within the error tolerance, and the calculated displacement of the implant can be visualized in 3D space.

1. Introduction

Femoral neck fractures (FNFs) constitute a common surgical trauma worldwide, with a patient population of 1.6 million per year and growing annually [1]. Aging people have a high incidence of femoral neck fractures due to their tendency to low-energy fall and high prevalence of osteoporosis prevalence [2]. Previous epidemiological studies have indicated that the incidence is approximately twice as high in females as in males [3]. Particularly with the aging of the population in all countries, the prevalence of femoral neck fractures increases over the years, accompanied by high mortality and disability rates [4]. It is expected that by 2050, the number of patients with FNFs will exceed 6 million worldwide [5], which will account for a significant proportion of orthopedic work and constitute a health care burden significant in terms of socio-economic impact.

Plastic surgery literature now agrees that cannulated screws and the dynamic hip screw (DHS) are the most commonly used treatment modalities for stable fractures classified as Garden I or II [6]. The internal fixation is minimally invasive compared to total hip arthroplasty (THA) and can significantly reduce the postoperative hospital stay [7,8]. Nevertheless, the use of internal fixation hip implants is associated with medium to long-term surgical complications such as chondrolysis, osteonecrosis, avascular necrosis, and mechanical failure of the implant, accompanied by a high rate of rehospitalization and reoperation [9,10,11].

Much literature has been presented to analyze and study internal fixation results with hip implants. Marco Bigoni et al. analyzed postoperative patient mortality, complications, and reoperation rates statistically by following 244 elderly patients who underwent internal fixation with implants over 10 years from January 2008 to December 2018 [12]. The results indicate that internal fixation effectively stabilizes femoral neck fractures, and the patient’s postoperative mortality and complication rates are only associated with individual patient comorbidities. Ref. [13] has conducted mechanical simulations of implants for the treatment of stable femoral neck fractures using synthetic bones. That study evaluated hip implants’ ability with different configurations by limiting relative motion at the fracture site using a manual hydraulic press to simulate hip compression, i.e., to measure the amount of resistance of the pins. In the experimental part, the femur’s stability was compared between two internal fixation systems by simulating force by the spine to the femur, with one of the internal fixation systems using two parallel cannulated screws and the other framework using three cannulated screws forming a pyramidal shape. In [14], the authors modeled the fracture with segmented human cadaveric femora. Using the servo-hydraulic device to simulate the force components in the frontal and sagittal planes in human gait motion, they measured the displacement of both internal fixation systems at different loads. Moreover, Jiantao Li et al. proposed a method analyzing five different implant configurations using a finite element approach [15]. They evaluated the mechanical properties such as stress and strain in different postoperative weight-bearing states of patients by computer simulation. In contrast, ref. [16] analyzed the stresses applied to the femur and related tissues under different fracture types after internal fixation of femoral neck fractures from a biomechanical perspective. It assessed the mechanical stability of mainstream fixation techniques such as cannulated screws, hip screw systems, proximal femur plates, and cephallomedullary nails.

Among these evaluation methods, the use of statistics to evaluate treatment effectiveness cannot provide specific reasons for pins’ failure and requires a large number of experimental samples as a basis for analysis. Mechanical simulations using synthetic bones, cadaveric femora, and finite element-based 3D models all use a simulated stress environment. Obviously, the simulated environment is different from the patient’s real behavioral movement after surgery and can only use a limited number of parameters and simulated stresses in a single scenario. What is more, it takes months or years of data collection to assess the effectiveness of internal fixation on patients accurately, which is very difficult for mechanical simulation experiments.

Computed tomography scans provide realistic information about the displacement of implants in the patient’s body after surgery. The acquisition of computerized tomography (CT) scan does not place an additional load on the patient’s daily movements. Traditionally, however, the measurement of fracture displacement or implant displacement in the body is mainly performed by the surgeon in CT medical images, selecting a reference, measuring the target object’s distance relative to the reference in different CT sequences, and calculating the displacement. In [17], the authors acquired two sets of chest computed tomography scans, CT1 and CT2, at an interval of 84 days. The distance of the rib’s outer cortex was measured in three planes, using the site of the rib fracture as a reference point to verify that rib fractures become more displaced over time. A similar approach was used by Bugaev and Nikolay et al. to measure axial displacement, sagittal displacement, and coronal displacement, and then they calculated the (Euclidean) distance using the Pythagorean formula [18]. In the literature [19], digital calipers and a protractor were used to measure foot fractures within the CT images’ coronal plane. This manual method of measuring implant displacement is complex and time-consuming to perform, and the accuracy of the measurement results is susceptible to subjective influence by the surgeon.

This paper presents a method for evaluating internal fixation implants based on CT medical image analysis. The Hansson pins system is used as the study object to quantify the implant’s movement after surgery and to analyze the effectiveness of the internal fixation system. Our method allows us to analyze the movement of the implant in a specific direction with low impact on the patient, based on real patient data.

2. Materials and Methods

2.1. Creation of Data Sets

In this paper, we constructed Dataset A and B for training the segmentation models of the left and right femur, respectively, and Dataset C for femur registration and displacement measurement of the implant. In the segmentation femur stage, this paper uses 3D-UNet, a semantic segmentation framework, which differs from the instance segmentation framework; 3D-UNet can only classify pixels in an image. In our study, if the dataset used for training does not partition anatomically left and right, the trained model will also label the healthy femur that is symmetrical with the injured femur as the target object. While in Section 2.2, the mask required for fine registration is only the femur that is fixed with the implant. To avoid registration failure, we construct the left femur and right femur datasets separately and train two segmentation models for the left and right femurs.

2.1.1. Dataset A and B

High-quality medical image collection is challenging due to the need to protect patient privacy and data security. Additionally, the annotation of images requires a large number of experienced surgeons; therefore, it is quite costly to integrate high-quality annotated data, limiting the development of medical image segmentation algorithms, which has become an accepted fact in the field of medical image research [20]. Consequently, no publicly available medical image datasets specifically for femoral fractures are available. However, some of the images in the dataset that are currently publicly available for other research purposes contain the femur we need. In these raw CT images, in addition to containing the entire left and right femur, adjacent tissues and organs such as the pelvis, spine, and ribs are also included. Which can considerably increase the GPU’s memory usage, making it necessary for us to extract regions of interest (ROIs), i.e., image information that contains only femur parts. Furthermore, we need to label the femur information as ground truth for training manually.

We used Pelvic Reference Data, a free dataset used for commercial, scientific, and educational purposes, as the original data [21,22]. This dataset was created to serve as a reference for the rigid registration of clinical images. The process of removing irrelevant information from CT images is shown in Figure 1.

Figure 2 displays the coronal images viewed from the anterior side. The complete image (b) is shared between the two datasets, and as a complement, each of the two datasets also contains unique left femur (a) and right femur (c). In this way, we can simply expand the dataset and accurately distinguish between left and right femur when segmenting CT images that contain a complete structure. By removing the data with poor image quality, in Dataset A, we ended up with 112 images of the left femur, of which 84 images were used for training, and 28 images formed the validation set. Similarly, 113 images were collected in Dataset B. A total of 85 images were used for training, and 28 images were used for validation. Subsequently, manual annotation of the femur for the constructed dataset.

2.1.2. Dataset C

In Dataset C, we collected 10 cases from March 2012 to January 2015 provided by Hyogo Prefectural Awaji Medical Center. Each case underwent two CT scans after internal fixation surgery and one year later.

All cases associated with intracapsular fractures and the patient’s age ranged from 69 to 65 years. These cases all used the Hansson Pin System (Hansson Pins, Swemac, Linköping, Sweden) [23]. Hansson pin is a 6.5-diameter unthreaded nail with various specifications and the length ranging from 75 to 150 mm. The implant has an integrated hook on each pin’s tip that deploys to provide rotational stability of the femoral head following insertion. Typically, using two pins for fixing and screwing out the hook pins when fixing (Figure 3).

Table 1. The length of pins and information of patients in each case.

Case No.	Length of Pins (mm)		Sex	Age	Location of Fracture
	Proximal	Distal
1	80	90	Female	78	Left femur
2	80	90	Female	79	Left femur
3	70	85	Female	90	Left femur
4	80	90	Female	65	Left femur
5	85	95	Female	76	Left femur
6	90	100	Female	64	Right femur
7	80	95	Female	81	Right femur
8	80	90	Female	80	Right femur
9	80	90	Female	67	Right femur
10	75	90	Female	85	Right femur

lists information such as the patient’s age and the pins used.

Figure 4 illustrates the different situations during a CT scan. Generally, in the examination during the recovery period from surgery, the CT data is cropped and transformed to provide a clearer and faster diagnosis, that is, a localized CT image of the left or right femur. In contrast, the complete femur and pelvis are sometimes retained in the CT image to visualize information around the fracture site. To obtain high robustness of the trained femur segmentation model and avoid the femur segmentation failure in the next step, we convert all images to the right, anterior, superior (RAS) anatomical coordinate system and use empty images to supplement the missing parts.

2.2. Registration of the Femur

For measuring the postoperative displacement of the Hansson pins, it is necessary to select a reference with rigid morphological properties. Depending on the setting in which the Hansson pin is used, the shape and X-ray absorptivity of the proximal femur does not change unless structural damage occurs, which satisfies the requirements for use as a reference. Femur image registration is the transfer of images containing femur and implants from the same patient at different times, different scanning devices, different scenes, and others to the same spatial coordinate system and strict alignment with femur as the reference. After this transformation, it is possible to measure the position information of Hansson pins at different times.

Typically for CT image alignment, at least two sets of images containing the same target information are required. The matching criterion is to achieve maximum similarity between the fixed and floating images, a combination of feature space matching algorithm, spatial search algorithm, optimization algorithm, and similarity measure. Commonly used rigid medical image registration algorithms include point set matching based, a genetic algorithm-based, and mutual information-based 3D image registration. The alignment of the femur image in this paper is rigid, i.e., no affine transformation of the graph is required, which requires high alignment accuracy.

We choose a multi-resolution framework, which uses negative mutual information as the similarity metric function. The framework contains four parts: spatial transformation module, similarity metric module, interpolator, and optimizer, as shown in Figure 5.

This framework is widely used in medical image registration due to its high accuracy, robustness, and fast alignment speed. Its core idea is to input the images in the fixed image pyramid and floating image pyramid into the framework layer by layer. An optimizer is used to drive the alignment process, and an interpolator is used to map the floating images into the new coordinate system [24,25].

The method of negative mutual information in the similarity measurement module was proposed by Mattes et al. in 2003. Mutual information forms a continuous histogram estimate of the underlying grayscale image using the Parzen window virtually eliminates the effect on the similarity calculation due to interpolation quantization and binary data discretization during the image space transformation. Image registration can be seen essentially as the process of minimizing the negative similarity function. When we define the set of discrete grayscales for a fixed image as $S_F$ and the set of discrete grayscales for a floating image as $S_R$, the negative similarity function model can be expressed as

$$S\left(\mu \right)=-{{\displaystyle \sum }}_{l\in S_F}{\sum }_{\kappa \in S_R}\frac{p\left(l,\kappa ;\mu \right){\log }_2\left(p\left(l,\kappa ;\mu \right)\right)}{p_F\left(l;\mu \right)p_R\left(\kappa \right)}.$$

(1)

where $p\left(l,\kappa ;\mu \right)$ is the joint distribution function, which can be calculated from the values of the Parzen window cubic spline and zero-order B-splines. $l,\kappa$ are the grayscale values in $S_F$ and $S_R$, respectively. $p_F$ is the floating image edge probability distribution and $p_R$ is the fixed image edge probability distribution. $\mu$ is the image transformation parameter.

Mattes mutual information function has continuously differentiable characteristics. The optimizer needs to meet the conditions of high speed, low resource consumption, and high robustness to obtain the optimal spatial transformation parameters. Therefore, we choose a multi-resolution algorithm as the optimization search strategy. In this section, we select the patient’s CT images before the one-year recovery period as fixed images and the CT data after the one-year recovery period as floating images to construct the Gaussian pyramid. Gaussian pyramids are constructed using Gaussian smoothing and downsampling to create a series of images of varying sizes. These images form a pyramid model from large to small and from bottom to top, as the fixed image and floating image modules present in Figure 5.

We take the original fixed image and floating image as the bottom level 0 of the pyramid, and after the discrete low-pass filter calculation, an upper level 1 of the upper pyramid is obtained, and the iterative process is repeated. For getting the Gaussian pyramid layer $G_i$, it is necessary to perform Gaussian low-pass filtering on its previous level image and then downsample it by inter-row and inter-column, usually for removing pixels in even rows and even columns of the image. The mathematical expression is

$$\begin{array}{cc}{G}_l\left(i,j\right)& ={\displaystyle \sum }_{m=-2}^2\sum _{n=-2}^2\omega \left(m,n\right)G_{l-1}\left(2i+m,2j+n\right)\\ & \left(1\le l\le N,0\le i\le R_l,0\le j\le C_l\right)\end{array}.$$

(2)

where $N$ is the number of Gaussian pyramid layers, $R_l$ and $C_l$ are the number of rows and columns of the image of the ith layer of the Gaussian pyramid, respectively; $\omega \left(m,n\right)$ is a two-dimensional 5 × 5 window function with the expression as

$$\omega =\frac{1}{256}\left(\begin{array}{ccccc}1& 4& 6& 4& 1\\ 4& 16& 24& 16& 4\\ 6& 24& 36& 24& 6\\ 4& 16& 24& 16& 4\\ 1& 4& 6& 4& 1\end{array}\right).$$

(3)

When two 3D images containing femur are input to the framework, the registration range defaults to the whole image, and the optimal spatial transformation parameters obtained after iteration corresponds to the global search space. In other words, although the spatially transformed floating image matches the fixed image, the purpose of our study is to align the femur. The tissues or organs around the pelvis, such as the pelvic bone, will significantly affect the similarity calculation. In this case, the spatial transformation mostly matches the pelvis’s position rather than the proximal femur. Therefore, we use the above framework to perform coarse alignment on the images containing the femur. After that, we use the femur’s mask and perform fine registration on the region containing the only femur.

2.3. D-UNet Framework

To extract the region of interest, we segmented the proximal femur without the femoral head in the patient’s CT images using a model trained by the 3D-UNet framework.

3D-UNet is an end-to-end training model proposed by Özgün Çiçek et al., which is mainly used for semantic segmentation of medical images [26]. The model inherits the features of the 2D-UNet network by using encoder and decoder structures to extract features and recover the semantic feature maps into volumetric images with the same resolution as the original images. Compared with the 2D-UNet network, 3D-UNet uses the image interlayer information to ensure the continuity of mask changes in adjacent images. Moreover, different from the fully convolutional network, which only deconvolutes the feature map, 3D-UNet achieves the multi-scale feature recognition of image features by the symmetric structure of four downsampling and four upsampling, and the skip connection method, i.e., it fuses the shallow features of the same scale in the encoder and the in-depth features from the upsampling to avoid the loss of edge information. The downsampled low-resolution information provides contextual information to the target, and the upsampled high-resolution features improve the network’s ability to recognize edge information such as gradients. Figure 6 illustrates the 3D UNet network framework used in this paper. In the encoder structure, we set the network structure to 5 layers, and the number of channels in each layer is 16, 32, 64, 128, and 256, respectively.

This study uses dice loss as the loss function, which is widely used in neural networks for medical image segmentation. Dice score coefficient (DSC) is used to evaluate the degree of overlap of two samples, and in binary semantic segmentation, the segmentation effect is evaluated based on ground truth [27]. Hence, we can maximize the overlap of two samples using $1-DSC$. Dice loss was first proposed and used in the VNet framework by Milletari et al. and is defined as

$$L_{dice}=1-\frac{2*{{\displaystyle \sum }}_{i=1}^N{p}_i{g}_i}{{{\displaystyle \sum }}_i^N{p}_i^2+{{\displaystyle \sum }}_{i=1}^N{g}_i^2}.$$

(4)

where $p_i$ and $g_i$ represent the predicted label and ground truth of each voxel, respectively, during the training process. $N$ is the number of voxels in the input image.

CT images vary depending on patient condition and scanner model. The output segmented femur is often accompanied by segmentation noise, as shown in Figure 7. The noise appears as small, separated coherent voxels. The model outputs segmented images in which the femur has the largest number of coherent voxels. To correct this problem of non-femur parts being incorrectly identified, we retain only the largest coherent components in the post-processing of the model output. The mathematical model can be expressed as

$$V_{\mathrm{opt}}=\arg \left(max\right)F\left(v\right).$$

(5)

where $F\left(v\right)$ is a function to calculate the maximum number of contiguous adjacent voxels.

2.4. Principal Component Analysis (PCA)

Based on the aligned CT images, the Hansson pins were reconstructed as a 3D model. We use the point cloud data to calculate the displacement distance after locating the endpoints of the pins. In this subsection, we apply the principal component analysis method to fit the pins’ axes, as shown in Figure 8. The point cloud size and the obtained axes are used to draw the outer envelope of the pins, and the point intersecting the envelope in the direction of the axes is the endpoint of the pins.

The principal component analysis is a multivariate statistical method commonly used for dimensionality reduction of multidimensional data [28]. The main principle is that the m-dimensional feature vector is mapped to the n-dimensional vector using the orthogonal transformation. This n-dimensional vector is an orthogonal vector constructed based on the original features, where the first vector is the direction with the most considerable variance in the original data.

The algorithm is:

• Input:

  •

  M-dimensional sample set $D=\left(x^{\left(1\right)},x^{\left(2\right)},\dots ,x^{\left(m\right)}\right)$;

  •

  The number of dimensions n to be dimensioned down to.

• Output: reduced-dimensional sample set $D\prime$.
• Steps:
  • Standardize all variables.

    $$x^{\left(i\right)}=x^{\left(i\right)}-\frac{1}{m}{{\displaystyle \sum }}_{j=1}^m{x}^{\left(j\right)}$$

    (6)
  • Calculation of covariance matrix.
  • Computes the eigenvectors and eigenvalues of the covariance matrix.
  • Select the largest n vectors normalized to form a new matrix $W$.
  • Transform the original matrix.

    $$z^{\left(i\right)}=W^T{x}^{\left(i\right)}.$$

    (7)
  • Output sample set $D^{\prime }=\left(z^{\left(1\right)},z^{\left(2\right)},\dots ,x{z}^{\left(m\right)}\right)$.

We define a single point in the point cloud as $P_i=\langle x_i,y,z_i\rangle$. The point cloud can be represented as a sample set $D=\left(P_1,P_2,\dots ,P_n\right)$, by inputting $D$ into the PCA algorithm above, we solve the feature vectors (${\epsilon }_1,{\epsilon }_2,{\epsilon }_3$) and form the orthogonal matrix $W=[{\epsilon }_1^{\prime },{\epsilon }_2^{\prime },{\epsilon }_3^{\prime }]$ after standard orthogonalization of each feature vector. ${\epsilon }_1^{\prime }$, ${\epsilon }_2^{\prime }$, and $,{\epsilon }_3^{\prime }$ are the main directions of the input point cloud, and the center of mass of the point cloud is taken as the origin of the new coordinate system. Using the point cloud’s center of mass as the origin of the coordinate system, ${\epsilon }_1^{\prime }$, ${\epsilon }_2^{\prime }$, and ${\epsilon }_3^{\prime }$ as the axes to form the coordinate system shown in Figure 8, where ${\epsilon }_1^{\prime }$ is the blue axis, ${\epsilon }_2^{\prime }$, is the green axis, and ${\epsilon }_3^{\prime }$ is the blue axis. The original point cloud data is converted to the new coordinate system using Equation (7). Subsequently, the red enclosing box is constructed according to the point cloud’s maximum and minimum values in the three directions of X, Y, and Z.

3. Experiment

3.1. Input of Images and Training of Segmentation Models

In general, CT images are obtained by data transformation of scanning devices according to the different X-ray beam absorption coefficients of human organs or tissues. The coordinate system origin and voxel spacing of the generated CT images vary depending on the scanning device. The 3D-UNet framework cannot extract voxel spacing as feature information, so we resample the data in Dataset A and B to the same voxel spacing, which is called the specified spacing. To minimize the loss of image detail due to the decrease in slices caused by increasing the specified spacing, we resampled the voxel spacing to 1.5 × 1.5 × 1.5 mm.

The images’ orientation may vary from case to case when the training data is input to the neural network. Here, we use the RAS coordinate system to ensure the input data orientation’s consistency. In addition, to simplify the feature extraction process of the 3D-UNet network and reduce the impact of femoral edge blur on the segmentation, we remove the information beyond 20HU-220HU from the CT images and convert the intensity range of the remaining information to (0,1). The processed images and labels are illustrated in Figure 9.

In this paper, the 3D-UNet network is set to five layers, so the size of each input image size should be a multiple of 16. Considering the image’s size after specified spacing resampling, we use cropping to augmentation the dataset in this section, and the cropped image size is (80, 80, 80). 3D-UNet has the features of requiring less training data and good results for medical image segmentation. Published literature indicates that the 3D-UNet network can still achieve good segmentation results when a small number of datasets are used for learning [26]. Thus, we did not use methods other than random crops for data augmentation.

The datasets A and B separately input into the 3D-UNet framework of the previous session for training. In this section, we implement the framework using MONAI, an open-source framework based on PyTorch [29]. The training yields the left femur segmentation model and the right femur segmentation model, respectively. The images in Dataset C were segmented using each of the two trained models according to the femur’s left and right positions with internal fixation using Hansson pins. The segmentation information of the injured femur was collected as mask data for the next step.

3.2. Registration of References

The CT data acquired postoperatively as fixed images, and the CT data scanned one year later as floating images were simultaneously inputted into the mutual information registration framework in the previous section for coarse alignment. The coarse alignment result is used as input, and the femur segmentation data from the previous step is used as masks for fine alignment of the images. To avoid the influence of mask edge information on the alignment, we use a kernel of size (27 × 27 × 27) to perform morphological collision processing on the mask.

To verify that segmenting femur by the model can provide an accurate mask for fine alignment, we add an experiment to manually labeled femur as the mask for fine alignment and compare the effect of both masks on fine registration.

The fixed image and floating image after fine registration are shown in Figure 10. In the two CT images after alignment, the proximal femur parts without the femoral head are overlapped and served as the reference. Observed in 3D space, the proximal pin and distal pin are in the same spatial coordinate system, and after 3D reconstruction, the voxel data of pins in CT are transformed into point cloud data, as shown in Figure 11.

3.3. Calculation of Implant Displacement

Using principal component analysis, we built the pins’ outer bounding box, obtained the coordinates of the endpoints of the four pins in Figure 11 separately, and calculated the displacement of the corresponding points. To analyze the displacement of Hansson pins more comprehensively, we take the axis of pins as the z axis, the direction of hook extension as y axis, and the direction of the cross product of y and z as x axis to establish a new coordinate system, as shown in Figure 12a,b are the proximal pin and distal pin coordinate systems, respectively. In the new coordinate system, the proximal pin and distal pin’s displacements in different directions are calculated.

3.4. Measurement of Implant Displacement Based on Conventional Methods

In this subsection, to verify our proposed measurement of implant displacement’s validity, we used the reference point-based distance measurement method described in [18]. We define the point on the outer surface of the Lesser trochanter farthest from the centerline of the medullary cavity as reference point A. As shown in Figure 13a–c, all steps to locate point A were performed entirely using the three planes in the CT images, and no 3D femur model was used to assist in localization. Figure 13d is used to assist in showing the position of the reference point in the CT image.

Point A is used as the origin, and a new coordinate system is formed with the intersecting lines of the planes, axial plane (green), coronal plane (blue), and sagittal plane (red), as shown in Figure 14. The coordinates of the target object are measured in the new coordinate system, and the Euclidean distance is calculated using the formula

$$d=\sqrt{{\left(x_1-x_2\right)}^2+{\left(y_1-y_2\right)}^2+{\left(z_1-z_2\right)}^2}.$$

(8)

where ($x_1$, $y_1$, $z_1$) and ($x_2$, $y_2$, $z_2$) represent the coordinates of the same point in the two CT image sequences, respectively.

To reduce the measurement error, we measure the coordinates of each point three times and take the average value to plug into Equation (8).

4. Results

Training Loss, Mean Dice, and Evaluation

Datasets A and B are used to train the left femur segmentation model, and the right femur segmentation model is derived from the same data source with the same image quality. The proportions of the images used for training and validation are also the same. The results of loss and the change of mean dice during the two models’ training are shown in Figure 15 and Figure 16, respectively.

The best metric was 0.9547 when the left femur segmentation model was trained with peoch of 580, and another model obtained a metric of 0.9450 when the peoch reached 552.

When the images in Dataset C are segmented according to the injury site using the two models that were trained, the left segmentation model’s evaluation metric is 0.85, and the evaluation metric of the right segmentation model is 0.81, using the manual labeling information as the ground truth. Figure 17 shows a comparison of the 3D images of the manually segmented proximal femur and the segmented femur using the model.

A multi-resolution framework using negative mutual information as the image similarity metric function is used to perform coarse registration of the images in each case, and their metric values are listed in

Table 2. Coarse registration metrics.

Left Femur	Case No.	1	2	3	4	5
	Metric Value	−0.271	−0.039	−0.285	−0.322	−0.467
Right Femur	Case No.	6	7	8	9	10
	Metric Value	−0.367	−0.259	−0.178	−0.097	−0.331

.

Figure 18 plots the coarse-aligned image with the 3D schematic of the femur. The yellow model presents the skeletal part of the fixed image, and the purple is the floating image. In the femur part, the two images do not entirely overlap; instead, some of the pelvic bones overlap.

In the experiments, the label obtained using the segmentation model and the manual annotation is used as the mask to fine align the images, respectively, and the mutual information from the two methods is presented in

Table 3. Results comparison of fine alignment using different masks.

Left Femur	Case No.		1	2	3	4	5
	Metric Value	Segmentation Model	−0.306	−0.065	−0.364	−0.346	−0.308
		Manual Labeling	−0.289	−0.046	−0.357	−0.370	−0.312
Right Femur	Case No.		6	7	8	9	10
	Metric Value	Segmentation Model	−0.337	−0.180	−0.256	−0.270	−0.339
		Manual Labeling	−0.360	−0.240	−0.231	−0.259	−0.342

. For the images in the same case, there is no significant difference in the precision-aligned metric value obtained when different masks are selected.

Figure 19 shows the 3D schematic diagram of the floating image obtained from the two experiments, where the yellow model is the bone in the fixed image, the green model is the result of alignment with the label generated by the segmentation model as the mask, and the red model is the result of manual labeling as the mask.

The fine-aligned CT images were transformed by 3D reconstruction to obtain the point cloud data, as in Figure 20.

To ensure that the 3D point cloud data can represent the actual size and accurate displacement calculation of Hansson pins, we compared the pins’ actual length with the measured length of the point cloud data, and the results are reported in

Table 4. Comparison of the actual length of Hansson pins with the point cloud data.

Fracture Site	Case No.	Proximal Pin (mm)			Distal Pin (mm)
		Actual Length	Fixed Images	Floating Images	Actual Length	Fixed Images	Floating Images
Left Femur	1	80	80.38	82.19	90	90.38	90.39
	2	80	80.28	80.46	90	90.31	90.14
	3	70	70.07	69.49	85	86.96	85.75
	4	80	81.07	81.32	90	90.98	90.10
	5	85	86.77	84.93	95	96.36	96.46
Right Femur	6	90	90.24	91.98	100	101.29	101.32
	7	80	79.84	79.89	95	95.29	95.00
	8	80	80.43	79.31	90	90.51	89.86
	9	80	80.61	79.38	90	90.82	89.27
	10	75	75.46	75.44	90	90.80	90.94

. Each case contains two sets of CT images, fixed images, and floating images; all the results are presented. The error between the length of the pins calculated by the point cloud and the actual length is within 2 mm.

Table 5. Hansson pins displacement measurement based on 3D point clouds.

Fracture Site	Case No.	Proximal Pin (mm)			Distal Pin (mm)
		Actual Length	Top Movement	Bottom Movement	Actual Length	Top Movement	Bottom Movement
Left Femur	1	80	19.49	17.44	90	16.51	16.11
	2	80	3.18	3.36	90	4.04	3.83
	3	70	0.49	0.26	85	0.55	0.88
	4	80	4.55	4.06	90	4.93	2.47
	5	85	7.62	4.78	95	7.47	7.58
Right Femur	6	90	8.41	8.99	100	8.87	10.59
	7	80	0.68	0.90	95	1.06	0.78
	8	80	1.98	0.93	90	1.25	0.82
	9	80	1.46	0.46	90	2.11	0.78
	10	75	0.36	0.39	90	1.96	2.09

shows the displacement distances of the two endpoints of pins calculated using the point cloud data. Additionally, as discussed in Section 3.3, a new spatial coordinate system was established with the fixed image pins as the reference. We converted the displacements to the new coordinate system for calculating Hansson pins’ displacement in a specific direction. The displacements of the pin’s endpoints in the three directions in space are listed in

Table 6. The displacement of Hansson pins’ endpoints in each direction after coordinate transformation.

Fracture Site

Case No.

Proximal Pin Displacement (mm)

Distal Pin Displacement (mm)

Top Endpoint

Bottom Endpoint

Top Endpoint

Bottom Endpoint

x axis

y axis

z axis

x axis

y axis

z axis

x axis

y axis

z axis

x axis

y axis

z axis

Left Femur

1

7.76

−7.11

−16.39

−5.54

3.10

−16.34

−1.59

−3.19

−16.22

−0.06

2.78

−15.98

2

0.23

−0.16

−3.18

−0.26

0.01

−3.36

0.00

−0.79

−3.97

−0.44

0.40

−3.79

3

−0.06

−0.06

−0.47

0.14

0.06

0.11

−0.22

−0.28

−0.39

0.14

−0.29

0.82

4

2.78

0.88

−3.50

−1.79

−0.73

−3.60

−0.66

−3.32

−3.42

0.37

0.27

−2.45

5

0.83

0.43

−7.71

−0.22

−0.43

−4.85

0.33

−1.38

−7.46

−0.30

1.73

−7.49

Right Femur

6

−1.81

−3.41

−7.28

1.19

2.02

−8.73

1.07

−1.78

−8.74

−0.06

0.94

−10.72

7

0.45

0.08

−0.49

−0.57

−0.40

−0.53

−0.23

0.05

−1.03

0.19

−0.16

−0.73

8

0.37

0.58

−1.84

−0.23

−0.50

−0.72

−0.15

0.10

−1.24

0.54

0.05

−0.58

9

−0.29

−0.30

−1.38

0.30

−0.26

−0.14

−0.51

−0.32

−2.01

0.09

0.60

−0.44

10

−0.05

0.31

−0.17

−0.30

−0.19

−0.15

−0.18

−0.34

−1.94

0.10

0.38

−2.07

.

We compared the manual measurement of implant displacement obtained in Section 3.4 with the femur registration-based measurements proposed in this paper. The difference between the two experimental results is shown in

Table 7. The difference between the implant displacement results obtained by the conventional method and femur registration-based method.

Fracture Site	Case No.	Proximal Pin (mm)			Distal Pin (mm)
		Actual Length	Top Movement	Bottom Movement	Actual Length	Top Movement	Bottom Movement
Left Femur	1	80	−0.77	0.54	90	−0.84	−0.18
	2	80	−0.08	−0.36	90	−0.81	−0.31
	3	70	−3.57	−1.89	85	−0.16	−0.34
	4	80	−1.87	−0.62	90	−1.67	−0.88
	5	85	0.55	0.57	95	1.04	2.26
Right Femur	6	90	−1.84	−1.68	100	−2.47	1.77
	7	80	1.37	1.91	95	1.61	1.53
	8	80	2.07	2.09	90	0.69	2.86
	9	80	1.98	1.25	90	2.68	2.64
	10	75	0.11	2.46	90	0.21	2.37

. The measurement error is within 3 mm, except in Case 3, where the measurement error is −3.57 mm.

Figure 21 and Figure 22 depict the errors of the lengths of Hansson pins obtained by the method based on femur registration and conventional method from the actual values, respectively. It is used to evaluate the stability line and accuracy of the measurement method.

5. Discussion

Internal fixation using implants is the standard management option for the treatment of stable femoral neck fractures. No evaluation method of the internal fixation system has been proposed that can use long-term postoperative examination data and measure pins displacement based on the postoperative patient’s realistic daily movement. In this paper, we use the Hansson pins system as the object of study. A neural network with a symmetrical structure was used to segment the injured femur automatically, and the segmented label was used as a mask to register the patient’s postoperative CT images at different time points. We calculated the displacement by obtaining the coordinate information of the hip implant from the 3D point cloud generated by the aligned CT images.

This study’s focus is to quantify and visualize the movement of the patient’s implanted pins after surgery and to evaluate the internal fixation system based on the effect of the patient’s regular behavioral habits on the migration of the implant. Generally, the pins’ axial displacement is a fundamental criterion, and the quantified displacement of the pins is the most visual data that can be read in radiographs, reflecting the pins’ ability to resist axial force. In many postoperative evaluations of orthopedic internal fixation procedures, the degree of implant movement is used as an indicator of the need for reoperation, the mechanical failure of the pins, or the internal fixation device’s reliability [30,31,32]. For example, in [33], the migration of screw and K-wire was used as one indicator to analyze the outcome after internal fixation of proximal humeral fractures. Due to the nature of radiographic images, surgeons judge implant failure based on experience only if significant implant displacement occurs. The pins displacement measurement method proposed in this study can quantify the implant’s displacement value in any specified direction in 3D space by transforming the coordinate system and visualizing it by 3D reconstruction. Furthermore, we use the segmented femur obtained from a neural network with symmetric properties as the mask for alignment and use the multi-resolution framework as the core for automatic alignment of CT images, which reduces human intervention and dramatically reduces time consumption. Therefore, it can reduce the workload of surgeons and their reliance on experience.

In this paper’s experiments, two sets of CT images for measurement with a one-year interval between scans. In contrast to many literature pieces that use hydraulic devices or finite element simulations to simulate a single motion scenario of the patient, the evaluation method proposed in this paper is based on the analysis of the pins data presented after the real behavior. Furthermore, we can fuse the patient’s CT data at multiple time points and combine the patient-specific health information for the evaluation of the internal fixation system, avoiding the simulation results that do not match with the real data due to simplified parameters. More importantly, the CT images we use can provide three-dimensional coordinate data with higher accuracy of results compared to two-dimensional radiological images.

We used negative mutual information for the experiments with the coarse and fine alignment of CT images as the evaluation function of image overlap. The metric values of fine registration in some cases listed in Table 3 are smaller than those of coarse registration. The fine alignment and the coarse alignment have different ranges of effect in the similarity function. In the coarse alignment, the mutual information is calculated with the whole input image as the range, while the fine alignment has a different range due to the mask’s use. The metric values in Table 3 are to verify whether there is a difference between the registration using manual annotation and the registration using the mask generated by 3D-UNet and are not correlated with the results shown in Table 2.

Another interesting finding in this paper is that in the 10 cases where Hansson pins were used as the internal fixation system, both proximal and distal pins shifted to varying degrees along the axis of the pins, away from the femoral head, and to a small degree in other directions compared to axial migration. Biomechanical analysis is that since the upper body’s major weight within the frontal and sagittal planes produce different components on the femur during normal walking [34]. In Case 2, the displacement in both axial and hook directions was significantly greater than in the other cases and may be related to the severity of the patient’s osteoporosis, which will continue to be discussed in future studies.

By comparing the lengths of Hansson pins calculated by the femur-based registration method with the traditional method in Figure 21 and Figure 22, we found that the measurement stability and accuracy of the proposed method are higher than the traditional method. Only in cases 4 and 6, the traditional method’s absolute errors are minor compard to the method proposed in this paper. Moreover, the traditional method’s error fluctuation is large, which indicates that the measurement effect is not stable. More importantly, according to Table 7, the method proposed in this paper consumes much less time than the manual measurement and obtains similar measurement results.

The cases in dataset C were obtained from Hyogo Prefectural Awaji Medical Center. We only collected records of intracapsular fractures in female patients during the four years from March 2012 to January 2015. This data supports the previous literature’s statistical findings that femoral fracture incidence is significantly higher in women than in men [3]. Since male and female femurs have the same symmetrical structure, with no significant differences in morphology and stress environment, it will not affect the experimental results’ applicability. Moreover, we will continue to focus on the treatment of intracapsular femur fractures, and in future studies, we will add medical image data from male patients for experimental comparisons.

In this paper, we use Hansson pins as the object of study, and Table 6 demonstrates the displacement of the pins along the axial and the hook pointing direction. The crucial step of the method is the registration of the rigid reference. Thus, the method is widely applicable to evaluating internal fixation implants in other fracture sites, and only different segmentation models need to be trained depending on the rigid reference. This paper is an improved solution based on a previous study, ref. [35] using a 3D point cloud alignment method for femur alignment. The registration method based on the multi-resolution framework proposed in this paper is less time consuming and higher accuracy.

A limitation of the current study is that the pins’ calculated displacement values are obtained without ground truth for comparison. For this reason, we use mutual information as the evaluation function in the process of registration of CT images and set the convergence minimum to 1e-6 to ensure the maximum overlap of the registered images. The error is limited to an acceptable range by the intuitive evaluation of the 3D images of the two data sets after alignment. Moreover, to ensure the pins’ displacement values’ validity, we use the actual length of Hansson pins as the parameter criterion for 3D reconstruction in generating 3D point clouds, and the length error is less than 2 mm. Despite the limitations, the results obtained by this method of measurement can accurately reflect the motion trend of the pins. For overcoming the limitations of this study, more biomechanical experiments are needed as a comparison of the results.

6. Conclusions

This study refers to address the problems of traditional internal fixation evaluation methods used for intracapsular fractures, which have low applicability, do not fully reflect the actual postoperative condition of patients, time-consuming, and low accuracy. We proposed a rapid evaluation method for internal fixation systems used for femoral neck fractures in the elderly. The method uses a 3D-UNet neural network to segment the injured femur and uses it as a mask to registers CT images scanned at different times. Further, the registered CT images are converted to point cloud data to quantify the implant’s displacement in a specific direction. This method does not affect the patient’s postoperative recovery and allows the fusion of long-term data for the evaluation of the internal fixation system.

Encouraging experimental results demonstrate that this method gives more reliable results than the traditional manual measurement of pins displacement. Although the evaluation given in this paper is preliminary, it is widely applicable to implants’ displacement after internal fixation procedures at other sites. A wide range of research prospects is available.