1. Introduction
The biomaterials used in medicine that are referred to as cements are used as binding materials in orthopedic surgery and dentistry. The origins of these materials date back to the early 1960s when Charnley performed hip alloplasty using polymethylmethacrylate (PMMA) cement to fix a femur [
1]. Such bone cement continues to this day to be produced from a mixture of a monomer (methyl methacrylate, MMA) and a polymer in the form of pre-polymerized polymethyl methacrylate. The process takes place in the presence of an initiator, an activator, and a stabilizer [
2,
3]. In orthopedic surgery, these materials are most often used in hip or knee alloplasty and when it is necessary to fill a cavity due to missing bone tissue in complicated fractures or in cases of neoplastic lesions in order to strengthen the internal structure [
4].
Bone cement, however, is most commonly used in arthroplasty of large lower limb joints, such as the knee or the hip. The number of procedures is increasing annually, with more than 1 million hip replacement surgeries performed each year worldwide [
5]. According to the Swedish Annual Report on Prosthesis Implantations (SKAR) for 2019, over the last four years, the number of primary knee replacement surgeries increased by 17%, and 92% of knee prostheses are fixed with bone cement [
6]. In the majority of publications, the opinions concerning the knee joint endoprosthesis procedure are very good among both doctors and patients, as they result in long-term subsidence or a significant reduction of pain or in the restoration of the range of motion [
7,
8,
9]. However, the 15-year survival of knee joint prostheses fixed with bone cement is estimated at 90% [
8]. In the United States alone, the number of revision arthroplasty procedures is expected to increase by 600% by 2030 [
10]. This means that a significant proportion of patients will have to undergo a revision arthroplasty procedure, which is a worse prognosis than the original joint arthroplasty procedure [
11]. Many factors, such as prosthesis placement, infections, and the mechanical strength of bone cement, which is the only element that connects the bone with the prosthesis, have an impact on the number of revision arthroplasty procedures. The environment surrounding the cement after the implantation is aggressive, which has a significant impact on the speed of aging of bone cement and, consequently, on the speed of changes in its mechanical properties [
4,
12].
The long-term properties of bone cement, including fatigue behavior, the viscoelasticity of creep, and stress relaxation, are central to the success of cemented hip replacement [
13]. The bone/cement interface is the key to the survival of a total hip replacement (THR) [
14,
15,
16]. The mechanical properties of bone cement are influenced by a number of factors, one of them being the method of preparation (mixing) [
17]. Impurities introduced during its implantation, such as blood or bone tissue residues, as well as deviations from the recommended ratio of components also greatly affect its mechanical parameters [
12,
18]. The contamination of cement with blood causes a decrease in its strength properties and a discontinuity of its structure [
12,
19,
20]. The basic strength properties, resistance to cracking, fatigue strength, and creep resistance of bone cement depend on its preparation process and its implantation, as well as on the influence of the environment of the human body [
21,
22,
23].
With a view to obtaining material of improved mechanical and biological properties, the effect of introducing modifying additives, including biodegradable additives, has been widely investigated (cellulose, chitosan, magnesium, polydioxanone, or tricalcium-phosphate) [
24], antibiotics (Ciprofloxacin and Vancomycin) [
25], hardening solutions (hardystonite (HT; Ca
2ZnSi
2O
7) mixed with brushite (B; CaHPO
4·2H
2O), calcium phosphate, and citrate liquid) [
26], and antimicrobial particles of silver sodium hydrogen zirconium phosphate (S–P) [
27].
An analysis of changes in mechanical parameters is extremely important from the viewpoint of implantation and estimation of implant lifetime and in the design of modern materials. However, it is often impossible to carry out a large number of physical tests due to high costs. A possible solution in this case is the use of predictive models, which enable the determination of the dependence between particular parameters and properties using earlier experimental data [
28]. However, there are better alternatives, such as the finite element method (FEM) [
29,
30,
31] or the boundary element method (BEM) and an analytical approach. The models developed in this way may indicate the most promising direction for research and help reduce the number of physical tests, thus significantly bring down the costs of the research [
28,
32,
33,
34].
Keeping in mind the abovementioned facts, first of all, the impact of changes in the strength of bone cements when various admixtures, both intended and accidental contaminants, is known. One must not forget the significant impact of the material properties of the cement on the failure of the reconstruction, and above all—on all the inconveniences resulting from revision procedures (re-risk and costs of surgery, inconvenience to the patient, etc.)—that the analysis of the influence of the extent of contamination with pollutants present in the operating field on the selected strength characteristics of the cement seems to be the most justified. Considering the fact that bone cement is rather expensive, surgical revision procedures carry additional risk to the patient, and any destructive experimental studies are long and cost-intensive, an attempt was undertaken to employ the available tools (deep learning networks and statistical modeling) to predict strength changes, as well as assess the effectiveness of the tools in question.
2. Materials and Methods
The experimental research included compression strength tests on bone cement samples to determine their mechanical properties depending on the degree of contamination. Based on the results of the experimental research, statistical models were developed and changes in the compressive strength of bone cements were predicted as the contamination increased. The obtained model results were compared with the results of the experimental research.
2.1. Material and Sample Preparation
Samples made of the DePuy CMW 3 Medium Viscosity Bone Cement were tested for compressive strength. The samples that underwent the compression test were prepared in accordance with the guidelines of the International Standard ISO 5388:2002 [
35]. The samples were cylindrical, their height was 12 ± 0.1 mm, and their diameter was 6 ± 0.1 mm. In order to determine the effect of contamination of the cement structure with 0.9% saline solution and with blood, at the stage of components’ joining, 1%, 2%, 4%, 6%, 8%, and 10% of the contaminants were introduced into the samples. The samples prepared in this way were seasoned in Ringer’s solution at 23 °C and tested before the seasoning, 24 h after they were made, and after 10, 20, and 30 days in the solution. The study was approved by the Scientific Research Ethics Committee of the Lublin University of Technology (No. 5/2016 of 19 December 2016). Detailed information on sample preparation is provided in publications [
12,
18].
2.2. Mechanical Testing
Compression strength tests were performed using the MTS Bionix-Servohydraulic Test System (Eden Prairie, MN, USA), which comprised a strength-testing machine with a dedicated MTS TestWorks software (Eden Prairie, MN, USA). The compression speed was selected in accordance with ISO 5388:2002 and was equal to 20 mm/min. Each of the sample series tested consisted of a minimum of five samples, according to the abovementioned standard. Detailed information on the compression strength test is provided in publications [
12,
18].
2.3. Deep Learning Networks’ Algorithms
The problem of the prediction of mechanical properties depending on the content of admixtures continues to be an interesting research area [
36,
37,
38]. However, it should be noted that the authors of the abovementioned publications had a slightly different approach to the problem of prediction. The prediction was performed on a single set of data, from which strength relationships were predicted. The present paper provides for one or, in the second variant, two full sets of data in the form of full-strength characteristics. Such formulation of the problem determined the choice of the neural network algorithm: DLN LSTM (Deep Learning Networks Long Short-Term Memory).
The practical experience [
39,
40] gained over time showed that the capabilities of classical recursive neural networks are of limited use due to the latency between important elements of the time series. Algorithms that store and transmit full information about the analyzed time series may be inefficient due to the need to adapt to a specific application and the difficulty in the scalability of the model for long-term time relationships [
41].
The Long Short-Term Memory (LSTM) is an architecture of recursive neural networks that is widely used in the field referred to as deep learning. The main assumption of the LSTM is to use a memory unit (cell) that stores its state over time, with the simultaneous presence of special nonlinear controllers in the form of gates, which are responsible for the correct flow of information. Gates are responsible for the proper flow of information between memory cells. Originally [
42], three gate structures were introduced: an input gate, an output gate, and a forget gate that resets the state of the memory. Later [
43], the LSTM architecture was simplified and reduced to the Gated Recurrent Unit (GRU), where the number of gates was reduced and the tasks of the forget gate were transferred to the output gate.
In order to determine the characteristics of the stress–strain relationship, for samples with modified input parameters in the form of defects in the composition of the mixture, a neural network algorithm referred to as deep learning, which contained a large number of hidden layers and gate structures, was used. The LSTM architecture contained in the MATLAB Deep Learning package R2020b with the Adam solver was used [
44,
45].
The calculation phase included predicting time series using the LSTM network. Stress values as a function of the deformation, obtained for individual samples, were declared as the input data. The pretraining data was preprocessed: The raw data set was reduced by removing the results of negligible low stress (the start of the linear range) together with the artifacts that resulted from the travel and the initial contact of the strength-testing machine with the samples. The length of the time series was then resampled due to the need to keep the same data lengths for each of the disturbance variants under further analysis. The data sets for each of the variants of the examined cases (disturbances) were combined into sequences: for blood admixture and saline content at the levels 0, 1, 2, 4, 6, 8 and 10% of the admixture.
Based on the data of the compression strength of cement with introduced contaminants in the share of 0, 1, 2, 4, 6, and 8%, constituting the training data set (72% of all data), separately for the addition of blood and saline, the full compression strength characteristics were predicted for the 10%-contaminated bone cement. In the second variant, two full compression strength characteristics were predicted from the training set obtained from cement characteristics obtained at the contamination of 0, 1, 2, 4, and 6% of the total cement volume. The obtained characteristics constituted the basis for the forecast of contamination in the shares of 8 and 10% by volume, respectively.
The predicting procedure using deep learning networks was carried out in two variants. In the case of the averaged input data, the data from all the experimental trials were averaged, separately for each contamination level and contaminant type. The second variant was the study based on random combinations of input time series: Adjusted raw data from each of the conducted experimental trials were used separately.
The data were standardized to prevent discrepancies during the learning process. The LSTM network was built to predict the value of each of the subsequent time steps. Data vectors without the last step were declared as the predicators. As a part of the declaration of the network architecture, the number of hidden units was declared to be 50 and the number of iterations covered 150 epochs. So as to prevent degeneration of the model describing the examined phenomena, a gradient threshold was declared and a dropout option was used to prevent the model from being attached to the analyzed data.
The next stage of the research was to obtain the force value at the point specified in ISO 5833: 2002 [
35]: A destructive force, the load shifted by 2% or load at the upper yield point (whichever occurred earlier), and transform it into the compressive tension. In the final stage, the forecast results were compared against the real experimental data. As part of the assessment of the quality of the obtained forecasts, the values obtained experimentally were compared with the data obtained using the deep learning networks tool.
2.4. Statistical Modeling
In order to predict changes in selected strength parameters, statistical analyses were carried out. These led to the generation of mathematical models that could describe changes in compression strength and microhardness of bone cement material enriched with physiological salt or human blood injected at a certain percentage concentration by weight. Developing such a model from experimental research results will enable estimating the value of these parameters for other admixture cases, e.g., for larger quantities of contaminants. To that end, an attempt was made to determine such models by analyzing and fitting typical variations (linear, polynomial function, different degrees). The accuracy of fitting of the selected models to the empirical results and the accuracy of prediction of the model were analyzed by comparing the predicted results with the actual ones in two variants: a model prepared from input data defined in the admixture range of 0–8% and 0–10%. The modeling was conducted using Statistica software (data analysis software system), version 13, from TIBCO Software Inc. (2017) (Palo Alto, CA, USA), and Microsoft Excel 2019 (Redmond, WA, USA).
4. Discussion
Load transmission through the stem of a cemented THR joint depends on multiple factors, including the mechanical properties of the cement. An increase in compressive stress in the cement and on the cement/bone interface should lead to the long-term stability of the joint [
13], but contamination of cement with blood or saline degrades its properties and may lead to joint damage with all the consequences this entails.
Methods for computer modeling methods of mechanical properties of biomaterials have great potential because they enable the preliminary estimation of the consequences of medical errors and often also a reduction of them. Knowledge of the scope of usefulness of the tested material may often protect patients from the need for repeated surgical interventions. This is also important from the point of view of physicians: Based on the estimated value of cement strength, they can present patients with probable scenarios of the outcome of the operation. This allows hospitals to reduce litigation costs in case of patient dissatisfaction.
The analysis of the reported results opens the way for further research related to the strength of biomaterials. The modeling results illustrate the fact and nature of a decrease in the strength of bone cements where contaminants are found. In the examined range of strength loss with an increase in the level of contamination, their close linear correlation was shown. Another important issue is the acceptable level of results from neural deep learning networks. This means that DLNs are suitable for predicting strength values and approximate characteristics, based on which the points determining the compressive strength of cement, which is the object of the research, were determined.
The analyses that were carried out show that a mathematical model that better fits actual data cannot always be used uncritically for precise predicting. In the case of bone cements, the prediction of a trend showing gradual changes in groups of strength characteristics proved to be quite well-described by means of a linear function. Due to the way neural networks work (predicting relationships in data difficult to describe with a single function and determining this function), there is a slight discrepancy between the results from random samples and the data from the combinations of these samples. Deep learning networks, compared to other types of networks in modeling mechanical properties (especially of materials with nonlinear strength characteristics), can be more effective due to the use of dropout (exclusion of certain neurons from the taught model during teaching, which reduces the interdependent fitting of neurons), which prevents the network from excessive fitting of the network to the training model. The advantages and disadvantages of using the network and statistical modeling are presented in
Table 9.
Although relatively promising results were obtained, the process of predicting changes in the strength of bone cements using DLN networks is burdened with certain limitations. The application of DLN in the analyzed case may seem disproportionate to the size of the problem. The generalization capacity allows analyzing the issue in a much wider scope. It is expected that the capabilities of DLN could help study the phenomenon of bone cements’ strength in the case of post-operative contamination, due to corrosion at the tissue–cement interface and other concomitant factors. The applied tool could also be used in the analysis of the problems of cement fatigue strength correlating with the occurrence of material corrosion. The approach that involves modeling the entire strength characteristics with a DLN to find the point that constitutes (according to ISO 5833:2002 [
35]) a destructive force, the load shifted by 2% or at the upper yield point (whichever occurred earlier), i.e., the strength characteristics of the material, may seem superfluous, but it paves the way for further works that will consider a substantially wider range of influencing factors. However, the nonstandard approach presented herein is aimed at extending the operation of strength analysis methods to include future research that will possibly also cover the plastic range and a full range of the strength characteristics as well as the research on the fatigue strength of contaminated bone cements. In conclusion, although the case of testing points that constitute the end of a linear range as a function of increasing admixture levels in the analyzed range can be easily described by means of a linear function, for a range extending beyond the analyzed admixture level area, the use of neural networks—including DLN—would be an effective solution, while a description with higher-level functions could work well for materials with nonlinear characteristics.
5. Conclusions
The generated higher-order polynomial models, despite their good fitting to the experimental data, are largely unsuitable for prediction purposes. Therefore, it is very important to wisely choose the type of model that we will use to try to describe the course of the changes. The model should result from observation of the nature of the variability of the data being modeled and should take into account the predicted boundary conditions. At the same time, it is important to make sure that the prediction range is reasonable. The further away our estimations are from the actual data, the greater the risk of errors and deviations from the actual values.
The results of contaminated bone cement compression strength predictions show that the use of DLNs for this purpose provides comparable results to those obtained with statistical modeling. However, very good models (linear, exponential) can give even more accurate results. It has been noted that the way networks are taught, i.e., the selection of input data, may slightly change the prediction accuracy. Due to the way neural networks work (predicting relationships based on data that is often difficult to describe with a single function, and determining this function), there is a slight discrepancy between the results obtained from random samples and the averaged data from the combinations of these samples. Thus, predicting based on the teaching of networks using a method that consists of a combination of random courses and then estimating the average value of the resultant courses usually gave more accurate results (the deviation from the average, experimental value: 2% saline, 7% blood) in comparison to the usage of averaged input data (5% saline, 9% blood), as presented in
Figure 9. The difference was about 2–3%. The biggest disadvantage of employing DLN in the presented research is the use of its relatively high computing power for an indirect prediction of the compressive strength (one point in the time course).
From the results reported here, it can be concluded that the general direction of further research should extend the scope of analyses presented in this paper. It would be interesting to confirm the observed change in prediction accuracy depending on how the teaching set for DLN is organized. Also, the authors plan to test the presented methods on data collected in similar research on different mechanical properties of biomaterials and compare their predictive strength.
Based on the presented results, it is possible to set out to construct an expert system. After accepting a given network, generating its structure with a complete processing algorithm, a system would be built to support the assessment, here, of the impact of a potential degree of contamination on the expected cement strength. After supplementing the existing database with new data, e.g., a higher degree of contamination, a system based on the generated information-processing algorithm would assess the potential strength of the cement with a new set of data.