Analytical Performance Evaluation of the New GEM® Premier™ 5000 in Comparison to the Epoc® Blood Gas Analyzer in Horses

Simple Summary Blood gas analyzers are stationary or hand-held machines used to measure blood gases and electrolytes in a blood sample taken from a patient. No machine has been specifically designed for use in horses, and therefore every machine needs to be validated before being used in equine practice. The aim of this study was to compare the newly marketed GEM5000 machine to the formerly validated epoc machine for blood gas analysis in horses. Blood samples taken from healthy and sick horses were run on both machines in alternate order, and various statistical tests were used to analyze if the two machines give similar results. Although the precision of the GEM5000 is good for most parameters, the agreement with the epoc machine is not always satisfactory. Therefore, data from different blood gas machines should not be used interchangeably. Abstract Different blood gas analyzers are used in equine practice. Every machine needs to be validated, as they have not been designed for use in horses. The aim of this study was to compare the newly marketed GEM5000 machine to the formerly validated epoc machine for blood gas analysis in horses. In this prospective, non-blinded, comparative laboratory analyzer study, 43 equine blood samples were analyzed on both analyzers and values were compared between the two machines via Lin’s concordance analysis, Passing–Bablok regression analysis and Bland–Altman plots. Duplicate measurements were conducted on the GEM5000 machine to evaluate precision. The GEM5000 failed to achieve the required precision for tHb, Hct and iCa2+, but achieved acceptable precision for all other parameters. Concordance correlation analysis revealed poor correlation for Na+, Cl−, iCa2+, K+, Hct and tHb, while there was an at least moderate agreement for all other parameters. Passing–Bablok regression revealed significant constant bias for pCO2, pO2, Cl−, and iCa2+ and significant proportional bias for pCO2, iCa2+ and SO2. Bland–Altman analysis revealed significant systematic bias for Na+, Cl−, iCa2+, K+, Hct, tHb and SO2. This study shows that while precision of the GEM5000 is good, values should not be used interchangeably with data obtained from other blood gas analyzers.


Introduction
The blood gas and electrolyte analysis (BGEA) is an essential tool in equine medicine and research as it helps the veterinarian to establish a clinical diagnosis, a prognosis and to provide an appropriate therapeutic monitoring of patients. Most machines offer the possibility to measure pH, partial pressures of oxygen (pO 2 ) and carbon dioxide (pCO 2 ), as well as electrolytes such as sodium (Na + ), potassium (K + ), chloride (Cl − ) and ionized calcium (iCa 2+ ).
Whole blood samples were collected from horses during their stay at the Equine Hospital of the University of Liege. Blood samples were taken from horses because of their primary clinical condition and only when deemed necessary by the clinician in charge. No extra blood was taken solely for the purpose of this study. In these conditions, no approval from the Ethical Committee of the institution was necessary. Owners of the horses agreed for the data of their horses and the results of the BGEA to be used for scientific purposes in an anonymized format. Forty-three blood samples from twenty-six horses admitted to the Equine Hospital of the University of Liege in January 2019 with various pathologies were used in this study. Horses with different diseases were recruited to ensure a wide range of blood gas partial pressures and electrolyte concentrations.

Sample Collection
Blood samples were taken in prefilled heparinized syringes (Blood gas-Monovette ® , Sarstedt AG & Co. KG, Nümbrecht, Germany) as a part of the clinical work-up of the patients or as a part of monitoring during general anesthesia. Blood samples were taken from the jugular vein or from the carotid artery in standing horses or from a catheter placed in the transverse facial artery in anaesthetized horses.

Sample Analysis
Analyses were immediately performed after blood sample collection and both machines were used in alternating sequence. Both instruments were maintained, calibrated and operated according to the manufacturers' guidelines. The devices were installed side by side to minimize the time interval between measurements. The following parameters Vet. Sci. 2023, 10, 114 3 of 11 were evaluated for the comparison of the two machines: pH, pCO 2 , pO 2 , Na + , Cl − , iCa 2+ , K + , Hct, tHb, BE, SO 2 and HCO 3 − . To evaluate the precision of the GEM5000 Premier machine, 29 samples were analyzed twice, in rapid succession. The following measured variables and calculated parameters were assessed for precision analysis: pH, pCO 2 , pO 2 , Na + , Cl − , iCa 2+ , K + , Hct, tHb, BE, SO 2 and HCO 3 − .

Instruments
In both machines, epoc (epoc Blood Analysis System, Siemens Healthcare SA, Groot-Bijgaarden, Belgium), and GEM5000 (Gem Premier 5000, Zaventem, Belgium), the pH, pCO 2 and electrolytes are measured potentiometrically with ion-selective electrodes and basing calculations on the Nernst-equation. Oxygen is measured with an ion-selective electrode of the Clarke type. The HCO 3 − is calculated by using the Henderson-Hasselbalch equation in both instruments using the following equation (log [HCO 3 − ] = pH + log pCO 2 − 7.608).

Statistical Analysis
Precision. To evaluate precision of the GEM5000 machine, samples were run in duplicate, and the mean, the standard deviation and the coefficient of variation (CV) in percent of each duplicate measurement were calculated for each parameter (Microsoft Excel). The means of all the CV were compared to preset clinical limits of acceptance with the assumption that they should be equal to or less than 25% of the defined allowable total error [16]. The allowable errors for blood gases and electrolyte concentrations were based on the Criteria for Acceptable Performance of the Clinical Laboratory Improvements Amendments [17].
Agreement. In order to determine the bias between the two analyzers, results obtained from the GEM5000 machine were compared to those obtained from the epoc machine using concordance correlation analysis, the Bland-Altman method, and the Passing-Bablok regression analysis (MedCalc, Ostende, Belgium). Lin's concordance correlation coefficient (r ccc ) has a measure for accuracy (bias corrected factor, Cb) and for precision (Pearson's ρ). It serves as an indicator of the strength of concordance between two measurements [18]. A value of the r ccc of less than 0.90 was considered as poor agreement, 0.90 to 0.95 as moderate agreement, 0.95 to 0.99 as substantial agreement and above 0.99 as almost perfect agreement between the methods [19].
The slope, the intercept and the residual standard deviation (RSD) of the Passing-Bablok regression analysis reflects the proportional, the constant bias and the random bias, respectively. If the 95% confidence intervals for intercept contained the value 0, and for slope the value 1, no significant constant or proportional bias between measurements is noticed. If data are normally distributed, 95% of random differences are expected to lie in the interval of −1.96 RSD to +1.96 RSD. A large interval is considered as an indicator for a high imprecision [20]. The linearity between compared results was assessed by the CUSUM test, which warrants the validity of Passing-Bablok procedure.
Additionally to regression analysis, Bland-Altman analysis determined the agreement between the compared methods by plotting the difference of two paired measurements against the mean of the two measurements. The mean difference between results of compared methods reflects the systematic bias. If the value 0 is included in the 95% confidence interval of the bias, it is assumed that there is no significant systematic bias [19,21]. Assuming that differences between measured values are normally distributed, 95% of all data points should lie within 1.96 times the standard deviation of the mean difference. This assumption allows for calculation of 95% limits of agreement [20,22]. The limits of agreement serve as an estimate of the total bias, which is composed of systematic and random bias and is assessed by the comparison with the pre-specified clinically allowable errors. Bias was assumed to be clinically relevant, if the 95% limits of agreement were outside the limits for the allowable total errors. The CLIA guidelines served as a basis for the assignment of limits for clinical acceptance [17].

Animals and Samples
A total of 43 blood samples including 23 arterial and 20 venous blood samples were collected and analyzed. These samples were taken from 26 horses of various breeds (19 Warmblood Horses, 2 Frisians, 2 Riding Ponies, 1 Draft horse, 1 Quarter Horse, 1 Standardbred) and ages. There were 3 stallions, 14 geldings and 9 mares, aged between 1 and 25 years. The mean body weight was 543 kg and ranged between 293 and 700 kg.

Precision (Repeatability)
Table 1 displays the mean coefficient of variation (CV) of each of the 29 duplicate measurements obtained from the GEM5000 analyzer, with comparison to clinical limits of acceptance, which are defined as equal to or less than 25% of the defined allowable total error.  Table 2 shows the mean values, standard deviations and ranges for the measured parameters of the two machines. Table 3 shows the results of the concordance correlation analysis between the epoc and the GEM5000 analyzer. At least moderate agreement was obtained for pH, pCO 2 , pO 2 , HCO 3 − , BE and SO 2 , while the analysis revealed poor agreement for all measured electrolytes and tHb. Table 4 shows the results of the Passing-Bablok regression and the Bland-Altman analysis from the comparison between epoc and GEM5000 for all measured parameters. Figure 1 displays the Bland-Altman plots for the comparison between the two analyzers for pH (A), pCO 2 (B) and pO 2 (C), Na + (D), Cl − (E), iCa 2+ (F), K + (G) Hct (H), tHb (I), BE (J), SO 2 (K) and HCO 3 − (L). The CUSUM analysis revealed no significant deviation from the linear relationship between the compared results for all determined parameters (p > 0.05). Table 2. Mean, standard deviation (SD) and measured concentration range for parameters determined by the two compared analyzers epoc and GEM5000 (n = 43).  Table 3. Concordance correlation analysis between the epoc and the GEM5000 analyzer, indicating the accuracy (bias correction factor, C b ) and the precision (Pearson ρ) as well as the Lin's concordance correlation coefficients (r ccc ) with 95% confidence intervals (95% CI).

Concordance Correlation Analysis
Lin's r ccc (95% CI)     pH: For the pH, the Bland-Altman plot revealed no significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.7682). The Passing-Bablok regression revealed no significant constant or proportional bias. pCO 2 : For the pCO 2 , the Bland-Altman plot revealed no significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.7512). The Passing-Bablok regression revealed a significant constant and proportional bias. pO 2 : For the pO 2 , the Bland-Altman plot revealed no significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.1356). The Passing-Bablok regression revealed a significant constant but no significant proportional bias. Na + , K + , Cl − , iCa 2+ : The Bland-Altman analysis indicated a significant systematic bias between results obtained from the epoc and the GEM5000 for Na + (p < 0.000), K + (p = 0.0056), Cl − (p = 0.00364), and iCa 2+ (p = 0.0006). The Passing-Bablok regression indicated no significant proportional bias for Na + , K + , Cl − , and no constant bias for Na + and K + ; however, there was a significant constant bias for Cl − . For iCa 2+ , the Passing-Bablok analysis revealed both significant constant and proportional bias.
Hct: For the Hct, the Bland-Altman plot revealed a significant systematic bias between results obtained from the epoc and the GEM5000 (p < 0.0001). The Passing-Bablok regression revealed no significant constant or proportional bias. tHb: For the tHb, the Bland-Altman plot revealed a significant systematic bias between results obtained from the epoc and the GEM5000 (p < 0.0001). The Passing-Bablok regression revealed no significant constant or proportional bias. BE: For the BE, the Bland-Altman plot revealed no significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.6059). The Passing-Bablok regression revealed no significant constant or proportional bias. SO 2 : For the SO 2 , the Bland-Altman plot revealed a significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.0002). The Passing-Bablok regression revealed no significant constant but a significant proportional bias. tween results obtained from the epoc and the GEM5000 (p < 0.0001). The Passing-Bablok regression revealed no significant constant or proportional bias. tHb: For the tHb, the Bland-Altman plot revealed a significant systematic bias between results obtained from the epoc and the GEM5000 (p < 0.0001). The Passing-Bablok regression revealed no significant constant or proportional bias. BE: For the BE, the Bland-Altman plot revealed no significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.6059). The Passing-Bablok regression revealed no significant constant or proportional bias. SO2: For the SO2, the Bland-Altman plot revealed a significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.0002). The Passing-Bablok regression revealed no significant constant but a significant proportional bias.
HCO3 − : For the HCO3 − , the Bland-Altman plot revealed no significant systematic bias between results obtained from the epoc and the GEM5000 (p = 0.8377). The Passing-Bablok regression revealed no significant proportional or constant bias.

Discussion
Blood gases and electrolytes play a vital role in a number of disorders affecting the homeostasis of the patient. The delay to obtain laboratory results for iCa 2+ , Na + , K + and Cl − concentrations is often too long to allow for timely decision making in critically ill patients. Therefore, reliable methods for on-site measurements are critical. In addition, determination of blood gases usually requires an analysis immediately after blood sample collection, since storage and transportation to the laboratory would affect the results. Several studies have investigated the effect of syringe type and storage temperature on blood gas analysis and underline the importance of standardized storage conditions and minimal delay from sampling to analysis [24,25]. Different analyzers are used for this purpose, but no machine has been designed for specific use in horses. Therefore, every machine needs to be validated before use in clinics. The epoc analyzer has recently been tested for equine blood, but cannot be considered as a gold standard technique [13,14]. Precision targets were met by the epoc machine for pH, PO 2 , Na + , K + , glucose, lactate, HCO 3 − , BE (ecf) and SO 2 , but not for PCO 2 , iCa 2+, HCT and Hb [14], while agreement was poor for Hct, Hb and BE when compared to the iSTAT or an ABL machine. Kirsch and co-workers [12] reported that the epoc was not meeting precision targets for pO 2 , pCO 2 and K + and that agreement was poor for pH, [HCO 3 − ] and [Na + ] when compared with the Cobas analyzer. Despite these variable results for precision and agreement with other machines, the epoc was chosen as a comparison machine for the GEM5000 in the present study, as its precision was generally accepted as satisfactory. Actually, no blood gas analyzer can be considered as a gold standard, and when comparing two machines, it is not clear which one provides data that are closer to the real values. Apart from storage temperature and duration, the body temperature itself may affect the measurement. In the current paper, the alpha-stat method has been used and all measurements were run on the machines, assuming a body temperature of 37 • C. Although, the importance of body temperature and temperature correction for blood gas analyses has been previously highlighted in exercising horses [26], it is probably less relevant for a method comparison study. The alpha-stat hypothesis suggests that we always interpret our blood gases as corrected to the same temperature (normal body temperature), regardless of how high or low the body temperature actually is. The pH-stat hypothesis, on the other hand, suggests that temperature should always be corrected to the core body temperature. Each approach has its advantages and disadvantages and deciding to use either one is especially important when clinical decisions are based on the results [27]. Since the current study compares the analytical performance of two different blood gas machines, we assume that any temperature related error would be the same for the two analyzers.
Precision analysis of the GEM5000 revealed that results were acceptable for pH, pCO 2 and for all electrolytes, while they were considered too high for Hct, tHb and iCa 2+ . When comparing to maximal CV% given by Westgaard [16], Hct and tHb were out of acceptable range. Hct and tHb are linked and a possible explanation is improper mixing. The importance of mixing the samples, especially for Hct, has been underlined by Mion and co-workers [28], who tested the GEM5000 machine at two different human hospitals.
They measured the effect of different handling of the sample during the pre-analytical phase before analyzing the sample in a GEM500 machine. They compared the effect of (1) automatic mixing using a rotary shaker for 5 min only to (2) 5 min of automatic mixing followed by 30 s of gentle turning and inverting by hand and (3) 5 min of automatic mixing using a rotary shaker followed by quick inversions by hand for an additional 30 s prior to analysis. Their results showed that additional hand mixing is superior to machine only mixing and that the best result is obtained when flipping the tube rapidly for 30 s before analyzing. In the present study, all samples were handled by the same operator and no machine mixing was used. The tubes where gently inverted by hand for 30 s before analyzing. The time from sampling to analysis was kept short, but results of precision for Hct and tHb indicate that precision could have been improved by better mixing of the samples. Although efforts were made to standardize the conditions, such as minimizing the delay from sampling to analysis, mixing the sample, and working in controlled room temperature and pressure, some errors might have occurred in the pre-analysis period. Available evidence strongly suggests that the pre-and post-analytical steps are more error-prone than the intra-analytical phase [28].
As with most machines, the precision is good, while the agreement is poor. It can be suggested that data from a single machine are reliable, while comparing data from different machines should be done cautiously. This may be of relevance in clinical practice, where analyses are run on different machines.
One of the principal limitations of the present study is the low number of samples investigated. In general, it is recommended that 40 to 100 samples should be used for method comparison studies [29], but this also depends on the range of samples, which should be as wide as possible. More than half of the results, however, were outside the reference intervals for equine venous blood, indicating a wide range of data. This is recommended by the Clinical Laboratory Improvement Amendments (CLIA) guidelines for evaluating agreement between two analyzers [30]. The reference ranges used for comparison were not generated by the machines in question, and therefore the exact number of samples out of reference range cannot be determined. Future studies should focus on the establishment of reference values for healthy horses on the GEM5000 machine. The importance of species and equipment specific reference values has been underlined by a recent study, which established reference values for equine BGEA [31].

Conclusions
Considering the overall satisfactory analytical performance, the good practicability, the intuitive and easy to use software interface and the rapid measuring time (about 45 s from sample aspiration to results), GEM5000 seems suitable to be used in equine practice, but results obtained with this machine should not be used interchangeably with those from other blood gas analyzers.
Funding: This research received no external funding.

Institutional Review Board Statement:
Only waste blood was used in this study and therefore no approval from the Institutional Committee for the Ethical Use of Animals was necessary (Commission Éthique Animale, University of Liege).

Informed Consent Statement:
Owners of the horses agreed that the data of their horses and the results of the BGEA may be used for scientific purposes in anonymized format.

Data Availability Statement:
On publication of the article, data will be made available on an open science framework.