Performance Evaluation of Diagnostic Assays for Detection and Classification of Carbapenemase-Producing Organisms

Rapid and accurate detection can help optimize patient treatment and improve infection control against nosocomial carbapenemase-producing organisms (CPO). In this study, a total of 217 routine clinical isolates (Enterobacterales and A. baumannii), including 178 CPOs and 39 non-CPOs, were tested to evaluate the performance of six phenotypic carbapenemase detection and classification assays, i.e., BD Phoenix CPO detect panel, Rapidec Carba-NP, O.K.N detection kit, and three carbapenem inactivation methods (CIMs; mCIM, eCIM, sCIM). The overall detection sensitivity and specificity were 98.78% (95.21–99.79%) and 79.49% (63.06–90.13%), respectively, for the BD phoenix CPO P/N test; 91.93% (86.30–95.45%) and 100% (88.83–100%), respectively, for the Rapidec Carba-NP; 98.06% (94.00–99.50%) and 97.44% (84.92–99.87%), respectively, for mCIM; and 96.89% (92.52–98.85%) and 94.87% (81.37–99.11%), respectively, for sCIM. The classification sensitivity and specificity for the BD phoenix CPO Ambler test, the O.K.N detection kit, and the mCIM and eCIM were 56.71% (48.75–64.34%) and 94.87% (81.37–99.11%), 99.28% (95.43–99.96%) and 100% (88.83–100%), and 92.90% (87.35–96.23%) and 97.44% (84.92–99.87%), respectively. All detection assays were reliable in detecting carbapenemase. However, the Rapidec Carba-NP and mCIM were insufficient in detecting OXA-48-like enzymes. The BD phoenix CPO detect panel had a strong ability to detect carbapenemase but failed to classify 48/59 (81.36%) KPC, 8/52 (15.38%) NDM, 8/22 (36.36%) OXA-23-like, and 6/11 (54.55%) dual enzymes. The O.K.N detection kit accurately detected and differentiated KPC, NDM, and OXA-48-like enzymes existing alone or in combination. The results of this study will support reliable laboratory work tools and promote therapeutic and infection control decisions.


Introduction
Carbapenems are considered as the last resort to defend against severe multidrugresistant Gram-negative bacterial infections owing to their broad spectrum of activity and stability to most β-lactamases [1]. Gram-negative organisms become carbapenemsresistant mainly through carbapenemase production, cephalosporinase (AmpC), and/or extended-spectrum β-lactamase (ESBL) expression in combination with porin-encoding genes (such as oprD) mutation and/or efflux pump (such as MexAB-OprM, MexXY-OprM, MexCD-OprJ) overexpression, which lead to limited cell membrane permeability [2][3][4][5]. Carbapenemase, a primary mechanism of carbapenem resistance in Gram-negative bacteria, is mainly encoded on plasmids and is highly transmissible [6]. Over the past decades, carbapenem-producing organisms (CPOs) have become increasingly prevalent worldwide, and infections caused by carbapenem-resistant organisms, especially CPOs, are associated with limited clinical treatment options, a high mortality rate, and a heavy healthcare burden.
Early screening and identification of carbapenem-producing organisms (CPO), at clinical infections or the carriage state, is efficient to prevent the development of hospital outbreaks and to instruct clinical treatment and intervention of these severe multidrugresistant bacteria infections. Phenotypic detection assays, including chromogenic media, the Carba-NP test [7], carbapenem inactivation method (CIM) [8], and matrix-assisted laser desorption ionization-time of flight mass spectrometry [9,10] analysis, are widely used to detect carbapenemases in clinical isolates, but cannot accurately classify the enzymes produced by the isolates. Commercial diagnostic kits based on immunochromatography principle, such as the OXA-48 K-Se T [11], O.K.N K-Se T [12], and CARBA-5 reagents [13], have high sensitivity and specificity. Genetic detection methods are considered to be the gold standard, such as traditional PCR, Xpert Carba-R, reverse transcription quantitative PCR, multiplex real-time PCR, and the microfluidic chip technology, which have high sensitivity and specificity, and can directly identify carbapenemase genes. However, the high cost and the requirement of specialized instruments and skilled staff limited their application for many laboratories worldwide without such resources [14][15][16]. Moreover, PCR-based methods may have false positives and false negatives, and may miss new variant types of carbapenemase.
To improve carbapenemase detection and classification, this study evaluated the performance of three commercial diagnostic kits-BD Phoenix CPO detect panel, Rapidec Carba-NP, and an O.K.N detection kit-and three carbapenem inactivation methods (CIMs)-mCIM, eCIM, and the simplified version sCIM-using clinical isolates collected in China.

Overall Performance
The sensitivity and specificity of the various assays for carbapenemase detection and classification are shown in Table 2. The BD phoenix CPO detect P/N test, mCIM, and sCIM had sensitivity of over 96% in detecting the carbapenemase production. The Rapidec Carba-NP had high specificity of 100%, but the sensitivity was 91.93% due to its low efficiency in detecting OXA-48-like carbapenemase. The O.K.N detection kit had high sensitivity and specificity of over 99% and 100% in detecting and classifying the carbapenemase within its detection range.

Discussion
Rapid and accurate detection and classification of CPOs is critical in avoiding treatment failures and guiding clinical antibiotics therapy and infection control decisions, including the use of expensive antibiotics and isolation measures. However, phenotypic identification of organisms with mu ltiple carbapenemase types, low-level carbapenemase expression, extended-spectrum beta-lactamases (ESBLs), AmpC beta-lactamases, growth impairment, and/or other potentially uncovered carbapenemase resistance mechanisms can be difficult [17].
The BD Phoenix CPO detect panel offers (semi)-automated antimicrobial susceptibility testing (AST) systems and phenotypic detection and classification of carbapenemase [18]. The high sensitivity of the BD Phoenix CPO detect panel P/N test observed in this study supports the direct reporting of CPO-negative strains without additional tests, and directly provides the true carbapenem MICs that can be used as a screening value for carbapenemase. However, the limitation was classification performance, especially in typing class A β-lactamase, with sensitivity of only 18.64%, which corresponds with the results of Simon et al. ( [21]. Eleven strains producing dual carbapenemases were determined as positive by the P/N test but without providing an Ambler classification or with attributing only one of the involved carbapenemases by the Ambler test. It is also noteworthy that 8 of 39 (20.51%) non-CPOs were false positive, which indicated downstream tests, such as a colorimetric assay and/or CIM tests, were needed to confirm the positive result detect by the BD Phoenix CPO detect P/N test. The performance of the BD Phoenix CPO detect P/N test could be challenged by the non-CPOs phenotypes, with decreased efficiency and specificity in detecting strains combining ESBLs and/or AmpCs with/without porin loss [21][22][23]. Among the eight false positive strains, 2/8 were AmpC producers, 3/8 were ESBLs producers, and 2/8 harbored both AmpC beta-lactamases and ESBLs.
Interpretation the result of the Rapidec Carba-NP can be challenging for an inexperienced operator, and inconclusive results can occur. Moreover, the detection of carbapenemases in mucoid isolates and the detection of OXA-48-like carbapenemase could be challenging, which was consistent with several previously published studies [24][25][26].
The O.K.N detection kit, a method with no special equipment and professionals required, interpretate easily and clearly, and could detect KPC, NDM, and OXA-48-like enzymes and their combination efficiently and rapidly. However, IMP (IMP-4, IMP-26), which is referred to as one of "the big five" and had prevalence in China [27], could not be identified. The recently launched Carba5 system [13], the RESIST-4 O.K.N.V. assay [28], and the RESIST-5 O.K.N.V.I. assay [29] could perform a multiple lateral flow immunoassay on NDM, KPC, IMP, VIM, and OXA-48-like enzymes, and can detect multiple enzymes produced by strains.
The mCIM and eCIM, recommended by CLSI guidelines for their high sensitivity and specificity in the detection of KPC, NDM, VIM, IMP, IMI, SPM, SME, and OXA enzymes carried by Enterobacterales bacteria [30], could tell serine β-lactamase (SBL) and metalloβ-lactamase (MBL), which will affect the clinical therapy decisions, such as the use of ceftazidime-avibactam [28]. However, it seemed insufficient when classified IMP and OXA-48-like enzyme. The sCIM test was an easy, rapid, efficient, and cheap method, but the results might be influenced by the amount of bacteria on the imipenem disk, since the result was negative with a small bacterial sample, but could become "uncertain" with more abundant bacteria in the tested sample.

Bacterial Strains
A total of 217 routine clinical isolates was tested to evaluate the performance of six phenotypic carbapenemase detection and classification assays, i.e., BD Phoenix CPO detect panel, Rapidec Carba-NP, an O.K.N detection kit, and three carbapenem inactivation methods (CIMs) (mCIM, eCIM, and sCIM). These isolates include 178 CPOs and 39 non-CPOs, the carbapenemase activity of which was previously tested by the mCIM and eCIM test according to CLSI guidelines [30], and then genetically confirmed by traditional PCR and sequencing. The primers used to amplify specific carbapenenmase genes are shown in Table 5. Strains collected from 66 general hospitals in 23 provinces of China from 2011 to 2018 were stored at −70 • C and incubated overnight on Columbia agar plate with 5% sheep blood at 35 ± 1 • C. To ensure the purity of strains, sub-incubation was carried out. Identification was conducted by matrix-assisted laser desorption ionizationtime of flight mass spectrometry (Bruker Daltonics, Bremen, Germany) using α-cyano-4hydroxycinnamic acid matrix and standard protocol. Species identification was determined by scores > 2.

BD Phoenix CPO Detect Panel
The BD Phoenix CPO detect panel (Becton, Dickinson and Company, Sparks, MD, USA) is a growth-based qualitative confirmatory method (ID/NMIC 503 panel, NMIC 502 panel), which could identify species and use drugs such as carbapenems and β-lactams, alone and in combination with various chelators and β-lactamase inhibitors required for CPO detection and classification to detect and confirm A, B, and D carbapenemase [17]. In the CPO detect panel, the ID/NMIC 503 panel aimed to identify species and detect carbapenemase activity (P/N test), and the NMIC 502 panel aimed to identify carbapenemase producers according to the Ambler classification (Ambler test). Not all isolates determined carbapenemase positive by the P/N test could be classified by the Ambler test. The P/N test-negative results with Ambler classification were previously demonstrated to be impossible [21]. Strains of 141 carbapenemase-producing Enterobacterales (CPE), 23 carbapenemase-producing Acinetobacter baumannii (CPAB), and 39 non-CPOs were incubated overnight using the BD M50 system, and results were interpreted by Epi-Centre software.

Rapidec Carba-NP Test
The Rapidec Carba-NP kit (bioMérieux, La Balme-les-Grottes, France) is a rapid biochemical test based on detecting the hydrolysis of the β-lactam ring of imipenem. The test was performed in accordance with the manufacturer's recommendations [31]. A full loop (10 µL) of bacterial colonies from a culture plate was mixed into the well c so that the turbidity is similar to well b. Visual observations then occurred after 30 min incubation at 37 • C and, if necessary, after 2 h. A color change from red-to-yellow or red-to-orange was considered positive. The visual assessment and results interpretation were conducted blindly by two technicians who had no information about carbapenemase production of these strains.

O.K.N Detection Kit
Then, 200 isolates were tested by O.K.N Detection kit (Beijing Gold Mountainriver Tech Development Co., Ltd., Beijing, China). A single colony on blood agar plate was suspended in 10 drops of lysis solution, and then three drops of the diluted sample were added onto the test strip. The results were read within 15 min with naked eyes according to the synopsis.

Carbapenem Inactivation Methods
The mCIM and eCIM tests were performed following the protocol recommended by the Clinical Laboratory Standards Institute (CLSI) guidelines [30]. In the sCIM test [32], one side of the imipenem disk was smeared with the tested strain instead of soaked in tryptic soy broth (TSB) containing the strain for 4 h, and then placed on the MHA plate inoculated with meropenem-susceptible Escherichia coli ATCC 25922.

Quality Control
K. pneumoniae BAA-1705 (producing KPC), E. coli ATCC-2452 (producing NDM), and E. coli BAA-2523 (producing OXA-48) were used as positive quality control strains, whereas K. pneumoniae ATCC 700603 was used as a negative quality control in evaluation of the BD Phoenix CPO detection panel. In the CIM tests, E. coli ATCC 25922 and K. pneumoniae BAA 1705 were used as negative and positive quality control strains, respectively.

Statistical Analysis
The sensitivity and specificity of each method were analyzed along with their 95% confidence intervals, and compared with the results from reference standard molecular characterization using the free software VassarStats (http://vassarstats.net/; 28 September 2021).

Conclusions
Taken together, the results of this study suggest that the BD Phoenix CPO detect P/N test, mCIM and sCIM were reliable for CPOs screening and detecting. However, due to the relatively low detection specificity and low classification sensitivity, additional tests and further improvements are required to confirm positive detection by the BD Phoenix CPO detect panel P/N test and to improve the performance of the Ambler test. Moreover, when the result of Rapidec Carba-NP was interpreted negative, additional tests such as mCIM and sCIM were needed to exclude the OXA-48-like enzyme. The O.K.N detection kit with high classification efficiency indicates a rapid, easy and reliable tool to classify KPC, NDM, and OXA-48-like enzymes.