Next Article in Journal
Preventive Antibiotic Prescribing Habits among Professionals Dedicated to Oral Implantology: An Observational Study
Previous Article in Journal
Mortality Following Clostridioides difficile Infection in Europe: A Retrospective Multicenter Case-Control Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Microscopic Analysis of Bacterial Inoculum Effect Using Micropatterned Biochip

1
Department of Electrical and Computer Engineering, Undergraduate School, Michigan State University, 426 Auditorium Road, East Lansing, MI 48824, USA
2
Department of Mechanical Engineering, Undergraduate School, Kookmin University, 77 Jeongneung-ro, Seongbuk-gu, Seoul 02707, Korea
3
School of Mechanical Engineering, Kookmin University, 77 Jeongneung-ro, Seongbuk-gu, Seoul 02707, Korea
*
Author to whom correspondence should be addressed.
Antibiotics 2021, 10(3), 300; https://doi.org/10.3390/antibiotics10030300
Submission received: 2 March 2021 / Revised: 10 March 2021 / Accepted: 12 March 2021 / Published: 13 March 2021
(This article belongs to the Section Antibiotics Use and Antimicrobial Stewardship)

Abstract

:
Antimicrobial resistance has become a major problem in public health and clinical environments. Against this background, antibiotic susceptibility testing (AST) has become necessary to cure diseases in an appropriate and timely manner as it indicates the necessary concentration of antibiotics. Recently, microfluidic based rapid AST methods using microscopic analysis have been shown to reduce the time needed for the determination of the proper antibiotics. However, owing to the inoculum effect, the accurate measurement of the minimal inhibitory concentration (MIC) is difficult. We tested four standard bacteria: Staphylococcus aureus, Pseudomonas aeruginosa, Escherichia coli, and Enterococcus faecalis, against five different antibiotics: piperacillin, cefotaxime, amikacin, levofloxacin, and ampicillin. The results showed that overall, the microfluidic system has a similar inoculum effect compared to the conventional AST method. However, due to the different testing conditions and determination protocols of the growth of the microfluidic based rapid AST, a few results are not identical to the conventional methods using optical density. This result suggests that microfluidic based rapid AST methods require further research on the inoculum effect for practical use in hospitals and can then be used for effective antibiotic prescriptions.

1. Introduction

The emergence of antimicrobial resistance is a major issue globally. The pathogens exhibiting this resistance are called “superbugs,” which pose a huge threat not only in hospitals but also in the community [1,2]. The World Health Organization announced that the dangers posed by antibiotic resistance have reached a serious threat level [3]. In addition, the US Centers for Disease Control and Prevention reported that people globally are at risk through the paradigm shift to a “post-antibiotic era” [4]. Antibiotic susceptibility testing (AST) is becoming increasingly important to counter this risk of antibiotic-resistant bacteria, not only in humans but also in animals [5]. AST is a clinical method for detecting bacterial resistance or observing the susceptibility of drugs as opposed to pathogens, enabling determination of the minimal inhibitory concentration (MIC) in each case [6]. Several AST methods have been established, such as the disk diffusion method based on solid-plate AST, the broth dilution method for liquid AST, and polymerase chain reaction-based methods [7]. Recently, numerous rapid AST methods based on microfluidic systems with microscopic analysis have been developed to reduce the time for AST and to increase the accuracy, but the inoculum effects of those methods have not been studied [8,9,10,11,12].
The inoculum effect is a phenomenon that is explained as an increase in MIC, which depends on the increase in the inoculated density of pathogens [13,14]. From a clinical perspective, the inoculum effect is very important because MIC values can be overestimated due to it. As a result of this, insufficient amounts of antibiotics can be administered, which would increase the mortality rate of patients [15]. In addition, owing to the overuse and misuse of antibiotics, bacterial strains are more likely to gain resistance to antibiotics, facilitating the emergence of highly resistant pathogens [16]. Given the development of new AST methods based on a microfluidic system using microscopic analysis, it is important to study the inoculum effect with these methods to prevent the associated misuse of antibiotics.
Here, we investigate the inoculum effect by image-based AST (IBAST), which is based on a microfluidic biochip. Although conventional AST methods require 16–20 h, as well as being labor-intensive and inconvenient, IBAST can be done in only 4–6 h, depending on the bacterial strain. In addition, the system utilizes an automated microscope, which saves a lot of time and labor compared with conventional AST methods [12,17]. As a rapid AST with an automated imaging system, this method has good potential to be used clinically. When we study the inoculum effect, we also change the volume of the culture medium with the antibiotic to investigate its effect on the inoculum effect. Consequently, IBAST was conducted against two Gram-negative strains, P. aeruginosa ATCC 27853 and E. coli ATCC 25922, and two Gram-positive strains, S. aureus ATCC 29213 and E. faecalis ATCC 29212, using five different antibiotics, namely, ampicillin, cefotaxime, piperacillin, levofloxacin, and amikacin, at three different bacterial concentrations: 5 × 105, 5 × 106, and 5 × 107 CFU/mL. We also investigated the effect of the volume of the culture medium with antibiotics on the inoculum effect. This research aids the development of rapid AST based on a microfluidic system with microscopic analysis to understand the inoculum effect and utilize the system in clinical areas appropriately.

2. Results

2.1. Inoculum Effect of Staphylococcus aureus ATCC 29213

IBAST was performed on S. aureus ATCC 29213 with three β-lactam antibiotics, ampicillin, cefotaxime, and piperacillin, and two non-β-lactam antibiotics, levofloxacin and amikacin. Figure 1a shows processed images of S. aureus ATCC 29213 of three different inocula upon 6 h of incubation after exposure to piperacillin. At 5 × 105 CFU/mL, MIC was determined to be 1 μg/mL. As the bacterial density increased to 5 × 106 CFU/mL and 5 × 107 CFU/mL, MIC increased to 2 and 4 μg/mL, respectively. Therefore, Figure 1a demonstrates the significant inoculum effect of S. aureus to piperacillin. In contrast, Figure 1b shows processed images of S. aureus exposed to cefotaxime after incubation for 6 h. As seen in the images, MIC was identical at 2 μg/mL and there were no changes of it among the three different bacterial concentrations.
S. aureus ATCC 29213 was also monitored after 6 h of incubation with different antibiotics. In a normal volume of the culture medium of 90 μL, the MIC values were identical in the case of cefotaxime (Table 1). When the bacterial density was increased tenfold from the initial concentration of 5 × 105 CFU/mL, the MIC values of ampicillin, levofloxacin, and piperacillin doubled. Moreover, when the bacterial concentration was increased to 5 × 107 CFU/mL, the MICs of amikacin and piperacillin doubled again, while that of ampicillin increased fourfold. On the other hand, that of levofloxacin did not change at 5 × 107 CFU/mL.
To investigate the variation of MICs in more detail to represent the inoculum effect, different volumes of culture media with antibiotics were used: 45 and 135 μL. Table 1 presents the MIC values in relation to the volumes of culture media with antibiotics. In the case of ampicillin, the MIC did not change with volume at a density of 5 × 105 CFU/mL. However, at 5 × 106 CFU/mL inoculum, the MIC increased to twofold at 45 μL compared to 90 and 135 μL. At the 5 × 107 CFU/mL inoculum, the MIC increased to 4 μg/mL at 45 μL. For levofloxacin, the MIC increased from 0.12 to 0.25 μg/mL in 5 × 105 and 5 × 106 CFU/mL at 90 and 135 μL. However, in the other cases, the values were static at 0.25 μg/mL, with no change. In the case of amikacin, when the bacterial density increased from 5 × 106 to 5 × 107 CFU/mL, all the MICs of the volumes of the culture media with antibiotics doubled. At the same bacterial concentration, the volume of the culture medium with antibiotics did not affect the MICs. In the case of piperacillin, when the bacterial density was increased tenfold from the initial level, the MICs were increased by twofold in 90 μL of culture medium. When 45 μL of the culture medium with piperacillin was injected, the MIC increased fourfold between 5 × 106 and 5 × 107 CFU/mL. In general, the low volume of the culture medium with antibiotics contributes to the inoculum effect in S. aureus with ampicillin, amikacin, and piperacillin. As shown in Table 1, when S. aureus was exposed to ampicillin at densities of 5 × 106 and 5 × 107 CFU/mL, the MIC values were decreased as the amount of the culture medium increased. In addition, at 5 × 105 CFU/mL for levofloxacin and at 5 × 105 CFU/mL for piperacillin, the MIC values were reduced. Given that all antibiotics used in our experiments have binding process mechanisms that are irreversible [18,19], we concluded that, if the amount of antibiotics increases, this means that the antibiotics are more likely to bind to the bacteria, which in turn leads to a decrease in the MIC.

2.2. Inoculum Effect of Pseudomonas aeruginosa ATCC 27853

In the case of P. aeruginosa ATCC 27853, all antimicrobial agents showed inoculum effects under normal conditions with 90 μL of culture medium. This pathogen was exposed to each antibiotic for 6 h. As seen in Table 2, all MICs increased twofold when the bacterial density was increased from 5 × 105 to 5 × 106 CFU/mL. However, when the density was increased from 5 × 106 to 5 × 107, the MIC of amikacin increased twofold as from 5 × 105 to 5 × 106, but the MIC of piperacillin increased over fourfold, while in the cases of levofloxacin and cefotaxime there was no change.
When P. aeruginosa was exposed to four different antimicrobial agents, namely, cefotaxime, levofloxacin, amikacin, and piperacillin, at 5 × 105 CFU/mL inoculum, all the MIC values were static with different amounts of the culture medium with antibiotics (Table 2). In the cases of levofloxacin, amikacin, and piperacillin, the MIC values were doubled when the bacterial inoculum was increased tenfold from the initial concentration. Cefotaxime showed different MIC values at each amount of the culture medium with antibiotics. The MIC was decreased when the amount of culture medium with antibiotics was increased from 5 × 106 to 5 × 107 CFU/mL. However, when 135 μL of cefotaxime was injected, the MIC was increased again at 5 × 107 CFU/mL. Levofloxacin did not change the MIC, even when the bacterial density was increased from 5 × 106 to 5 × 107 CFU/mL. The MIC of amikacin was doubled, and piperacillin was increased more than fourfold as the inoculum size increased from 5 × 106 to 5 × 107 CFU/mL, regardless of the amount of antibiotics. Overall, P. aeruginosa showed an inoculum effect for all four antibiotics and was not affected by the amount of the culture medium with antibiotics, except in the case of cefotaxime.
As seen in Table 2, in the case of cefotaxime, the MIC value was decreased with the volume of the culture medium at the same bacterial density: 5 × 106 CFU/mL. This can be interpreted as showing that an increased amount of antibiotics influences the bactericidal effect more efficiently. Owing to the binding mechanism of antibiotics, cefotaxime has an irreversible inhibition process [19]. However, at a density of 5 × 107 CFU/mL, the MIC was increased again when the inoculated antibiotics increased from 90 to 135 μL. We guessed that this was because of the increased volume of the culture medium. The antibiotics were diluted with culture medium, so that if we inoculated antimicrobial agents, we also inoculated more of the medium into the well. If the medium increased, this meant that the bacteria had good conditions to grow in, even if the overall amount of antibiotics increased. Therefore, positive effects on growth occurred, resulting in MIC augmentation.

2.3. Inoculum Effect of Escherichia coli ATCC 25922

E. coli ATCC 25922 was monitored after 4 h of incubation with antimicrobial agents (Table 3). When the pathogen was exposed to ampicillin, levofloxacin, and piperacillin, the MICs were not changed under the normal conditions of 90 μL of culture medium. However, in the case of cefotaxime, the MIC was increased at 5 × 106 CFU/mL. Similarly, when amikacin was applied to E. coli, the MIC doubled at 5 × 107 CFU/mL.
When E. coli was exposed to cefotaxime with different amounts of the culture medium, the MIC changed (Table 3). At a density of 5 × 105 CFU/mL, the MIC decreased as the amount of antibiotics increased from 45 to 90 μL. As the bacterial density increased tenfold, the MIC increased at doses of 90 and 135 μL. In addition, when the bacterial concentration was 5 × 107 CFU/mL, the MIC significantly increased at doses of 45 and 135 μL. However, at a 90 μL dose of cefotaxime, the MIC remained at 0.12 μg/mL. In addition, amikacin was also tested with different amounts of the culture medium. The only feature that changed when amikacin was used occurred at a bacterial density of 5 × 107 CFU/mL. Specifically, the MIC value at 5 × 107 CFU/mL was increased twofold compared with that at 5 × 106 CFU/mL. Overall, E. coli did not show an inoculum effect among the four antibiotics, except cefotaxime.
As shown in Table 3, at 5 × 105 CFU/mL, the MIC of cefotaxime decreased as the amount of the culture medium increased. This phenomenon is also shown in Table 2, like in the case of P. aeruginosa against cefotaxime at 5 × 106 CFU/mL. In addition, at 5 × 107 CFU/mL, the MIC was increased as the amount of antibiotics increased from 90 to 135 μL.

2.4. Inoculum Effect of Enterococcus faecalis ATCC 29212

As seen in Table 4, E. faecalis ATCC 29212 was observed after 6 h of incubation with antibiotics. Only levofloxacin showed an inoculum effect, while the results for the other agents were static, even if the number of pathogens increased. When the bacterial organisms were increased tenfold from the initial density with levofloxacin, the MIC doubled from 5 × 105 to 5 × 106 CFU/mL, but it did not change from 5 × 106 to 5 × 107 CFU/mL. In the case of levofloxacin, different amounts of the culture medium were applied to E. faecalis. Among the three volumes of culture media, the MICs doubled when the bacterial density was increased tenfold from the initial concentration. However, unlike the other cases, the only change occurred at 5 × 107 CFU/mL at a dose of 45 μL where the MIC was doubled again.

3. Discussion

IBAST is a system that combines microfluidic chips, microscopic images, and image processing technology. Through this, we can get AST results quickly compared to the conventional methods. The conventional AST method based on turbidity occurred inoculum effect, so the experiment was conducted in anticipation of this phenomenon in IBAST. As a result, there were cases of inoculum effect depending on the combination of bacteria and antibiotics, and in this case, the experiment was conducted by changing the amount of badges.
In IBAST, S. aureus showed an inoculum effect against ampicillin and piperacillin (Table 1). These results may be due to the mechanism of action of β-lactam antibiotics, such as ampicillin and piperacillin. β-Lactam antibiotics bind with penicillin-binding proteins (PBPs), which inhibit cell wall construction [19]. They not only inhibit bacterial cell wall synthesis but also cause morphological changes. For instance, the filamentous elongation or swelling of bacteria occurs when β-lactams are exposed; this results in cell lysis, eventually leading to cell death. In accordance with this, the MIC results of ampicillin and piperacillin against S. aureus showed clear inoculum effect according to the inoculum size (Table 1). This corresponds with the findings of several other experiments [20,21]. However, although cefotaxime is one of the β-lactam antibiotics, it did not show any inoculum effect in S. aureus in IBAST, which is in agreement with other previous experiments [22,23]. Previous studies showed that, when the bacterial inoculum increased, the MIC of cefotaxime tended not to increase [22,23]. These results may be due to the antibiotic class of cefotaxime. While ampicillin and piperacillin are within the penicillin group, cefotaxime is a subclass of cephalosporins. Given their inhibition of cell wall construction, the two antibiotics broadly have the same mechanism of action, but they have different PBP affinities. Mizuguchi et al. showed the difference of PBP affinity between ampicillin and cefazolin, which is one of the cephalosporins like cefotaxime [24]. In the case of non-β-lactam antibiotics on S. aureus, amikacin showed an inoculum effect (Table 1). Amikacin is a subclass of aminoglycosides, which are known as inhibitors of protein synthesis. The primary target of amikacin is the 30S ribosomal subunit, which is necessary for protein synthesis. Unlike tetracyclines, aminoglycoside inhibition is irreversible, which means that if binding has been completed, recycling is not possible. Therefore, the bacterial densities were found to depend on the antibiotic concentration [20,25]. Studies have suggested that this may be due to natural selection. As the bacterial density increases, it will enhance the possibility of bacterial subpopulations gaining resistance, and eventually a number of mutations will emerge, reducing antibiotic susceptibility. Levofloxacin also showed an inoculum effect against S. aureus (Table 1). Levofloxacin belongs to the subclass of fluoroquinolones, which are DNA synthesis inhibitors whose primary target is DNA gyrase. Levofloxacin was proven to be effective against Gram-positive and Gram-negative bacteria. Furthermore, in the present study, the MICs of levofloxacin against Gram-positive bacteria were significantly similar to those of ciprofloxacin. Several studies have shown that ciprofloxacin had a weak inoculum effect, as did levofloxacin [26,27], which is in agreement with our experimental results shown in Table 1.
P. aeruginosa showed an inoculum effect in all cases, regardless of the antibiotic (Table 2). In the case of piperacillin, a subclass of β-lactam antibiotics, previous studies also indicated that it showed an inoculum effect against P. aeruginosa [28,29]. When pathogens are present in a large number, they are more likely to survive because of the resistant mutants that develop via selection. Because of the combined production of β-lactamase, the bacteria become less susceptible to the antibiotics because the agents are broken down by inactivating the enzymes of other antibiotic-targeted pathogens. In addition, the preferential affinity of PBPs could be another possible reason for this. In this regard, the augmentation of bacterial concentration may cause inadequate saturation of the protein binding sites, which can result in a bactericidal effect. In our experiment, as shown in Table 2, at bacterial density of 5 × 107 CFU/mL, we observed an abrupt increase in the MIC. Moreover, even if the bacterial growth was inhibited and there was potentially a decrease in the count of viable pathogens, they were still metabolically active. Therefore, the entire bacterial mass increased via morphological changes such as elongation. As such, the MIC result can be recognized as significantly high at a density of 5 × 107 CFU/mL. In the case of cefotaxime, several studies have indicated that it showed an inoculum effect [29,30,31,32]. A large difference of MIC values was also previously identified, which agreed with our results. These results may be due to the mechanism of action of β-lactam antibiotics and can be explained in the same way as for piperacillin. Amikacin showed an inoculum effect against P. aeruginosa in IBAST. Unlike piperacillin, amikacin is a subclass of aminoglycosides, so it does not have any β-lactam rings in its chemical structure. Meanwhile, an inoculum effect is largely related to the inactivation of antibiotics by β-lactamase enzymes. Therefore, amikacin showed less inoculum effect in comparison with piperacillin, which is an agreement with our experiment [28,33]. Levofloxacin showed a limited inoculum effect against P. aeruginosa. With respect to non-β-lactam antibiotics, the pattern of the MIC values of levofloxacin is like that of amikacin. As we mentioned previously, levofloxacin is a subclass of fluoroquinolones and its activity is like that of ciprofloxacin [26,34,35,36]. Therefore, it was concluded that fluoroquinolones were little affected by inoculum size, which corresponded with our experiment. In addition, another study showed that ciprofloxacin had little inoculum effect [37].
E. coli is associated with an inoculum effect in the case of cefotaxime (Table 3). Several studies have shown that when E. coli was exposed to cefotaxime, the MIC tended to increase [38,39,40]. Two factors can explain the in vitro activity of the β-lactam drug with different inoculum sizes. The first is the intrinsic activity of the antimicrobial agents against the inoculated pathogens. The second is that susceptibility of the antibiotics to hydrolysis by β-lactamase of the pathogens. In other words, it was proposed that the inoculum effect depends on the amount of hydrolysis. However, another study clarified that β-lactamase activity was not the only factor behind the inoculum effect [38]. Moreover, it was suggested that β-lactam antibiotics can exhibit different degrees of morphological changes with different pathogens, which may result in different degrees of inoculum effect. In this regard, we concluded that E. coli has an inoculum effect against cefotaxime. Our results agree with those of other experiments showing moderate inoculum effects of cefotaxime [38,39,40]. Amikacin also showed an inoculum effect in IBAST. Amikacin, a subclass of aminoglycoside, combines with the 30S ribosomal subunit, which is an irreversible process. Owing to this characterization, we concluded that E. coli has a moderate inoculum effect against amikacin, which corresponds with the findings of other studies [20,25,41]. Unlike cefotaxime, ampicillin, and piperacillin did not show a significant inoculum effect in IBAST. According to previous studies, ampicillin had very little or no inoculum effect and piperacillin showed a moderate inoculum effect [15,42]. In one of these studies, it was asserted that there are two possible explanations for this: antibiotic destruction by β-lactamases and morphological changes such as filamentous transformation with continued growth [42]. The results of IBAST for ampicillin corresponded to the findings of other studies, while the results for piperacillin did not [20,39,42]. Neither of these antibiotics changed the MIC values. Moreover, in our experiment, we determined the MIC as 80% inhibition compared with that for a control well. We guessed that this evaluation standard may result in different results compared with other studies. In addition, penicillin and piperacillin are subclasses of penicillin, so we guessed that these two antimicrobial agents have different PBP affinities from cefotaxime, which may cause different results for the MIC. While levofloxacin had no inoculum effect against E. coli in IBAST, it had a minimal inoculum effect against E. coli in previous study [41]. We guessed that the fact that the determination of the MIC by IBAST involves counting the colonies from a single bacterium and the 80% evaluation standard would impact on the calculation of the MIC during image processing.
E. faecalis is a Gram-positive bacterium resembling S. aureus. However, unlike S. aureus, E. faecalis had no inoculum effect against ampicillin and piperacillin, as shown in Table 4. These two are β-lactam antibiotics, which are subclasses of penicillin. We guessed that penicillin tends not to be affected by the inoculum size of E. faecalis. In the previous study, experiments on the inoculum effect of E. faecalis against ampicillin and piperacillin were conducted [43], the results of which do not agree with our results. We concluded that these different results could be due to the method of determining the MIC in IBAST or due to the different isolated strains of bacteria. We used the E. faecalis ATCC 29212 strain, but other studies did not. While one study used the Enterococcal strain HH22 [44], another study utilized Enterococcus faecalis with resistance to aminoglycoside and showing β-lactamase-producing ability [45], and another study also used β-lactamase-producing strains. On the other hand, E. faecalis presents a minimal inoculum effect against levofloxacin. As we discussed before, levofloxacin exhibits activity similar to that of ciprofloxacin. According to previous studies, ciprofloxacin had an inoculum effect against E. faecalis. Therefore, we concluded that levofloxacin may have an inoculum effect against E. faecalis, which is in agreement with previous study [37,43].

4. Materials and Methods

4.1. Fabrication of the IBAST Chip

The IBAST chip was based on a 96-well plate format and was designed using 3D CAD software (SolidWorks v2014; Dassault Systemes SolidWorks Corp., Vélizy-Villacoublay, France). This micropatterned chip was made by injection molding process (NT2–120; Woojin Plaimm, Republic of Korea) of polystyrene (K-RESIN; Chevron Phillips Chemical, Baytown, TX, United States) (Figure 2a). In advance of IBAST, air plasma treatment (CUTE-MP; FemtoScience, Hwaseong-si, Republic of Korea) was performed for 1 min to provide hydrophilicity to the chips, which helps uniform distribution of agarose mixture on the chip.

4.2. Stock Preparation and Subculture of Bacteria

Four standard bacterial strains (P. aeruginosa ATCC 27853, E. coli ATCC 25922, S. aureus ATCC 29213, and E. faecalis ATCC 29212) of Clinical and Laboratory Standards Institute (CLSI) were purchased from MicroBioLogics, Inc. (Saint Cloud, MN, USA). We conducted all the experiments in accordance with the ethical standards from American Type Culture Collection and performed all the experiments in the biosafety cabinet. Bacterial stock solutions were made by mixing 50 mL of glycerol (Sigma-Aldrich, St. Louis, MO, USA) and 50 mL of deionized water. A total of 400 μL of 50% glycerol-deionized water mixture was mixed with 800 μL of bacterial cation adjusted Mueller–Hinton broth (CAMHB; BD Biosciences, San Jose, CA, USA) mixture. These stock solutions were placed in microcentrifuge tubes (SPL Life Sciences, Seoul, Korea) and cryopreserved at −70 °C in a deep freezer. For the IBAST, subcultures were performed before the experiment. Stock solutions from aliquots were inoculated with one loop on Luria–Bertani agar plates (KisanBio Co., Ltd., Seoul, Korea) and were incubated in a 37 °C incubator overnight. After 20 to 24 h of incubation, several colonies were formed and used to generate target concentration of bacteria.

4.3. Antibiotic Preparation

Five antimicrobial agents were used to determine the MIC values of each bacterium. Three β-lactam antibiotics were used: ampicillin, cefotaxime, and piperacillin (Sigma-Aldrich). In addition, levofloxacin (Sigma-Aldrich), a representative quinolone, and amikacin (Sigma-Aldrich), a representative aminoglycoside, were used. These antibiotics were diluted with deionized water to make aliquots and stored at −70 °C in a deep freezer. Each aliquot was made up to a concentration of 10 mg/mL. For the IBAST and to accurately determine the MIC values, these stock solutions were adjusted by mixing MHB medium to match the concentrations of quality control range of CLSI, so that twofold dilutions were made.

4.4. Inoculum Sizes of Bacteria

After incubation, four CLSI standard strains of bacteria were diluted to three concentrations each. To measure inoculum effects, bacterial colonies on agar plates were collected in a loop and mixed with MHB medium to make McFarland 0.5 at approximately 1.5 × 108 CFU/mL. The cell density was calibrated using ultraviolet spectrophotometric devices (DensiCHEK™; bioMerieux Inc., Marcy l’Étoile, France). Then, the suspension was mixed with different amounts of MHB medium to achieve concentrations of 1.5 × 107, 1.5 × 106, and 1.5 × 105 CFU/mL. A total of 200 μL of bacterial solution was mixed with 600 μL of 0.5% agarose in a liquid state at 37 °C. Then, 10 μL of bacterial mixtures were inoculated in each main well of micropatterned chip, followed by solidification at room temperature. A total of 90 μL of antimicrobial agents in culture medium were injected into each side well (Figure 2a). Antibiotics and culture medium diffused toward the bacteria, which was captured using a charge coupled device camera with a 20× microscopic lens (Figure 2b). After the inoculation, the micropatterned chip was incubated in the chamber at 37 °C. In the case of E. coli ATCC 25922, incubation was performed for 4 h. Other pathogens such as P. aeruginosa ATCC 27853, S. aureus ATCC 29213, and E. faecalis ATCC 29212 were incubated for 6 h. Imaging was performed immediately after the incubation, utilizing an automated microscope. Subsequently, all MIC values were compared with results obtained by the broth microdilution (BMD) method to determine whether the results were reliable.

4.5. Determination of Growth of Bacteria

In the IBAST experiment, growth was determined by comparing its bacterial area with a threshold of 20% of the growth in control wells containing only culture medium without antibiotics. This standard was also applied to the broth microdilution test.

4.6. Image Processing

A 20× microscope objective lens was used for imaging and the field of view of one image was 1126.4 × 594 μm. Microfabricated chips were inserted into a microscope device (QuantaMatrix, Inc., Seoul, Korea) and monitored using a complementary metal–oxide–semiconductor camera. To accurately determine the MIC values of each case, images were processed digitally. With the help of MATLAB (version 2019; Natick, MA, USA), the image processing program was coded. RGB raw images were then changed to binary format in order to count the area of bacterial colonies in the image (Figure 3). Then, black and white inversion was performed, so that the background of each image could be discriminated from pathogens. Specifically, the pixel number of white area was counted as the area of bacterial colony. If the imaging data of bacterial growth were lower than 20% compared with the control well, the result was recorded as no growth. Red dots in Figure 3b represent colonies recognized by image processing. In addition, to remove a main well line that presented a shape like a curve at the far right end, a parameter of circularity was used. If the circularity was lower than 0.3, MATLAB eliminated the curve line and did not count it as a bacterium.

5. Conclusions

In this study, we demonstrated IBAST, which is a method of rapid AST utilizing microfluidic chip and microscopic imaging. The entire AST process takes 4–6 h, enabling the fast determination of the proper antibiotics for the patients. It also helps to accurately determine the MIC, considering the inoculum effect. While conventional methods depend on optical density by checking turbidity, IBAST can monitor the colony formation of single bacteria in more detail. Overall, IBAST showed similar patterns of inoculum effect compared with conventional methods. However, in some cases, IBAST showed a different pattern of inoculum effect, due to the different method of determination of the growth of bacteria. To apply IBAST in a clinical context, broad studies related to the inoculum effect are needed. After that, the IBAST system can be applied in numerous clinical environments to avoid the overuse or misuse of antibiotics and save numerous patients from severe diseases.

Author Contributions

Conceptualization, J.C. and J.H.H.; methodology, J.H.H.; software, S.Y.L.; validation, J.H.H. and S.Y.L.; formal analysis, J.C. and J.H.H.; investigation, J.H.H.; resources, J.H.H.; data curation, J.H.H.; writing (original draft preparation), J.C. and J.H.H.; writing (review and editing), J.C. and J.H.H.; visualization, J.H.H.; supervision, J.C.; project administration, J.C.; funding acquisition, J.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (NRF-2019R1A2C1084419).

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors are grateful to QuantaMatrix, Inc., for provision of the imaging analysis system.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Blair, J.M.A.; Webber, M.A.; Baylay, A.J.; Ogbolu, D.O.; Piddock, L.J.V. Molecular mechanisms of antibiotic resistance. Nat. Rev. Microbiol. 2015, 13, 42–51. [Google Scholar] [CrossRef]
  2. Coll, F.; Harrison, E.M.; Toleman, M.S.; Reuter, S.; Raven, K.E.; Blane, B.; Palmer, B.; Kappeler, A.R.M.; Brown, N.M.; Torok, M.E.; et al. Longitudinal genomic surveillance of MRSA in the UK reveals transmission patterns in hospitals and the community. Sci. Transl. Med. 2017, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. WHO. Antimicrobial Resistance; Global report on surveillance; World Health Organization: Geneva, Switzerland, 2014. [Google Scholar] [CrossRef]
  4. CDC. Antibiotic Resistance Threats in the United States, 2013; CDC: Atlanta, GA, USA, 2013. [Google Scholar] [CrossRef] [Green Version]
  5. Balouiri, M.; Sadiki, M.; Ibnsouda, S.K. Methods for in vitro evaluating antimicrobial activity: A review. J. Pharm. Anal. 2016, 6, 71–79. [Google Scholar] [CrossRef] [Green Version]
  6. Wiegand, I.; Hilpert, K.; Hancock, R.E.W. Agar and broth dilution methods to determine the minimal inhibitory concentration (MIC) of antimicrobial substances. Nat. Protoc. 2008. [Google Scholar] [CrossRef]
  7. Jorgensen, J.H.; Turnidge, J.D. Susceptibility Test Methods: Dilution and Disk Diffusion Methods. Man. Clin. Microbiol. 2015. [Google Scholar] [CrossRef]
  8. Kara, V.; Duan, C.H.; Gupta, K.; Kurosawa, S.; Stearns-Kurosawa, D.J.; Ekinci, K.L. Microfluidic detection of movements of Escherichia coli for rapid antibiotic susceptibility testing. Lab Chip 2018, 18, 743–753. [Google Scholar] [CrossRef]
  9. Li, Y.Y.; Yang, X.; Zhao, W.A. Emerging Microtechnologies and Automated Systems for Rapid Bacterial Identification and Antibiotic Susceptibility Testing. Slas. Technol. 2017, 22, 585–608. [Google Scholar] [CrossRef] [Green Version]
  10. Rossello, G.A.M.; Perez, M.A.B. Rapid antibiotic susceptibility test in Clinical Microbiology. Enferm. Infec. Micr. Clin. 2016, 34, 61–68. [Google Scholar] [CrossRef]
  11. Wistrand-Yuen, P.; Malmberg, C.; Fatsis-Kavalopoulos, N.; Lubke, M.; Tangden, T.; Kreuger, J. A Multiplex Fluidic Chip for Rapid Phenotypic Antibiotic Susceptibility Testing. Mbio 2020, 11. [Google Scholar] [CrossRef] [Green Version]
  12. Choi, J.; Yoo, J.; Lee, M.; Kim, E.G.; Lee, J.S.; Lee, S.; Joo, S.; Song, S.H.; Kim, E.C.; Lee, J.C.; et al. A rapid antimicrobial susceptibility test based on single-cell morphological analysis. Sci. Transl. Med. 2014, 6. [Google Scholar] [CrossRef]
  13. Lukačišinová, M.; Bollenbach, T. Toward a quantitative understanding of antibiotic resistance evolution. Curr. Opin. Biotechnol. 2017. [Google Scholar] [CrossRef]
  14. Nicoloff, H.; Andersson, D.I. Indirect resistance to several classes of antibiotics in cocultures with resistant bacteria expressing antibiotic-modifying or -degrading enzymes. J. Antimicrob. Chemother. 2016, 71, 100–110. [Google Scholar] [CrossRef]
  15. Soriano, F.; Ponte, C.; Santamaria, M.; Jimenez-Arriero, M. Relevance of the inoculum effect of antibiotics in the outcome of experimental infections caused by Escherichia coli. J. Antimicrob. Chemother. 1990, 25, 621–627. [Google Scholar] [CrossRef]
  16. Martinez, J.L.; Baquero, F. Mutation frequencies and antibiotic resistance. Antimicrob. Agents Chemother. 2000. [Google Scholar] [CrossRef] [Green Version]
  17. Choi, J.; Jung, Y.G.; Kim, J.; Kim, S.; Jung, Y.; Na, H.; Kwon, S. Rapid antibiotic susceptibility testing by tracking single cell growth in a microfluidic agarose channel system. Lab Chip 2013, 13, 280–287. [Google Scholar] [CrossRef]
  18. Georgopapadakou, N.H. Penicillin-binding proteins and bacterial resistance to β-lactams. Antimicrob. Agents Chemother. 1993. [Google Scholar] [CrossRef] [Green Version]
  19. Kohanski, M.A.; Dwyer, D.J.; Collins, J.J. How antibiotics kill bacteria: From targets to networks. Nat. Rev. Microbiol. 2010. [Google Scholar] [CrossRef] [Green Version]
  20. Konig, C.; Simmen, H.P.; Blaser, J. Bacterial concentrations in pus and infected peritoneal fluid--implications for bactericidal activity of antibiotics. J. Antimicrob. Chemother. 1998, 42, 227–232. [Google Scholar] [CrossRef]
  21. Song, K.H.; Jung, S.I.; Lee, S.; Park, S.; Kim, E.S.; Park, K.H.; Park, W.B.; Choe, P.G.; Kim, Y.K.; Kwak, Y.G.; et al. Inoculum effect of methicillin-susceptible Staphylococcus aureus against broad-spectrum beta-lactam antibiotics. Eur. J. Clin. Microbiol. Infect. Dis. 2019, 38, 67–74. [Google Scholar] [CrossRef]
  22. Palmer, S.M.; Kang, S.L.; Cappelletty, D.M.; Rybak, M.J. Bactericidal killing activities of cefepime, ceftazidime, cefotaxime, and ceftriaxone against Staphylococcus aureus and beta-lactamase-producing strains of Enterobacter aerogenes and Klebsiella pneumoniae in an in vitro infection model. Antimicrob. Agents. Chemother. 1995, 39, 1764–1771. [Google Scholar] [CrossRef] [Green Version]
  23. Steckelberg, J.M.; Rouse, M.S.; Tallan, B.M.; Osmon, D.R.; Henry, N.K.; Wilson, W.R. Relative efficacies of broad-spectrum cephalosporins for treatment of methicillin-susceptible Staphylococcus aureus experimental infective endocarditis. Antimicrob. Agents Chemother. 1993, 37, 554–558. [Google Scholar] [CrossRef] [Green Version]
  24. Mizuguchi, Y.; Ogawa, M.; Udou, T. Morphological changes induced by beta-lactam antibiotics in Mycobacterium avium-intracellulare complex. Antimicrob. Agents Chemother. 1985, 27, 541–547. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Li, R.C.; Ma, H.H. Parameterization of inoculum effect via mathematical modeling: Aminoglycosides against Staphylococcus aureus and Escherichia coli. J. Chemother. 1998, 10, 203–207. [Google Scholar] [CrossRef] [PubMed]
  26. Li, J.; Xie, S.; Ahmed, S.; Wang, F.; Gu, Y.; Zhang, C.; Chai, X.; Wu, Y.; Cai, J.; Cheng, G. Antimicrobial activity and resistance: Influencing factors. Front. Pharmacol. 2017. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Mizunaga, S.; Kamiyama, T.; Fukuda, Y.; Takahata, M.; Mitsuyama, J. Influence of inoculum size of Staphylococcus aureus and Pseudomonas aeruginosa on in vitro activities and in vivo efficacy of fluoroquinolones and carbapenems. J. Antimicrob. Chemother. 2005, 56, 91–96. [Google Scholar] [CrossRef] [Green Version]
  28. Chan, E.; Zhou, S.; Srikumar, S.; Duan, W. Use of in vitro critical inhibitory concentration, a novel approach to predict in vivo synergistic bactericidal effect of combined amikacin and piperacillin against Pseudomonas aeruginosa in a systemic rat infection model. Pharm. Res. 2006, 23, 729–741. [Google Scholar] [CrossRef]
  29. Eng, R.H.; Smith, S.M.; Cherubin, C. Inoculum effect of new beta-lactam antibiotics on Pseudomonas aeruginosa. Antimicrob. Agents Chemother. 1984, 26, 42–47. [Google Scholar] [CrossRef] [Green Version]
  30. Brook, I. Inoculum effect. Rev. Infect. Dis. 1989, 11, 361–368. [Google Scholar] [CrossRef]
  31. Corrado, M.L.; Landesman, S.H.; Cherubin, C.E. Influence of inoculum size on activity of cefoperazone, cefotaxime, moxalactam, piperacillin, and N-formimidoyl thienamycin (MK0787) against Pseudomonas aeruginosa. Antimicrob. Agents Chemother. 1980, 18, 893–896. [Google Scholar] [CrossRef] [Green Version]
  32. Masuyoshi, S.; Arai, S.; Miyamoto, M.; Mitsuhashi, S. In vitro antimicrobial activity of cefotaxime, a new cephalosporin. Antimicrob. Agents Chemother. 1980, 18, 1–8. [Google Scholar] [CrossRef] [Green Version]
  33. Kresken, M.; Korber-Irrgang, B.; Lauffer, J.; Decker-Burgard, S.; Davies, T. In vitro activities of ceftobiprole combined with amikacin or levofloxacin against Pseudomonas aeruginosa: Evidence of a synergistic effect using time-kill methodology. Int. J. Antimicrob. Agents 2011, 38, 70–75. [Google Scholar] [CrossRef]
  34. Gombert, M.E.; Aulicino, T.M. Comparison of agar dilution, microtitre broth dilution and tube macrodilution susceptibility testing of ciprofloxacin against several pathogens at two different inocula. J. Antimicrob. Chemother. 1985, 16, 709–712. [Google Scholar] [CrossRef]
  35. Odenholt, I.; Lowdin, E.; Cars, O. Bactericidal effects of levofloxacin in comparison with those of ciprofloxacin and sparfloxacin. Clin. Microbiol. Infect. 1998, 4, 264–270. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Firsov, A.A.; Vostrov, S.N.; Kononenko, O.V.; Zinner, S.H.; Portnoy, Y.A. Prediction of the effects of inoculum size on the antimicrobial action of trovafloxacin and ciprofloxacin against Staphylococcus aureus and Escherichia coli in an in vitro dynamic model. Antimicrob. Agents Chemother. 1999, 43, 498–502. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Fass, R.J. In vitro activity of ciprofloxacin (Bay o 9867). Antimicrob. Agents Chemother. 1983, 24, 568–574. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Queenan, A.M.; Foleno, B.; Gownley, C.; Wira, E.; Bush, K. Effects of inoculum and beta-lactamase activity in AmpC-and extended-spectrum beta-lactamase (ESBL)-producing Escherichia coli and Klebsiella pneumoniae clinical isolates tested by using NCCLS ESBL methodology. J. Clin. Microbiol. 2004, 42, 269–275. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Soriano, F.; Garcia-Corbeira, P.; Ponte, C.; Fernandez-Roblas, R.; Gadea, I. Correlation of pharmacodynamic parameters of five beta-lactam antibiotics with therapeutic efficacies in an animal model. Antimicrob. Agents Chemother. 1996, 40, 2686–2690. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Thomson, K.S.; Moland, E.S. Cefepime, piperacillin-tazobactam, and the inoculum effect in tests with extended-spectrum beta-lactamase-producing Enterobacteriaceae. Antimicrob. Agents Chemother. 2001, 45, 3548–3554. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Ogawa, Y.; Nakano, R.; Kasahara, K.; Mizuno, T.; Hirai, N.; Nakano, A.; Suzuki, Y.; Kakuta, N.; Masui, T.; Yano, H.; et al. Comparison of the inoculum size effects of antibiotics on IMP-6 beta-lactamase-producing Enterobacteriaceae co-harboring plasmid-mediated quinolone resistance genes. PLoS ONE 2019, 14, e0225210. [Google Scholar] [CrossRef]
  42. Goldstein, E.J.; Citron, D.M.; Cherubin, C.E. Comparison of the inoculum effects of members of the family Enterobacteriaceae on cefoxitin and other cephalosporins, beta-lactamase inhibitor combinations, and the penicillin-derived components of these combinations. Antimicrob. Agents Chemother. 1991, 35, 560–566. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Patterson, J.E.; Zervos, M.J. Susceptibility and bactericidal activity studies of four beta-lactamase-producing enterococci. Antimicrob. Agents Chemother. 1989, 33, 251–253. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Murray, B.E.; Mederski-Samoraj, M.; Foster, S.K.; Brunton, J.L.; Harford, P. In vitro studies of plasmid-mediated penicillinase from Streptococcus faecalis suggest a staphylococcal origin. J. Clin. Investig. 1986. [Google Scholar] [CrossRef]
  45. Markowitz, S.M.; Wells, V.D.; Williams, D.S.; Stuart, C.G.; Coudron, P.E.; Wong, E.S. Antimicrobial susceptibility and molecular epidemiology of β-lactamase-producing, aminoglycoside-resistant isolates of Enterococcus faecalis. Antimicrob. Agents Chemother. 1991. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Processed images of S. aureus with three different concentrations, 5 × 105, 5 × 106, and 5 × 107 CFU/mL, after 6 h of incubation exposed to piperacillin and cefotaxime. Red dots in the images illustrate the colonies of bacteria that were recognized by image processing. (a) Representative images of S. aureus with piperacillin. (b) Representative images of S. aureus with cefotaxime. The scale bar represents 200 μm.
Figure 1. Processed images of S. aureus with three different concentrations, 5 × 105, 5 × 106, and 5 × 107 CFU/mL, after 6 h of incubation exposed to piperacillin and cefotaxime. Red dots in the images illustrate the colonies of bacteria that were recognized by image processing. (a) Representative images of S. aureus with piperacillin. (b) Representative images of S. aureus with cefotaxime. The scale bar represents 200 μm.
Antibiotics 10 00300 g001
Figure 2. (a) Schematics of micropatterned biochip based on 96-well plate format. Detailed structure of a single well, containing a main well and a side well. The larger main well is inoculated with a mixture of bacteria and agarose. The smaller side well is for antimicrobial agents with MHB medium. (b) Schematic of cross-section view of testing well. Antibiotic solutions diluted with MHB are applied to the side well, after which diffusion occurs toward the main well. Bacteria are fixed with agarose and the culture media with antibiotic diffuse into the agarose.
Figure 2. (a) Schematics of micropatterned biochip based on 96-well plate format. Detailed structure of a single well, containing a main well and a side well. The larger main well is inoculated with a mixture of bacteria and agarose. The smaller side well is for antimicrobial agents with MHB medium. (b) Schematic of cross-section view of testing well. Antibiotic solutions diluted with MHB are applied to the side well, after which diffusion occurs toward the main well. Bacteria are fixed with agarose and the culture media with antibiotic diffuse into the agarose.
Antibiotics 10 00300 g002
Figure 3. (a) RGB raw image from charge coupled device (CCD)camera. (b) Processed binary image with red crosses indicating the bacterial colonies.
Figure 3. (a) RGB raw image from charge coupled device (CCD)camera. (b) Processed binary image with red crosses indicating the bacterial colonies.
Antibiotics 10 00300 g003
Table 1. Minimal inhibitory concentration (MIC) values of S. aureus according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
Table 1. Minimal inhibitory concentration (MIC) values of S. aureus according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
AntibioticsAmpicillinCefotaximeLevofloxacinAmikacinPiperacillin
Inoculum size
(CFU/mL)
5 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 107
Volume of culture media w/antibiotics
(μL)
450.25140.250.250.25112128
900.250.522220.120.250.25112124
1350.250.520.120.250.25112122
Table 2. MIC values of P. aeruginosa according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
Table 2. MIC values of P. aeruginosa according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
AntibioticsCefotaximeLevofloxacinAmikacinPiperacillin
Inoculum size
(CFU/mL)
5 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 107
Volume of culture media w/antibiotics
(μL)
45832320.5110.51224>16
90816160.5110.51224>16
135816>640.5110.51224>16
Table 3. MIC values of E. coli according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
Table 3. MIC values of E. coli according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
AntibioticsAmpicillinCefotaximeLevofloxacinAmikacinPiperacillin
Inoculum size
(CFU/mL)
5 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 107
Volume of culture media w/antibiotics
(μL)
450.120.12>0.250.50.51
908880.060.120.120.0150.0150.0150.50.51444
1350.060.12>0.250.50.51
Table 4. MIC values of E. feacalis according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
Table 4. MIC values of E. feacalis according to different bacterial inoculum sizes and volumes of culture media with antibiotics. The unit of MIC is μg/mL.
AntibioticsAmpicillinLevofloxacinPiperacillin
Inoculum size
(CFU/mL)
5 × 1055 × 1065 × 1075 × 1055 × 1065 × 1075 × 1055 × 1065 × 107
Volume of culture media w/antibiotics
(μL)
450.512
901110.511222
1350.511
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hwang, J.H.; Lee, S.Y.; Choi, J. Microscopic Analysis of Bacterial Inoculum Effect Using Micropatterned Biochip. Antibiotics 2021, 10, 300. https://doi.org/10.3390/antibiotics10030300

AMA Style

Hwang JH, Lee SY, Choi J. Microscopic Analysis of Bacterial Inoculum Effect Using Micropatterned Biochip. Antibiotics. 2021; 10(3):300. https://doi.org/10.3390/antibiotics10030300

Chicago/Turabian Style

Hwang, Jung Ho, Sang Young Lee, and Jungil Choi. 2021. "Microscopic Analysis of Bacterial Inoculum Effect Using Micropatterned Biochip" Antibiotics 10, no. 3: 300. https://doi.org/10.3390/antibiotics10030300

APA Style

Hwang, J. H., Lee, S. Y., & Choi, J. (2021). Microscopic Analysis of Bacterial Inoculum Effect Using Micropatterned Biochip. Antibiotics, 10(3), 300. https://doi.org/10.3390/antibiotics10030300

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop