Next Article in Journal
Comparison of Media Company Performance Efficiency Based on the Search Engine’s Method of Providing News Content (External Links vs. Internal Links)
Previous Article in Journal
Preventive and Therapeutic Effects of Punica granatum (Pomegranate) in Respiratory and Digestive Diseases: A Review
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An In Vitro Model Using TRIS-Buffered Plasma-Activated Water to Reduce Pathogenic Microorganisms Involved in Digital Dermatitis Infection in Cattle

1
Institute of Food Quality and Food Safety, University of Veterinary Medicine Hannover, Foundation, 30173 Hannover, Germany
2
Faculty Engineering and Health, University of Applied Sciences and Arts, 37085 Göttingen, Germany
3
Clinic for Cattle, University of Veterinary Medicine Hannover, Foundation, 30173 Hannover, Germany
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(23), 12325; https://doi.org/10.3390/app122312325
Submission received: 11 November 2022 / Revised: 28 November 2022 / Accepted: 29 November 2022 / Published: 2 December 2022
(This article belongs to the Section Applied Microbiology)

Abstract

:
Bovine digital dermatitis is an important infectious claw disease caused by multimicrobial infections with bacteria such as Fusobacterium (F.) necrophorum or Porphyromonas (P.) levii. To analyze the antibacterial effects of a TRIS-buffered plasma-activated water (Tb-PAW) on the bacterial number of F. necrophorum, P. levii, Escherichia (E.) coli, Staphylococcus (S.) aureus and Clostridium (C.) sporogenes 1 mL of each bacterial solution (106–108 CFU/mL) was incubated with 9 mL Tb-PAW up to 15 min. E. coli, F. necrophorum and P. levii were significantly reduced by 5.0 log after 1 min of treatment, while S. aureus and C. sporogenes required 15 min to reach a 3.0 log reduction. The addition of bovine serum albumin did not negatively affect the bactericidal effect. Tb-PAW storage at 7 °C and 21 °C is possible for up to 24 h without any change in the bactericidal effect, while Tb-PAW stored at 30 °C can only be used for a period of 12 h. The present data indicate that Tb-PAW can be used to reduce various bacteria even under the influence of different parameters. However, due to the complexity of Tb-PAW, further studies are required to ensure its microbicidal activity before practical application.

1. Introduction

Bovine digital dermatitis (DD) is a claw disease in cattle which often causes lameness due to painful lesions. These lesions are characterized by erosive ulcerations often presented as moist, circular, strawberry-like structures with long hair surrounding them [1,2]. The reduced cattle welfare is reflected in decreased milk yield, lower body condition scores, poor reproductive performances and increased losses due to early culling [3,4]. DD has a multifactorial etiology. Causative factors, leading to pre-damage of the skin are manifold. These are e.g., unhygienic stable conditions, slatted floors without scrapers, as well as standing in wet manure, which allow the infiltration of pathogens [2,5,6]. In particular, pathogenic bacteria, including Treponema spp., Fusobacterium necrophorum, Porphyromonas levii, Mycoplasmataceae, Escherichia coli and staphylococci are considered to play an important role in the development of DD [2,6,7,8,9,10].
For decades, the treatment of DD has been a major challenge. Research is still being conducted on treatment alternatives to the current commonly used systemic or topical antibiotic applications. Oxytetracycline or lincomycin are often used as a topical therapy whereas erythromycin is frequently used for group treatments with footbaths [2,11,12]. In addition, the use of antibiotics causes economic losses for farmers, due to the transient prohibition of milk delivery and meat production [13]. Novel insights into the treatment of DD show that non-antibiotic agents also have a beneficial effect on the lesions. These include formaline, copper sulfate, dragonhyde or phenytoin, but these chemicals often have environmentally damaging potentials [13,14,15].
It Is important to reduce the existing pathogenic organisms involved in DD to achieve prevention or effective healing of the specific lesions. Plasma-activated liquids (PAL) may provide an antibiotic-free option to minimize pathogens and prevent or at least mitigate DD. A wide variety of PALs are now being produced and studied for their antimicrobial activity. Plasma-activated water (PAW) is made from pure distilled water, whereas various buffers are used to produce plasma-activated buffers for example phosphate buffered saline (PBS) or citrate-phosphate buffer [16,17].
Nowadays, PAW in particular is used in a variety of ways, for example in microbiology for bacterial reduction or in medicine for wound healing, treatment of skin diseases or cancer therapy [18,19,20,21], even insecticidal effects of PAW have been demonstrated [22]. PAW contains various reactive oxygen species (ROS), as well as reactive nitrogen species (RNS). ROS include hydrogen peroxide (H2O2), hydroxyl radicals (OH) or superoxides (O2), whereas peroxynitrite (ONOO-) and nitrogen dioxide (NO2) radicals belong to the RNS [23,24,25]. It has been widely documented that these reactive species can damage the microorganisms at various cell components and thus inactivate them. For example, Yusupov et al. [26] found that oxidative stress can break the cell wall due to the destruction of peptide-glycan bonds. Similarly, it was shown by Chen et al. [27] hat after intracellular entry of reactive species, both DNA and proteins in the bacterial cell were destroyed.
In this study, we investigated the influence of a TRIS-buffered PAW (Tb-PAW) on Gram-positive and Gram-negative aerobic, as well as anaerobic microorganisms associated with DD and frequently found on the claw or in the stable, using an in vitro model. These investigations were followed by an analysis of specific factors influencing the antimicrobial effectivity of the Tb-PAW, such as proteins, temperature or storability. The aim of the research was to evaluate the microbicidal activity of Tb-PAW for its applicability in an agricultural facility to reduce pathogenic microorganisms under various conditions, thus providing an alternative therapy or prophylaxis for the management of DD in the stable.

2. Materials and Methods

2.1. Plasma Device and Production of Tb-PAW

The Tb-PAW was produced by the faculty of engineering and health (HAWK, University of Applied Science and Arts, Göttingen, Germany) using an in-house development of the HAWK for the production of Tb-PAW based on the principle of a double-insulated DBD discharge. The complete setup for the generation of Tb-PAW consists of an arrangement of 10 individual “plasma tubes” which are integrated into an array (2 × 5) in a corresponding assembly (see Figure 1a).
The single plasma tube consists of an outer radially symmetrical silica tube (total length 300 mm, outer diameter 12 mm inner diameter 10 mm), wrapped with Cu-foil, which serves as the ground electrode (GND), and a centrally positioned ceramic tube (Al2O3) (Figure 1b). The ceramic tube has an outer diameter of 3 mm, an inner diameter of 1.6 mm and is filled with a brass rod (diameter 0.8 mm), the cavities are filled with brass powder. The ceramic tube acts as the high-voltage electrode. The chosen geometry results in a discharge gap of approx. 3.5 mm, which has a discharge length of approx. 100 mm. The discharge gap is streamed with pressure air as process gas at a gas volume flow rate of 5 Lmin−1. This results in an average exposition time of the air to the discharge conditions of ≈0.1 s. The outer quartz tube protrudes approx. 5 cm into a beaker filled with deionized water. The distance between the end of the discharge section and the water surface is approx. 160 mm, so that the plasma exhaust contacts water after approx. 0.15 s after exiting the plasma zone.
The high voltage power supply provides alternating pulses (U = 16.6 kV peak-peak) with a pulse repetition frequency of 17 kHz, a pulse duration of approx. 2 µs, an in-coupled power of approx. 400 W and a power density of approx. 6 W/cm³ to the plasma-array. The characteristic U-I-envelope of the plasma source is depicted in Figure 2.
The goal was to achieve a neutral pH-value of the Tb-PAW to allow an application on living animals in the future. Thus, TRIS-buffer (0.5 mol/L), made with TRIS(hydroxymethyl)aminomethan (TRIS, Trometamol, ≥99.8%, VWR International, Darmstadt, Germany) and TRIS-HCl (TRIS(hydroxymethyl)aminomethan hydrochlorid, ≥99.0%, VWR), were treated with the plasma source for 20 min to obtain 250 mL Tb-PAW. The Tb-PAW and the (untreated, control) TRIS-buffer were transported at room temperature within 4 h after preparation to the Institute for Food Quality and Food Safety (LMQS, University of Veterinary Medicine Hannover, Germany). The experiments started within 4 to 5 h after preparation of the Tb-PAW. At first, the ph-values of the TRIS-buffer control (7.6 ± 0.2) and the Tb-PAW sample (7.3 ± 0.2) were measured with a pH-meter (Jenway, Cole-Parmer, Stone, Staffordshire, ST15 OSA, UK) equipped with a glass electrode (InLab Semi-Micro, Mettler Toledo, Gießen, Germany). For the experiments, the pH-values of all TRIS-buffer samples were adjusted to the respective pH-value of the Tb-PAW in order to exclude a pH-effect. Additionally, prior to the transport to the LMQS a Reflectoquant (Merck KGaA, Darmstadt, Germany) was used to measure the concentrations of nitrate anions (NO3; approx. 5540 mg/L) and nitrite anions (NO2; approx. 440 mg/L), as well as hydrogen peroxide (H2O2; approx. 4.5 mg/L) concentrations in the Tb-PAW.

2.2. Bacterial Strains and Culture Conditions

Escherichia (E.) coli (DSM 682), Staphylococcus (S.) aureus (DSM 799, MSSA reference strain), Fusobacterium (F.) necrophorum (DSM 21784), Porphyromonas (P.) levii (DSM 23370), and Clostridium (C.) sporogenes were selected for the experiments, whereas Treponema spp. was left out because it is more difficult to cultivate. The bacterial strains E. coli (origin unknown), S. aureus (isolated from a human lesion) and the two anaerobes F. necrophorum (isolated from a bovine hepatic abscess) and P. levii (isolated from bovine rumen) were obtained from the German Collection of Microorganisms and Cell Cultures GmbH (DSMZ, Braunschweig, Germany). C. sporogenes was taken from an own stock (originated from a milk sample from an older test series) of the LMQS. E. coli, S. aureus and C. sporogenes were plated on Columbia blood agar with sheep blood (Oxoid GmbH, Wesel, Germany). F. necrophorum and P. levii were plated on Schaedler Anaerobic Agar with Sheep Blood, Haemin and Vitamin K 1 (Oxoid GmbH, Wesel, Germany). The incubation period for E. coli and S. aureus was 24 h, for C. sporogenes and P. levii 48 h and for F. necrophorum 72 h. All microorganisms were incubated at 37 °C. During incubation F. necrophorum, P. levii and C. sporogenes were kept in an anaerobic jar (Merck KGaA, Darmstadt, Germany) with resazurin anaerobic indicator paper (Oxoid GmbH, Wesel, Germany) and AnaeroGen 3.5 L bags (Oxoid GmbH, Wesel, Germany). At each experiment, to prevent negative effects of the oxygen, the latter bacteria strains were exposed to air for a maximum of 30 min before the transfer to the anaerobic environment. For safekeeping of the bacterial strains, colonies of each bacterial strain were transferred to a separate cryotube (Carl Roth, Karlsruhe, Germany) and frozen at −80 °C. Before starting the experiments, one bead from each cryotube was spread onto a blood agar plate and grown as described above.

2.3. Tb-PAW Treatment on Different Bacterial Strains

All microbiological tests were performed in accordance with ISO-16649-2 and ISO-11290-2. For each bacterial strain, 9 mL Tb-PAW and 9 mL TRIS-buffer were added to sterile test tubes. The untreated TRIS-buffer was used as a control. Bacterial colonies were suspended in sterile saline (0.9% NaCl) and adjusted to McFarland turbidity standards of 1.5 (E. coli, S. aureus, C. sporogenes), 2.0 (P. levii), and 3.0 (F. necrophorum) to achieve a concentration of the colony forming units (CFU) of approx. 107–108 CFU/mL (106 CFU/mL for C. sporogenes). One mL of each bacterial suspension was added to the Tb-PAW and TRIS-buffer samples (final concentrations of 106–107 CFU/mL resp. 105 CFU/mL), mixed and incubated at room temperature for 1 min. In addition, S. aureus and C. sporogenes were incubated for 5 min and 15 min. Serial dilutions were performed by adding 1 mL of the sample solution to 9 mL of sterile saline solution containing peptone-buffered water (0.85% NaCl, 0.1% peptone, VWR). To identify the surviving bacteria, 100 µL of the appropriate dilution stage was spread in duplicate on selective agar plates (Oxoid, GmbH, Wesel, Germany). Colic Brilliance™ E. coli/Coliform selective agar (ColiC agar) was used for E. coli, Baird–Parker agar for S. aureus, Tryptose Cycloserine Agar for C. sporogenes and Schaedler Anaerobic Agar with Sheep Blood, Haemin & Vitamin K 1 (Schaedler agar) for F. necrophorum and P. levii. The agar plates were incubated as described above.

2.4. Application of BSA as a Protein Factor

For analysis of the effect of protein addition E. coli, F. necrophorum and P. levii were used, due to their significant reductions in the previous experiment after one minute (see Section 3.1). For the experimental setup, the final solution was adjusted to contain a concentration of approx. 5% BSA and a bacteria number of approx. 106–107 CFU/mL. Thus, the McFarland turbidity standard was adjusted to 3.0 (E. coli), 4.0 (P. levii) and 6.0 (F. necrophorum). For the samples with 5% BSA, 1.66 mL of a 30% BSA solution (Sigma-Aldrich, St. Louis, MO, USA) and 0.5 mL of the bacterial strains were each added to separate sterile test tubes. After the addition of 7.84 mL Tb-PAW, the samples were vortexed and incubated for one minute at room temperature. Then, 1 mL of the solution was added to 9 mL sterile saline solution containing peptone-buffered water (0.85% NaCl, 0.1% peptone, VWR) and again a dilution series were performed. Subsequently 100 µL of the appropriate dilutions were spread in duplicate on ColiC (E. coli) and Schaedler agar (P. levii, F. necrophorum). The agar plates were incubated and evaluated as described in Section 2.2. and Section 2.6. As controls, the Tb-PAW was replaced with untreated TRIS-buffer. As the 30% BSA solution contained sodium azide (NaN3) for preservation, additional control samples were run including 0.1% NaN3 (as an equivalent for 5% BSA) and Tb-PAW.

2.5. Tb-PAW Storage Trials at Different Temperatures

The effect of different storage temperatures and time periods on the antimicrobial properties of the Tb-PAW were analyzed 4, 8, 12 and 24 h after preparation at 7 °C, 21 °C, and 30 °C emulating different weather conditions. In four independent experiments 9 mL of Tb-PAW was placed in 12 sterile test tubes and stored in a 30 °C incubator, in a refrigerator at 7 °C and at room temperature in a cabinet. All samples were kept in the dark. Immediately before the measurement points of time, the E. coli concentration was adjusted to 107–108 CFU/mL using McFarland turbidity standards. Subsequent to addition of 1 mL of bacterial suspension the Tb-PAW test samples, were mixed and incubated for 1 min and analyzed as described in Section 2.3. An identical number of 9 mL TRIS-buffer controls were investigated.

2.6. Bacterial Enumeration

To perform a colony count assay, all agar plates showing up to 300 colonies were included in the scoring. In case of shortfall of bacterial counts below the detection limit of of 10 CFU/mL (E. coli) and 100 CFU/mL (all other bacteria) bacteria numbers of 5 CFU/mL (E. coli) and 50 CFU/mL (all other bacteria) were considered for the statistical analysis. Results are expressed in common logarithm of CFU/mL.

2.7. Statistical Analysis of the Data

All experiments were repeated at least three times and the results are presented as the mean ± standard deviation. Statistical analysis was performed with SAS Enterprise Guide 7.1 (SAS Institute Inc., Cary, NC, USA) using a one-way analysis of variance (ANOVA), followed by a Tukey’s range test (HSD). All data were visualized with GraphPad Prism (GraphPad software, San Diego, CA, USA). Probabilities of p ≤ 0.05 were considered significant.

3. Results and Discussion

3.1. Bactericidal Efficacy of Tb-PAW on Different Microorganisms

The antibacterial effect of Tb-PAW compared to the untreated TRIS-buffer control for the respective pathogens is shown in Figure 3. The Tb-PAW was able to achieve significant reductions of the CFU of Gram-negative bacteria species E. coli (5.12 ± 0.46 log), F. necrophorum (5.04 ± 0.03 log) and P. levii (4.98 ± 0.61 log) after a 1 min exposure time (Figure 3a). Whereas the CFU of E. coli, F. necrophorum and P. levii were decreased below the detection limit, only low but not significant reduction rates of the bacteria numbers were obtained for Gram-positive bacteria C. sporogenes (0.20 ± 0.75 log) and S. aureus (0.36 ± 0.40 log) within 1 min compared to the control samples. After increasing the Tb-PAW exposure time of C. sporogenes and S. aureus to 5 min (Figure 3b) S. aureus was significantly reduced by about 1.76 ± 0.27 log. C. sporogenes showed insignificant decreases of the CFU (0.36 ± 0.40 log). For both bacterial species, an additional exposure time of 15 min was applied, functioning as highest possible practice-oriented treatment duration. The prolonging contact time did enhance the antimicrobial efficacy, leading to significant reductions of 3.06 ± 0.38 log for C. sporogenes and 3.66 ± 0.16 log for S. aureus (Figure 3b).
We observed that the Tb-PAW used in this study could reduce both Gram-negative and Gram-positive bacteria up to at least 3.0 log. However, S. aureus and C. sporogenes were more resistant to the Tb-PAW treatment, which reflected in elevated treatment times compared to E. coli, F. necrophorum and P. levii. A comparison of the present results with previously published studies [28,29] is difficult as antimicrobial effects of different PALs are quite variable. This high variability might be due to the application of different parameters such as voltage, working gases and gas flow rate, the treatment regime (used liquid and plasma treatment time), the content of reactive species, the acidity of the PAL or differences of the bacterial strains [24,30,31,32]. For example, in the present study the inactivation for S. aureus within 15 min was higher compared to the study of Tsoukou et al. [28]. Rothwell et al. [29] presented an increased efficacy for E. coli (6 log) within 1 min compared to our results. An effect which might be attributed to a lower initial CFU number in the present study. Some studies showed that the utilized bacterial species (E. coli, S. aureus) were only inactivated after high exposure times. Furthermore, other authors also reported a higher sensitivity of Gram-negative bacteria against PALs [24,28,33,34]. For example, Zhao et al. [24] observed that S. aureus, treated with PAW, needed 5 h to be significantly reduced by 3 log steps, whereas E. coli reached this reduction within 30 min. Li et al. [34] compared the reduction of the Gram-negative bacterium Porphyromonas (P.) gingivalis and the Gram-positive bacterium Actinomyces viscosus after PAW treatment and described a higher inactivation rate for P. gingivalis within a shorter treatment time.
The higher effect on the Gram-negative compared to the Gram-positive bacteria may be due to their differences in the cell wall structure. Gram-positive bacteria show a reduced susceptibility to reactive species due to their significantly thicker cell wall (20–80 nm vs. 10–15 nm) [33]. Furthermore, we have shown that the treatment of anaerobes with Tb-PAW also leads to different inactivations. Our findings depicting an increased impact of Tb-PAW on anaerobic bacteria is supported by the study of Li et al. [34]. They also found that aerobic bacteria, such as Streptococcus mutans, are less sensitive than anaerobic bacteria when exposed to comparable concentrations of different ROS species. A strong interdependency of antibacterial efficacy of the applied Tb-PAW’s and bacteria species is apparent.
To date, the exact mode of action of PAW has not been fully elucidated. As described by Laroussi et al. [35] the effect of other ROS and RNS produced in PTW besides NO3 is not well understood. According to Li et al. [36], plasma-activated chemical solutions (PACS) containing H+, NO2 and H2O2 and especially their conversion to peroxynitrite, play a key role in microbial decontamination. Zouh et al. [25], who emphasized the role of peroxynitrite in their work, also supported this statement. However, other authors have noted that H2O2, hydroxyl radicals (•OH), and ozone (O3) for example, play important roles (Zhang et al. [37]). Thus, further research is necessary to understand the mode of action of PAW.
The TRIS-buffer used in the present study caused the ph-value of the Tb-PAW to be neutral, giving the opportunity for their in situ application on open wounds and skin of living animals. Some studies compared the results between PAW and plasma-activated buffer in relation to their acidity and figured a lower pH-value would increase the inactivation rate [16,17,38]. Consequently, we decided on the one hand to buffer the PAL, on the other hand to adjust the control solution to the pH-value of the Tb-PAW, which was also associated with high inactivations.

3.2. Tb-PAW Inactivation Ability of E. coli, F. necrophorum and P. levii under the Influence of Bovine Serum Albumin

In the present study the impact of bovine serum albumin (BSA) on the antimicrobial capacities of Tb-PAW against E. coli, F. necrophorum and P. levii was tested evaluating if proteins, which are present for example, on the skin of the cattle with DD might influence the overall Tb-PAW efficacy. The reduction rates of Tb-PAW applications containing 5% BSA and without BSA are shown in Figure 4. As described above, the Tb-PAW could significantly (p ≤ 0.05) reduce all bacterial species below the detection limit. Under the influence of 5% BSA E. coli was significantly reduced by 4.48 ± 0.42 log values. The reductions for F. necrophorum and P. levii using 5% BSA were 4.39 ± 0.28 and 4.55 ± 0.37 log values, respectively. The addition of 5% BSA to the Tb-PAWs revealed no significant impact on occurring reduction rates.
The 30% BSA solution contained NaN3 as an antimicrobial agent, thus control samples with the existing concentrations of NaN3 (0.1 mg/L) were additionally analyzed during the BSA-impact study. No significant differences between all groups could be obtained. The results are reported in the Supplementary Material (see Table S1).
The presented results disagree with other studies revealing a buffer effect of organic materials, which can considerably reduce antimicrobial activities of PALs [39,40]. BSA is described as a protein that can not only form a physical barrier between microorganisms and reactive species, but also actively interacts with free radicals leading to a decrease in the antimicrobial properties of PALs [39,41]. For example, Zhang et al. [39] demonstrated that BSA reduced the antimicrobial efficacy of PAW against S. aureus after a treatment time of 10 min. In the study by Xiang et al. [40], bacterial suspensions were added after combining beef extract or peptone with PAW (different protein concentrations) and after a waiting time of 15 min resulting in decreasing bacterial reduction properties with increasing protein concentrations. In contrast to the previous studies in the present experimental setup the bacteria were exposed to Tb-PAW and BSA at the same time which might explain the different effects between the present and other studies. However, the present setup seems more logical. For example, if a PAL is applied to the skin of living animals, the PAL is likely to come into contact with the microorganisms as well as with proteins and other (probably disturbing) substances simultaneously.
Other studies indirectly support the present results. If liquids containing bacteria were treated with plasma to obtain a PAL, the proteins and other components in the media had no impact on the bacterial inactivation capacity [42,43]. For example, Rowan et al. [42] inactivated microbial pathogens, such as E. coli and Campylobacter jejuni in chilled poultry wash water, achieving an increased reduction compared to samples in distilled water. Gurol et al. [43] showed in their study that after treating different types of milk (whole, semi skimmed and skimmed milk) with low pressure plasma, the inactivation of E. coli decreased by 54% after 3 min.
In conclusion, the protective effect of proteins against microbial inactivation by PAL seems to depend on several parameters, including the protein source, the amount of protein, the experimental setup (treatment time and treatment procedure), as well as the plasma source and its settings, as evidenced by these various study results.

3.3. Tb-PAW Inactivation of E. coli after Different Storage Times and Temperatures

In a third study, Tb-PAW was stored at three different temperatures (7 °C, 21 °C and 30 °C) and monitored at four time points (4 h, 8 h, 12 h and 24 h). The bactericidal efficacy of those Tb-PAWs against E. coli was tested (Table 1). The comparisons between untreated TRIS-buffer control samples and the corresponding Tb-PAW samples showed significant reductions of the CFU in Tb-PAW at all temperatures and storage times. These ranged from 5.08 ± 0.52 log steps (30 °C, 8 h) to 0.73 ± 0.23 log steps (30 °C, 24 h). The residual microbial content of the Tb-PAW sample at 30 °C and 24 h was significantly higher compared to the 7 °C and 21 °C samples, whereas the other time points revealed no differences between the temperature groups. In addition, we found significantly higher bacterial counts in the 30 °C treatment group at 24 h compared to the other three time points. We therefore verified that Tb-PAWs stored at 7 °C and 21 °C can be used for a period of 24 h and for at least 12 h when stored at 30 °C.
These results are consistent with findings of the study by Shen et al. [44], who also reported prolonged preservation of bactericidal activity when PAW was stored at low temperatures. In this study, the bactericidal properties of a PAW used against S. aureus also increased with decreasing temperatures (25 °C < 4 °C < −20 °C < −80 °C). After one day of storage, the reduction rates dropped from 5.0 log CFU/mL (room temperature) to approx. 3.7 log CFU/mL (−80 °C) and 1.8 log CFU/mL (25 °C), respectively. The PAW at −80 °C retained a significantly higher reduction rate and revealed a better way of storage than 25 °C.
Traylor et al. [17] investigated the bactericidal activity of PAW and plasma-activated PBS (PAPBS). They found that storage of PAW resulted in a decreasing reduction effect with time. The PALs were stored for seven days after preparation. Each day, E. coli was incubated with the PALs for 15 min as well as for 3 h. Over two days, the 3 h exposure resulted in a 5.0 log reduction before it began to decline, while the 15 min exposure dropped to 2.4 log on the first day (30 min after production) and remained at about 1.0 log until the end of the second day. In contrast to the PAW, the PAPBS did not reach a high reduction at any of the time points and remained below 1.0 log.
The effect of the PALs in these studies appears to decrease with time and at elevated temperature, which is in good agreement with our own findings. However, the decrease varies greatly depending on the study. In the researches described, much longer exposure times (4 min to 3 h) were used compared to our study. In addition, we were able to achieve high reductions rates with Tb-PAW compared to the PAPBS of Traylor et al. [17] using only 1 min exposure time resulting in reduction rates of up to 2.61 logs (7 °C) and 3.18 logs (21 °C) after 24 h.

4. Conclusions

The results of this study demonstrate that Tb-PAW can effectively reduce a range of bacteria involved in the development of DD. The level of reduction is depending on several parameters, including the exposure time and the treated bacterial species. Gram-positive bacteria (S. aureus and C. sporogenes) were more resistant to the Tb-PAW than Gram-negative bacteria (E. coli, F. necrophorum and P. levii). As the Tb-PAW in the present study could achieve a fast and high inactivation of the bacterial strains despite a neutral ph-value, the antimicrobial effect of the Tb-PAW seems to be only due to the action of reactive species within the Tb-PAW. Furthermore, Tb-PAW is maintaining high efficacy levels even under the presence of organic substances. Thus, our results indicate that Tb-PAW might constitute a potential DD-therapy in dairy cows. Due to the high time stability (12 h), even under elevated temperatures with up to 30 °C, Tb-PAW seems to be a promising tool being able to perform under real-life conditions in agricultural holdings, e.g., as a claw bath or spray solution. Further work in progress is focusing on the analysis of Tb-PAW applied directly to skin samples or in situ on wounds infected with DD.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/app122312325/s1, Table S1: Means and standard deviation of Escherichia coli, Fusobacterium necrophorum and Porphyromonas levii counts (log10 CFU/mL) after 1 min treatment of Tb-PAW, Tb-PAW with bovine serum albumin and Tb-PAW with sodium azide.

Author Contributions

Conceptualization, M.P., G.A., L.S., C.K., M.H. (Martina Hoedemaker) and V.G.-P.; methodology, V.G.-P., L.S. and G.A.; software, V.G.-P.; validation, V.G.-P., L.S., M.P. and C.K.; formal analysis, V.G.-P.; investigation, V.G.-P.; resources, C.O., M.H. (Marcus Harms) and R.O.; data curation, V.G.-P.; writing—original draft preparation, V.G.-P.; writing—review and editing, L.S., C.K., G.A., M.P., M.H. (Martina Hoedemaker), C.O., M.H. (Marcus Harms), R.O., B.A., K.A.R., W.V. and L.t.B.; visualization, V.G.-P.; supervision, M.P., L.S., C.K. and G.A.; project administration, L.S.; funding acquisition, B.A., K.A.R., G.A., W.V. and L.t.B. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the European Innovation Partnership (EIP-AGRI) project “PlaWaKiRi” and funded by the European Agricultural Fund for Rural Development (EAFRD 276 03 254 021 0329, http://ec.europa.eu/agriculture/rural-development-2014-2020/index_de.htm (accessed on 1 November 2022)), the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—491094227 “Open Access Publication Funding” and the University of Veterinary Medicine Hannover, Foundation.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Read, D.H.; Walker, R.L. Papillomatous Digital Dermatitis (Footwarts) in California Dairy Cattle: Clinical and Gross Pathologic Findings. J. Vet. Diagn. Investig. 1998, 10, 67–76. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Refaai, W.; Van Aert, M.; Abd El-Aal, A.M.; Behery, A.E.; Opsomer, G. Infectious Diseases Causing Lameness in Cattle with a Main Emphasis on Digital Dermatitis (Mortellaro Disease). Livest. Sci. 2013, 156, 53–63. [Google Scholar] [CrossRef]
  3. Melendez, P.; Bartolome, J.; Archbald, L.F.; Donovan, A. The Association between Lameness, Ovarian Cysts and Fertility in Lactating Dairy Cows. Theriogenology 2003, 59, 927–937. [Google Scholar] [CrossRef]
  4. Warnick, L.D.; Janssen, D.; Guard, C.L.; Gröhn, Y.T. The Effect of Lameness on Milk Production in Dairy Cows. J. Dairy Sci. 2001, 84, 1988–1997. [Google Scholar] [CrossRef] [PubMed]
  5. Fiedler, A. Mortellarosche Krankheit—Therapie und Management der digitalen Dermatitis beim Rind. Veterinär Spieg. 2022, 32, 32–37. [Google Scholar] [CrossRef]
  6. Moe, K.K.; Yano, T.; Misumi, K.; Kubota, C.; Nibe, K.; Yamazaki, W.; Muguruma, M.; Misawa, N. Detection of Antibodies against Fusobacterium Necrophorum and Porphyromonas Levii-like Species in Dairy Cattle with Papillomatous Digital Dermatitis: Antibodies Responses in Cattle with PDD. Microbiol. Immunol. 2010, 54, 338–346. [Google Scholar] [CrossRef]
  7. Krull, A.C.; Shearer, J.K.; Gorden, P.J.; Cooper, V.L.; Phillips, G.J.; Plummer, P.J. Deep Sequencing Analysis Reveals Temporal Microbiota Changes Associated with Development of Bovine Digital Dermatitis. Infect. Immun. 2014, 82, 3359–3373. [Google Scholar] [CrossRef] [Green Version]
  8. Nielsen, M.W.; Strube, M.L.; Isbrand, A.; Al-Medrasi, W.D.H.M.; Boye, M.; Jensen, T.K.; Klitgaard, K. Potential Bacterial Core Species Associated with Digital Dermatitis in Cattle Herds Identified by Molecular Profiling of Interdigital Skin Samples. Vet. Microbiol. 2016, 186, 139–149. [Google Scholar] [CrossRef] [Green Version]
  9. Nordhoff, M.; Moter, A.; Schrank, K.; Wieler, L.H. High Prevalence of Treponemes in Bovine Digital Dermatitis-A Molecular Epidemiology. Vet. Microbiol. 2008, 131, 293–300. [Google Scholar] [CrossRef] [Green Version]
  10. Wilson-Welder, J.; Alt, D.; Nally, J. Digital Dermatitis in Cattle: Current Bacterial and Immunological Findings. Animals 2015, 5, 1114–1135. [Google Scholar] [CrossRef]
  11. Berry, S.L.; Read, D.H.; Famula, T.R.; Mongini, A.; Döpfer, D. Long-Term Observations on the Dynamics of Bovine Digital Dermatitis Lesions on a California Dairy after Topical Treatment with Lincomycin HCl. Vet. J. 2012, 193, 654–658. [Google Scholar] [CrossRef] [PubMed]
  12. Laven, R.A.; Hunt, H. Evaluation of Copper Sulphate, Formalin and Peracetic Acid in Footbaths for the Treatment of Digital Dermatitis in Cattle. Vet. Rec. 2002, 151, 144–146. [Google Scholar] [CrossRef] [PubMed]
  13. Laven, R.A.; Logue, D.N. Treatment Strategies for Digital Dermatitis for the UK. Vet. J. 2006, 171, 79–88. [Google Scholar] [CrossRef] [PubMed]
  14. El-Shafaey, E.-S.; Hamed, M.A.; Elfadl, E.A.; Gomaa, N.A.; Rizk, M.A. Phenytoin: A Promising Non-Antibiotic Drug for the Topical Treatment of Digital Dermatitis in Dairy Cows. Vet. World 2021, 14, 2907–2912. [Google Scholar] [CrossRef] [PubMed]
  15. Teixeira, A.G.V.; Machado, V.S.; Caixeta, L.S.; Pereira, R.V.; Bicalho, R.C. Efficacy of Formalin, Copper Sulfate, and a Commercial Footbath Product in the Control of Digital Dermatitis. J. Dairy Sci. 2010, 93, 3628–3634. [Google Scholar] [CrossRef] [Green Version]
  16. Joshi, I.; Salvi, D.; Schaffner, D.W.; Karwe, M.V. Characterization of Microbial Inactivation Using Plasma-Activated Water and Plasma-Activated Acidified Buffer. J. Food Prot. 2018, 81, 1472–1480. [Google Scholar] [CrossRef]
  17. Traylor, M.J.; Pavlovich, M.J.; Karim, S.; Hait, P.; Sakiyama, Y.; Clark, D.S.; Graves, D.B. Long-Term Antibacterial Efficacy of Air Plasma-Activated Water. J. Phys. D Appl. Phys. 2011, 44, 472001. [Google Scholar] [CrossRef] [Green Version]
  18. Gan, L.; Duan, J.; Zhang, S.; Liu, X.; Poorun, D.; Liu, X.; Lu, X.; Duan, X.; Liu, D.; Chen, H. Cold Atmospheric Plasma Ameliorates Imiquimod-Induced Psoriasiform Dermatitis in Mice by Mediating Antiproliferative Effects. Free Radic. Res. 2019, 53, 269–280. [Google Scholar] [CrossRef]
  19. Kamgang-Youbi, G.; Herry, J.-M.; Meylheuc, T.; Brisset, J.-L.; Bellon-Fontaine, M.-N.; Doubla, A.; Naïtali, M. Microbial Inactivation Using Plasma-Activated Water Obtained by Gliding Electric Discharges. Lett. Appl. Microbiol. 2009, 48, 13–18. [Google Scholar] [CrossRef]
  20. Subramanian, P.S.G.; Jain, A.; Shivapuji, A.M.; Sundaresan, N.R.; Dasappa, S.; Rao, L. Plasma-activated Water from a Dielectric Barrier Discharge Plasma Source for the Selective Treatment of Cancer Cells. Plasma Process. Polym. 2020, 17, 1900260. [Google Scholar] [CrossRef]
  21. Xu, D.; Wang, S.; Li, B.; Qi, M.; Feng, R.; Li, Q.; Zhang, H.; Chen, H.; Kong, M.G. Effects of Plasma-Activated Water on Skin Wound Healing in Mice. Microorganisms 2020, 8, 1091. [Google Scholar] [CrossRef]
  22. ten Bosch, L.; Pfohl, K.; Avramidis, G.; Wieneke, S.; Viöl, W.; Karlovsky, P. Plasma-Based Degradation of Mycotoxins Produced by Fusarium, Aspergillus and Alternaria Species. Toxins 2017, 9, 97. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Khlyustova, A.; Labay, C.; Machala, Z.; Ginebra, M.-P.; Canal, C. Important Parameters in Plasma Jets for the Production of RONS in Liquids for Plasma Medicine: A Brief Review. Front. Chem. Sci. Eng. 2019, 13, 238–252. [Google Scholar] [CrossRef]
  24. Zhao, Y.-M.; Ojha, S.; Burgess, C.M.; Sun, D.-W.; Tiwari, B.K. Inactivation Efficacy and Mechanisms of Plasma Activated Water on Bacteria in Planktonic State. J. Appl. Microbiol. 2020, 129, 1248–1260. [Google Scholar] [CrossRef]
  25. Zhou, R.; Zhou, R.; Prasad, K.; Fang, Z.; Speight, R.; Bazaka, K.; Ostrikov, K. Cold Atmospheric Plasma Activated Water as a Prospective Disinfectant: The Crucial Role of Peroxynitrite. Green Chem. 2018, 20, 5276–5284. [Google Scholar] [CrossRef]
  26. Yusupov, M.; Bogaerts, A.; Huygh, S.; Snoeckx, R.; van Duin, A.C.T.; Neyts, E.C. Plasma-Induced Destruction of Bacterial Cell Wall Components: A Reactive Molecular Dynamics Simulation. J. Phys. Chem. C 2013, 117, 5993–5998. [Google Scholar] [CrossRef]
  27. Chen, H.; Bai, F.; Xiu, Z. Oxidative Stress Induced in Saccharomyces cerevisiae Exposed to Dielectric Barrier Discharge Plasma in Air at Atmospheric Pressure. IEEE Trans. Plasma Sci. 2010, 38, 1885–1891. [Google Scholar] [CrossRef]
  28. Tsoukou, E.; Delit, M.; Treint, L.; Bourke, P.; Boehm, D. Distinct Chemistries Define the Diverse Biological Effects of Plasma Activated Water Generated with Spark and Glow Plasma Discharges. Appl. Sci. 2021, 11, 1178. [Google Scholar] [CrossRef]
  29. Rothwell, J.G.; Alam, D.; Carter, D.A.; Soltani, B.; McConchie, R.; Zhou, R.; Cullen, P.J.; Mai-Prochnow, A. The Antimicrobial Efficacy of Plasma-Activated Water against Listeria and E. Coli Is Modulated by Reactor Design and Water Composition. J. Appl. Microbiol. 2022, 132, 2490–2500. [Google Scholar] [CrossRef]
  30. Vlad, I.E.; Martin, C.; Toth, A.R.; Papp, J.; Anghel, S.D. Bacterial Inhibition Effect of Plasma Activated Water. Rom. Rep. Phys. 2019, 71, 602. [Google Scholar]
  31. Xiang, Q.; Fan, L.; Li, Y.; Dong, S.; Li, K.; Bai, Y. A Review on Recent Advances in Plasma-Activated Water for Food Safety: Current Applications and Future Trends. Crit. Rev. Food Sci. Nutr. 2022, 62, 2250–2268. [Google Scholar] [CrossRef] [PubMed]
  32. Royintarat, T.; Seesuriyachan, P.; Boonyawan, D.; Choi, E.H.; Wattanutchariya, W. Mechanism and Optimization of Non-Thermal Plasma-Activated Water for Bacterial Inactivation by Underwater Plasma Jet and Delivery of Reactive Species Underwater by Cylindrical DBD Plasma. Curr. Appl. Phys. 2019, 19, 1006–1014. [Google Scholar] [CrossRef]
  33. Ursache, M.; Moraru, R.; Hnatiuc, E.; Nastase, V.; Mares, M. Comparative Assessment of the Relation between Energy Consumption and Bacterial Burden Reduction Using Plasma Activated Water. In Proceedings of the 2014 International Conference on Optimization of Electrical and Electronic Equipment (OPTIM), Bran, Romania, 22–24 May 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 1036–1041. [Google Scholar]
  34. Li, Y.; Pan, J.; Zhang, Q.; Wang, J.; Zhang, J.; Fang, J. In Vitro Studies of the Antimicrobial Effect of Non-Thermal Plasma-Activated Water as a Novel Mouthwash. Eurpean J. Oral Sci. 2017, 125, 463–470. [Google Scholar] [CrossRef] [PubMed]
  35. Laroussi, M.; Bekeschus, S.; Keidar, M.; Bogaerts, A.; Lu, X.; Hori, M.; Stapelmann, K.; Miller, V.; Reuter, S.; Laux, C.; et al. Low Temperature Plasma for Biology, Hygiene, and Medicine: Perspective and Roadmap. IEEE Trans. Radiat. Plasma Med. Sci. 2022, 64, 127–157. [Google Scholar] [CrossRef]
  36. Li, Y.; Nie, L.; Liu, D.; Kim, S.; Lu, X. Plasma-activated Chemical Solutions and Their Bactericidal Effects. Plasma Process. Polym. 2022, 19, 2100248. [Google Scholar] [CrossRef]
  37. Zhang, Q.; Liang, Y.; Feng, H.; Ma, R.; Tian, Y.; Zhang, J.; Fang, J. A Study of Oxidative Stress Induced by Non-Thermal Plasma-Activated Water for Bacterial Damage. Appl. Phys. Lett. 2013, 102, 203701. [Google Scholar] [CrossRef]
  38. Baik, K.Y.; Kim, Y.H.; Ryu, Y.H.; Kwon, H.S.; Park, G.; Uhm, H.S.; Choi, E.H. Feeding-Gas Effects of Plasma Jets on Escherichia Coli in Physiological. Plasma Process. Polym. 2013, 10, 235–242. [Google Scholar] [CrossRef]
  39. Zhang, Q.; Ma, R.; Tian, Y.; Su, B.; Wang, K.; Yu, S.; Zhang, J.; Fang, J. Sterilization Efficiency of a Novel Electrochemical Disinfectant against Staphylococcus aureus. Environ. Sci. Technol. 2016, 50, 3184–3192. [Google Scholar] [CrossRef]
  40. Xiang, Q.; Kang, C.; Zhao, D.; Niu, L.; Liu, X.; Bai, Y. Influence of Organic Matters on the Inactivation Efficacy of Plasma-Activated Water against E. coli O157:H7 and S. aureus. Food Control 2019, 99, 28–33. [Google Scholar] [CrossRef]
  41. Di Simplicio, P.; Cheeseman, K.H.; Slater, T.F. The Reactivity of the Sh Group of Bovine Serum Albumin with Free Radicals. Free Radic. Res. Commun. 1991, 14, 253–262. [Google Scholar] [CrossRef]
  42. Rowan, N.J.; Espie, S.; Harrower, J.; Anderson, J.G.; Marsili, L.; macGregor, S.J. Pulsed-Plasma Gas-Discharge Inactivation of Microbial Pathogens in Chilled Poultry Wash Water. J. Food Prot. 2007, 70, 2805–2810. [Google Scholar] [CrossRef] [PubMed]
  43. Gurol, C.; Ekinci, F.Y.; Aslan, N.; Korachi, M. Low Temperature Plasma for Decontamination of E. coli in Milk. Int. J. Food Microbiol. 2012, 157, 1–5. [Google Scholar] [CrossRef] [PubMed]
  44. Shen, J.; Tian, Y.; Li, Y.; Ma, R.; Zhang, Q.; Zhang, J.; Fang, J. Bactericidal Effects against S. Aureus and Physicochemical Properties of Plasma Activated Water Stored at Different Temperatures. Sci. Rep. 2016, 6, 28505. [Google Scholar] [CrossRef] [PubMed]
Figure 1. (a) Scheme of the plasma tube array to generate Tb-PAW (b) Scheme of the single plasma tube.
Figure 1. (a) Scheme of the plasma tube array to generate Tb-PAW (b) Scheme of the single plasma tube.
Applsci 12 12325 g001
Figure 2. U–I–characteristics of the plasma source at a power of approx. 400 W.
Figure 2. U–I–characteristics of the plasma source at a power of approx. 400 W.
Applsci 12 12325 g002
Figure 3. (a) Inactivation of Escherichia (E.) coli, Staphylococcus (S.) aureus, Fusobacterium (F.) necrophorum, Porphyromonas (P.) levii and Clostridium (C.) sporogenes after 1 min of Tb-PAW treatment (b) Inactivation of C. sporogenes and S. aureus after 1 min, 5 min and 15 min of Tb-PAW treatment. Results represent the mean ± standard deviation. Significant differences are defined as * p ≤ 0.05. Ctrl = Control (TRIS-buffer); Tb-PAW = TRIS-buffered plasma-activated water.
Figure 3. (a) Inactivation of Escherichia (E.) coli, Staphylococcus (S.) aureus, Fusobacterium (F.) necrophorum, Porphyromonas (P.) levii and Clostridium (C.) sporogenes after 1 min of Tb-PAW treatment (b) Inactivation of C. sporogenes and S. aureus after 1 min, 5 min and 15 min of Tb-PAW treatment. Results represent the mean ± standard deviation. Significant differences are defined as * p ≤ 0.05. Ctrl = Control (TRIS-buffer); Tb-PAW = TRIS-buffered plasma-activated water.
Applsci 12 12325 g003
Figure 4. Comparison of the inactivation capacity of Tb-PAW on E. coli, F. necrophorum and P. levii under the influence of BSA. The treatment time for each bacterium was 1 min. Results represent the mean ± standard deviation. * Significant differences between the TRIS-buffer (control) and Tb-PAW samples are defined as p ≤ 0.05. Not significant differences between Tb-PAW and Tb-PAW with 5% BSA are defined as ns. ns = not significant; BSA = bovine serum albumin; Tb-PAW = TRIS-buffered plasma-activated water.
Figure 4. Comparison of the inactivation capacity of Tb-PAW on E. coli, F. necrophorum and P. levii under the influence of BSA. The treatment time for each bacterium was 1 min. Results represent the mean ± standard deviation. * Significant differences between the TRIS-buffer (control) and Tb-PAW samples are defined as p ≤ 0.05. Not significant differences between Tb-PAW and Tb-PAW with 5% BSA are defined as ns. ns = not significant; BSA = bovine serum albumin; Tb-PAW = TRIS-buffered plasma-activated water.
Applsci 12 12325 g004
Table 1. Means and standard deviation of E. coli counts (log CFU/mL) following 1 min Tb-PAW treatment.
Table 1. Means and standard deviation of E. coli counts (log CFU/mL) following 1 min Tb-PAW treatment.
Storage (hours)
Treatment481224
7 °C treatment temperature
Control 14.93 ± 0.81 A5.02 ± 0.74 A5.71 ± 0.59 A5.05 ± 0.63 A
Tb-PAW 21.23 ± 0.50 B2.40 ± 0.39 B2.40 ± 0.80 B2.43 ± 0.77 B,x
21 °C treatment temperature
Control5.31 ± 0.17 A5.56 ± 0.58 A5.71 ± 0.47 A5.27 ± 0.49 A
Tb-PAW1.14 ± 0.42 B1.18 ± 0.83 B2.94 ± 0.57 B2.09 ± 0.88 B, x
30 °C treatment temperature
Control5.65 ± 0.22 A6.03 ± 0.28 A5.95 ± 0.32 A5.76 ± 0.30 A
Tb-PAW1.16 ± 0.80 a,B0.96 ± 0.45 a,B2.25 ± 0.42 a,B5.03 ± 0.22 b,B,y
a,b Different small letters within the same row differ significantly (p ≤ 0.05). A,B Different capital letters within the same column and treatment temperature differ significantly (p ≤ 0.05). x,y Different letters within the same column differ significantly (p ≤ 0.05). 1 TRIS-buffer was used as a control; 2 Tb-PAW = TRIS-buffered plasma-activated water.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Große-Peclum, V.; Siekmann, L.; Krischek, C.; Avramidis, G.; ten Bosch, L.; Harms, M.; Ochs, C.; Ortmann, R.; Hoedemaker, M.; Ahlfeld, B.; et al. An In Vitro Model Using TRIS-Buffered Plasma-Activated Water to Reduce Pathogenic Microorganisms Involved in Digital Dermatitis Infection in Cattle. Appl. Sci. 2022, 12, 12325. https://doi.org/10.3390/app122312325

AMA Style

Große-Peclum V, Siekmann L, Krischek C, Avramidis G, ten Bosch L, Harms M, Ochs C, Ortmann R, Hoedemaker M, Ahlfeld B, et al. An In Vitro Model Using TRIS-Buffered Plasma-Activated Water to Reduce Pathogenic Microorganisms Involved in Digital Dermatitis Infection in Cattle. Applied Sciences. 2022; 12(23):12325. https://doi.org/10.3390/app122312325

Chicago/Turabian Style

Große-Peclum, Vanessa, Lisa Siekmann, Carsten Krischek, Georg Avramidis, Lars ten Bosch, Marcus Harms, Christian Ochs, Rinat Ortmann, Martina Hoedemaker, Birte Ahlfeld, and et al. 2022. "An In Vitro Model Using TRIS-Buffered Plasma-Activated Water to Reduce Pathogenic Microorganisms Involved in Digital Dermatitis Infection in Cattle" Applied Sciences 12, no. 23: 12325. https://doi.org/10.3390/app122312325

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop