Next Article in Journal
A Review on Pathophysiology, and Molecular Mechanisms of Bacterial Chondronecrosis and Osteomyelitis in Commercial Broilers
Next Article in Special Issue
Inheritance-Specific Dysregulation of Th1- and Th17-Associated Cytokines in Alopecia Areata
Previous Article in Journal
Shapes and Patterns of Heme-Binding Motifs in Mammalian Heme-Binding Proteins
Previous Article in Special Issue
Eosinophils, Basophils, and Neutrophils in Bullous Pemphigoid
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Biomolecules
Volume 13
Issue 7
10.3390/biom13071030
biomolecules-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Combining 16S Sequencing and qPCR Quantification Reveals Staphylococcus aureus Driven Bacterial Overgrowth in the Skin of Severe Atopic Dermatitis Patients

by
Amedeo De Tomassi
1,†,
Anna Reiter
1,†,
Matthias Reiger
1,
Luise Rauer
1,2,3,
Robin Rohayem
1,
CK-CARE Study Group
4,‡,
Claudia Traidl-Hoffmann
1,2,3,4,5,†,
Avidan U. Neumann
1,2,4,† and
Claudia Hülpüsch
1,3,4,*,†
1
Environmental Medicine, Faculty of Medicine, University of Augsburg, 86156 Augsburg, Germany
2
Institute of Environmental Medicine, Helmholtz Zentrum München, 86156 Augsburg, Germany
3
Environmental Medicine, Technical University of Munich, 86156 Augsburg, Germany
4
CK CARE, Christine-Kühne Center for Allergy Research and Education, 7265 Davos, Switzerland
5
ZIEL—Institute for Food and Health, Technical University of Munich, 85354 Freising, Germany
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
‡
Membership of the CK-CARE study group is provided in the Appendix A.
Biomolecules 2023, 13(7), 1030; https://doi.org/10.3390/biom13071030
Submission received: 25 April 2023 / Revised: 11 June 2023 / Accepted: 18 June 2023 / Published: 23 June 2023
(This article belongs to the Special Issue Pathogenesis and Drug Development for Non-communicable Inflammatory Skin Diseases)
Browse Figures
Review Reports Versions Notes

Abstract

:
Atopic dermatitis (AD) is an inflammatory skin disease with a microbiome dysbiosis towards a high relative abundance of Staphylococcus aureus. However, information is missing on the actual bacterial load on AD skin, which may affect the cell number driven release of pathogenic factors. Here, we combined the relative abundance results obtained by next-generation sequencing (NGS, 16S V1-V3) with bacterial quantification by targeted qPCR (total bacterial load = 16S, S. aureus = nuc gene). Skin swabs were sampled cross-sectionally (n = 135 AD patients; n = 20 healthy) and longitudinally (n = 6 AD patients; n = 6 healthy). NGS and qPCR yielded highly inter-correlated S. aureus relative abundances and S. aureus cell numbers. Additionally, intra-individual differences between body sides, skin status, and consecutive timepoints were also observed. Interestingly, a significantly higher total bacterial load, in addition to higher S. aureus relative abundance and cell numbers, was observed in AD patients in both lesional and non-lesional skin, as compared to healthy controls. Moreover, in the lesional skin of AD patients, higher S. aureus cell numbers significantly correlated with the higher total bacterial load. Furthermore, significantly more severe AD patients presented with higher S. aureus cell number and total bacterial load compared to patients with mild or moderate AD. Our results indicate that severe AD patients exhibit S. aureus driven increased bacterial skin colonization. Overall, bacterial quantification gives important insights in addition to microbiome composition by sequencing.

1. Introduction

Atopic eczema (also called atopic dermatitis, AD) is an inflammatory skin disease affecting up to 20% of children and 5% of adults [1,2]. The skin barrier is an interconnected network consisting of physical, chemical, immunological, and microbial aspects and is disrupted on multiple levels in AD patients [3]. Typically, a dysbiosis in the skin microbiome towards Staphylococcus aureus is observed and its relative abundance is associated with disease severity [4,5,6]. S. aureus encodes different virulence factors, such as toxins and proteases in its genome, which have a negative effect on the skin barrier. The expression of virulence factors such as toxins and biofilms are connected to AD severity and are regulated by quorum sensing and, hence, by the bacterial cell numbers [7,8,9,10,11].
While the analysis of the microbiome via 16S amplicon sequencing (next-generation sequencing, NGS) is a powerful method to analyze the microbial composition of a sample, the absolute bacterial load cannot be assessed with this standard procedure in scientific studies [12]. Even though the relative abundance may remain stable under certain conditions, the dynamics of absolute numbers can be very different [13] and have a biological implication due to the cell density-regulated expression of virulence factors via quorum sensing [14]. Thus, by neglecting absolute quantification data, underlying physiological or pathological mechanisms could remain hidden when only considering relative abundance [15].
To quantify the overall bacterial load or single taxa, additional cellular- and molecular-based methods such as fluorescence spectroscopy, flow cytometry, targeted quantitative PCR (qPCR), 16S qRT-PCR, ddPCR, and reference spike-in can be applied. While fluorescence spectroscopy, flow cytometry, and 16S qRT-PCR are suitable for the detection of viable cells, these methods are time-consuming in sample preparation and quantification [16]. As qPCR allows sensitive and specific enumeration of broad and specific microbial populations in a wide range of samples at the gene level in a rapid manner, it is a commonly used method in large scientific studies [16]. For the detection of the total bacterial load, the 16S rRNA is a typical target due to its conserved region across bacterial taxa. However, the 16S rRNA copy numbers vary strongly between different bacterial species [17,18,19]. Therefore, the copy number needs to be considered when calculating cell numbers. Alternatively, a unique gene for the identification of certain species can be chosen [20].
The aim of this study was to evaluate the consensus of S. aureus detection via qPCR and NGS. Furthermore, the relevance of bacterial load in the context of AD was assessed in a longitudinal and cross-sectional skin microbiome study comparing AD patients to healthy controls.

2. Materials and Methods

2.1. Microbiome Datasets

LONGITUDINAL DATASET. The longitudinal study investigated the skin microbiome of n = 6 healthy individuals (HE) and n = 6 AD patients weekly over the course of eight weeks while applying different emollients at the two body sides by using skin swabs. Patients were matched by age, gender, and sampling location. The BampH study was approved by the ethics committee of the Technical University of Munich (187/17S) [6]. Additionally, the raw microbiome data is publicly available at the European Nucleotide Archive under the study accession number PRJEB37663. In total, 108 AD samples were analyzed by qPCR and NGS, of which 1 AD sample did not pass NGS quality control. Furthermore, 90 HE samples were analyzed by qPCR after the exclusion of 1 individual in the HE due to an atopic condition. NGS of HE samples was not used here.
CROSS-SECTIONAL DATASET. The cross-sectional study includes samples from the ProRaD study which was approved by the ethics committee of Switzerland (EK2016-00301) and the local ethics committee of the Technical University of Munich 112/16S. ProRaD is a Prospective longitudinal study to investigate the Remission phase in patients with Atopic Dermatitis (AD) and other allergy-associated diseases, such as asthma, food allergies, and allergic rhinitis of the CK CARE consortium [21]. Here, n = 135 lesional skin swabs and n = 135 non-lesional skin swabs from the first study visit of AD patients from Augsburg (Ethics Technical University Augsburg 112/16S) were analyzed by NGS and qPCR. The lesional sample was taken from a region with the current lesion, preferably from the antecubital fossa (Ac). Two non-lesional samples did not pass NGS quality control.
ADDITIONAL DATASET. To increase the number of samples from healthy (HE) individuals, additional samples from n = 14 healthy individuals were included here from the UV study, which was approved by the ethics committee of the Technical University of Munich (57/18S, 112/16S). The UV study was set up to analyze the UV-protective effect of emollients on the skin physiology and microbiome in healthy participants. Here, only the non-UV radiated, non-treated, baseline visit samples from the volar arm (Va) of 14 HE participants were analyzed by qPCR.
AD SEVERITY. AD severity was determined via SCORingAtopicDermatitis (SCORAD), and patients were separated into three AD severity categories: mild (SCORAD from 0 to ≤25), moderate (SCORAD from >25 to ≤50), and severe (SCORAD >50) [22].

2.2. Microbiome Analysis

SAMPLE PREPARATION. For 16S rRNA gene amplicon sequencing, samples of all three skin microbiome datasets were prepared, as previously published [6]. In brief, skin swabs (Sigma-swab, MWE, Corsham, England) were taken and stored in 500 μL of Stool DNA Stabilizer solution (Stratec, Berlin, Germany). The DNA was extracted with the QIAamp UCP Pathogen kit (Qiagen: Hilden, Germany).
16S rRNA GENE AMPLICON SEQUENCING. The V1–V3 region was amplified using the 27F-YM (5-AGAGTTTGATYMTGGCTCAG-3) and 534R (5-ATTACCGCGGCTGCTGG-3) primers, and barcodes were added in a second PCR step. Subsequently, AMPure XP beads (Beckman Coulter, Fullerton, CA, USA) were used for amplicon purification. Samples were sequenced with the Illumina MiSeq®® platform (Illumina Inc., San Diego, CA, USA) using 2 × 300 bp paired-end reads (MiSeq®® Reagent Kit v3 600 cycles; Illumina Inc.). Denoising of the sequences was performed with DADA2 [23] and annotation with AnnotIEM [24] as well as with the standard RDP database formatted for DADA2 (https://doi.org/10.5281/zenodo.4310151 accessed on 25 April 2023).
QUANTIFICATION VIA QPCR. Absolute quantification of the 16S rRNA gene and the unique S. aureus gene nuc were carried out via a TaqMan assay using the following primers and probes: S. aureus forward primer: GTTGCTTAGTGTTAACTTTAGTTGTA, reverse primer: AATGTCGCAGGTTCTTTATGTAATTT, and probe: FAM-AAGTCTAAGTAGCTCAGCAAATGCA-BHQ1 [25]; 16S rRNA gene copies forward primer: TGGAGCATGTGGTTTAATTCGA, reverse primer: TGCGGGACTTAACCCAACA, and probe: Cy5-CACGAGCTGACGACARCCATGCA-BHQ2 (Eurogentec S.A., Seraing, Belgium) [26]. The reactions were performed in 10 µL final volume using the PerfeCTa Multiplex qPCR ToughMix (Quantabio, Beverly, MA, USA) with a 100 nM concentration for each primer and probe in the multiplex setup. After a 2-min denaturation–activation step at 95 °C, 45 cycles were performed with a denaturation step of 15 s at 95 °C and an annealing–elongation step of 60 s at 60 °C in a CFX384 Real Time System (Bio-Rad Laboratories, Inc., Hercules, CA, USA). The quantity cycles (Cqs) were determined as the average of independent triplicates.

2.3. Statistical Analysis

The S. aureus cell number, also termed here as the qPCR-absolute-abundance, on the skin was assessed by determining the number of nuc gene copies, whereas the total bacterial load on the skin was assessed by determining the number of 16S gene copies, which notably is only a relative estimate since there is more than one 16S gene copy per bacterial cell. To compare the S. aureus relative abundance values obtained with NGS to qPCR results, the proportion of S. aureus cells (number of nuc gene copies) out of the total 16S gene copies per sample (nuc gene copies * 6/total 16S copies) was determined which we term as qPCR-relative-abundance, where the number of nuc gene copies is multiplied by 6 since the nuc gene appears only once per S. aureus cell, while S. aureus has, on average, 6 16S gene copies per cell [18].
Correlations between continuous variables were assessed using the Pearson test. Statistical significance of differences in continuous variable results between 2 and 3 groups was assessed with the non-parametric Mann–Whitney U Test or the Kruskal–Wallis test with Dunn’s pairwise post hoc test, respectively. Moreover, the statistical significance of differences in relative abundance between groups was assessed with the Fisher exact test for 2 × 2 contingency tables and with the Freeman–Halton extension for 2 × 3 contingency tables. p-values were considered significant at the alpha-error two-tailed level of p < 0.05. No correction for multiple testing was applied since we had only tested 3 pre-defined variables in this analysis (S. aureus relative abundance, S. aureus absolute abundance, and 16S copy number). However, even if the strict Bonferroni correction had been applied for the 3 variables, all the relevant p-values would still have been highly significant as they are mostly p < 0.0001. Unless otherwise stated, median values are shown in the figures. The statistical analyses and plots were performed and created, respectively, either with R Version R-4.2.1 or GraphPad Prism version 9.5.0 for Windows (GraphPad Software, La Jolla CA, USA).

3. Results

3.1. Consensus in S. aureus Abundance Results Via NGS and qPCR

To assess the consensus of the results obtained via NGS and qPCR, the relative and/or absolute abundance of S. aureus were determined with both methods in lesional skin samples from AD patients in both the longitudinal skin microbiome dataset (n = 6) and the cross-sectional dataset (n = 135) (demographic data is included in Table S1).
The absolute S. aureus abundance measured by qPCR of the nuc gene correlated strongly and significantly with the relative abundance of S. aureus observed with NGS for lesional AD samples with detectable S. aureus in the longitudinal and cross-sectional datasets (Figure 1). Similarly, the detected S. aureus relative abundance of AD lesional samples was highly and significantly correlated by qPCR and NGS in both datasets (Figure S1). However, in both datasets, a proportion of the samples had a discrepancy in S. aureus detection between the two methods, with S. aureus detected either only by qPCR or by NGS. This phenomenon was especially observed in the low range of relative S. aureus abundance <1%. While the longitudinal dataset had a higher number of samples with undetectable S. aureus by NGS, the cross-sectional dataset had a higher number of samples with undetectable S. aureus via qPCR (Figure 1). The sensitivity of qPCR relative to NGS was 97.8% in the longitudinal dataset and 72.1% in the cross-sectional dataset, while the corresponding sensitivity of NGS relative to qPCR was 88.1% in the longitudinal dataset (Figure 1A) and 100% in the cross-sectional dataset (Figure 1B).
Moreover, the intra-individual differences in S. aureus relative abundance were strongly reproducible between the NGS and qPCR methods when comparing the difference between body sides in the longitudinal dataset (Figure 2A) or between lesional and non-lesional samples in the cross-sectional dataset (Figure 2B).
The changes in the longitudinal behavior of S. aureus follow the same pattern in relative abundance as measured by NGS, relative abundance as estimated by qPCR, or absolute abundance measured by qPCR (Figure 3A,B). However, the amplitude of fluctuations is higher when measured via qPCR than via NGS, as seen in the example in Figure 3A, which was validated for all patients (Figure 3B, Figure S2).

3.2. Bacterial Load in Atopic Dermatitis

The total bacterial load in healthy individuals and in atopic dermatitis patients was estimated by the number of 16S rRNA gene copies measured by qPCR. The longitudinal dataset with six AD patients and five healthy individuals revealed highly similar 16S rRNA gene copy numbers between body sides per individual in the HE and AD groups (Figure 4A,B). The individual bacterial load remained relatively stable throughout the study period of eight weeks. Interestingly, a higher variation in bacterial load was observed between the healthy individuals as compared to AD patients, but nevertheless, all AD patients showed higher bacterial load than HE controls (Figure 4A,B).
Combining data from the longitudinal, cross-sectional, and additional healthy datasets, the total bacterial load was significantly higher in AD non-lesional (n = 135) and lesional (n = 135) skin compared to healthy (n = 20) individuals (HE median = 8.3 × 104, AD NL median = 3.5 × 105, AD LS median = 5.5 × 105) (Figure 4C).
To understand whether the higher bacterial load measured in AD patients is due to an overgrowth of S. aureus or a general increase in bacterial load, the number of measured 16S rRNA gene copies was correlated with the number of S. aureus cells. Strikingly, a significant correlation between the total bacterial load (16S copy number) and S. aureus absolute abundance (number of nuc gene copies) was observed only in the lesional samples of AD patients with S. aureus absolute abundance larger than 105 but not for patients with S. aureus absolute abundance lower than 105 (Figure 4D). This observation indicates that a high bacterial load in AD skin is driven by an increasing number of S. aureus cells.

3.3. Association between S. aureus Bacterial Colonization and Atopic Dermatitis Severity

To investigate whether AD disease severity is driven by the proportion of S. aureus within the bacterial community or by absolute S. aureus cell numbers, we analyzed S. aureus abundance and the total bacterial load in the lesional samples from the cross-sectional dataset (n = 135) based on the following AD severity categories: mild, moderate, and severe, as defined by the SCORAD index. Higher S. aureus relative abundance, measured by NGS, as well as S. aureus absolute abundance measured by qPCR, showed a significant association with AD severity (Figure 5A,B). In addition, significantly more mild AD patients (68.9%) had undetectable S. aureus load by qPCR as compared to patients with moderate (33.3%) and severe (15.4%) AD (Figure 5D). The fraction of patients with undetectable S. aureus load was not significantly different between moderate and severe patients.
Interestingly, while the median S. aureus absolute abundance in mild AD patients was undetectable via qPCR, the median was slightly higher in moderate AD (70 cells) and significantly higher in severe AD (median 3 × 105 cells) patients (Figure 5B). Significantly more severe AD patients (61.5%) also had absolute S. aureus abundance larger than 105, the threshold identified for correlation with total bacterial load (Figure 4D), as compared to patients with moderate (17.6%) and mild (11.1%) AD (Figure 5D).
Moreover, the total bacterial load (16S copy number) was significantly higher in severe AD patients (median 2 × 106) as compared to moderate AD (5 × 105) and mild AD (6 × 105) (Figure 5C). A total bacterial load larger than 106 was observed in significantly more (66.7%) severe AD patients, as compared to patients with moderate (35.3%) and mild (37.8%) AD (Figure 5D). Thus, severe AD patients demonstrate a higher S. aureus abundance associated with a higher total bacterial load.
S. aureus absolute and relative abundance as well as 16S copy numbers were not significantly associated with gender and not correlated with age (Figure S3). Furthermore, the association of severe AD with higher levels of S. aureus absolute and relative abundance, as well as with higher 16S copy numbers, is observed independently of the age range (Figure S3).

4. Discussion

Our results suggest a high level of consistency in the relative abundance of S. aureus between data generated via qPCR and NGS. Nonetheless, the absolute quantification of total bacterial load and of S. aureus gave additional relevant biological insights, hinting towards an S. aureus driven total bacterial overgrowth in absolute cell numbers in severe AD patients.
Even though there was a strong correlation between the S. aureus relative and absolute abundances determined via NGS and qPCR, some samples had S. aureus detected either only by qPCR or only by NGS. Both methods used in this study were DNA-based approaches and therefore are comparable techniques. While DNA-based methods such as NGS and qPCR may be affected by biases introduced during sampling, cell lysis and DNA extraction, these steps would not explain the differences between the two detection methods since both use the same extracted bacterial DNA. Therefore, the integrity of the DNA is important for reliable results in both methods [27].
There is a wide range of possible explanations for the discrepancy of observed S. aureus relative and absolute abundances between the two methods. First, both use different amounts of input DNA, where stochastic effects can particularly vary the detection of low-range S. aureus abundance. Discrepancies were mainly observed in low-level S. aureus abundance and therefore close to the technical limit of detection and quantification [28]. Furthermore, two different target genes were used for qPCR detection of S. aureus (nuc, single copied) and NGS detection of S. aureus (16S rRNA gene copies, median of six copies). The difference in copy numbers could explain the failed qPCR detection of S. aureus, while NGS gave a positive result for S. aureus based on the multi-copy 16S rRNA gene [18,19]. Additionally, mutations in one of the target genes per method could lead to reduced amplification in PCR for NGS or qPCR, and would affect only one of the two methods. Moreover, a higher calculated percentage of S. aureus in relation to 16S rRNA gene copies was found in some samples, possibly due to S. aureus strains with either more than six copy numbers of 16S rRNA gene copies or reduced efficiency of the 16S rRNA gene qPCR reaction.
A particular challenge for NGS data is the taxonomic (mis)annotation of sequences. In this study, the V1–V3 region of the 16S rRNA was sequenced and used for species annotation. Using only parts of the 16S rRNA gene for species annotation is not as precise as using the full-length 16S rRNA gene [29]. However, the V1–3 hypervariable region represents bacterial communities similarly well as the whole genome sequencing [30], and is able to distinguish S. aureus from most other Staphylococci [29,30]. To achieve a highly accurate species-level classification, we used taxonomic assignment by AnnotIEM, which combines several databases for annotation [24,31]. To verify the annotation given by AnnotIEM based on multiple databases, each sequence was re-annotated using the standard RDP database through DADA2, which confirmed the S. aureus annotation.
Even though qPCR is frequently used in science due to its aforementioned advantages, qPCR also suffers from challenges, such as reduced sensitivity due to PCR inhibitors or the multiplex set-up. Particularly when close to the detection limit, the precision of qPCR might be hampered as low-level presence of the target gene can result in stochastic amplification and measurement uncertainties [32]. A coefficient of variation between 28% and 79% was observed for three test organisms near the detection limit due to instrument repeatability in one study [33]. Another study revealed that a relative abundance of <10% was associated with high rates of errors when inferring the original species concentration [34].
Regarding the biological implications of total bacterial cells on the skin, the longitudinal dataset revealed a participant-specific bacterial load in healthy individuals between 103 and 106 copy numbers of 16S rRNA. A bacterial load between 103 and 107 16S rRNA gene copies has been reported on healthy skin, with major site-specific differences between moist, sebaceous, and dry body sides [35]. Additionally, in the longitudinal dataset, different body locations were sampled, which might have affected the large variation in bacterial colonization observed in healthy individuals. However, these differences between either individuals or body sides were lost in the AD patients, where all patients had a median of 105 bacterial cells on the sampled skin location.
The total bacterial load in AD patients was increased 10-fold compared to healthy individuals in our data as well as in previous studies [35,36]. In other environments, such as the gut, the importance of the overall bacterial load in human physiology and health has already been reported [13,37,38]. Differences in bacterial load are plausibly associated with disease status, as known from single-pathogen infectious diseases and the effect of quorum sensing on the pathogenicity of bacteria. In the context of AD, an increase in S. aureus relative abundance, especially during disease flares in lesional skin, is already well known and could be confirmed in this study [4]. S. aureus virulence factors, such as toxins, proteases, and biofilms, are known to negatively impact the skin barrier and exacerbate AD severity [39]. The release of toxins and proteases by S. aureus highly depends on quorum sensing, and therefore the absolute bacterial cell numbers might hypothetically be more relevant than the relative composition of microbes on the skin [7,14]. However, other bacterial species, fungal infections, IgA and IgE levels, and other clinical factors (such as other allergy-associated diseases, asthma, food allergies and/or allergic rhinitis) may also be associated with AD severity but were not further analyzed here.
In this study, we show, for the first time, that in AD patients, not only the bacterial composition is changed towards S. aureus but also that there is an S. aureus driven increase of the total bacterial load, especially in severe patients. Strikingly, the bacterial load increased in an S. aureus driven manner only above a threshold of 105 S. aureus cells. This could be a critical cell number to activate the expression of toxins and proteases, leading to inflammation and skin barrier disruption, which can lead to an increased skin pH [40]. Changes in the microenvironment such as an increased skin pH are in favor of S. aureus, leading to an increased growth rate, followed by disease exacerbation due to further irritation of the skin by virulence factors [6,9,40]. In line, successful AD treatment with the monoclonal, IL4-Rα- blocking antibody Dupilumab has been reported to reduce absolute bacterial load and S. aureus load and to increase microbial diversity while reducing disease severity [41,42].
The evidence presented here for S. aureus driven bacterial overgrowth in AD provides mechanistic insights into the development and exacerbation of AD symptoms and may indicate S. aureus and its quorum sensing system as a target for new therapeutics in AD. We demonstrated in this study that NGS data on the skin microbiome composition can be importantly complemented by information on absolute cell number provided by quantitative methods, such as qPCR in the context of AD research.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/biom13071030/s1, Figure S1: Correlation between the relative S. aureus abundance observed via NGS or qPCR in the longitudinal dataset (A) and cross-sectional dataset (B); Figure S2: Kinetics of S. aureus absolute and relative abundance for all 6 AD patients in the longitudinal dataset; Figure S3: S. aureus absolute (A) and relative (B) abundance and 16S copy number (C) as function of age and AD severity in the cross-sectional dataset; Table S1: Demographic data for the longitudinal dataset (A) and cross-sectional dataset (B).

Author Contributions

Conceptualization, C.T.-H., A.U.N. and C.H.; data curation, A.R., A.U.N. and C.H.; formal analysis, A.D.T., A.R., L.R., R.R., A.U.N. and C.H.; funding acquisition, M.R. and C.T.-H.; methodology, A.D.T., A.R., M.R., L.R., A.U.N. and C.H.; project administration, C.T.-H., A.U.N. and C.H.; resources, ProRaD study concept and design, CK-CARE Study Group supervision, A.U.N. and C.H.; validation, A.U.N. and C.H.; visualization, A.R., A.U.N. and C.H.; writing—original draft, A.D.T., A.R. and C.H.; Writing—review and editing, A.D.T., A.R., M.R., L.R., R.R., C.T.-H., A.U.N. and C.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Christine Kühne-Center for Allergy Research and Education (CK-CARE).

Institutional Review Board Statement

This study was conducted in accordance with the Declaration of Helsinki. The longitudinal BampH study was approved by the ethics committee of the Technical University Munich (187/17S). The cross-sectional ProRaD study was approved by the ethics committee of Switzerland (EK2016-00301). The additional UV study was approved by the ethics committee of the Technical University Munich (57/18S, 112/16S).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data has been partly published already and is accessible in the European Nucleotide Archive under the accession number PRJEB37663.

Acknowledgments

We are grateful to all our study participants and, in particular, to the patients of the ProRaD study funded by the CK-CARE.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

CK-CARE Study Group investigators share the authorship:
Thomas Bieber, MD, PhD1,2,3
Peter Schmid-Grendelmeier, MD2,10
Claudia Traidl-Hoffmann, MD2,8,9
Cezmi Akdis, PhD2,4
Roger Lauener, MD2,5
Marie-Charlotte Brüggen, MD, PhD2,6,7
Claudio Rhyner, PhD2,3,4
Laura Maintz, MD1,2,3
Nadine Herrmann, PhD1,2, 11
Eugen Bersuch, MD2,12
Avidan Neumann, PhD2,8,9
Gertrud Hammel, MA2,8,9
Daria Luschkova, MD2,8,9
Claudia CV Lang, MD2,3,7
Anita Dreher, M.Sc2
Matthias Reiger, PhD2,8,9
  • Department of Dermatology and Allergy, University Hospital Bonn, Venusberg-Campus 1, 53127 Bonn, Germany
  • Christine Kühne-Center for Allergy Research and Education Davos (CK-CARE), Herman-Burchard-Str. 1, 7265 Davos, Switzerland
  • Davos Biosciences, Herman-Burchard-Str. 1, 7265 Davos, Switzerland
  • Swiss Institute of Allergy and Asthma Research (SIAF), Davos, Herman-Burchard-Str. 9, 7265 Davos, Switzerland
  • Children’s Hospital of Eastern Switzerland, Claudiusstrasse 6, 9006 St.Gallen, Switzerland
  • Hochgebirgsklinik Davos, Herman-Burchard-Str. 1, 7265 Davos, Switzerland
  • Faculty of Medicine, University of Zürich, Switzerland
  • Environmental Medicine, Faculty of Medicine, University of Augsburg, Stenglinstraße 2, 86156 Augsburg, Germany
  • Institute of Environmental Medicine, Helmholtz Zentrum Muenchen, German Research Center for Environmental Health, Augsburg, Germany
  • Allergy Unit, Dept. of Dermatology, University Hospital of Zürich, Rämistrasse 100, 8091 Zürich, Switzerland
  • Department of Dermatology and Allergy, University Hospital Bonn, Bonn, Germany

References

  1. Barbarot, S.; Auziere, S.; Gadkari, A.; Girolomoni, G.; Puig, L.; Simpson, E.L.; Margolis, D.J.; de Bruin-Weller, M.; Eckert, L. Epidemiology of atopic dermatitis in adults: Results from an international survey. Allergy 2018, 73, 1284–1293. [Google Scholar] [CrossRef] [Green Version]
  2. Bylund, S.; Kobyletzki, L.B.; Svalstedt, M.; Svensson, Å. Prevalence and Incidence of Atopic Dermatitis: A Systematic Review. Acta Derm. Venereol. 2020, 100, adv00160. [Google Scholar] [CrossRef] [PubMed]
  3. Eyerich, S.; Eyerich, K.; Traidl-Hoffmann, C.; Biedermann, T. Cutaneous Barriers and Skin Immunity: Differentiating A Connected Network. Trends Immunol. 2018, 39, 315–327. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Kong, H.H.; Oh, J.; Deming, C.; Conlan, S.; Grice, E.A.; Beatson, M.A.; Nomicos, E.; Polley, E.C.; Komarow, H.D.; Murray, P.R.; et al. Temporal shifts in the skin microbiome associated with disease flares and treatment in children with atopic dermatitis. Genome Res. 2012, 22, 850–859. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Rauer, L.; Reiger, M.; Bhattacharyya, M.; Brunner, P.M.; Krueger, J.G.; Guttman-Yassky, E.; Traidl-Hoffmann, C.; Neumann, A.U. Skin microbiome and its association with host cofactors in determining atopic dermatitis severity. J. Eur. Acad. Dermatol. Venereol. 2023, 37, 772–782. [Google Scholar] [CrossRef]
  6. Hülpüsch, C.; Tremmel, K.; Hammel, G.; Bhattacharyya, M.; de Tomassi, A.; Nussbaumer, T.; Neumann, A.U.; Reiger, M.; Traidl-Hoffmann, C. Skin pH–dependent Staphylococcus aureus abundance as predictor for increasing atopic dermatitis severity. Allergy 2020, 75, 2888–2898. [Google Scholar] [CrossRef]
  7. Tam, K.; Torres, V.J. Staphylococcus aureus Secreted Toxins and Extracellular Enzymes. Microbiol. Spectr. 2019, 7, 16. [Google Scholar] [CrossRef]
  8. Ogonowska, P.; Gilaberte, Y.; Barańska-Rybak, W.; Nakonieczna, J. Colonization With Staphylococcus aureus in Atopic Dermatitis Patients: Attempts to Reveal the Unknown. Front. Microbiol. 2020, 11, 567090. [Google Scholar] [CrossRef]
  9. Le, K.Y.; Otto, M. Quorum-sensing regulation in staphylococci-an overview. Front. Microbiol. 2015, 6, 1174. [Google Scholar] [CrossRef] [Green Version]
  10. Gonzalez, T.; Stevens, M.L.; Baatyrbek Kyzy, A.; Alarcon, R.; He, H.; Kroner, J.W.; Spagna, D.; Grashel, B.; Sidler, E.; Martin, L.J.; et al. Biofilm propensity of Staphylococcus aureus skin isolates is associated with increased atopic dermatitis severity and barrier dysfunction in the MPAACH pediatric cohort. Allergy 2021, 76, 302–313. [Google Scholar] [CrossRef]
  11. Blicharz, L.; Michalak, M.; Szymanek-Majchrzak, K.; Młynarczyk, G.; Skowroński, K.; Rudnicka, L.; Samochocki, Z. The Propensity to Form Biofilm in vitro by Staphylococcus aureus Strains Isolated from the Anterior Nares of Patients with Atopic Dermatitis: Clinical Associations. Dermatology 2021, 237, 528–534. [Google Scholar] [CrossRef] [PubMed]
  12. Knight, R.; Vrbanac, A.; Taylor, B.C.; Aksenov, A.; Callewaert, C.; Debelius, J.; Gonzalez, A.; Kosciolek, T.; McCall, L.-I.; McDonald, D.; et al. Best practices for analysing microbiomes. Nat. Rev. Microbiol. 2018, 16, 410–422. [Google Scholar] [CrossRef] [Green Version]
  13. Barlow, J.T.; Bogatyrev, S.R.; Ismagilov, R.F. A quantitative sequencing framework for absolute abundance measurements of mucosal and lumenal microbial communities. Nat. Commun. 2020, 11, 2590. [Google Scholar] [CrossRef] [PubMed]
  14. Miller, M.B.; Bassler, B.L. Quorum sensing in bacteria. Annu. Rev. Microbiol. 2001, 55, 165–199. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Tkacz, A.; Hortala, M.; Poole, P.S. Absolute quantitation of microbiota abundance in environmental samples. Microbiome 2018, 6, 110. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Wang, X.; Howe, S.; Deng, F.; Zhao, J. Current Applications of Absolute Bacterial Quantification in Microbiome Studies and Decision-Making Regarding Different Biological Questions. Microorganisms 2021, 9, 1797. [Google Scholar] [CrossRef] [PubMed]
  17. Louca, S.; Doebeli, M.; Parfrey, L.W. Correcting for 16S rRNA gene copy numbers in microbiome surveys remains an unsolved problem. Microbiome 2018, 6, 41. [Google Scholar] [CrossRef]
  18. Větrovský, T.; Baldrian, P. The Variability of the 16S rRNA Gene in Bacterial Genomes and Its Consequences for Bacterial Community Analyses. PLoS ONE 2013, 8, e57923. [Google Scholar] [CrossRef] [Green Version]
  19. Klappenbach, J.A.; Saxman, P.R.; Cole, J.R.; Schmidt, T.M. rrndb: The Ribosomal RNA Operon Copy Number Database. Nucleic Acids Res. 2001, 29, 181–184. [Google Scholar] [CrossRef] [Green Version]
  20. Wang, S.; Ventolero, M.; Hu, H.; Li, X. A revisit to universal single-copy genes in bacterial genomes. Sci. Rep. 2022, 12, 14550. [Google Scholar] [CrossRef]
  21. Bieber, T.; Traidl-Hoffmann, C.; Schäppi, G.; Lauener, R.; Akdis, C.; Schmid-Grendlmeier, P. Unraveling the complexity of atopic dermatitis: The CK-CARE approach toward precision medicine. Allergy 2020, 75, 2936–2938. [Google Scholar] [CrossRef]
  22. Severity scoring of atopic dermatitis: The SCORAD index. Consensus Report of the European Task Force on Atopic Dermatitis. Dermatology 1993, 186, 23–31. [CrossRef] [PubMed]
  23. Callahan, B.J.; McMurdie, P.J.; Rosen, M.J.; Han, A.W.; Johnson, A.J.A.; Holmes, S.P. DADA2: High-resolution sample inference from Illumina amplicon data. Nat. Methods 2016, 13, 581. [Google Scholar] [CrossRef] [Green Version]
  24. Bhattacharyya, M.; Reiger, M.; Rauer, L.; Huelpuesch, C.; Traidl-Hoffman, C.; Neumann, A.U. AnnotIEM: A Novel Algorithm for Species Level Annotation of 16S Gene Based Microbial OTUs [Version 1; not Peer Reviewed], in F1000 Research; ISMB/ECBB: Basel, Switzerland, 2019. [Google Scholar]
  25. Kilic, A.; Muldrew, K.L.; Tang, Y.W.; Basustaoglu, A.C. Triplex real-time polymerase chain reaction assay for simultaneous detection of Staphylococcus aureus and coagulase-negative staphylococci and determination of methicillin resistance directly from positive blood culture bottles. Diagn. Microbiol. Infect. Dis. 2010, 66, 349–355. [Google Scholar] [CrossRef]
  26. Yang, S.; Lin, S.; Kelen, G.D.; Quinn, T.C.; Dick, J.D.; Gaydos, C.A.; Rothman, R.E. Quantitative multiprobe PCR assay for simultaneous detection and identification to species level of bacterial pathogens. J. Clin. Microbiol. 2002, 40, 3449–3454. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Manzari, C.; Oranger, A.; Fosso, B.; Piancone, E.; Pesole, G.; D’Erchia, A.M. Accurate quantification of bacterial abundance in metagenomic DNAs accounting for variable DNA integrity levels. Microb. Genom. 2020, 6, e000417. [Google Scholar] [CrossRef] [PubMed]
  28. Kralik, P.; Ricchi, M. A Basic Guide to Real Time PCR in Microbial Diagnostics: Definitions, Parameters, and Everything. Front. Microbiol. 2017, 8, 108. [Google Scholar] [CrossRef] [Green Version]
  29. Johnson, J.S.; Spakowicz, D.J.; Hong, B.-Y.; Petersen, L.M.; Demkowicz, P.; Chen, L.; Leopold, S.R.; Hanson, B.M.; Agresta, H.O.; Gerstein, M.; et al. Evaluation of 16S rRNA gene sequencing for species and strain-level microbiome analysis. Nat. Commun. 2019, 10, 5029. [Google Scholar] [CrossRef] [Green Version]
  30. Meisel, J.S.; Hannigan, G.D.; Tyldsley, A.S.; SanMiguel, A.J.; Hodkinson, B.P.; Zheng, Q.; Grice, E.A. Skin Microbiome Surveys Are Strongly Influenced by Experimental Design. J. Investig. Dermatol. 2016, 136, 947–956. [Google Scholar] [CrossRef]
  31. Gwak, H.-J.; Rho, M. Data-Driven Modeling for Species-Level Taxonomic Assignment From 16S rRNA: Application to Human Microbiomes. Front. Microbiol. 2020, 11, 570825. [Google Scholar] [CrossRef]
  32. Taylor, S.C.; Nadeau, K.; Abbasi, M.; Lachance, C.; Nguyen, M.; Fenrich, J. The Ultimate qPCR Experiment: Producing Publication Quality, Reproducible Data the First Time. Trends Biotechnol. 2019, 37, 761–774. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Hospodsky, D.; Yamamoto, N.; Peccia, J. Accuracy, Precision, and Method Detection Limits of Quantitative PCR for Airborne Bacteria and Fungi. Appl. Environ. Microbiol. 2010, 76, 7004–7012. [Google Scholar] [CrossRef] [Green Version]
  34. Boshier, F.A.T.; Srinivasan, S.; Lopez, A.; Hoffman, N.G.; Proll, S.; Fredricks, D.N.; Schiffer, J.T. Complementing 16S rRNA Gene Amplicon Sequencing with Total Bacterial Load To Infer Absolute Species Concentrations in the Vaginal Microbiome. mSystems 2020, 5, e00777-00719. [Google Scholar]
  35. Gao, Z.; Perez-Perez, G.I.; Chen, Y.; Blaser, M.J. Quantitation of major human cutaneous bacterial and fungal populations. J. Clin. Microbiol. 2010, 48, 3575–3581. [Google Scholar] [CrossRef] [Green Version]
  36. Morton, J.T.; Marotz, C.; Washburne, A.; Silverman, J.; Zaramela, L.S.; Edlund, A.; Zengler, K.; Knight, R. Establishing microbial composition measurement standards with reference frames. Nat. Commun. 2019, 10, 2719. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Vandeputte, D.; Kathagen, G.; D’hoe, K.; Vieira-Silva, S.; Valles-Colomer, M.; Sabino, J.; Wang, J.; Tito, R.Y.; De Commer, L.; Darzi, Y.; et al. Quantitative microbiome profiling links gut community variation to microbial load. Nature 2017, 551, 507–511. [Google Scholar] [CrossRef]
  38. Jian, C.; Luukkonen, P.; Yki-Järvinen, H.; Salonen, A.; Korpela, K. Quantitative PCR provides a simple and accessible method for quantitative microbiota profiling. PLoS ONE 2020, 15, e0227285. [Google Scholar] [CrossRef] [Green Version]
  39. Seiti Yamada Yoshikawa, F.; Feitosa de Lima, J.; Notomi Sato, M.; Álefe Leuzzi Ramos, Y.; Aoki, V.; Leao Orfali, R. Exploring the Role of Staphylococcus Aureus Toxins in Atopic Dermatitis. Toxins 2019, 11, 321. [Google Scholar] [CrossRef] [Green Version]
  40. Proksch, E. pH in nature, humans and skin. J. Dermatol. 2018, 45, 1044–1052. [Google Scholar] [CrossRef]
  41. Callewaert, C.; Nakatsuji, T.; Knight, R.; Kosciolek, T.; Vrbanac, A.; Kotol, P.; Ardeleanu, M.; Hultsch, T.; Guttman-Yassky, E.; Bissonnette, R.; et al. IL-4Rα Blockade by Dupilumab Decreases Staphylococcus aureus Colonization and Increases Microbial Diversity in Atopic Dermatitis. J. Investig. Dermatol. 2020, 140, 191–202.e197. [Google Scholar] [CrossRef] [Green Version]
  42. Olesen, C.M.; Ingham, A.C.; Thomsen, S.F.; Clausen, M.L.; Andersen, P.S.; Edslev, S.M.; Yüksel, Y.T.; Guttman-Yassky, E.; Agner, T. Changes in Skin and Nasal Microbiome and Staphylococcal Species Following Treatment of Atopic Dermatitis with Dupilumab. Microorganisms 2021, 9, 1487. [Google Scholar] [CrossRef] [PubMed]
Biomolecules 13 01030 g001 550
Figure 1. High consensus of qPCR and NGS in estimating S. aureus abundance. There is a strong correlation between the relative S. aureus abundance observed via NGS and the absolute S. aureus abundance measured by qPCR in both the longitudinal (A) and the cross-sectional (B) datasets. Pearson correlations were performed for samples with detectable S. aureus. Colors in the longitudinal dataset correspond to samples from the 6 different patients. UD = undetectable.
Figure 1. High consensus of qPCR and NGS in estimating S. aureus abundance. There is a strong correlation between the relative S. aureus abundance observed via NGS and the absolute S. aureus abundance measured by qPCR in both the longitudinal (A) and the cross-sectional (B) datasets. Pearson correlations were performed for samples with detectable S. aureus. Colors in the longitudinal dataset correspond to samples from the 6 different patients. UD = undetectable.
Biomolecules 13 01030 g001
Biomolecules 13 01030 g002 550
Figure 2. Reflection of intra-individual differences is similar between NGS and qPCR. The difference in S. aureus relative abundance between body sides (A) or between lesional and non-lesional samples (B) of same patients at the same timepoints was strongly and highly significantly correlated (Pearson test).
Figure 2. Reflection of intra-individual differences is similar between NGS and qPCR. The difference in S. aureus relative abundance between body sides (A) or between lesional and non-lesional samples (B) of same patients at the same timepoints was strongly and highly significantly correlated (Pearson test).
Biomolecules 13 01030 g002
Biomolecules 13 01030 g003 550
Figure 3. Change in S. aureus abundance over time as reflected by NGS and qPCR. The changes in relative and absolute S. aureus abundance over time followed the same pattern as measured by NGS or by qPCR (A). The differences in S. aureus relative abundance between consecutive time points significantly correlated between NGS and qPCR (B). Blue reflects absolute S. aureus abundance (qPCR), red reflects relative abundance of S. aureus by qPCR, and orange indicates relative abundance of S. aureus by NGS. Correlation analysis was performed using the Pearson test.
Figure 3. Change in S. aureus abundance over time as reflected by NGS and qPCR. The changes in relative and absolute S. aureus abundance over time followed the same pattern as measured by NGS or by qPCR (A). The differences in S. aureus relative abundance between consecutive time points significantly correlated between NGS and qPCR (B). Blue reflects absolute S. aureus abundance (qPCR), red reflects relative abundance of S. aureus by qPCR, and orange indicates relative abundance of S. aureus by NGS. Correlation analysis was performed using the Pearson test.
Biomolecules 13 01030 g003
Biomolecules 13 01030 g004 550
Figure 4. Bacterial overgrowth driven by S. aureus in AD patients. The bacterial load measured as 16S rRNA gene copies remained relatively stable in AD patients (A) and healthy controls (B) over nine weeks on two body sides. Overall, bacterial load is significantly higher in AD non-lesional (NL) and lesional (LS) skin compared to healthy (HE) controls (C). Total bacterial load (16S copy number) is correlated with S. aureus absolute abundance in lesional skin of AD patients only for patients with S. aureus absolute abundance larger than 105 (D). In (A,B), colors represent different patients and shapes show body sides. ns = non-significant, UD = undetectable.
Figure 4. Bacterial overgrowth driven by S. aureus in AD patients. The bacterial load measured as 16S rRNA gene copies remained relatively stable in AD patients (A) and healthy controls (B) over nine weeks on two body sides. Overall, bacterial load is significantly higher in AD non-lesional (NL) and lesional (LS) skin compared to healthy (HE) controls (C). Total bacterial load (16S copy number) is correlated with S. aureus absolute abundance in lesional skin of AD patients only for patients with S. aureus absolute abundance larger than 105 (D). In (A,B), colors represent different patients and shapes show body sides. ns = non-significant, UD = undetectable.
Biomolecules 13 01030 g004
Biomolecules 13 01030 g005 550
Figure 5. Association between S. aureus and AD severity. S. aureus relative abundance measured via NGS (A) and S. aureus absolute abundance measured via qPCR (B)) were significantly higher in severe AD patients. Bacterial load of severe patients determined via 16S rRNA gene copies was significantly higher in severe patients compared to mild and moderate patients (C). Significant differences were observed in numbers of patients with undetectable or high S. aureus and total bacterial load between mild, moderate, and severe patients (D). Mild= SCORAD between 0 and 25, moderate= SCORAD between >25 and 50, severe= SCORAD >50. Mann–Whitney U test was used for statistical analysis. Dotted horizontal lines indicate the threshold for detection limit (UD = undetectable), and dashed horizontal lines denote high numbers of S. aureus (105 for absolute S. aureus abundance and 10% for relative S. aureus abundance in qPCR or NGS). ns = not-significant, UD = undetectable.
Figure 5. Association between S. aureus and AD severity. S. aureus relative abundance measured via NGS (A) and S. aureus absolute abundance measured via qPCR (B)) were significantly higher in severe AD patients. Bacterial load of severe patients determined via 16S rRNA gene copies was significantly higher in severe patients compared to mild and moderate patients (C). Significant differences were observed in numbers of patients with undetectable or high S. aureus and total bacterial load between mild, moderate, and severe patients (D). Mild= SCORAD between 0 and 25, moderate= SCORAD between >25 and 50, severe= SCORAD >50. Mann–Whitney U test was used for statistical analysis. Dotted horizontal lines indicate the threshold for detection limit (UD = undetectable), and dashed horizontal lines denote high numbers of S. aureus (105 for absolute S. aureus abundance and 10% for relative S. aureus abundance in qPCR or NGS). ns = not-significant, UD = undetectable.
Biomolecules 13 01030 g005
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

De Tomassi, A.; Reiter, A.; Reiger, M.; Rauer, L.; Rohayem, R.; CK-CARE Study Group; Traidl-Hoffmann, C.; Neumann, A.U.; Hülpüsch, C. Combining 16S Sequencing and qPCR Quantification Reveals Staphylococcus aureus Driven Bacterial Overgrowth in the Skin of Severe Atopic Dermatitis Patients. Biomolecules 2023, 13, 1030. https://doi.org/10.3390/biom13071030

AMA Style

De Tomassi A, Reiter A, Reiger M, Rauer L, Rohayem R, CK-CARE Study Group, Traidl-Hoffmann C, Neumann AU, Hülpüsch C. Combining 16S Sequencing and qPCR Quantification Reveals Staphylococcus aureus Driven Bacterial Overgrowth in the Skin of Severe Atopic Dermatitis Patients. Biomolecules. 2023; 13(7):1030. https://doi.org/10.3390/biom13071030

Chicago/Turabian Style

De Tomassi, Amedeo, Anna Reiter, Matthias Reiger, Luise Rauer, Robin Rohayem, CK-CARE Study Group, Claudia Traidl-Hoffmann, Avidan U. Neumann, and Claudia Hülpüsch. 2023. "Combining 16S Sequencing and qPCR Quantification Reveals Staphylococcus aureus Driven Bacterial Overgrowth in the Skin of Severe Atopic Dermatitis Patients" Biomolecules 13, no. 7: 1030. https://doi.org/10.3390/biom13071030

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop