Abstract
The present umbrella review of four systematic reviews with meta-analysis aimed to assess whether clear aligners are associated with better periodontal conditions compared with fixed appliances in patients undergoing orthodontic treatment. The present study protocol was developed in accordance with the PRISMA statement before the literature search, data extraction, and analysis and was registered on PROSPERO (CRD42023401808). The question formulation, search, and study selection strategies were developed according to the PICO model. Systematic reviews with a meta-analysis published in English without date restriction were electronically searched across the Cochrane Library, Web of Science (Core Collection), Scopus, EMBASE, and MEDLINE/PubMed databases until 10 February 2023. An assessment of study quality was performed using the AMSTAR 2 tool. Differences in the PI, GI, and BOP in the short- and medium-term follow-ups, in the PPD in long-term follow-up, and the gingival recessions in the short-term follow-up were found between subjects with clear aligners and fixed appliances, revealing a slight tendency for clear aligners to be associated with healthier periodontal conditions. However, even if statistically significant, such differences would be negligible in a clinical environment. Therefore, the impact of orthodontic treatment with clear aligners and fixed appliances on periodontal health status should be considered comparable.
1. Introduction
Periodontal health is defined by the absence of microscopically and macroscopically detectable signs of inflammation that may affect periodontal physiology [1].
Periodontal health maintenance in patients undergoing orthodontic treatment depends on several factors, including the patient’s oral hygiene habits and biofilm control, periodontal host–microbe homeostasis, periodontal phenotype, especially with regard to buccal bone plate width, systemic conditions and diseases (e.g., diabetes mellitus) directly and indirectly affecting the periodontal status and oral microbiome, and personal habits (e.g., smoking) [2,3].
Traditional orthodontic treatment encourages tooth movement to correct dental malocclusion through appliances fixed to teeth surfaces, such as orthodontic bands and brackets, archwires, ligatures, and auxiliaries [4,5].
Fixed orthodontic appliances often complicate oral hygiene procedures [6] and facilitate plaque accumulation on both teeth and the appliances’ surfaces [7,8,9]. Indeed, the biofilm control and clinical periodontal inflammatory parameters are generally worse in patients undergoing orthodontic treatment with fixed appliances than in non-orthodontic patients [9]. Hence, a combination of manual, orthodontic, or powered brushing, motivational aids, and organic products, or the short-term use of chlorhexidine mouthwashes could be recommended for biofilm control and gingival inflammation reduction in subjects with fixed orthodontic treatment [10].
Clear aligners were introduced in 1999 to overcome some limitations of fixed appliances and to satisfy the esthetic and comfort requirements of patients. Indeed, orthodontic treatment with clear aligners is based on removable thermoplastic splints covering the entire dental arch, progressively moving the teeth into an ideal position [11]. The current literature describes orthodontic treatment with clear aligners as safe, comfortable, and aesthetic [12,13]. Moreover, clear aligners have been reported to offer an advantage over fixed appliances in the segmented movement of teeth and shortened treatment duration [14]. In contrast, fixed appliances seemed more effective in producing adequate occlusal contacts and controlling teeth torque and rotation [15]. Nevertheless, recent technological developments have made it possible to treat many complex malocclusions with clear aligners [16].
In addition, clear aligners can be easily removed during meals and oral hygiene procedures, allowing patients to effectively control gingival biofilm [2,13] and, thus, presumably maintain healthier periodontal conditions during orthodontic treatment as compared to traditional fixed appliances. This is particularly important for adults, who have a higher prevalence of periodontitis, seeking orthodontic treatment [17].
However, the evidence on the periodontal health status of subjects undergoing orthodontic treatment with clear aligners and fixed appliances remains contradictory [18].
Hence, the aim of the present umbrella review was to summarize the current evidence in order to assess whether clear aligners are associated with a more beneficial impact on periodontal health status compared to fixed appliances in patients undergoing orthodontic treatment.
The null hypothesis was that the impact of orthodontic treatment with clear aligners and fixed appliances on periodontal health is comparable.
2. Materials and Methods
2.1. Study Protocol
The present study protocol, registered in the PROSPERO International Prospective Register of Systematic Reviews (CRD42023401808), was developed before the literature search, data extraction, and analysis and performed in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) statement (Supplementary Material) [19,20].
The study question definition, search strategies, and study selection criteria were developed according to the PICO model [21]. The study question was “Are clear aligners more beneficial for a healthy periodontal status than fixed appliances?” and focused on:
P—Population: patients undergoing orthodontic treatment (with no age or gender restriction) with fixed orthodontic appliances or clear aligners;
I—Intervention: orthodontic treatment with clear aligners (any type);
C—Comparison: orthodontic treatment with fixed (vestibular or lingual) appliances;
O—Outcome(s): periodontal health status as measured by clinical indices (excluding self-report).
2.2. Search Strategy
An electronic literature search for systematic reviews with meta-analysis, published in English, without date restriction, and related to the periodontal health status of patients undergoing orthodontic treatment was independently conducted by two independent reviewers (D.C. and F.D.A.), through February 2023, across the Cochrane Library, Web of Science (Core Collection), Scopus, EMBASE, and MEDLINE/PubMed databases. The search strategy was performed according to medical subject heading terms (Mesh), if any, and non-mesh terms; the mesh and non-mesh terms were also combined with Boolean operators, as shown in Table 1.
Table 1.
Search strategy.
The following filters were applied: “Review (English)” and “Topic” in the Web of Science database; “Review (English) in the Scopus database; “Systematic Review (English)” and “Meta-analysis (English)” in the MEDLINE/PubMed database; “Systematic Review (English)” and “Meta-analysis (English)” in the EMBASE database; “Review” and “Title, Abstract, Keyword” in the Cochrane Library.
Moreover, an exploration of the grey literature (unpublished studies) was performed in the OpenGrey database.
2.3. Study Selection and Eligibility Criteria
Collected citations were recorded, and duplicates were eliminated using the reference management tool EndNoteTM (Clarivate). Two reviewers (D.C. and F.D.A.) independently screened the remaining records and identified potentially relevant titles and abstracts eligible for further analysis.
The full texts of those records that met the eligibility criteria and the ambiguous title/abstracts were obtained without requiring contact with the study authors. The two authors (D.C. and F.D.A.) reviewed the full texts independently. The level of agreement between authors in the study selection process was assessed through the Cohen-weighted kappa (κ) coefficient, considering a minimum threshold value of 0.61 (substantial) [22]. The opinion of a third author (F.D.S.) was sought, if necessary, in case of disagreement.
The inclusion and exclusion criteria are reported in Table 2.
Table 2.
Inclusion and exclusion criteria.
2.4. Data Extraction and Collection
Data were extracted independently by two authors (D.C. and F.D.A.). A dedicated data extraction form was used which was developed before the start of the study and followed the models proposed for the intervention reviews of RCT and non-RCTs [23]; a third author (F.D.S.) was consulted when necessary. The Cohen kappa coefficient was used to assess the inter-examiner reliability in the data extraction and collection process [24].
For each systematic review included in the present review, the following data were collected:
- -
- The first author, year, journal, and funding;
- -
- The design and number of studies included in the qualitative syntheses of each review;
- -
- The number of studies included in the quantitative meta-analysis;
- -
- The population characteristics: sample size, age range, and gender ratio;
- -
- The intervention group data: treatment performed (type, characteristics, duration, and follow-up of the orthodontic treatment with clear aligners);
- -
- The control group data: orthodontic treatment performed (type, characteristics, duration, and follow-up of the orthodontic treatment with fixed appliances);
- -
- The statistically significant periodontal outcomes (clinical parameters, such as the plaque index “PI”, gingival index “GI”, periodontal probing depth “PPD”, gingival recession, and others; crevicular parameters; any other parameters to assess the periodontal health status or gingival biofilm accumulation reported in the systematic reviews with a meta-analysis included);
- -
- The conclusion(s) of the study.
2.5. Data Synthesis
A narrative synthesis was performed, focusing on the population studied, the intervention, and periodontal outcomes. Microsoft Excel 2016 software (Microsoft Corporation, Redmond, WA, USA) was used to qualitatively summarize the data from the included studies in descriptive statistical analysis and evaluate the clinical periodontal outcomes concerning the orthodontic appliances used (fixed vs. clear aligners).
2.6. Quality Assessment
Two authors (D.C. and F.D.A.) assessed the included reviews independently, using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) 2 tool composed of 16 items [25]. Any disagreement was initially resolved by discussion or in conjunction with a third author (S.M.) if necessary.
3. Results
3.1. Study Selection
The electronic search of all databases yielded 166 references. After duplicate elimination, 134 references remained. In reviewing the titles of the 134 entries, 121 were excluded because the subject was not relevant or because the type of the article was not a systematic review.
Abstracts of the remaining 13 articles were obtained, and 6 did not meet the eligibility criteria and were therefore excluded.
Full texts of the remaining seven articles were screened. Contacting the authors to obtain the full text or further information was unnecessary. Three articles were additionally excluded because they did not meet the inclusion criteria for this study. Specifically, two systematic reviews synthesized findings from studies comparing different types of orthodontic aligners [2,26] and one study did not provide a quantitative meta-analysis [27] (Table 3).
Table 3.
Studies excluded and the reasons for their exclusion.
Finally, four articles [28,29,30,31] from the electronic search were included in this umbrella review. No additional studies that were compatible with the eligibility criteria were found by screening grey literature or manually reviewing the reference lists of the included articles. The level of agreement between the examiners in the selection process (Cohen kappa coefficient) was 0.79 (substantial).
Figure 1 shows the flow diagram from the study selection, which included electronic searching databases and registers.
Figure 1.
PRISMA 2020 flow diagram for selecting new systematic reviews, including searches of databases and registers only.
3.2. Study Characteristics
None of the four systematic reviews with a meta-analysis [24,25,26,27] included in this umbrella review reported being externally funded [28,29,30,31].
Two systematic reviews [29,30] provided qualitative and quantitative data synthesis from all their included studies. The quantitative meta-analysis of the other two systematic reviews [28,31] included only a subset of the reviewed studies.
The total sample size was 2042 orthodontic patients, including 681 males and 850 females, aged 10 to 51 years, although gender and age were not reported for 551 and 464 subjects, respectively.
Participants were grouped as follows: 978 fell into the intervention group treated with clear aligners and 1064 comprised the control group treated with fixed vestibular or lingual appliances. Specifically, of the 978 subjects in the intervention group, 411 were treated with Invisalign®, 15 with PET-G (polyethylene terephthalate glycol) aligners, 20 with AirNivol aligners, and the remaining 532 participants with unspecified types of aligners. Of the 1064 subjects in the control group, 30 underwent orthodontic treatment with a fixed lingual appliance; specifically, 461 with a fixed buccal appliance; 35 with elastomeric ligated brackets; and 15 with self-ligating fixed appliances; no further information on fixed appliances was provided for the remaining 523 participants.
None of the included systematic reviews reported the total orthodontic treatment duration for clear aligners and fixed appliances. Follow-up for both the intervention and control groups ranged from 1 month to more than two years.
In all included reviews, clinical periodontal parameters were assessed. Specifically, the plaque index (PI), gingival index (GI), and periodontal probing depth (PPD) were measured in all four included studies [28,29,30,31]. One systematic review recorded the sulcus bleeding index (SBI) [29,31] and another systematic review reported gingival recessions [31]. Any other parameter (e.g., radiographic, crevicular) assessing periodontal health status was registered.
A subgroup analysis of data according to study type (RCTs and non-randomized studies) was performed only in one study [28], while a subgroup analysis based on follow-up time was performed in all four meta-analyses [28,29,30,31].
The inter-examiner reliability (Cohen kappa coefficient) in the data extraction and collection process was 0.87 (almost perfect).
The extracted data are reported in Table 4.
Table 4.
Data collected from the studies included in the present systematic review of systematic reviews: general information: First author, year, journal of publication, reference number, and funding; methods: study characteristics (number and design of studies included in the qualitative synthesis; number of studies included in the qualitative meta-analysis), participants (sample size, age, and gender), intervention (type, characteristics, duration, and follow-up), and comparison (type, characteristics, duration, and follow-up); periodontal outcomes; and conclusion(s).
3.3. Quality of the Included Systematic Reviews
According to the AMSTAR 2 checklist, the quality of the included reviews was variable: two [29,30] were classified as being of low-quality evidence, one [28] as being of moderate-quality evidence, and one [31] as being of high-quality evidence. Most of the AMSTAR 2 items were covered to varying degrees.
The two systematic reviews considered low-quality evidence [29,30] had the same critical deficiency: the authors had not provided a list of excluded studies and had not justified the reasons for exclusion. There was complete agreement among reviewers on the quality assessment.
The quality assessment and level of evidence of the systematic reviews included in the present study according to the AMSTAR 2 tool are shown in Table 5 and Table 6.
Table 5.
Methodological quality assessment based on the AMSTAR 2 items.
Table 6.
Level of evidence of systematic reviews with meta-analysis included according to the AMSTAR 2 tool.
3.4. Synthesis Results for Periodontal Clinical Parameters
The five periodontal clinical parameters recorded in the systematic reviews included the plaque index (PI), gingival index (GI), sulcus bleeding index (SBI), periodontal probing depth (PPD), and gingival recessions. No data concerning the clinical attachment level (CAL) were retrieved from the systematic reviews currently considered.
3.4.1. Plaque Index
All four systematic reviews included in the present umbrella review [28,29,30,31] examined the PI in groups treated with clear aligners and fixed orthodontic appliances.
A significant difference in the PI between subjects undergoing orthodontic treatment with clear aligners and fixed appliances was reported in the short-term (from baseline to 2–3 months) follow-up (low to moderate evidence), ranging from −0.35 to −0.69 [28,29,30], and the medium-term (from baseline to 6–9 months) follow-up (low to high evidence), ranging from −0.91 to −1.10 [29,30,31].
At the long-term follow-up (from baseline to 12 months or more), no difference in the PI was found between treatment with clear aligners and fixed appliances [28,31].
3.4.2. Gingival Index
All four meta-analyses included in this review assessed differences between the GI in participants undergoing orthodontic treatment with clear aligners and fixed appliances [28,29,30,31].
A significant difference in the GI between the intervention group (clear aligners) and the control group (fixed appliances) was reported at the short-term (from baseline to 2–3 months) follow-up (moderate evidence) (at 1 month: MD, −0.24, 95% CI, −0.35 to −0.12; at 3 months MD, −0.63, 95% CI, −1.22 to −0.04) [28] and in the medium-term follow-up (from baseline to 6–9 months) (low evidence) (MD, −0.14, 95% CI, −1.95 to −0.34) [30]. No difference was found in the long-term follow-up (from baseline to 12 months or longer) (high evidence) [31].
3.4.3. Periodontal Probing Depth
All systematic reviews included evaluated the PPD values in clear aligners vs. fixed orthodontic appliance groups [28,29,30,31].
Some findings, albeit of low evidence, pointed out better PPD values in orthodontic patients treated with clear aligners compared to those who underwent fixed orthodontic treatment in the short-term follow-up (from baseline to 2–3 months) (at 3 months: MD, −0.26, 95% CI, −0.52 to −0.01 [30] and in the medium-term (from baseline to 6–9 months) follow-up (at 6 months: MD, −0.42, 95% CI, −0.83 to −0.01) [30].
Evidence ranging from moderate to high supported the role of clear aligners in reducing the worsening of PPD values during orthodontic treatment in the long-term follow-up (from baseline to 12 months or more), with a mean difference between the intervention and control groups of −0.45 to −0.9 [28,31].
3.4.4. Sulcus Bleeding Index
One meta-analysis [26] included in this review compared the SBI values in subjects undergoing orthodontic treatment with clear aligners and fixed appliances and revealed that clear aligners, compared with fixed orthodontic appliances, had a significantly lower SBI status in the short-term (after 1 month: SMD, −0.44; 95% CI: −0.70 to −0.19; after 3 months: SMD, −0.49, 95% CI, −0.93 to −0.05) and medium-term (after 6 months: SMD, −0.91, 95% CI, −1.47 to −0.35) follow-ups.
3.4.5. Gingival Recession
Gingival recession was reported in only one of the systematic reviews presently considered [31], but no meta-analysis was performed because this outcome was recorded in only one study included in the systematic review.
In patients who underwent orthodontic treatment with fixed appliances, the gingival recession was statistically significantly higher at the 3-month follow-up (−0.85 mm ± 0.45) than at baseline (−0.67 mm ± 0.51). In contrast, no differences in the position of the gingival margin between the baseline and follow-up were observed in the group of subjects with clear aligners.
Table 7a–d shows a summary of the periodontal outcomes used to assess the periodontal health status of patients undergoing orthodontic treatment with clear aligners and fixed appliances, based on the timing of the follow-up examination.
Table 7.
(a) Synthesis results for the PI based on follow-up. (b) Synthesis results for the GI based on follow-up. (c) Synthesis results for the PPD based on follow-up. (d) Synthesis results for the SBI based on follow-up.
4. Discussion
Periodontal complications have been reported to be one of the most common side effects associated with orthodontic treatment [32].
In periodontally healthy orthodontic patients, gingivitis is the most critical periodontal complication and rarely progresses to periodontal disruption during orthodontic treatment [33].
However, periodontitis prevalence increases with age [17]. Consequently, as more adults have sought orthodontic treatment in recent decades, the number of periodontitis patients undergoing orthodontic treatment has increased significantly [34]. In these patients, the periodontal complications of orthodontic treatment may include additional attachment loss and the progression or recurrence of periodontitis [35]. Indeed, orthodontic forces may increase periodontal tissue and especially bone loss at periodontitis sites by further upregulating Interleukin (IL)-6, which is already elevated by bacterially induced periodontal inflammation [36].
Recent studies suggest that clear aligners may be the first treatment option in patients at risk for gingivitis or periodontitis, favoring the maintenance of better periodontal health conditions in patients at risk for gingivitis or periodontitis, especially adults [37,38,39,40].
Considering the importance of limiting periodontal complications during orthodontic treatment, the present umbrella review of systematic reviews and meta-analyses aimed to summarize the current evidence on the impact of clear aligners on periodontal health status compared to fixed appliances in patients undergoing orthodontic treatment.
A total of four studies [28,29,30,31] were included, with a sample of 2042 participants aged 10 to 51 years undergoing orthodontic treatment with clear aligners or fixed appliances. This study population is representative of the sociodemographic characteristics of the current population of orthodontic patients [41]. Indeed, currently, both young people and adults seek orthodontic treatment. Moreover, in all age groups of patients undergoing orthodontic treatment, there is a higher prevalence of female patients, which is consistent with the sample of the present study that has an M:F ratio of 1:1.3 [41].
None of the studies reported the total orthodontic treatment duration with clear aligners or fixed appliances. However, this does not seem relevant, as soft tissue inflammation can develop rapidly within the first few months of treatment, depending more on individual susceptibility than treatment duration [42].
Participant follow-up ranged from 1 month to more than 2 years, including the assessment of periodontal outcomes at a short-term follow-up (from baseline to 2–3 months), medium-term follow-up (from baseline to 6–9 months), and long-term follow-up (from baseline to 12 months or more). The latter approximates the end time of orthodontic treatment, as the mean duration is 19.9 months (MD, 19.9, 95% CI, 19.58 to 20.22 months).
4.1. Impact of Clear Aligners versus Fixed Appliances on Periodontal Conditions
Investigators in the included studies used five clinical indices to assess periodontal health status: the plaque index (PI), gingival index (GI), sulcus bleeding index (SBI), periodontal probing depth (PPD), and gingival recession.
The plaque index (PI) is a commonly used clinical index to assess oral hygiene status based on the accumulation of gingival biofilm around the teeth, gingiva, and gingival sulcus or periodontal pockets [43]. Gingival biofilm is a polymicrobial biofilm composed of diverse bacterial complexes that mutually benefit from coaggregation, adhesion, and metabolic interactions [44,45] and is the primary etiologic factor in the development, progression, and recurrences of periodontal inflammation [46,47,48]. The main biofilm-induced periodontal diseases are gingivitis, a non-destructive inflammation of the gingiva that is reversible by controlling the biofilm, and periodontitis, which instead leads to the irreversible loss of attachment, periodontal ligament, and alveolar bone [49,50,51] and is also dependent on the individual’s susceptibility and responsiveness to the inflammatory insult [52,53,54].
Orthodontic patients always have significantly worse PI scores, indicating a worse oral hygiene status compared to non-treated individuals [55]. In turn, there is a good consensus in the literature, confirmed by the present results, that clear aligners provide significantly better control of biofilm accumulation than fixed orthodontic appliances, especially during the first year of treatment. Indeed, differences in the PI between patients with clear aligners and fixed appliances were reported at short-term (from −0.35 to −0.69) and medium-term follow-ups (−0.91 to −1.10) (Table 7a). These findings may be explained by the evidence that orthodontic brackets, bands, and archwires promote biofilm accumulation, retain more plaque, and hinder its effective removal [46]. In addition, it may be speculated that after the first year of fixed orthodontic treatment, on average, patients with fixed appliances become more adept at oral hygiene. On the other hand, clear aligners cover at least most of the crown, prevent biofilm accumulation [27], and can be removed, allowing patients to perform their oral hygiene procedures under optimal conditions [27].
The clinical evaluation of gingival inflammation was recorded through the gingival index (GI) and sulcus bleeding index (SBI), measuring gingival edema and erythema and assessing the presence and severity of periodontal bleeding [43,56].
Because orthodontic fixed appliances generally reduce the effectiveness of biofilm control in patients, the associated risk of local gingivitis occurrence is expected to increase [46,57]. Furthermore, biofilm accumulation may favor subgingival periodontal pathogens and induce the release of periodontal proinflammatory cytokines, which in turn leads to periodontal tissue destruction [58,59]. Conversely, clear aligners, which prevent biofilm accumulation, may potentially limit the risk of inflammation and subsequent destruction of periodontal tissue during orthodontic treatment [27].
Accordingly, the included meta-analyses’ results reported evidence of the significantly better control of gingival inflammation among patients with removable rather than fixed appliances. Indeed, differences in the GI were reported in the short-term and medium-term follow-ups (at 1 month: MD, −0.24, 95% CI, −0.35 to −0.12; at 3 months MD, −0.63, 95% CI, −1.22 to −0.04; at 6 months: MD, −0.14, 95% CI, −1.95 to −0.34) (Table 7b). Similarly, differences in the SBI were found in the short-term (at 1 month: SMD, −0.44; 95% CI: −0.70 to −0.19; at 3 months: SMD, −0.49, 95% CI, −0.93 to −0.05) and mid-term (at 6 months: SMD, −0.91, 95% CI, −1.47 to −0.35) follow-ups (Table 7d).
Recent studies have found that the worsening of periodontal probing depth values (PPD) during orthodontic treatment is mainly due to the bacterial biofilm-induced inflammation of the gingiva, which can lead to gingival overgrowth and periodontal pseudopockets [60].
Retrieved findings supported that clear aligners are associated with better PPD values at short- and medium-term follow-ups (after 3 months: MD, −0.26, 95% CI, −0.52 to −0.01; after 6 months: MD, −0.42, 95% CI, −0.83 to −0.01) (Table 7c). In addition, moderate to high evidence supported the role of clear aligners in limiting the worsening of PPD values during orthodontic treatment at the long-term follow-up (from −0.45 to −0.93) (Table 7c) compared with fixed orthodontic appliances.
Some authors suggested that orthodontic treatment could be associated with gingival recession and loss of clinical attachment level [61]. Crego-Ruiz et al. [31] reported a significant increase in gingival recession at the 3-month follow-up in patients with fixed orthodontic appliances but not in subjects treated with clear aligners. However, no data concerning the clinical attachment level (CAL) were retrieved from the systematic reviews currently considered.
4.2. Clinical Considerations and Implications
Almost all periodontal indices showed a slight tendency for clear aligners to be associated with healthier periodontal conditions by limiting gingival biofilm accumulation and periodontal inflammation. However, some clinical considerations should be made.
Although statistically significant, the difference in PPD between subjects with clear aligners and patients undergoing fixed orthodontic treatment was less than one millimeter (from −0.45 to −0.93) (Table 7c), which would be practically negligible in a clinical setting [62,63,64].
In addition, when considering the clinical relevance of the differences in the PI, GI, and SBI between patients with clear aligners and patients with fixed appliances, it is important to remember that these indices are ordinal variables. Therefore, if the mean difference between subjects with clear aligners and those with fixed appliances is between 0 and 1, both approaches would likely receive the same index value because the difference is less than a whole point. Consequently, the differences in periodontal indices between the two treatment groups should be considered clinically significant only if they are greater than 1. Accordingly, only the difference in the PI score reported during the medium-term follow-up can assume clinical relevance.
Regarding gingival recessions, although clear aligners seemed to be associated with a more stable gingival margin position, this outcome was recorded in a short-term follow-up and only one study, and thus, it needs to be confirmed by further studies with a long-term follow-up.
Based on the above, the current evidence remains insufficient to determine whether clear aligners are associated with healthier periodontal conditions. In addition, no evidence suggested that clear aligners should be a first-line treatment option in patients, especially adult ones, at risk for gingivitis or periodontitis.
Accordingly, rather than the choice of treatment modality, the establishment of an appropriate periodontal surveillance and health promotion program for the adequate control of periodontal biofilm and inflammation should be considered as an effective preventative measure for periodontal complications during orthodontic treatment [10].
Furthermore, since uncontrolled periodontal inflammation during orthodontic treatment is known to promote the progression of periodontitis and tissue destruction [65,66,67], a comprehensive diagnosis that takes into account not only the patient’s orthodontic problems but also their periodontal needs to achieve and maintain periodontal health is strongly recommended before starting orthodontic treatment [68].
When interpreting the clinical implications of the results, some limitations of the present systematic review of systematic reviews should be considered.
First, several studies were included in more than one of the systematic reviews currently included. Second, high heterogeneity was found in almost all the meta-analyses performed in the included systematic reviews. This could be due to differences in patients’ periodontal self-care and home care instructions given by physicians. In addition, heterogeneous data, especially regarding the timing of follow-up, and lack of data on the characteristics and duration of orthodontic treatment precluded the possibility of performing a meta-analysis.
Further studies investigating the clinical relevance of the differences between the impact of clear aligners and fixed appliances on periodontal health status are needed for definitive conclusions.
5. Conclusions
The present study included four systematic reviews examining the impact of clear aligners compared with fixed appliances on the periodontal health status of patients undergoing orthodontic treatment.
Clear aligners provided a significantly better control of biofilm accumulation than fixed orthodontic appliances, especially during the first year of treatment (PI: MD from −0.35 to −1.10); however, no differences were found during the long-term follow-up. Similarly, the gingival inflammatory status was significantly better controlled in patients with removable rather than fixed appliances at short- and medium-term follow-ups (GI at 1 month: MD, −0.24, 95% CI, −0.35 to −0.12; at 3 months MD, −0.63, 95% CI, −1.22 to −0.04; at 6 months: MD, −0.14, 95% CI, −1.95 to −0.34). In addition, there was moderate to strong evidence that clear aligners limited the worsening of PPD values during orthodontic treatment at the long-term follow-up (from −0.45 to −0.93).
However, such differences in periodontal outcomes between subjects with clear aligners and fixed appliances were statistically significant, but practically negligible in the clinical context. Indeed, differences in the GI and SBI (ordinal variables) were less than 1, and differences in the PPD were less than 1 mm (hardly measurable).
Regarding gingival recession, there is not enough evidence supporting that clear aligners might increase or decrease gingival recessions compared to fixed appliances.
Given the current state of knowledge, the impact of orthodontic treatment with clear aligners and fixed appliances on periodontal health should be considered comparable, and there is no evidence to support the choice of clear aligners as the first treatment option in patients at risk for gingivitis or periodontitis.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare11091340/s1, PRISMA 2020 Checklist.
Author Contributions
Conceptualization, F.D.S. and S.M.; methodology, F.D.S. and S.M.; validation, V.D., F.D. and F.G.; investigation, D.C., F.D.S. and S.M.; data curation, D.C., F.D. and V.D.; writing—original draft preparation, D.C., F.D. and F.G.; writing—review and editing, F.D.S. and V.D.; supervision, S.M. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Data supporting the reported results can be found in the Cochrane Library, Web of Science (Core Collection), Scopus, and MEDLINE/PubMed databases.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Lang, N.P.; Bartold, P.M. Periodontal Health. J. Periodontol. 2018, 89, S9–S16. [Google Scholar] [CrossRef]
- Rossini, G.; Parrini, S.; Castroflorio, T.; Deregibus, A.; Debernardi, C.L. Periodontal Health during Clear Aligners Treatment: A Systematic Review. Eur. J. Orthod. 2015, 37, 539–543. [Google Scholar] [CrossRef] [PubMed]
- Talic, N.F. Adverse Effects of Orthodontic Treatment: A Clinical Perspective. Saudi Dent. J. 2011, 23, 55–59. [Google Scholar] [CrossRef]
- Giuca, M.R.; Pasini, M.; Drago, S.; del Corso, L.; Vanni, A.; Carli, E.; Manni, A. Influence of Vertical Facial Growth Pattern on Herbst Appliance Effects in Prepubertal Patients: A Retrospective Controlled Study. Int. J. Dent. 2020, 2020, 1–8. [Google Scholar] [CrossRef]
- Martina, S.; Martini, M.; Bordegoni, M.; Razionale, A.V. Predictability of Root Movements Using Virtual Root Setup in a Patient With Periodontal Disease Treated With Clear Aligners. Open Dent. J. 2021, 15, 605–611. [Google Scholar] [CrossRef]
- Heintze, S.D.; Shalini, G.; Varsha, S.A.; Abhishek, D.; Neetu, J. Oral Health for the Orthodontic Patient; Quintessence: Berlin, Germany, 1999. [Google Scholar]
- Megha, S.; Shalini, G.; Varsha, S.A.; Abhishek, D.; Neetu, J. Effect of Short-Term Placebo-Controlled Consumption of Probiotic Yoghurt and Indian Curd on the Streptococcus Mutans Level in Children Undergoing Fixed Interceptive Orthodontic Therapy. Turk. J. Orthod. 2019, 32, 16–21. [Google Scholar] [CrossRef]
- Carli, E.; Pasini, M.; Lardani, L.; Giuca, G.; Miceli, M. Impact of Self-Ligating Orthodontic Brackets on Dental Biofilm and Periodontal Pathogens in Adolescents. J. Biol. Regul. Homeost. Agents 2021, 35, 107–115. [Google Scholar] [CrossRef]
- Karkhanechi, M.; Chow, D.; Sipkin, J.; Sherman, D.; Boylan, R.J.; Norman, R.G.; Craig, R.G.; Cisneros, G.J. Periodontal Status of Adult Patients Treated with Fixed Buccal Appliances and Removable Aligners over One Year of Active Orthodontic Therapy. Angle Orthod. 2013, 83, 146–151. [Google Scholar] [CrossRef] [PubMed]
- Di Spirito, F.; Amato, A.; Di Palo, M.P.; Cannatà, D.; Giordano, F.; D’Ambrosio, F.; Martina, S. Periodontal Management in Periodontally Healthy Orthodontic Patients with Fixed Appliances: An Umbrella Review of Self-Care Instructions and Evidence-Based Recommendations. Dent. J. 2023, 11, 35. [Google Scholar] [CrossRef]
- Martina, S.; Rongo, R.; Bucci, R.; Razionale, A.V.; Valletta, R.; D’Antò, V. In Vitro Cytotoxicity of Different Thermoplastic Materials for Clear Aligners. Angle Orthod. 2019, 89, 942–945. [Google Scholar] [CrossRef] [PubMed]
- Robertson, L.; Kaur, H.; Fagundes, N.C.F.; Romanyk, D.; Major, P.; Flores Mir, C. Effectiveness of Clear Aligner Therapy for Orthodontic Treatment: A Systematic Review. Orthod. Craniofac. Res. 2020, 23, 133–142. [Google Scholar] [CrossRef] [PubMed]
- Lagravère, M.O.; Flores-Mir, C. The Treatment Effects of Invisalign Orthodontic Aligners: A Systematic Review. J. Am. Dent. Assoc. 2005, 136, 1724–1729. [Google Scholar] [CrossRef] [PubMed]
- Buschang, P.H.; Shaw, S.G.; Ross, M.; Crosby, D.; Campbell, P.M. Comparative Time Efficiency of Aligner Therapy and Conventional Edgewise Braces. Angle Orthod. 2014, 84, 391–396. [Google Scholar] [CrossRef]
- Ke, Y.; Zhu, Y.; Zhu, M. A Comparison of Treatment Effectiveness between Clear Aligner and Fixed Appliance Therapies. BMC Oral Health 2019, 19, 24. [Google Scholar] [CrossRef]
- D’Antò, V.; Valletta, R.; De Simone, V.; Pisano, M.; Martina, S. Clear Aligners Treatment of Class III Subdivision with an Extraction of a Lower Bicuspid. Int. J. Environ. Res. Public Health 2023, 20, 3550. [Google Scholar] [CrossRef] [PubMed]
- Billings, M.; Holtfreter, B.; Papapanou, P.N.; Mitnik, G.L.; Kocher, T.; Dye, B.A. Age-Dependent Distribution of Periodontitis in Two Countries: Findings from NHANES 2009 to 2014 and SHIP-TREND 2008 to 2012. J. Clin. Periodontol. 2018, 45, S130–S148. [Google Scholar] [CrossRef]
- Shokeen, B.; Viloria, E.; Duong, E.; Rizvi, M.; Murillo, G.; Mullen, J.; Shi, B.; Dinis, M.; Li, H.; Tran, N.C.; et al. The Impact of Fixed Orthodontic Appliances and Clear Aligners on the Oral Microbiome and the Association with Clinical Parameters: A Longitudinal Comparative Study. Am. J. Orthod. Dentofac. Orthop. 2022, 161, e475–e485. [Google Scholar] [CrossRef]
- Moher, D.; Shamseer, L.; Clarke, M.; Ghersi, D.; Liberati, A.; Petticrew, M.; Shekelle, P.; Stewart, L.A.; PRISMA-P Group. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 Statement. Syst. Rev. 2015, 4, 1. [Google Scholar] [CrossRef]
- Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gøtzsche, P.C.; Ioannidis, J.P.A.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med. 2009, 6, e1000100. [Google Scholar] [CrossRef]
- Richardson, W.S.; Wilson, M.C.; Nishikawa, J.; Hayward, R.S. The Well-Built Clinical Question: A Key to Evidence-Based Decisions. ACP J. Club 1995, 123, A12–A13. [Google Scholar] [CrossRef]
- Landis, J.R.; Koch, G.G. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef] [PubMed]
- The Cochrane Collaboration. Data Extraction Forms. Data Collection Form for Intervention Reviews for RCTs and Non-RCTs-Template. [Online] Cochrane Developmental, Psychosocial and Learning Problems, 3 April 2014. 2014. Available online: https://training.cochrane.org (accessed on 20 March 2023).
- Viera, A.J.; Garrett, J.M. Understanding Interobserver Agreement: The Kappa Statistic. Fam. Med. 2005, 37, 360–363. [Google Scholar] [PubMed]
- Shea, B.J.; Reeves, B.C.; Wells, G.; Thuku, M.; Hamel, C.; Moran, J.; Moher, D.; Tugwell, P.; Welch, V.; Kristjansson, E.; et al. AMSTAR 2: A Critical Appraisal Tool for Systematic Reviews That Include Randomised or Non-Randomised Studies of Healthcare Interventions, or Both. BMJ 2017, 358, j4008. [Google Scholar] [CrossRef]
- Oikonomou, E.; Foros, P.; Tagkli, A.; Rahiotis, C.; Eliades, T.; Koletsi, D. Impact of Aligners and Fixed Appliances on Oral Health during Orthodontic Treatment: A Systematic Review and Meta-Analysis. Oral Health Prev. Dent. 2021, 19, 659–672. [Google Scholar] [CrossRef]
- Partouche, A.J.D.; Castro, F.; Baptista, A.S.; Costa, L.G.; Fernandes, J.C.H.; de Fernandes, G.V.O. Effects of Multibracket Orthodontic Treatment versus Clear Aligners on Periodontal Health: An Integrative Review. Dent. J. 2022, 10, 177. [Google Scholar] [CrossRef]
- Jiang, Q.; Li, J.; Mei, L.; Du, J.; Levrini, L.; Abbate, G.M.; Li, H. Periodontal Health during Orthodontic Treatment with Clear Aligners and Fixed Appliances: A Meta-Analysis. J. Am. Dent. Assoc. 2018, 149, 712–720.e12. [Google Scholar] [CrossRef] [PubMed]
- Lu, H.; Tang, H.; Zhou, T.; Kang, N. Assessment of the Periodontal Health Status in Patients Undergoing Orthodontic Treatment with Fixed Appliances and Invisalign System: A Meta-Analysis. Medicine 2018, 97, e0248. [Google Scholar] [CrossRef] [PubMed]
- Wu, Y.; Cao, L.; Cong, J. The Periodontal Status of Removable Appliances vs Fixed Appliances: A Comparative Meta-Analysis. Medicine 2020, 99, e23165. [Google Scholar] [CrossRef]
- Crego-Ruiz, M.; Jorba-García, A. Assessment of the Periodontal Health Status and Gingival Recession during Orthodontic Treatment with Clear Aligners and Fixed Appliances: A Systematic Review and Meta-Analysis. Med. Oral Patol. Oral Cir. Bucal 2023, 25760. [Google Scholar] [CrossRef]
- Alfuriji, S.; Alhazmi, N.; Alhamlan, N.; Al-Ehaideb, A.; Alruwaithi, M.; Alkatheeri, N.; Geevarghese, A. The Effect of Orthodontic Therapy on Periodontal Health: A Review of the Literature. Int. J. Dent. 2014, 2014, 1–8. [Google Scholar] [CrossRef]
- Willmot, D. Orthodontic Treatment and the Compromised Periodontal Patient. Eur. J. Dent. 2008, 2, 1–2. [Google Scholar] [CrossRef]
- Confer, J.C.; Easton, J.A.; Fleischman, D.S.; Goetz, C.D.; Lewis, D.M.G.; Perilloux, C.; Buss, D.M. Evolutionary Psychology. Controversies, Questions, Prospects, and Limitations. Am. Psychol. 2010, 65, 110–126. [Google Scholar] [CrossRef] [PubMed]
- Rego, R.O.; Oliveira, C.A.; dos Santos-Pinto, A.; Jordan, S.F.; Zambon, J.J.; Cirelli, J.A.; Haraszthy, V.I. Clinical and Microbiological Studies of Children and Adolescents Receiving Orthodontic Treatment. Am. J. Dent. 2010, 23, 317–323. [Google Scholar]
- Rath-Deschner, B.; Nogueira, A.V.B.; Beisel-Memmert, S.; Nokhbehsaim, M.; Eick, S.; Cirelli, J.A.; Deschner, J.; Jäger, A.; Damanaki, A. Interaction of Periodontitis and Orthodontic Tooth Movement—An in Vitro and in Vivo Study. Clin. Oral Investig. 2022, 26, 171–181. [Google Scholar] [CrossRef] [PubMed]
- Miethke, R.-R.; Vogt, S. A Comparison of the Periodontal Health of Patients during Treatment with the Invisalign® System and with Fixed Orthodontic Appliances. J. Orofac. Orthop./Fortschr. Der Kieferorthopädie 2005, 66, 219–229. [Google Scholar] [CrossRef]
- Azaripour, A.; Weusmann, J.; Mahmoodi, B.; Peppas, D.; Gerhold-Ay, A.; van Noorden, C.J.F.; Willershausen, B. Braces versus Invisalign®: Gingival Parameters and Patients’ Satisfaction during Treatment: A Cross-Sectional Study. BMC Oral Health 2015, 15, 69. [Google Scholar] [CrossRef] [PubMed]
- Levrini, L.; Mangano, A.; Montanari, P.; Margherini, S.; Caprioglio, A.; Abbate, G.M. Periodontal Health Status in Patients Treated with the Invisalign® System and Fixed Orthodontic Appliances: A 3 Months Clinical and Microbiological Evaluation. Eur. J. Dent. 2015, 09, 404–410. [Google Scholar] [CrossRef] [PubMed]
- Abbate, G.M.; Caria, M.P.; Montanari, P.; Mannu, C.; Orrù, G.; Caprioglio, A.; Levrini, L. Periodontal Health in Teenagers Treated with Removable Aligners and Fixed Orthodontic Appliances. J. Orofac. Orthop./Fortschr. Der Kieferorthopädie 2015, 76, 240–250. [Google Scholar] [CrossRef] [PubMed]
- Proffit, W.R.; Fields, H.W.; Sarver, D.M. Ortodonzia Moderna, 4th ed.; Masson, E., Ed.; Elsevier: Milan, Italy, 2013. [Google Scholar]
- Ren, Y.; Jongsma, M.A.; Mei, L.; van der Mei, H.C.; Busscher, H.J. Orthodontic Treatment with Fixed Appliances and Biofilm Formation—A Potential Public Health Threat? Clin. Oral Investig. 2014, 18, 1711–1718. [Google Scholar] [CrossRef]
- Löe, H. The Gingival Index, the Plaque Index and the Retention Index Systems. J. Periodontol. 1967, 38, 610–616. [Google Scholar] [CrossRef]
- Kwon, T.; Lamster, I.B.; Levin, L. Current Concepts in the Management of Periodontitis. Int. Dent. J. 2020, 71, 462–476. [Google Scholar] [CrossRef] [PubMed]
- Amato, M.; Di Spirito, F.; D’Ambrosio, F.; Boccia, G.; Moccia, G.; De Caro, F. Probiotics in Periodontal and Peri-Implant Health Management: Biofilm Control, Dysbiosis Reversal, and Host Modulation. Microorganisms 2022, 10, 2289. [Google Scholar] [CrossRef]
- Genco, R.J.; Borgnakke, W.S. Risk Factors for Periodontal Disease. Periodontology 2000 2013, 62, 59–94. [Google Scholar] [CrossRef]
- Amato, A. Periodontitis and Cancer: Beyond the Boundaries of Oral Cavity. Cancers 2023, 15, 1736. [Google Scholar] [CrossRef]
- Ramaglia, L.; Di Spirito, F.; Sirignano, M.; La Rocca, M.; Esposito, U.; Sbordone, L. A 5-year Longitudinal Cohort Study on Crown to Implant Ratio Effect on Marginal Bone Level in Single Implants. Clin. Implant Dent. Relat. Res. 2019, 21, 916–922. [Google Scholar] [CrossRef]
- Ballini, A.; Cantore, S.; Farronato, D.; Cirulli, N.; Inchingolo, F.; Papa, F.; Malcangi, G.; Inchingolo, A.D.; Dipalma, G.; Sardaro, N.; et al. Periodontal Disease and Bone Pathogenesis: The Crosstalk between Cytokines and Porphyromonas Gingivalis. J. Biol. Regul. Homeost. Agents 2015, 29, 273–281. [Google Scholar] [PubMed]
- Amato, A. Oral-Systemic Health and Disorders: Latest Advances on Oral–Gut–Lung Microbiome Axis. Appl. Sci. 2022, 12, 8213. [Google Scholar] [CrossRef]
- Di Spirito, F. Oral-Systemic Health and Disorders: Latest Prospects on Oral Antisepsis. Appl. Sci. 2022, 12, 8185. [Google Scholar] [CrossRef]
- Murakami, S.; Mealey, B.L.; Mariotti, A.; Chapple, I.L.C. Dental Plaque-Induced Gingival Conditions. J. Periodontol. 2018, 89, S17–S27. [Google Scholar] [CrossRef] [PubMed]
- Di Spirito, F.; La Rocca, M.; De Bernardo, M.; Rosa, N.; Sbordone, C.; Sbordone, L. Possible Association of Periodontal Disease and Macular Degeneration: A Case-Control Study. Dent. J. 2020, 9, 1. [Google Scholar] [CrossRef] [PubMed]
- Di Spirito, F.; Sbordone, L.; Pilone, V.; D’Ambrosio, F. Obesity and Periodontal Disease: A Narrative Review on Current Evidence and Putative Molecular Links. Open Dent. J. 2019, 13, 526–536. [Google Scholar] [CrossRef]
- Contaldo, M.; Lucchese, A.; Lajolo, C.; Rupe, C.; Di Stasio, D.; Romano, A.; Petruzzi, M.; Serpico, R. The Oral Microbiota Changes in Orthodontic Patients and Effects on Oral Health: An Overview. J. Clin. Med. 2021, 10, 780. [Google Scholar] [CrossRef]
- Mühlemann, H.R.; Son, S. Gingival Sulcus Bleeding--a Leading Symptom in Initial Gingivitis. Helv. Odontol. Acta 1971, 15, 107–113. [Google Scholar] [PubMed]
- Di Spirito, F.; Pelella, S.; Argentino, S.; Sisalli, L.; Sbordone, L. Oral Manifestations and the Role of the Oral Healthcare Workers in COVID-19. Oral Dis. 2022, 28, 1003–1004. [Google Scholar] [CrossRef] [PubMed]
- Ren, Y.; Vissink, A. Cytokines in Crevicular Fluid and Orthodontic Tooth Movement. Eur. J. Oral Sci. 2008, 116, 89–97. [Google Scholar] [CrossRef] [PubMed]
- Di Spirito, F.; Argentino, S.; Martuscelli, R.; Sbordone, L. Mronj Incidence after Multiple Teeth Extractions in Patients Taking Oral Bis-Phosphonates without “Drug Holiday”: A Retrospective Chart Review. Oral Implantol. 2019, 12, 105–110. Available online: https://www.oimplantology.org/oimp/article/view/282 (accessed on 20 March 2023).
- Vincent-Bugnas, S.; Borsa, L.; Gruss, A.; Lupi, L. Prioritization of Predisposing Factors of Gingival Hyperplasia during Orthodontic Treatment: The Role of Amount of Biofilm. BMC Oral Health 2021, 21, 84. [Google Scholar] [CrossRef]
- Bollen, A.-M.; Cunha-Cruz, J.; Bakko, D.W.; Huang, G.J.; Hujoel, P.P. The Effects of Orthodontic Therapy on Periodontal Health: A Systematic Review of Controlled Evidence. J. Am. Dent. Assoc. 2008, 139, 413–422. [Google Scholar] [CrossRef]
- Mombelli, A.; Graf, H. Depth-Force-Patterns in Periodontal Probing. J. Clin. Periodontol. 1986, 13, 126–130. [Google Scholar] [CrossRef]
- Watts, T. Constant Force Probing with and without a Stent in Untreated Periodontal Disease: The Clinical Reproducibility Problem and Possible Sources of Error. J. Clin. Periodontol. 1987, 14, 407–411. [Google Scholar] [CrossRef]
- Walsh, T.F.; Saxby, M.S. Inter-and Intra-Examiner Variability Using Standard and Constant Force Periodontal Probes. J. Clin. Periodontol. 1989, 16, 140–143. [Google Scholar] [CrossRef]
- Wennström, J.L.; Stokland, B.L.; Nyman, S.; Thilander, B. Periodontal Tissue Response to Orthodontic Movement of Teeth with Infrabony Pockets. Am. J. Orthod. Dentofacial. Orthop. 1993, 103, 313–319. [Google Scholar] [CrossRef] [PubMed]
- D’Ambrosio, F.; Caggiano, M.; Schiavo, L.; Savarese, G.; Carpinelli, L.; Amato, A.; Iandolo, A. Chronic Stress and Depression in Periodontitis and Peri-Implantitis: A Narrative Review on Neurobiological, Neurobehavioral and Immune-Microbiome Interplays and Clinical Management Implications. Dent. J. 2022, 10, 49. [Google Scholar] [CrossRef] [PubMed]
- Caggiano, M.; Amato, A.; Acerra, A.; D’Ambrosio, F.; Martina, S. Evaluation of Deviations between Computer-Planned Implant Position and In Vivo Placement through 3D-Printed Guide: A CBCT Scan Analysis on Implant Inserted in Esthetic Area. Appl. Sci. 2022, 12, 5461. [Google Scholar] [CrossRef]
- Martin, C.; Celis, B.; Ambrosio, N.; Bollain, J.; Antonoglou, G.N.; Figuero, E. Effect of Orthodontic Therapy in Periodontitis and Non-Periodontitis Patients: A Systematic Review with Meta-Analysis. J. Clin. Periodontol. 2022, 49 (Suppl. 24), 72–101. [Google Scholar] [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).