Online Audio-Visual Information on the Treatment of OSA with Mandibular Advancement Devices: Analysis of Quality, Reliability and Contents

: The Internet is a preferred source of health information. This study evaluated the quality, reliability and content of online audio-visual information on the mandibular advancement device (MAD) for the treatment of obstructive sleep apnea (OSA). “Mandibular advancement device” was searched on four online platforms. A total of 63 videos (51% from healthcare professionals, 22% from commercial companies, 21% from laypeople, and 6% from hospitals/universities) were evaluated using metrics, the video information and quality index (VIQI), modiﬁed-DISCERN, and the Journal of American Medical Association (JAMA) benchmark criteria and contents. VIQI had signiﬁcantly lower scores for videos uploaded by laypeople (mean 7.92, 95%CI 5.90–9.95) versus healthcare professionals (12.38, 95%CI 11.28–13.47) and commercial companies (11.21, 95%CI 9.61–12.81). The m-DISCERN scores were signiﬁcantly lower for laypeople (1.15, 95%CI 0.93–1.40) versus healthcare professionals (2.13, 95%CI 1.73–2.52) and hospitals/universities (3.00, 95%CI 1.70–4.30), as well as for commercial companies (1.43, 95%CI 1.13-1.73) versus hospitals/universities. Contents were signiﬁcantly less complete for laypeople (1.54, 95%CI 0.60–2.48) versus healthcare professionals (3.25, 95%CI 2.66–3.84). The results of the present study suggest that the quality and completeness of online audio-visual information with respect to the use of MAD for the treatment of OSA are generally poor, and the currently available videos, especially those uploaded by laypeople, may determine misinformation and/or unrealistic treatment expectations.


Introduction
Obstructive sleep apnea (OSA) is a sleep-related breathing disorder characterized by repetitive episodes of reduced airflow during sleep due to a complete or incomplete obstruction in the upper airways [1].It is an increasingly prevalent condition that greatly impacts public health, affecting roughly one billion people between the ages of 30 and 69 years all over the world [2,3].The gold-standard treatment for OSA is continuous positive airway pressure (CPAP), but in recent years, the mandibular advancement device (MAD) has become an increasingly common treatment [4][5][6].MAD is recommended for people affected by mild to moderate OSA and for those with primary snoring.MAD is also accepted as an alternative therapy for patients with severe OSA not tolerating or not responding to CPAP [5].Currently, the number of people affected by OSA who are unaware of that condition and, consequently, do not access treatment is still high.This mismanagement leads to multiple adverse health outcomes, including decreased quality of life, increased cardiovascular morbidity and mortality, and increased risk of motor vehicle and work accidents.Promoting accurate information among people about OSA and related treatment options is a fundamental step to facing this potentially lifethreatening disease [7].Nowadays, the Internet is considered an ideal communication tool among laypeople seeking health-related information due to easy and fast access.One of the most frequently used search engines is YouTube, which was created in 2005 and counted more than 2.5 billion users performing billions of daily views in 2022 [8][9][10][11].The strength of YouTube videos lies in the combined use of audio-visual communication, which makes it quickly accessible to individuals from all demographic backgrounds.However, as the educational content of Internet videos is not subjected to any formal peer review process, and everybody can upload any kind of video clip, it is imperative to investigate the completeness and trustworthiness of Internet videos as a source of health-related information.The quality of YouTube videos for different types of medical and dental information has been investigated previously, with the majority of studies concluding that videos contain scientifically inaccurate and often misleading health-related information, which might cause confusion among patients, negatively influencing their expectations and adherence to treatment [8,[12][13][14][15][16][17][18].It has already been demonstrated that the online written information provided by websites does not fulfil the most important aspects of MAD therapy and may be difficult to understand by laypeople [19].However, no study has so far analyzed the role of online videos.The aim of this study was, therefore, to critically evaluate the quality, reliability and educational content of online audio-visual information regarding MAD.

Video Search Strategy
This study is a cross-sectional evaluation of online audio-visual information.On 30 November 2022 YouTube, Google Video, Microsoft Bing and Watch Facebook were searched using the keyword "mandibular advancement device".Each browser, with its cache cleared, was searched for the first 50 videos sorted "by relevance" in accordance with previous studies referencing that more than 90% of YouTube users scan only the first 3 pages of search results [20].All 200 videos were screened and categorized simultaneously by two researchers experienced in dental sleep medicine (E.B. and A.F.).Any discrepancies were discussed with a third researcher (S.I.P.) until a consensus was reached.The following exclusion criteria were applied: (1) language not in English, (2) video content not related to the topic, (3) poor audio-visual quality, (4) duplicates, (5) duration shorter than 2 min, assuming a video shorter than that cannot provide any useful information, and (6) longer than 15 min, assuming users might not have the time or patience to watch a video for more than that length of time, in agreement with previous studies [17,21,22].

Video Metrics Analysis
The first analysis involved video metrics, in-detail source locators (URLs), video title, duration, upload date, number of views, number of likes and dislikes, and number of comments.Based on these metrics, the Video Power Index (VPI) and Interaction index (II), which assess video popularity, were calculated [21,23]: VPI = [(view ratio × like ratio)/100]; where view ratio = views per day, and like ratio = [(likes × 100)/(likes + dislikes)]; II (%) = [(number of likes − number of dislikes)/(total number of views)] × 100.

Video Sources and Target Audience
All videos were categorized by the source of upload into 4 groups: healthcare professionals, hospitals/universities, commercial companies (dental manufacturing company or dental supply company) and laypeople.The target audience was assessed according to the language and the detailing of scientific information in the videos: when the video included clinical applications and/or scientific explanations and utilized a medical language, the target audience was considered dental professionals and dental students.The other remaining videos that explained MADs comprehensibly in plain terminology were intended for laypeople.

Assessment of Video Quality and Reliability
The video information and quality index (VIQI) was used to analyze the overall educational quality and content of videos [16][17][18]21,24].The VIQI scale, consisting of the contents of the Global Quality Scale (GQS), uses a 5-point Likert scale ranging from 1 (poor quality, poor flow, most information missing, not useful) to 5 (high quality, excellent flow, very useful) to evaluate the 4 following video features: the flow of information, information accuracy, quality (1 point each for the presence of still images, animations, interviews with individuals in the community, video captions, and a report summary), and precision (level of coherence between the video title and content).The total score ranges from 4 to 20 points [21,25,26].Reliability was assessed using a modified 5-point DISCERN tool [27] and the Journal of American Medical Association (JAMA) benchmark criteria [28,29].The DISCERN system was created by a specialized group at Oxford University (UK) as a tool to assess the quality of written consumer health information on treatment choices.It includes 16 questions ranging from 1 to 5, and the total score is between 16 and 80 points [30].In this study, we used the short form of the DISCERN that was adapted by Singh et al.The modified-DISCERN (m-DISCERN) questioning tool consists of 5 items about clarity, reliability of the source of information, lack of bias, reference supplementation and mention of uncertainty.According to m-DISCERN, video scores > 3 points indicate good reliability, a score of 3 points shows moderate reliability, and scores < 3 points refer to poor reliability [27].The JAMA criteria evaluate the reliability of online health information, assessing authorship, attribution, disclosure and currency of the video with the total score ranging from 0 to 4 [28,29].

Video Content Analysis
As there was no available checklist evaluating the quality of online video content on MADs, a customized MAD scoring scheme was a priori developed specifically for this study (Figure 1).The aim of that checklist was to assess videos on the presence of content domains and comprehensiveness.The specific criteria included: (1) a definition of MAD, (2) indications about the use of MAD, (3) contraindications about the use of MAD, (4) instructions on the usage of MAD, (5) adverse effects of MAD, (6) comparison of treatment alternatives (CPAP, surgery, etc.), (7) the need to consult a dentist qualified in dental sleep medicine, (8) the diagnostic and therapeutic process of OSA described as a multidisciplinary care team endeavor, and (9) the cost and duration of therapy with MAD.The presence of any individual component was equated to the score of 1, whereas its absence was equated to the score of 0. Based on the presence or absence of information, a video could reach a score between 0 and 9.We categorized the videos into three levels of usefulness based on the scores: 'useful' (3+), 'moderately useful' (1-2) and 'not useful' (0).

Statistical Analysis
50 randomly selected videos were used to investigate reliability for the quality and content analysis scores using the intraclass correlation coefficient.The same examiner assigned the scores twice over 2 weeks for intra-examiner reliability, while 2 different examiners independently evaluated the same 50 videos for inter-examiner reliability.

Figure 1.
Checklist for content analysis of videos on MAD, mandibular advancement device.Scores were attributed as follows for each criterion: 0 points, not mentioned; 1 point, mentioned; CPAP, continuous positive airway pressure; TMDs, temporomandibular disorders.

Statistical Analysis
50 randomly selected videos were used to investigate reliability for the quality and content analysis scores using the intraclass correlation coefficient.The same examiner assigned the scores twice over 2 weeks for intra-examiner reliability, while 2 different examiners independently evaluated the same 50 videos for inter-examiner reliability.
The normality of data was assessed with the Kolmogorov-Smirnov test.The differences between the groups for content, quality and reliability of the information were analyzed using a Kruskal-Wallis H test and a Bonferroni-corrected Mann-Whitney U post hoc test.The normality of data was assessed with the Kolmogorov-Smirnov test.The differences between the groups for content, quality and reliability of the information were analyzed using a Kruskal-Wallis H test and a Bonferroni-corrected Mann-Whitney U post hoc test.
Statistical analyses were performed using the statistical software SPSS for Windows (version 18.0; 2009; SPSS Inc.).The limit for statistical significance was set at p < 0.05.

Results
A total number of 200 videos were initially screened; 63 videos were finally included in the study after applying the eligibility criteria (Figure 2).
Appl.Sci.2023, 13, x FOR PEER REVIEW Statistical analyses were performed using the statistical software SPSS for Wi (version 18.0; 2009; SPSS Inc.).The limit for statistical significance was set at p < 0.0

Results
A total number of 200 videos were initially screened; 63 videos were finally in in the study after applying the eligibility criteria (Figure 2).

General Characteristics
As shown in Table 1, the mean length of videos was 4.64 min, the mean num views was 34,149.21, the mean number of likes was 701.56, the mean number of d was 0, and the mean number of comments was 45.71.The mean Video Power Ind Interaction Index, indicating the popularity of videos based on daily views and li dislike amounts, were 34.73 and 35.06, respectively.The video that reached the VPI score was uploaded by healthcare professionals (1297.02).

General Characteristics
As shown in Table 1, the mean length of videos was 4.64 min, the mean number of views was 34,149.21, the mean number of likes was 701.56, the mean number of dislikes was 0, and the mean number of comments was 45.71.The mean Video Power Index and Interaction Index, indicating the popularity of videos based on daily views and like and dislike amounts, were 34.73 and 35.06, respectively.The video that reached the highest VPI score was uploaded by healthcare professionals (1297.02).

Source of Upload and Target Audience
Of the 63 videos analyzed, 32 were uploaded by healthcare professionals (51%), 4 by hospitals or universities (6%), 14 by commercial companies (22%) and 13 by laypeople (21%) (Figure 3).As for the target audience, 71% of videos were intended for laypeople, while 29% were specifically for dental professionals.

Quality, Reliability and Content
Intra-examiner and inter-examiner reliability was good to excellent, with all intraclass correlation coefficients being above 0.86 for quality and content analysis scores.
The mean VIQI total score was 11.24.The mean m-DISCERN was 1.83.The mean JAMA score was 1.89.The mean total content score was 2.67.As for the target audience, 71% of videos were intended for laypeople, while 29% were specifically for dental professionals.

Quality, Reliability and Content
Intra-examiner and inter-examiner reliability was good to excellent, with all intraclass correlation coefficients being above 0.86 for quality and content analysis scores.
The mean VIQI total score was 11.24.The mean m-DISCERN was 1.83.The mean JAMA score was 1.89.The mean total content score was 2.67.
Figure 4 shows the individual scores for videos categorized by different sources.As for the VIQI, the videos uploaded by laypeople (mean value = 7.92) showed significantly lower values compared with those uploaded by healthcare professionals (mean value = 12.38) (p < 0.0001) and by commercial companies (mean value = 11.21)(p = 0.009).Regarding m-DISCERN, videos uploaded by laypeople had significantly lower scores (mean value = 1.15) compared to videos by healthcare professionals (mean value = 2.13) (p = 0.003) and by hospitals/universities (mean value = 3.00) (p = 0.002); m-DISCERN was significantly lower also for videos by commercial companies (mean value = 1.43) compared to those uploaded by hospitals/universities (p = 0.005).Regarding JAMA, no statistically significant difference was found among the source categories, with all showing low scores (mean value = 2.00).As for the content, the videos uploaded by laypeople (mean value = 1.54) showed significantly lower scores compared to videos uploaded by healthcare professionals (mean value=3.25)(p = 0.003).The results are summarized in Table 2.
lower also for videos by commercial companies (mean value = 1.43) compared to those uploaded by hospitals/universities (p = 0.005).Regarding JAMA, no statistically significant difference was found among the source categories, with all showing low scores (mean value = 2.00).As for the content, the videos uploaded by laypeople (mean value = 1.54) showed significantly lower scores compared to videos uploaded by healthcare professionals (mean value=3.25)(p = 0.003).The results are summarized in Table 2.In detail, the analysis of single components of the customized content score revealed that, among all videos, the most mentioned topics were the definition and indication regarding the use of MAD (respectively, 75% and 77% of videos).The comparison with alternative treatments was quoted in 41% of videos.A total of 17% of videos cited long-or short-term side effects of treatment with MAD.The duration and cost of the therapy with MAD were cited only in 9.5% of the videos.The instructions on the usage of MAD were provided in 20% of the videos.Contraindications about the use of MAD were mentioned in 9.5% of the videos.The need for a multidisciplinary care team was mentioned in 9.5% of the videos, and the need to consult a dentist qualified in sleep medicine was reported only in 8% of the videos.In detail, the analysis of single components of the customized content score revealed that, among all videos, the most mentioned topics were the definition and indication regarding the use of MAD (respectively, 75% and 77% of videos).The comparison with alternative treatments was quoted in 41% of videos.A total of 17% of videos cited long-or short-term side effects of treatment with MAD.The duration and cost of the therapy with MAD were cited only in 9.5% of the videos.The instructions on the usage of MAD were provided in 20% of the videos.Contraindications about the use of MAD were mentioned in 9.5% of the videos.The need for a multidisciplinary care team was mentioned in 9.5% of the videos, and the need to consult a dentist qualified in sleep medicine was reported only in 8% of the videos.

Discussion
This is the first study that aimed to investigate the quality, reliability and content of online audio-visual information on the treatment of OSA with MAD.The Internet has gained more and more importance among patients willing to learn about medical conditions and desirous of an active role in healthcare decision-making.Research reports that more than 80% of patients use online resources to learn and find information about their health state, and 70% of them assert that it consistently influences their treatment decisions and judgements [31,32].In addition, it has been proven that audio-visual media makes content and messages more engaging and much easier to remember for people compared with other forms of communication, increasing the capability to retain information [33].In detail, Al-Silwadi et al. suggested that audio-visual information could improve orthodontic patient knowledge better than oral communication or information provided by leaflets [34].Nowadays, there is no standardized regulation or peer review process, and anyone can freely and easily upload videos on many online platforms.For that reason, the identification of the video uploaders has a substantial impact on the reliability and quality of the videos.An evaluation tool like the Health-on-the-Net (HON) certification (used to assess the trustworthiness of online information on websites) should be developed and validated in order to verify and guarantee the reliability of online audio-visual health information [35].
The content and usefulness of YouTube videos concerning various medical issues and health conditions have been investigated by several researchers [17,29,36,37].This is the first study evaluating the overall quality of videos related to MAD on four Internet platforms of common use.We decided to extend the research not only on YouTube but also on Google Video, Microsoft Bing and Watch Facebook in order to include a wider and more varied pool of users.Most of the videos included were from YouTube, and most of those searched on the three other platforms referred to videos originally posted on YouTube and then shared on Google, Bing or Facebook.
Our investigation showed that most of the videos were uploaded by healthcare professionals, in agreement with the results obtained by Hassona et al., Leong et al. and Kazi et al. [25,38,39] but in contrast with those by Hatipo glu et al. and Lena et al., where laypeople were the first source of upload [18,21].The explanation could be that clinicians specialized in sleep medicine are inclined to bring their experience and knowledge online in order to allow easy access to information for people looking for professional opinions.The finding that commercial companies were also well-represented in our research (22%) might be due to the increasing interest of modern manufacturing houses in social media management and advertizement engagement.
In our sample, the target audience was laypeople in 71% of videos, in agreement with the current literature [16,21,36].If we assume that common people regard the Internet as a preferential route of information, it can be interpreted positively because common users could obviously better understand easy and plain language and concepts.
The VIQI revealed that the overall quality, precision and educational content provided by specialists was higher (although considered overall moderate) compared with those provided by common people.The finding that dental manufacturing companies also showed moderate scores means that the commercial industry is investing in communication and valuable advertizement.
Regarding m-DISCERN, videos uploaded by laypeople and commercial companies had significantly lower scores compared to videos by healthcare professionals and by hospitals/universities (considered moderate).In detail, the scores obtained by laypeople and commercial companies were classified as poor, in accordance with Singh et al. [27].This finding is consistent with the current literature and may reflect that the information used by hospitals/universities and health professionals is from more reputable sources and is therefore presented in a clear and unbiased manner [40].
As for JAMA, all categories of uploaders reached mean values of 2, reflecting a lack of accurate and reliable information provided.
As for the content analysis, videos uploaded by healthcare professionals and hospitals/universities were considered "useful" according to the scoring scheme adopted in the present study, while commercial and laypeople videos showed "moderately useful" scores.In general, these values, failing to exceed the cut-off of 3.00, detected a low level of educational content.In detail, it is noteworthy that no video mentioned all the topics of the customized checklist adopted in the present study.In accordance with the current evidence from the available literature, many videos answered only one or two of the analyzed criteria [16,21].The most quoted topics in the videos were the definition and indication of MAD, which was clearly the underlying and implied information to start a video dealing with that topic.Less than half of the videos mentioned an alternative treatment like CPAP or surgery, and no information was provided in order to make the viewer educated on the pros and cons of them.The instructions on the usage of MAD were provided in 20% of videos, consisting of wearing the appliance during the night.Only 17% of the videos mentioned short-or long-term side effects like hypersalivation, xerostomia, tooth discomfort, muscular pain or temporomandibular disorders (TMDs) and occlusal changes [41][42][43][44].As for short-term effects, they are temporary and are going to vanish in days or months.Long-term adverse effects must be clearly explained to patients in order to secure their compliance during treatment.The duration of treatment with MAD is likely to be life-long, and in our study, this aspect proved to be stressed only in 9.5% of videos.We think such a lack of information can be really misleading for patients, who often expect fast and decisive therapy.Another topic that was mentioned only in 9.5% of the videos regards contraindications that should be considered by dentists when proposing MAD for the treatment of OSA.An insufficient number of teeth or general precarious oral health (inadequate prosthetics or restorations, periodontal disease), the presence of active TMDs and a reduction in mandibular protrusion ability are conditions that can impede the usage of MAD [44,45].The need for a multidisciplinary care team including an otolaryngologist, cardiologist, pneumologist, neurologist and a qualified dentist was mentioned only in 9.5% of videos, and specifically, the figure of a dentist qualified in dental sleep medicine was reported only in 8% of videos.Indeed, the qualified dentist is not only the specialist who confirms the feasibility and manages over time the treatment with MAD in cooperation with the sleep medicine physician but is also a diagnostic sentinel for OSA.In fact, daily, the qualified dentist can encounter many patients and, by collecting anamnesis and performing the oral examination, can pay attention to the signs and symptoms predictive of OSA and then refer the patient to specialists for further examination [45].
The present study has some limitations.Firstly, because of the dynamic and changing nature of YouTube and other online platforms, the results of this cross-sectional study are time-dependent.Videos are uploaded and deleted continuously on the platforms, and that affects the reproducibility of the study.Secondly, the search included the first 50 videos for each platform, according to the current literature, which affirms that 95% of users will look no further than the first three pages of the results [20].Finally, the quality and reliability of the videos were measured using three widely approved tools such as VIQI, m-DISCERN and JAMA benchmark criteria, while the video content was categorized according to a customized checklist covering the main aspects of the treatment of OSA with MAD.Further studies using a variety of instruments, including a validated tool for the analysis of the video contents to triangulate the validity of the present findings, should, therefore, be encouraged in the future.

Conclusions
The results of the present study suggest that although specialists and healthcare professionals contributed to better quality videos compared with laypeople, the overall reliability, quality and completeness of the contents of videos regarding the treatment of OSA with MAD were quite low.As the Internet offers healthcare professionals the opportunity to disseminate correct and precise information, an effort should be made by specialists in order to improve the level of accuracy, and policymakers should consider developing regulations on a standardized peer-review process.The quality of videos can be improved through a more complete reporting of the side effects, contraindications and duration of therapy, as well as through the importance of having a qualified dentist involved in the multidisciplinary diagnostic and therapeutic process.This information could aid patients in making timely treatment decisions, reducing the occurrence of side effects, facilitating realistic expectations and decreasing the risk of treatment discontinuation.

Figure 1 .
Figure 1.Checklist for content analysis of videos on MAD, mandibular advancement device.Scores were attributed as follows for each criterion: 0 points, not mentioned; 1 point, mentioned; CPAP, continuous positive airway pressure; TMDs, temporomandibular disorders.

Figure 3 .
Figure 3. Sources of upload of included videos.

Figure 4
shows the individual scores for videos categorized by different sources.As for the VIQI, the videos uploaded by laypeople (mean value = 7.92) showed significantly lower values compared with those uploaded by healthcare professionals (mean value = 12.38) (p < 0.0001) and by commercial companies (mean value = 11.21)(p = 0.009).Regarding m-DISCERN, videos uploaded by laypeople had significantly lower scores (mean value = 1.15) compared to videos by healthcare professionals (mean value = 2.13) (p = 0.003) and by hospitals/universities (mean value = 3.00) (p = 0.002); m-DISCERN was significantly

Figure 3 .
Figure 3. Sources of upload of included videos.

Table 1 .
General characteristics of videos.SD, standard deviation.

Table 1 .
General characteristics of videos.SD, standard deviation.

Table 2 .
Individual scores for videos categorized by different sources.VIQI, video information and quality index; m-DISCERN, modified-DISCERN; JAMA, Journal of the American Medical Association benchmark criteria.

Table 2 .
Individual scores for videos categorized by different sources.VIQI, video information and quality index; m-DISCERN, modified-DISCERN; JAMA, Journal of the American Medical Association benchmark criteria.