Evaluation of the Quality and Influence of YouTube as a Source of Information on Robotic Myomectomy

Background: We aimed to evaluate the usefulness of YouTube videos for helping patients become informed about robotic myomectomy. Methods: We searched YouTube using “robotic myomectomy” and “Da Vinci myomectomy.” Videos were sorted by view count, with the 150 most highly viewed videos being selected. From each video, content type, source, view count, video length, time on YouTube, likes, and dislikes were extracted. A scoring system was used to evaluate video quality. Results: The most prevalent content was recordings of actual surgical procedures of robotic myomectomy, and the most common provider was gynecologic surgeons. Videos directly related to robotic myomectomy were mainly provided by medical groups, had been present on YouTube for a significantly longer time (p = 0.003), and had a higher rate of no responses from viewers (p = 0.014) than videos indirectly related to robotic myomectomy. Videos uploaded by nonmedical groups had more likes, more dislikes, and a higher view ratio (p = 0.029, 0.042, and 0.042, respectively). Scores reflecting video quality did not differ between the two groups. Multiple logistic regression revealed that low-quality videos (less than score 5) were significantly correlated with content indirectly related to robotic myomectomy, poor general quality, fewer views, fewer likes, and no response by viewers. Conclusions: Patients who want to get informed about robotic myomectomy on YouTube should exclude low-quality videos according to such parameters as content, views, and response by viewers. In addition, medical groups should provide videos of good quality for instructing patients about this procedure.


Introduction
Robotic myomectomy has been increasingly performed for treating uterine myoma in women who want to preserve their fertility. It provides a qualified visual field with a 3D camera and wrist-like movement of instruments without tremor, which makes it easy to repair the myometrium after removing uterine fibroids. To prevent uterine rupture in a future pregnancy, primary repair of the myometrium is particularly important. Therefore, robotic myomectomy is commonly preferred for women who wish to become pregnant.
With the widespread use of the internet, patients can easily obtain health-related information from various online resources. In recent surveys, it has been shown that healthrelated information is commonly accessed by internet users, with 8 of 10 having accessed such information online [1]. Among the diverse resources from which such information is available online, YouTube is the most popular website, with an abundance of videos [2]. Although many physicians may agree on the importance of YouTube as a source of medical information for the public, the quality and reliability of such videos remain controversial. Given that such health-care-related videos can be uploaded without any oversight or peer review, they may contain inappropriate or misleading information.
To the best of our knowledge, no studies have evaluated the usefulness of YouTube videos for helping patients become informed about robotic myomectomy. Therefore, in this study, we evaluated the reliability and quality of the most viewed videos about this procedure uploaded to YouTube. We also analyzed the responses of viewers to determine whether viewers "liked" videos of good quality.

Search Strategy
A search of YouTube as described above was performed on 18 November 2021 using the term "robotic myomectomy." Sorting of the videos in terms of their view count was performed to identify those that had been watched most frequently. Videos were included in this study if they met the following inclusion criteria: (1) in English and (2) primarily related to robotic myomectomy. They were not used in this study if they met the following exclusion criteria: (1) in languages other than English, (2) irrelevant content, (3) duplicated videos, and (4) lacked audio. Of the 300 videos that were initially identified, the top 150 were included for review.

Video Assessment
We obtained the following data from each video: (1) video title, (2) uploader's name, (3) number of views, (4) length of the video (minutes), (5) uploaded date, (6) days since upload, (7) the counts of "likes" and "dislikes" (represented by the "thumbs up" and "thumbs down" icons), and (8) details of the robotic myomectomy procedure shown in the video and of the operators and associated institution.
Based on the content, we categorized the videos into four groups: (1) explanations of the surgery (providing general information about robotic myomectomy), (2) surgical procedure (detailing the techniques and processes used during the surgery, or describing the particular instruments used), (3) personal experiences (sharing personal feelings and experiences related to robotic myomectomy, e.g., pregnancy after surgery), and (4) preand postoperative care (e.g., bowel preparation, laboratory tests, pain control, exercise, and wound care). In addition, videos were classified into directly related (including explanations of surgery and surgical procedures) and indirectly related groups (including personal experiences or postoperative care).
According to their authorship, we classified the videos into four groups: (1) those made by academics (i.e., those affiliated with a university), (2) those made by physicians (i.e., those not affiliated with a university but working as physicians), (3) those made by patients (i.e., women who had undergone or were going to undergo robotic hysterectomy), and (4) those made by commercial entities (i.e., videos marketing a product or service). Videos were further divided into medical (uploaded by academics and physicians) and nonmedical groups (uploaded by patients and commercial entities).

Quality Assessment
Given the lack of established standards for evaluating the quality of videos such as those on YouTube, we used a scoring system established in a previous study [3]. For this evaluation, we employed factors related to general video quality, factors related to the inclusion of important information about robotic myomectomy and explanations of such information, and the extent to which scientific evidence was provided (Box 1). Regarding the general quality of videos and their flow, scoring was performed on a scale from 1 to 3. Meanwhile, regarding the information about robotic myomectomy, six different elements were used (indication, complication, surgical process, preoperative preparation, anesthesia, and postop management) and allocated 0 points if they were not mentioned and 2 points if they were. Regarding the provision of scientific evidence (clear statements about the sources of information and the inclusion of details about where further information about the topic of the video could be obtained) in the videos, this was assigned 0 points if such information was not provided and 3 points if it was. This gave an overall score ranging from 1 to 18 points. We determined that videos scoring less than 5 points were of low quality.

Box 1. Factors for evaluating the quality of robotic myomectomy-related YouTube videos.
General quality (poor: 1 point; moderate: 2 points; good: 3 points) Overall quality (visual and audio) Flow of content Degree to which information in helpful to viewers (not mentioned: 0 points; mentioned: 2 points) Indication Complication Surgical process Preoperative preparation, anesthesia Postoperative management and care Scientific evidence (no: 0 points; yes: 3 points) Clearly discloses sources of information Provides clues of where additional information on the video topic available The quality of each video was independently assessed by two professional gynecologists, with the average of their scores being used for analysis. The two gynecologists had performed more than 300 cases of robotic surgery (e.g., robotic myomectomy, robotic hysterectomy, and robotic ovarian cystectomy) each.

Statistical Analysis
In this study, SPSS Version 26.0 (SPSS, Inc., Chicago, IL, USA) and Medcalc software 15.2.2 (Medcalc, Ostend, Belgium) were used for the statistical analysis. Data are described as median (range) for continuous variables and n (%) for categorical ones. Cohen's kappa coefficient was used to assess the agreement between the two reviewers, with values >0.8 representing excellent agreement, values of 0.6-0.8 representing substantial agreement, values of 0.4-0.6 representing moderate agreement, and those <0.4 representing poor agreement [5]. The ratings of the reviewers were found to be highly correlated (Cohen's kappa coefficient 0.9).
Pearson's chi-squared test or Fisher's exact test was used to compare the results for categorical variables. The Kolmogorov-Smirnov test was used to assess whether the data of continuous variables were normally distributed. Variables that were normally distributed were compared using the independent t-test, while the Mann-Whitney U test was used otherwise. A p-value < 0.05 was used to indicate a statistically significant difference.
Multivariate analysis was performed using binary logistic regression. The hazard ratio (HR) and 95% confidence intervals (CI) were calculated. Cutoff values of parameters in discrimination of low-quality videos (score < 5) from others were determined by receiveroperating characteristic (ROC) curve analysis.

Main Results
Among the 150 most viewed videos related to robotic myomectomy, 121 were recordings of the actual surgical procedure, mainly performed by gynecologic surgeons. The most common source of the videos was medical groups, including physicians and academics (Table 1). Videos indirectly related to robotic myomectomy mainly involved patients sharing their experiences and feelings (Table 1). Videos sourced from commercial channels presented surgical instruments (e.g., CO 2 laser and ultrasound) that can be used during the surgery. The medical groups mainly contributed videos directly related to robotic myomectomy (113/120; 94.2%), whereas the nonmedical groups mainly contributed videos indirectly related to robotic myomectomy, including those on personal experiences and pre-and postoperative care (13/30; 43.3%).
Descriptive features of the videos are shown in Table 2. The top 150 videos related to robotic myomectomy had been viewed 5,445,322 times (median 919.5, range 124-4,268,240). The most viewed video had been uploaded by a medical media channel in 2014, which provides not only the detailed surgical procedure but also general information about robotic myomectomy, such as the indication for surgery, type of fibroids, and other treatment options. This video had also received the highest numbers of likes (10,000) and dislikes (1400). Among the 130 videos directly related to robotic myomectomy, 27 videos received no responses (no dislikes or likes) from viewers. To evaluate the quality of the videos, we used a scoring method described previously [6]. To increase the accuracy of the information, we gave 3 points when scientific evidence was presented in the video. The median score was 8 (range 3.5-16). One of the three videos with the highest scores was the video with the highest numbers of views, likes, and dislikes. The other two videos were provided by physicians, which comprehensively described the surgical procedure and pre-and postoperative care.
Videos directly related to robotic myomectomy were significantly associated with a longer time on YouTube (p = 0.003) and more commonly had no responses from viewers (p = 0.014) than videos indirectly related to robotic myomectomy (Table 3). Videos uploaded by nonmedical groups had higher numbers of likes and dislikes, and a higher view ratio (p = 0.029, 0.042, and 0.042, respectively) (Table 4). Also, VPI tended to be higher for videos uploaded by nonmedical groups than those uploaded by medical groups (p = 0.062). Quality scores were similar between the two groups (Table 4).  In addition, the characteristics of the videos were compared depending on the time of upload (until 31 December 2016 vs. from 1 January 2017, onwards). The like ratio tended to be higher for videos uploaded after 2016 (p = 0.059). Although quality scores were higher for videos uploaded after 2016, this was not statistically significant (p = 0.237) ( Table 5). Multiple logistic regression revealed that low-quality videos (less than score 5) were significantly correlated with content (indirectly related to robotic myomectomy), poor general quality, views (fewer than 1472), likes (fewer than 10), and no response by viewers ( Table 6).

Discussion
Since the introduction of the Da Vinci surgical system (Intuitive Surgical Inc) to gynecologic surgery in 2005, robotic myomectomy has become one of the best options for treating uterine myoma for women who want to become pregnant [7]. Robotic myomectomy can provide a safe and precise approach to the surgical field with minimal incision, as well as less pain and faster recovery.
With the trends in minimal invasive procedures, robotic myomectomy has also been increasingly attracting women with symptomatic uterine myoma.
Therefore, we reviewed the most viewed 150 videos on YouTube, and evaluated the quality and influences of each content.
The video quality scores were similar regardless of content (directly vs. indirectly related to robotic myomectomy), source (medical vs. nonmedical), and the time of upload (until 31 December 2016 vs. from 1 January 2017, onwards).
Since the videos most commonly featured recordings of the actual surgical procedure of robotic myomectomy by gynecologic surgeons, these might be less informative for members of the public interested in robotic myomectomy. In addition, some videos uploaded by surgeons had poor visual and audio quality. Moreover, most of the videos uploaded by medical groups did not provide patient-friendly information, which is valuable for those preparing to undergo robotic myomectomy.
Interestingly, no responses by viewers (no likes or dislikes) were significantly common for videos directly related to robotic myomectomy. In addition, videos uploaded by nonmedical groups were more frequently liked and disliked by viewers than those uploaded by medical groups. The rate of a lack of responses by viewers was higher for videos uploaded by medical groups than for those uploaded by nonmedical groups, although the statistical significance was marginal. Since the time on YouTube was significantly longer for videos directly related to robotic myomectomy and videos uploaded by medical groups, it is disappointing that viewers were not interested in those videos. We consider that those videos directly related to robotic myomectomy and uploaded by medical groups were not attractive to viewers because such videos contain difficult medical terms and only feature the surgical procedure, making them difficult for the general public to understand.
However, recent videos uploaded after 2016 showed slight advances compared with videos from the preceding period. The like ratio and quality score tended to be higher for recent videos, although this was not statistically significant. The rate of having no responses from viewers was lower for recent videos, although this was not statistically significant.
Most research evaluating the quality of YouTube videos about surgical procedures suggested that YouTube is an inappropriate source of public information on such procedures, which corresponds with our results [8].
Several studies have evaluated YouTube videos as a source of information about obstetric and gynecologic surgeries [9]. For example, a recent study assessed the quality of information in YouTube videos regarding hysterectomy [8]. It reported that only 6% of videos could be considered excellent, 43% moderate, and 51% poor in terms of information content [8]. In addition, most patient-made videos were critical of hysterectomy (71.72%) [8]. Meanwhile, most academic-or physician-made videos were educational and focused on the surgical techniques, and thus were aimed at doctors, not patients [8].
Another study focused on YouTube to evaluate the overall quality of the top 100 videos on the portal about cesarean delivery [6]. Among these 100 videos, 47 were directly related to cesarean delivery, and most (n = 30) of the videos had been produced by physicians [6]. It was also found that most videos directly related to cesarean delivery had been produced by medical groups, which received higher quality scores than the videos indirectly related to such delivery mainly provided by nonmedical groups [6]. The results also revealed that the videos directly related to cesarean delivery had often been provided at an earlier date and had lower like ratios than the videos indirectly associated to cesarean delivery [6].
This study is the first to analyze the quality and influence of YouTube videos about robotic myomectomy. However, it has some limitations. First, we analyzed only videos using the English language, which may have caused selection bias. In future study, it will be necessary to include videos using other languages to achieve a more comprehensive evaluation. Second, we chose to use a quality scoring system referred to in previous reports, as there are no validated criteria for video analysis. We added essential information on robotic myomectomy for the quality evaluation of videos. In addition, to increase the accuracy of the information, we gave 3 points when scientific evidence was presented in the video. However, there is a need for more validated criteria to comprehensively evaluate video quality.

Conclusions
In conclusion, YouTube is not yet a valuable source of adequate information about robotic myomectomy. Although medical groups have long uploaded many videos directly related to this procedure, most of these videos just feature recordings of the procedure in real time, thus providing no easily understandable information for the general public seeking health-care information on YouTube. Therefore, medical groups who are responsible for general health care should provide videos of good quality for patients.
In addition, videos indirectly related to robotic myomectomy with poor general quality, fewer views (fewer than 1472), fewer likes (fewer than 10), and no response by viewers are not recommended for patients who want to get informed about robotic myomectomy on YouTube.