Analysis of the quality of vulvar cancer-related videos on YouTube

Introduction

Globally, Vulvar cancer is a relatively rare malignant tumor that receives less attention than other gynecological malignancies such as cervical cancer and ovarian cancer (1). However, the incidence of vulvar cancer is increasing at a rate of 0.6 percent per year and the survival rate is decreasing (1). Similar to cervical cancer, infection with the human papillomavirus (HPV) is closely related to the occurrence of vulvar cancer in a significant proportion of patients (2). The HPV vaccination, regular screening, and several other precautionary measures may have potential value in the prevention of vulvar cancer (3). Of note, a biopsy under the guidance of vulvoscopy is the gold standard for the diagnosis of precancerous vulvar squamous cell carcinoma (VSCC), which is the most common pathological type of vulvar cancer (4). However, globally, the popularity of the relevant specialized technology and the understanding of this condition among high-risk patients is low (5). In the case of malignant tumors without specific symptoms, vulvar cancer is commonly misdiagnosed as inflammatory or infectious diseases (6), which may delay the timing of treatment and influence patient prognosis.

With the rapid development of information technology, short-video apps are playing an important role in health promotion (7,8). However, because the quality of videos varies greatly, some users may be misled by such online information (9-11). Therefore, evaluation of the quality of health-related videos on media platforms may help to improve the efficiency of health promotion and the understanding of these diseases among patients. YouTube, as the most widely used short-video media platform, contains a bulk of health information. Accordingly, YouTube played a huge role during the coronavirus disease 2019 (COVID-19) pandemic and provided users with a vast amount of knowledge in relation to prevention and treatment (12). To date, several studies have explored the quality of YouTube videos on gynecological cancers such as ovarian cancer (13), cervical cancer (14), and breast cancer (15). The quality of videos related to the above diseases is hard to achieve the excellent grade. As a kind of rare gynecological cancer, the quality of videos about vulvar cancer also need to be made a comprehensive evaluation, because of the fact that the misinformation or low-quality health information may increase the burden of disease including the incidence and mortality. Videos which can provide accurate information about prevention, screening, behavioral intervention, diagnosis, and treatment of vulvar cancer are demanded by general public. This study fills this gap by evaluating the quality of videos about vulvar cancer on YouTube. The goal is to make facts-based recommendations for promoting better public health engagement. We present this article in accordance with the STROBE reporting checklist (available at https://tcr.amegroups.com/article/view/10.21037/tcr-24-1411/rc).

Methods

Data extraction

On May 15, 2024, a search of YouTube was conducted using the term “vulvar cancer”. On Google Chrome, the cookie-free, cache-cleared, and private browsing pattern was set and the search was conducted in default search mode of proposed relevance after erasing the search history on the YouTube. The top 100 results were selected for the identification of relevant videos. Duplicate, irrelevant, commercial, operational, and audio-free videos were excluded, such as videos demonstrating surgical techniques without knowledge outputs for the general public, or having similar contents. Finally, a total of 65 videos were included in this study. Data were gathered on each video’s length, language, days watched online, views, comments, and likes, as well as the source and uploader’s location. Regarding the author, the video source was categorized as physician, education and training institute, news media, profit organization, or non-profit organization. Figure 1 depicts the process used to select YouTube videos. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Figure 1 Flowchart of the selection of YouTube videos for analysis. DISCERN, Quality Criteria for Consumer Health Information; PEMAT, Patient Education Materials Assessment Tool; GQS, Global Quality Scale; JAMA, Journal of the American Medical Association.

Video evaluation

The video quality was assessed using the DISCERN instrument (Quality Criteria for Consumer Health Information), the Journal of the American Medical Association (JAMA) benchmark criteria, the Patient Education Materials Assessment Tool (PEMAT), and the Global Quality Scale (GQS). The above-mentioned four instruments were widely used in researches about the evaluation of videos’ quality on YouTube (10,11).

DISCERN has been frequently utilized to assess the caliber of health information (16). DISCERN consists of 16 questions broken down into three sections: questions 1 through 8 deal with the publication’s dependability; questions 9 through 15 deal with the quality of the information about treatment options; and question 16 deals with the publication’s total score. Each question receives a score ranging from 1 (poor) to 5 (good). The 16 questions are summed to obtain the final DISCERN score. In this study, the videos were categorized into groups based on their overall DISCERN scores: very poor [27], poor [27–38], fair [38–50], good [51–62], and excellent [63–80]. Although DISCERN was designed to evaluate textual publications, it is important to note that it has been widely employed for evaluating health videos (17). The DISCERN questionnaire can be found in Table S1.

Four JAMA benchmark criteria (authorship, attribution, disclosure, and currency) were used to assess the quality of online health information in this study. The total available score was 4 (18), with 1 point awarded for each requirement met. Each criterion is thoroughly explained in Table S2.

The PEMAT, which consists of 17 questions, was used to evaluate the actionability and understandability of each video (19). Each item in the PEMAT is rated as agree (1 point), disagree (0 points), or not applicable (no point and stated as not applicable), with 13 items related to understandability and four related to actionability. The results are expressed as a percentage for each domain. Higher percentages indicate more actionability or understandability. Scores of more than 70% suggest that the material is clear or applicable.

The GQS is a tool that uses a five-point scale to rate a video’s effectiveness, flow, and quality. High quality is indicated by four points, medium quality by three points, and poor quality by one or two points (20). A detailed description of the measure is provided in Table S3.

All videos were rated by T.X. and H.Z., two independent raters, and only 3 (4.6%) videos were controversial. When disagreements between reviewers occurred, discussion with the third rater (S.P.) was performed to reach a consensus.

The completeness of the video material was evaluated using six dimensions: the definition of disease, symptoms/signs, risk factors, evaluation, management, and outcomes. Based on this assessment, each dimension was given a rating ranging from 0 (no content) to 2 (extensive content). The mean score for each dimension was then represented using a hexagonal radar chart (21).

Statistical analysis

All statistical analyses were performed using SPSS software version 26.0 (SPSS Inc., Chicago, IL, USA). The means and standard deviations (SDs), frequencies, and ratios (%) for continuous and categorical data, respectively, were reported. The Kruskal-Wallis test was used to determine whether there were any significant differences between the median scores of more than two groups. The relationships between the variables were assessed using Spearman’s correlation coefficients. P<0.05 was considered statistically significant.

Results

Video features, quality, and content

After the application of the exclusion criteria, a total of 65 relevant videos were identified and included in this study. The majority of the videos were uploaded by an organization (50/76.9%), while individual uploads accounted for a minor portion of the videos (15/23.1%). Profit organizations uploaded the least amount of videos (2/3.1%), while education and training institutes posted the most (42/64.6%) (Table 1). The majority of the videos were in English (61/93.8%), making them widely understandable.

The average video length was 1,240.23 s, with a range of 24.00 to 6,394.00 s. The oldest video was posted more than 10 years ago, while the most recent was posted 2 months before data collection. The videos received 21.00 to 1,223,363.00 views, 0.00 to 4,118.00 likes, and 0.00 to 460.00 comments. Table 2 provides a detailed description of the characteristics of the videos.

The average DISCERN total score, JAMA score, PEMAT understandability score, PEMAT actionability score, and GQS score were 40.78 (range, 16.00–75.00), 3.05 (range, 2.00–4.00), 89.90% (range, 23.08–100.00%), 52.95% (range, 0.00–100.00%), and 3.06 (range, 1.00–5.00), respectively (Table 2).

The hexagonal radar chart allowed us to determine the unevenness of the information on YouTube. With a mean score of 1.55, the primary subject matter of the videos was the symptoms of vulvar cancer. The average ratings for the other five aspects of the material were 1.18 (definition), 0.99 (risk factors), 0.80 (evaluation), 0.98 (management), and 0.63 (outcomes). Figure 2 presents the hexagonal radar chart.

Figure 2 Completeness of video content.

Correlations between video features and quality

According to the results of the Kruskal-Wallis tests, there were no correlations between the source of the video and the DISCERN score, JAMA score, PEMAT actionability score, or comments per day. There were significant correlations between the source of the video and the duration, views per day, likes per day, PEMAT understandability, and GQS score (P=0.04, 0.004, 0.001, 0.03, and 0.045, respectively). The GQS score was considerably higher for videos published by physicians as compared to those published by non-profit organizations, after Bonferroni correction (P=0.02). In comparison to videos from education and training institutes, the PEMAT understandability score was considerably lower for videos from normal users (P=0.03). Additionally, videos released by non-profit organizations were considerably shorter than videos uploaded by physicians (P=0.03). Further, the views per day and likes per day of videos published by physicians were considerably higher than those of videos produced by education and training institutes and non-profit organizations (P=0.004, 0.048, 0.001, and 0.03, respectively) (Table 3).

The DISCERN categorization ratings were 13.85% “excellent”, 15.38% “good”, 29.23% “fair”, 20.00% “poor”, and 21.54% “very poor”. The DISCERN categorization demonstrated statistically significant associations with video length (P<0.001), views per day (P=0.02), likes per day (P<0.001), comments per day (P=0.03), PEMAT actionability (P<0.001), and GQS scores (P<0.001), according to the Kruskal-Wallis test (Table 4).

The JAMA score was significantly correlated with the DISCERN total score (r=0.320, P=0.009) and PEMAT understandability score (r=0.308, P=0.01), according to the correlation test. The GQS score (r=0.873, P<0.001), video duration (r=0.703, P<0.001), views per day (r=0.311, P=0.01), number of likes (r=0.274, P=0.03), likes per day (r=0.474, P<0.001), and PEMAT actionability score (r=0.829, P<0.001) were all positively correlated with the DISCERN total score. In addition, there were significant positive associations between the GQS score and the length of the video (r=0.702, P<0.001), views per day (r=0.253, P=0.04), likes per day (r=0.391, P=0.001), PEMAT understandability score (r=0.296, P=0.02), and PEMAT actionability score (r=0.796, P<0.001) (Table 5).

Discussion

Although it is the most common pathological type of vulvar cancer, VSCC is still regarded as the forgotten women’s cancer (22). Not all cases of VSCC are related to HPV; however, the occurrence of VSCC is closely linked to persistent infection with a high-risk type of HPV for many patients. Statistics indicate that HPV-associated VSCC accounts for approximately 40% of all patients with VSCC, and HPV has been identified as the factor underlying the recent increases in the incidence of VSCC worldwide (23). The 80–90% of patients with precancerous VSCC lesions, i.e., high-grade squamous intraepithelial lesions (HSILs), were found to be infected with HPV (24). Hence, widespread vaccination with the currently available HPV vaccines has the potential to reduce the incidence of vulvar cancer (25). Cervical cancer, which is also an HPV-associated gynecological cancer, has a normalized screening process that is broadly performed among women of reproductive age. On the contrary, both physicians and high-risk patients, such as women with a low genital tract infection with high-risk HPV, lack a basic understanding of vulvar cancer. Actually, videos produced by physicians and normal users (23.1%) also only contributed a small part of included videos in the research. Physicians are not commonly producing contents on vulvar cancer, because of the lack of proficiency, there may be a risk of misinformation or incomplete information being disseminated. With more high-quality videos about VSCC being uploaded in media platforms, it may exert positive effects on improving the gap in knowledge. As an open-access video-sharing platform, YouTube contains a plethora of videos conveying health information (26). In terms of influencing users’ behaviors and attitudes, YouTube has been shown to be advantageous over other platforms such as Twitter and Facebook (27). Hence, videos on YouTube may have the potential to help promote the prevention and understanding of vulvar cancer among the public.

In contrast to a previous study (10), in this study, only a small proportion of videos were produced by physicians (15.4%, 10/65). Education and training institutes were the major publishers of vulvar cancer videos (42/65, 64.6%). Remarkably, there were also relatively few videos uploaded by normal users (5/65, 7.7%), which suggests that not only professionals but also the general public, pay little attention to and have little understanding of this rare cancer. Videos related to common diseases such as hypertension (28) and diabetes (29) have more likes and comments than videos about vulvar cancer on YouTube. According to the DISCERN scoring standard, the videos included in the current study could also only be ranked as fair (40.78/80), and there was a lack of excellent quality videos (9/65, 13.85%). Nonetheless, as a media platform, YouTube might still have the potential to become a medium between people and vulvar cancer information to improve public health.

Although there were no statistically significant differences, videos from physicians had higher DISCERN scores than those from other origins. For the GQS score, however, videos from physicians had significantly higher scores (P=0.045). This is similar to previous studies (10,11). Videos from physicians seemed to have higher quality and gained acceptance by more viewers, with more views and likes. Another interesting finding was that the duration of the video was positively associated with the quality of the video. This could be because longer-duration videos can convey more details about the disease and can provide patients with better guidance (30), which will improve the rating of the video’s quality. Undoubtedly, the outstanding video works should not only have high quality, but also be acceptable by viewers. However, long videos may lose the attention of viewers. In the future study, the optimal video length can be explored, which may make a balance between the detailed content and viewers’ engagement. On the other hand, videos ranked ‘fair’ seemed to have higher views, likes, and comments than ‘excellent’ videos, which is a potential problem. The common users of YouTube may lack the ability to accurately discern health information. Videos with low-quality content might contain incorrect information (31) which could influence the behavior of viewers and result in serious consequences, such as delayed diagnosis and treatment. In similar with other vulvar disorders, such as inflammation and infection, for instance, pruritus is usually the dominant symptom for VSCC in relatively early stage, which lack the specificity (4). The poor-quality online health information might lead to the neglection of symptoms and delay of treatment for high-risk population. In the current study, the correlation analysis indicated that higher DISCERN scores were closely related to higher GQS and JAMA scores. Although different marking systems may have different emphases, there is also an intrinsic link between the various tools used for the evaluation of video quality. The results also indicated that higher DISCERN scores were closely related to more views and likes, suggesting that users have the basic ability to judge the quality of videos. However, the number of comments seemed to have no correlation with the quality of the video. The comments might be driven by factors like the popularity of the topic rather than the accuracy or quality of the content, which may skew the interaction metrics. The hexagonal radar chart revealed that the included videos were mostly focused on the symptoms of vulvar cancer, neglecting information on the evaluation and outcomes of this disease. In line with anal intraepithelial neoplasm (10), some precancerous lesions for VSCC are not visible to the naked eye and require diagnosis with specialized examinations, such as vulvoscopy (4). Although surgical excision is the main treatment approach for vulvar cancer, for patients with advanced disease stages, the poor survival prognosis and complications of surgery, such as sexual or urinary dysfunction, should be taken into account (32). Hence, information about evaluation and outcomes, such as survival rates, quality of life after treatment, or post-operative complications, should be included in more videos about vulvar cancer in the future to improve the quality of these videos.

Based on the results of this study, there is still some work to be done to improve the quality of YouTube videos about vulvar cancer. This may have positive effects on increasing public awareness of this relatively rare disease. Firstly, the platform can set a filtering mechanism to help exclude low-quality videos and show high-quality videos, such as videos with a relatively long duration and those containing information about the evaluation and outcomes of vulvar cancer. On the other hand, YouTube should encourage videos from professional physicians, especially those who specialize in gynecological oncology. In addition, on the health science channel of YouTube, a special column should be established for vulvar cancer to catch users’ attention, especially for patients with high-risk HPV infection in the lower genital tract. Nonetheless, there are several limitations in this study. Firstly, observer bias is inevitable because of the subjective characteristics of the evaluation instruments. In the future, more objective measures of video quality, such as user engagement metrics or viewer satisfaction surveys, should be incorporated in researches to complement the current evaluation tools. Secondly, as a cross-sectional analysis of a limited sample of videos, the findings only reflect the quality of YouTube videos at one point in time. Further, imposing a restriction to identify only English language videos during the search may have excluded videos produced in other languages.

Conclusions

As a relatively rare gynecological oncology, vulvar cancer generally lacks public attention, which impacts the prevention, diagnosis, and treatment of vulvar cancer. YouTube, as one of the most widely used media platforms, may serve as a helpful platform for disseminating scientific information about vulvar cancer. However, the quality of videos about vulvar cancer on YouTube is unsatisfactory. The findings of this study offer ideas for the improvement of videos’ quality and future implementation to make YouTube a more practical tool for promoting the prevention and cure of vulvar cancer.

Acknowledgments

We would like to express our deepest gratitude to Shaorong Pan, MD (Department of Gastrointestinal Surgery, Peking University First Hospital) for guidance in the evaluation of videos’ quality.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://tcr.amegroups.com/article/view/10.21037/tcr-24-1411/rc

Peer Review File: Available at https://tcr.amegroups.com/article/view/10.21037/tcr-24-1411/prf

Funding: None.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://tcr.amegroups.com/article/view/10.21037/tcr-24-1411/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Akhtar-Danesh N, Elit L, Lytwyn A. Trends in incidence and survival of women with invasive vulvar cancer in the United States and Canada: a population-based study. Gynecol Oncol 2014;134:314-8. [Crossref] [PubMed]
2. Weinberg D, Gomez-Martinez RA. Vulvar Cancer. Obstet Gynecol Clin North Am 2019;46:125-35. [Crossref] [PubMed]
3. Kechagias KS, Kalliala I, Bowden SJ, et al. Role of human papillomavirus (HPV) vaccination on HPV infection and recurrence of HPV related disease after local surgical treatment: systematic review and meta-analysis. BMJ 2022;378:e070135. [Crossref] [PubMed]
4. Kesić V, Vieira-Baptista P, Stockdale CK. Early Diagnostics of Vulvar Intraepithelial Neoplasia. Cancers (Basel) 2022;14:1822. [Crossref] [PubMed]
5. Bevilacqua F, Selk A, Stockdale C, et al. The International Society for the Study of Vulvovaginal Disease (ISSVD) Vulvar Awareness Day Campaign: Knowledge of Vulvovaginal Diseases Among Italian Obstetrics and Gynecology Residents. J Low Genit Tract Dis 2024;28:91-4. [PubMed]
6. Selk A. Vulvar Squamous-Cell Carcinoma. N Engl J Med 2022;386:e3. [Crossref] [PubMed]
7. Javidan A, Nelms MW, Li A, et al. Evaluating YouTube as a Source of Education for Patients Undergoing Surgery: A Systematic Review. Ann Surg 2023;278:e712-8. [Crossref] [PubMed]
8. Loeb S, Reines K, Abu-Salha Y, et al. Quality of Bladder Cancer Information on YouTube. Eur Urol 2021;79:56-9. [Crossref] [PubMed]
9. Roozenbeek J, van der Linden S, Goldberg B, et al. Psychological inoculation improves resilience against misinformation on social media. Sci Adv 2022;8:eabo6254. [Crossref] [PubMed]
10. Chen Z, Pan S, Zhou M, et al. Evaluation of the quality and reliability of anal cancer and its precancerous lesions-related content on YouTube: a cross-sectional study. BMJ Open 2023;13:e074787. [Crossref] [PubMed]
11. Chen Z, Pan S, Zuo S. TikTok and YouTube as sources of information on anal fissure: A comparative analysis. Front Public Health 2022;10:1000338. [Crossref] [PubMed]
12. Ostrovsky AM, Chen JR. TikTok and Its Role in COVID-19 Information Propagation. J Adolesc Health 2020;67:730. [Crossref] [PubMed]
13. Ekmez F, Ekmez M. Evaluation of the Quality and Reliability of YouTube Videos With Turkish Content as an Information Source for Gynecological Cancers During the COVID-19 Pandemic. Cureus 2023;15:e44581. [Crossref] [PubMed]
14. Adhikari J, Sharma P, Arjyal L, et al. YouTube as a Source of Information on Cervical Cancer. N Am J Med Sci 2016;8:183-6. [Crossref] [PubMed]
15. Chai BS, Ingledew PA. Characteristics Assessment of Online YouTube Videos on Radiotherapy for Breast Cancer. Clin Breast Cancer 2023;23:e230-8. [Crossref] [PubMed]
16. Charnock D, Shepperd S, Needham G, et al. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53:105-11. [Crossref] [PubMed]
17. Aydin MF, Aydin MA. Quality and reliability of information available on YouTube and Google pertaining gastroesophageal reflux disease. Int J Med Inform 2020;137:104107. [Crossref] [PubMed]
18. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. JAMA 1997;277:1244-5. [Crossref] [PubMed]
19. Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 2014;96:395-403. [Crossref] [PubMed]
20. Bernard A, Langille M, Hughes S, et al. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol 2007;102:2070-7. [Crossref] [PubMed]
21. Goobie GC, Guler SA, Johannson KA, et al. YouTube Videos as a Source of Misinformation on Idiopathic Pulmonary Fibrosis. Ann Am Thorac Soc 2019;16:572-9. [Crossref] [PubMed]
22. Clancy AA, Spaans JN, Weberpals JI. The forgotten woman's cancer: vulvar squamous cell carcinoma (VSCC) and a targeted approach to therapy. Ann Oncol 2016;27:1696-705. [Crossref] [PubMed]
23. Forman D, de Martel C, Lacey CJ, et al. Global burden of human papillomavirus and related diseases. Vaccine 2012;30:F12-23. [Crossref] [PubMed]
24. Hampl M, Deckers-Figiel S, Hampl JA, et al. New aspects of vulvar cancer: changes in localization and age of onset. Gynecol Oncol 2008;109:340-5. [Crossref] [PubMed]
25. Lindquist S, Frederiksen K, Petersen LK, et al. The risk of vaginal, vulvar and anal precancer and cancer according to high-risk HPV status in cervical cytology samples. Int J Cancer 2024;155:61-70. [Crossref] [PubMed]
26. Langford A, Loeb S. Perceived Patient-Provider Communication Quality and Sociodemographic Factors Associated With Watching Health-Related Videos on YouTube: A Cross-Sectional Analysis. J Med Internet Res 2019;21:e13512. [Crossref] [PubMed]
27. Carneiro B, Dizon DS. Prostate Cancer Social Media: In YouTube We Trust? Eur Urol 2019;75:568-9. [Crossref] [PubMed]
28. Kaya E, Solak Y, Şahin M, et al. Is YouTube useful as a source of information for approaches to reducing blood pressure and hypertension treatment? Hypertens Res 2023;46:386-94. [Crossref] [PubMed]
29. Mylavarapu M, Maheta D, Clarke S, et al. Diabetes Mellitus on YouTube: A Cross-Sectional Observational Study to Assess the Quality and Reliability of Videos. Cureus 2023;15:e43704. [Crossref] [PubMed]
30. Bai G, Fu K, Fu W, et al. Quality of Internet Videos Related to Pediatric Urology in Mainland China: A Cross-Sectional Study. Front Public Health 2022;10:924748. [Crossref] [PubMed]
31. Li HO, Pastukhova E, Brandts-Longtin O, et al. YouTube as a source of misinformation on COVID-19 vaccination: a systematic analysis. BMJ Glob Health 2022;7:e008334. [Crossref] [PubMed]
32. Lawrie TA, Nordin A, Chakrabarti M, et al. Medical and surgical interventions for the treatment of usual-type vulval intraepithelial neoplasia. Cochrane Database Syst Rev 2016;2016:CD011837. [Crossref] [PubMed]