Internet Health Resources for Diabetic Foot Search: The Content, Quality, and Readability of Information on Web Sites

Abstract

Background: As the incidence of diabetes mellitus increases, the incidence of diabetic foot also increases. This situation, which may lead to devastating complications and progress to limb loss for patients, exposes patients and their social environments to a big crisis. Thus, patients may seek secondary opinions from online sources about information they initially obtained from health institutions. We aimed to evaluate the information content related to diabetic foot on the Internet that is probably used by patients for Internet searching. Methods: After software optimization and reset, related queries with the keyword diabetic foot were determined on Google Trends. Selected keywords were searched in three search engines, and the results were examined. Web sites were classified into five subcategories (nongovernmental health institution, governmental institution, academic, blog, and university) and evaluated with an information content scale (ICS) based on the literature, Journal of the American Medical Association benchmark criteria, the Flesch-Kincaid readability test, and presence of the Health On the Net Foundation Code of Conduct certificate. The search engines, keywords, and Web site subcategories were investigated with the evaluation criteria. Results: In terms of finding Web sites eligible for assessment, the Google search engine listed more eligible Web sites than did Bing and Yahoo. Concerning the ICS, there was no significant difference between search engines for total scores (P > .05). Concerning ICS diagnosis and evaluation and ICS total score, academic Web sites scored significantly higher than other subcategories. Conclusions: Results that can be obtained with an Internet search for diabetic foot depend on the proper keyword selection, Web site type, and search engine to help patients reach more appropriate content.

The prevalence of diabetes mellitus is increasing every year, and it is predicted that the total number of patients will reach approximately 629 million in 2045 [1,2]. Diabetes mellitus may lead to severe morbidity with extremity complications. The extremity complications, which may progress to loss of limb due to diabetes mellitus, are most frequently seen in the lower extremities. The lifetime incidence of foot ulcers in diabetic individuals has been reported to be 34% [3,4]. Foot ulcers, many of which are considered to be within the definition of diabetic foot, can be seen in severity ranging from superficial ulcers to critically infected wounds that require amputation. Approximately 1 million people each year are treated with amputation for diabetes mellitus– related causes [5], which has a devastating effect on patients and their relatives, as well as a serious socioeconomic loss.

Diabetic foot has very negative effects on general health status and quality of life in patients [6,7]. In this disease process, which has an advanced morbidity potential, patients may need to seek information outside of the patient-physician relationship. Areas of technology use in daily health-care applications are gradually expanding. Use of the Internet as a source of information is increasing daily. This situation has caused the Internet to become an important source of information today for individuals regarding their health status [8,9,10,11].

It is possible that this group of patients, who may undergo traumatic problems, such as fear of limb loss and psychosocial influence, may seek information and alternative suggestions on the Internet. We aimed to specify the parameters that may affect the content and quality of the information obtained through Internet search engines about the diabetic foot.

Materials and Methods

A flowchart was created to investigate the information content, readability, and certification status of Web sites related to the subject of diabetic foot (Figure 1). Hence, first, the keywords related to diabetic foot were chosen. Because this was a Web-based study, it was decided to define the related queries about diabetic foot as keyword/keywords in search engine/engines.

The Process of Determining the Related Queries

To choose the Web site/sites to determine the keyword/keywords, the search engines were investigated. The market share was determined to decide on the search engine/engines. The top five search engines (Google, Bing, Yahoo, Baidu, and Yandex) with the highest market share today have been active since June 2009. The market share of all of the search engines worldwide was evaluated between June 2009 and December 2020 [12]. The search engine that seemed to have the highest market share at all times for this period was Google. Thus, it was decided to examine the data of Google Trends to determine the related queries as keywords.

Inclusion and Exclusion Criteria for Determining the Keywords Within the Related Queries

The related queries were determined at the Google Trends Web site by searching for diabetic foot (

Table 1. Related Queries for Google Trends Search Related to Diabetic Foot and the Ratio of Search Volumes Compared with Diabetic Foot.

Abbreviation: ICD-10, International Classification of Diseases, 10th Revision. ^aSelected keywords that met the specified criteria.

). Of these related queries, those that did not contain the words diabetic and foot simultaneously (eg, diabetic ulcer, foot ulcer, foot care, foot pain, diabetes, foot ulcers, diabetic ulcers, foot neuropathy, diabetic neuropathy, and neuropathy), which led to broader meanings, were excluded from this study. In addition, because the phrases that concern health-care professionals, such as the International Classification of Diseases (ICD) phrase, were not suitable for general diabetic patient searches (ICD-10 diabetic foot, diabetic foot ulcer ICD-10), repetition of a different related query with the plural (diabetic foot ulcers), related queries containing the (the diabetic foot), and synonyms (diabetes foot, diabetic feet) were also excluded.

Nine of 25 related queries met the specified criteria. The three most frequently searched related queries—diabetic foot ulcer, diabetic foot care, and diabetic foot pain—were found to represent 63.31% of the total search volume of nine related queries. Thus, these three related queries and the main idea of this study, diabetic foot, were accepted as keywords of the search (Figure 2).

Determining Which Search Engines and Web Sites to Evaluate

Google, Bing, and Yahoo were chosen for this study because they are the most preferred search engines in English-speaking countries [13,14]. After each search engine’s history and single sign-on settings were reset on January 19, 2021, a search was performed to show ten results per page. Because the results on the first pages were known to be preferred [15] and to use a similar method as in previous medical studies [16,17], 30 results for each keyword were determined and recorded, including the first three pages.

Microsoft Edge, Version 86.0.622.69 (Official build) (64-bit) (Microsoft Corp, Redmond, Washington) was used for Bing and Yahoo searches. The search engine optimization effect was fixed before the search by resetting the search history for each browser. First, the search history and temporary files were deleted, and the language was divided into English. Then, keywords were typed into the search engine. Calls were made and recorded on January 19, 2021.

Inclusion and Exclusion Criteria for Determining the Web Sites

In evaluating the first 30 results in each search engine, we excluded repetitive results, studies containing insufficient text (Web sites containing <300 words), sites with advertisements, and sites with only video content, although they were not specified in the search engine result. In determining the word count, the number of words in the items in the information content scale (ICS) was considered. Because the comprehensibility of sentences with eight words or less was almost complete [12], it was calculated that 344 words would be required if each item was expressed with a single sentence in the 43-item ICS. Thus, it was decided that the minimum number of words in the expression to be included in the plain text should be approximately 300.

Categorizing the Web Sites

Primarily, the given Web sites as a result of search engines were categorized as previous studies that were related to medical searches on the Internet [13]. Accordingly, the sites were divided into five categories: nongovernmental health institution Web sites, governmental institution Web sites, academic Web sites, blogs, and university Web sites.

Determining Evaluation of Information Content of the Web Sites Related to Diabetic Foot

An ICS was created to make an evaluation similar to previous studies [18,19,20,21]. While creating this ICS, it was decided to use the information contained in the “Diabetic Foot” chapter of Surgery of the Foot and Ankle as a basic reference source to determine the basic information about the diabetic foot [22]. In the light of these data, a table with 43 points was prepared, including three subtitles (diagnosis and evaluation, treatment and rehabilitation, complications and outcome) (

Table 2. Information Content Scale with Subtitles.

). Evaluations were made by the two of us.

Determining the Quality and Readability of the Web Sites

Presence of the Health On the Net Foundation Code of Conduct (HONcode) certificate on the Web site, Journal of the American Medical Association (JAMA) benchmark criteria, and the Flesch-Kincaid readability test were used to evaluate the quality and readability of the Web sites.

Whether the Web sites had a HONcode certificate was evaluated and recorded by a single evaluator using the 2020 Health On the Net official site search engine [23].

The JAMA benchmark criteria [14] were also evaluated by a single evaluator. This assessment consisted of four categories to reveal whether a source of information was credible: 1) indication of the author, 2) indication of the source of information, references, 3) date of update, and 4) sponsorship, advertising policy, conflict of interest statement.

The text content of the Web sites was evaluated using the Flesch-Kincaid readability test. Text content was transferred to Microsoft Word (Microsoft Office 2010 Proofing Tools, v14.0.7263.5000, 64 bit; Microsoft Corp) for evaluation. Flesch-Kincaid [15,16] readability grade level was calculated with the total number of words, the total number of sentences, and the total number of syllables:

Statistical Analysis

Descriptive analyses were performed to provide information about the general characteristics of the study groups. Continuous variables are given as mean 6 SD; categorical variables are given as number (percentage). Comparing the means of quantitative variables between groups, independent samples t test and one-way analysis of variance were used. Paired sample t test was used for repeated measurements. Cross tables and X² tests were used to evaluate whether there was a relationship between qualitative variables. A P < .05 was considered statistically significant. A statistical software program (IBM SPSS Statistics for Windows, Version 22.0; IBM Corp, Armonk, New York) was used for calculations.

Comparing the Information Content, Quality, and Readability of the Web Sites

It was evaluated whether there was a significant difference between the categories of the Web Sites, selected keyword(s), and selected search engines concerning JAMA benchmark criteria, HONcode certification, readability, and information content. Significance of difference between two means and one-way analysis of variance were used for all evaluations.

Results

For four search words, an evaluation was made with 276 Web sites with appropriate content from 360 results Web pages obtained from three different search engines.

After evaluating the distribution of the number of Web sites accepted for the study according to search engines, it was observed that there was a significant difference between search engines concerning whether the obtained results were suitable for evaluation (P < .005). The Google search engine offered more relevant results than did Bing and Yahoo, but there was no significant difference between Bing and Yahoo (

Table 3. Distribution of the Number of Web Sites Accepted for this Study by Search Engine.

^aThe significant differences between the two means were used. ^bStatistically significant (P < .05).

).

There was no significant difference in the comparison between search engines except that the JAMA benchmark criteria were significantly higher only in the Yahoo search engine compared with the Google search engine (

Table 4. Evaluating the Differences in the Obtained Data Between Search Engines and Evaluators.

Note: Data are given as mean 6 SD. Common letters (ab) on a line express statistical nonsignificance. Abbreviations: ICS, information content scale; JAMA, Journal of the American Medical Association. ^aComparison between groups (one-way analysis of variance). ^bStatistically significant (P < .05). ^cIntragroup comparison (significance test of the difference between two observers).

).

Concerning the ICS, it was observed that there were no significant differences between search engines except for the Bing search engine treatment and rehabilitation subtitle (P = .02) (Table 4).

Regarding the information content, quality, and readability criteria according to the keywords, the Bing search engine statistically showed no significant difference, but the Google search engine statistically significantly showed a difference in all of the criteria except the JAMA benchmark criteria (

Table 5. Analysis of the Results Obtained According to the Selected Keywords.

Note: Data are given as mean 6 SD. Common letters (ab) on a line express statistical nonsignificance. Abbreviations: ICS, information content scale; JAMA, Journal of the American Medical Association. ^aComparison between groups (one-way analysis of variance). ^bStatistically significant (P < .05). ^cIntragroup comparison (significance test of the difference between two observers).

). The Yahoo search engine showed a statistically significant difference for ICS treatment and rehabilitation, ICS complications and outcome, and total ICS scores (Table 5). It was observed that there was no statistically significant difference in the evaluation of the validity between the evaluators (Table 5, P₂ values).

Web sites with HONcode certification had a statistically significantly higher JAMA benchmark score, but there were no statistically significant differences concerning other criteria (

Table 6. Distribution of Variables According to HONcode Certification.

Note: Data are given as mean 6 SD. Common letters (ab) on a line express statistical nonsignificance. Abbreviations: HONcode, Health On the Net Foundation Code of Conduct; ICS, information content scale; JAMA, Journal of the American Medical Association. ^aComparison between groups (one-way analysis of variance). ^bStatistically significant (P < .05). ^cIntragroup comparison (significance test of the difference between two observers).

).

After categorization, the distribution of Web sites was as follows: 172 nongovernmental health institutions, 38 governmental institutions, 36 academic, 26 blogs, and four university Web sites (

Table 7. Quantitative Variable Distribution According to Web Site Categorization.

Note: Data are given as mean 6 SD. One-way analysis of variance was used. Common letters (abc) on a line express statistical nonsignificance. Abbreviations: ICS, information content scale; JAMA, Journal of the American Medical Association. ^aStatistically significant (P < .05).

). Comparing the Web sites concerning ICS diagnosis and evaluation and ICS total score, academic Web sites scored significantly higher than other Web site categories (P < .005). Academic sites were also found to have higher readability grades than the nongovernmental health institution and governmental institution groups (P < .005). Nongovernmental health sites had significantly higher readability grades than the governmental institution sites (P < .005). Blog sites had significantly higher readability grades than official sites (P < .005).

Discussion

This study focuses on Web sites related to the diabetic foot regarding information content, readability, quality of information, and medical certification. Although there is no statistically significant difference between search engine choice regarding information content, readability, quality of information, and medical certification, Web site type and keyword choice differences have led to variable effects.

The main result of the present study is that the choice of search engine did not show a statistically significant difference in the readability, information content, and medical certification in Internet searches related to the diabetic foot. In addition, the search validity of the Google search engine was statistically significantly high. In the literature, in studies that evaluate the information quality of specific medical subjects, usually a single information pool is obtained using all of the search engines [13,17,18,19,20,21,22,23,24], so there are relatively few studies [25,26] that offer a similar comparison. Wang et al [26] rated the results they obtained after the breast cancer search insert with eight volunteer raters and compared four different search engines. When the data obtained after this comparison were evaluated, it was seen that the search engine with the highest search validity was Google, and the highest user satisfaction was Bing. The higher search validity of the Google search engine in the present study is similar to that of the study by Wang et al. Given that use of the Google search engine has continuously increased since publication of the studies, this difference can be considered normal.

Another result of the present study seems to be the effects of Web site subtype on information content, readability, and JAMA benchmark scores. Although this change has variable effects and several dimensions, it seems that it causes a change more remarkably than other variables. Given that academic Web sites show a significant elevation in ICS total and ICS diagnosis and evaluation subtitles make us think that such sites can provide more information to the reader.

Griffiths et al [27] evaluated 50 Web sites obtained from three different search engines with the keyword robotic-assisted total knee. As a grade of their study, they found that academic sites had higher information content scores, similar to the present study. There are also studies in the literature in which academic Web sites do not have the best performance. Shen et al [13] focused on patient questions about total joint arthroplasty, evaluated Web sites in six categories (commercial, academic, medical practice, single surgeon, government, social media), and revealed that government and commercial sites had the highest JAMA benchmark scores.

Another finding of the present study, which is that academic sites have higher readability grades, more detailed explanations, and higher information content, can be considered a normal finding. It made us think that the evaluation scale we developed specific to the present study was suitable for validity between users. Although such a comparison was not made in studies conducted using similar evaluation scales [20,21,22] in the literature, we found it appropriate to present such a comparison to keep the errors caused by the evaluation difference between the users to a minimum.

In the comparison in which the results returned by the selected search word in the search engine were evaluated, there was no significant change in the results obtained with the Bing search engine and the word selection, but there were changes in all of the criteria except the JAMA benchmark criteria with the word selection in the Google search engine.

Similar to the result of the present study, many studies in the literature reported that the importance of keyword selection was emphasized and that richer content could be obtained with different keywords. Johnson et al [23] studied Tommy John injuries, and Dy et al studied distal radius fractures [28] and lateral epicondylitis [18]; they all obtained different results in content, quality, and readability with the different selected keywords.

Although the difference in the results with the selected keywords in the present study varies by search engine, it is an important and distinct finding of the present study and draws attention as a generally accepted fact supported by different studies in the literature [18,23,28].

Another result of the present study is that HONcode certification does not cause a statistically significant difference, except for the JAMA benchmark criteria, which has similar subevaluation criteria. This finding suggests that the current certification is not sufficient to reflect the information content of the diabetic foot search results.

O’Neill et al [21] noted that the sites with HONcode certification for cauda equina syndrome have higher DISCERN and JAMA benchmark scores. Their study differs from the present study in this aspect, and this difference may be because the search term diabetic foot represents a more general medical problem that is more sought after and seen more frequently in society.

The main limitation of the present study is the small number of evaluators. Note that this limitation does not adversely affect the results of the study because the ICS evaluation has similar results in the interpersonal comparison, and the two evaluators performed only the ICS evaluation together.

Today, the Internet, which plays an important role in social medical awareness, will continue to be a crucial source of information for patients. The method of obtaining information about diabetic foot with search engines is affected by many parameters. Internet search results for diabetic foot may lead to lack of information and inequality in this sensitive patient group because of different readability, information content, and information quality rates.

Conclusions

The results that can be obtained using the Internet regarding the diabetic foot problem depend on many factors: the proper keyword selection, Web site type, and search engine can help patients reach more appropriate content.