Abstract
The Health Online Service Provision Index (HOSPI) is an instrument to assess and monitor hospitals’ websites. The index comprises four criteria—Content, Services, Community Interaction and Technology Features—each with a subset of indicators and sub-indicators. HOSPI was applied to the Portuguese hospitals’ websites in 2023, originating the dataset described in this article. The article also provides a detailed account of the data collection process, which involved direct observation of the websites and specific treatment methods, ensuring the reliability and validity of the dataset. It underscores the relevance of having this data available and how it can improve service provision online in health facilities and support policymaking.
Dataset: https://doi.org/10.34622/datarepositorium/ENHJQZ (accessed on 18 November 2024).
Dataset License: CC0
1. Introduction
Hospital websites have become critical infrastructures for healthcare service delivery and patient communication in the digital age. As healthcare institutions increasingly leverage online platforms to engage with patients, provide services, and disseminate information, the systematic evaluation of these digital interfaces has become paramount. This dataset presents a comprehensive assessment of Portuguese hospital websites in 2023 using the Health Online Service Provision Index (HOSPI), a methodological framework developed and maintained by the United Nations University Operating Unit on Policy-Driven Electronic Governance (UNU-EGOV).
The significance of this data collection effort is underscored by several key factors. First, it aligns with the United Nations Sustainable Development Goals (SDGs), particularly SDG 3 (ensuring healthy lives and promoting well-being) and SDG 16 (promoting strong institutions). Second, it addresses the growing trend of patients utilizing healthcare websites for information gathering, treatment research, and healthcare provider selection [1]. Third, it responds to the European Commission’s long-standing policy priorities regarding information society development, from the eEurope initiative through the current EU2020 Strategy [2,3,4].
Communication dissemination by hospitals must consistently make use of their online presence, counting on professional websites that convey useful, timely, and correct information [5] while also offering personalized services to their patients. There has been a growing effort from health institutions to design strategies for the implementation of more robust electronic services [6] to increase patient satisfaction and strengthen trust in health sector institutions [7]. Healthcare institutes’ websites can facilitate better patient-centred care via improving healthcare procedures, clinical results provision, patients’ requirements and preferences satisfaction, collaborative decision-making, interaction between doctors and patients, and access to medical data [8,9,10]. This raises efficiency, reduces medical errors, improves disease management, and decreases healthcare system load, reducing hospital overcrowding and inappropriate health interventions.
The development of websites that better serve patients’ needs is thus a key element in the evolution of hospital services provided through the internet. To deliver the expected level of service, it is important to understand how users perceive and evaluate health sector web presence.
This dataset provides a unique and comprehensive evaluation of hospital websites based on scientifically derived indicators. It offers three primary benefits: (1) it provides hospital management with aggregated information for service improvement, (2) it enables patients to make informed comparisons between healthcare providers, and (3) it assists policymakers in monitoring and developing eHealth policies. The data encompass multiple dimensions of website assessment, including inclusivity, usability, transparency, and service provision capabilities.
HOSPI offers hospital management aggregated and scientifically sound information to improve provided services, it offers patients the possibility to compare and select health service providers, and, finally, it assists health sector policymakers in monitoring and further developing eHealth policy. It is built on identifying relative evaluation dimensions and conceived measurements of health sector websites’ services.
The remainder of this paper is organized as follows. Section 2 describes the studies that influenced HOSPI development. Section 3 presents data description related to HOSPI instrument data collection. Section 4 describes the methodology. Finally, Section 5 discusses the relevance of such an instrument and data collection, and it concludes the paper with limitations of the study and suggestions for future research.
2. Related Work
HOSPI is the result of the development of an instrument that compiles a set of indicators used to evaluate hospitals’ websites [11]. More than 20 frameworks and methods for health sector websites have been reviewed, and their suggestions have been used in HOSPI design [11].
Some authors have looked into multiple dimensions of hospital websites. For instance, Llinás et al. [12] study and assess the user-orientation of American, British, and Spanish hospital websites. They assess websites according to readability, accessibility, and the quality of information provided. Similarly, Huerta et al. [13] evaluate the web presence of hospitals and their health systems based on five criteria, accessibility, content, marketing, technology, and usability. Randeree and Rao [14] examine access/usability, audience, accuracy, timeliness, content, authority, and security. The research of Moslehifar et al. [15] centres on four different categories: general information, accessibility of websites, functionality of websites, and facilities information provided in websites, whilst the suggested schema of Patsioura et al. [16] focuses on three main dimensions: information, communication, and electronic services.
Several studies focus on specialized health areas, as is the case of Lewiecki et al. [17], that exploit and assess measurement tools to measure the quality of osteoporosis websites for patients. They use indicators in the areas of content, credibility, navigability, currency, and readability. Also, Guardiola-Wanden-Berghe et al. [18] conducted an observational, descriptive and cross-sectional study implemented using systematic analysis techniques that focuses on evaluation of documentary and content quality assessment of eating disorder websites. And Rezniczek et al. [19] assess the quality of websites of obstetrics and gynecology departments in German-speaking countries using Google search rank, technical aspects, navigation, and content as objective criteria.
Different approaches have been used by Moreno et al. [20] that suggest a qualitative and user-oriented method for evaluating quality of health-related websites based on a 2-tuple fuzzy linguistic approach, and Bilsel et al. [21] present a quality assessment framework, which comprises seven major e-service quality elements, including tangibles, reliability, responsiveness, confidence, empathy, quality of information, and integration of communication issues of websites.
There are studies focused on specific countries. Maifredi et al. [22] examine the attributes of the contents and the user-orientation of Italian hospital websites. Norum [23] assesses the quality of Norwegian cancer hospitals’ Websites according to general information, hospital details, and technical aspects. Mira et al. [24] focus on the readability and accessibility of Spanish hospital websites and conclude that they need to be more patient oriented because the websites visited did not fulfil even half of the readability and accessibility attributes required by widely used standards. Also in Spain, Calvo [25] evaluates the quality and illustrates features of websites of large hospitals evaluating the global quality, accessibility, usability, interactivity, updating, quality model, and information. Mancini et al. [26] discovered that the enforcement of accessibility regulations has helped to significantly improve hospital website accessibility in Italy.
Additional factors in website assessment can be found in other research such as the ones conducted by Garcia-Lacalle et al. [27] that specify which factors have an influence on website adoption and level of development over time, or Gruca & Wakefield [28] that assess the status of US hospital websites examining the following features: electronic documents, providing decision aids, linkages to partners, building trust via external verification, facilitating transactions, multiparty targeting, self-service information, and discussion forums. Liu et al. [29], focus on the assessment of the quality of hospital websites in China using a preconcerted objective criterion based on content, function, design, and management and usage. Finally, Tsai and Chai [30] produced an assessment questionnaire for nursing websites covering overall impression, download and switch speed, accessibility and convenience, web page content, and compatibility with common browsers.
All these instruments and techniques were analysed for HOSPI development with the intention to be generally applicable to any hospital website. It encompasses four criteria: content, services, community interaction, and technology features.
3. Data Description
This section describes the data available in the dataset files in open format [31]. The dataset includes three CSV files and one Excel (XLSX) file aggregating the data in the three CSVs. The three CSV files are the information about the dataset, the metadata file, and the data file. As data was stored in tabular form, each column of the data table will be addressed, focusing on what data is contained, its format, and how to read and interpret it.
Data was collected in the application of the HOSPI instrument in Portugal in 2023 in which 102 hospitals were assessed to determine the state of online service provision. HOSPI is divided into four criteria, as shown in Table 1. Each criterion comprises a set of indicators that further encompass sub-indicators. The latter are the items being assessed and to which the data is available.
Table 1.
HOSPI criteria and indicators.
More detailed information about the dataset and how it is structured can be found in Table 2. All columns are mandatory throughout the dataset. Special conditions exist for some of the sub-indicators, but information regarding these rules is presented when needed.
Table 2.
Information about the dataset.
Table 3 presents the information about the data collected in the preparation phase. All items are mandatory, and the type of hospital must be one of the options presented in the Data Values column.
Table 3.
Metadata of the preparation phase.
The rest of the dataset metadata is divided into four tables for readability purposes, presented in the Appendix A, one for each criterion. These tables contain the column name, a short description of the sub-indicator that helps to understand what is being assessed, the possible data values for that sub-indicator, and in what conditions it must be different from NA.
Criterion 1 information is presented in Table A1. All items are mandatory in this criterion, but private hospitals can receive value NA in some of the sub-indicators as these relate to information that is mandatory by law only to public hospitals. Indicator C1.i5—Research and/or Teaching is mandatory only for hospitals identified as being part of research and teaching institutions. All other hospitals can receive value NA in this indicator.
This criterion evaluates the presence of information relevant to the user. It evaluates the quality, availability, relevance, completeness, and concise representation of specific information that it is expected to be provided in a hospital website. Content criterion includes five indicators: Health institution information available on the website (C1.i1), Quality Metrics (C1.i2), Organisational Structure and Medical Information (C1.i3), Patient Information (C1.i4), and Research and/or Teaching (C1.i5).
Table A2 presents the information for criterion 2—Services. All items are mandatory for all types of hospitals in this criterion, which means the possible data values are only 1 and 0. This criterion considers how the growth of consumerism and the proliferation of internet-accessible sources of health-related information have modified the traditional roles of provider and patient. The trend towards providing personalized electronic services can bring many benefits to both hospitals and patients. Personalized content can be provided during interactions with all users, and this might improve loyalty to a particular hospital. This criterion includes electronic healthcare scheduling, prescription request, automation of hospital’s back-office procedures, forms availability on the website, electronic completion of administrative transactions, and online appointments. Services criterion is assessed in three indicators: Administration Procedures (C2.i1), Appointments (C2.i2), and Patient Care (C2.i3).
Criterion 3 metadata is detailed in Table A3. Some sub-indicators may receive value NA depending on the value attributed to an antecedent sub-indicator. Details for these cases are presented in the “Conditions to be different from NA”. This criterion describes the interaction between hospital, patients, and online communities on the web. Online communities often involve members to provide content to the website and contribute in some way. Examples of such involvement include forums, complaints forms, interaction with the media, and the hospital’s marketing activities. Hospital sites can host patient support groups, interact with community organisations, and become a portal for physician organisations and private medical offices. Community Interaction criterion includes three indicators: Participation (C3.i1), Media (C3.i2), and Advertising/Marketing (C3.i3)
Information for criterion 4—Technology Features is described in Table A4. Some sub-indicators may receive value NA depending on the existence of a specific feature as detailed in the “Conditions to be different from NA”. This criterion encompasses mainly technical items related to easy navigation, website quality, visual appeal, functionality, and reliability. The Technology Features criterion is related to how the content and services are assembled and made available on a website. Technology Features criterion includes five indicators: Navigability (C4.i1), Accessibility (C4.i2), Usability/Readability (C4.i3), Credibility (C4.i4), and Privacy/Security (C4.i5).
4. Methods
HOSPI was developed in 2017 based on an extensive literature review [31], test application, and expert group meetings [11]. It has been applied in different countries (Portugal, Greece, Malaysia, and Ecuador) since then and updated and improved with each application.
This application of the HOSPI instrument in Portugal took place from June to August 2023. A total of 147 hospitals were identified. After grouping private hospitals from the same group, which use the same website, and removing hospitals without a working website, a total of 102 websites were assessed.
4.1. Data Collection
Data collection is based on direct observation of the hospitals’ websites. The direct observation is conducted by a team of two assessors under the supervision of a third one, considered the supervisor, who is an expert on the HOSPI assessment process [32,33] and can resolve any discrepancies between the two assessors’ assessment results. This means that for each hospital website, there are two observations (one from each assessor), which will be analysed by the supervisor.
The assessors are instructed to assume, during the assessment, the logic and attitude that would typically be of an average citizen/user/patient when navigating the website. This means that the effort put into the search for the assessment sub-indicators should be similar to the one performed, on average, by a citizen while using the website and not an exaggerated and extensive effort. Although it is possible to meticulously look for certain elements and find them on a website, this does not seem the best approach because it disregards the fact that the user needs to be able to find what he wants quickly and intuitively, assuring the website is useful and effectively used.
The employed assessors have the typical characteristics of a hospital website user. They should be able to navigate the website into different segments and able to understand HOSPI guidelines. The assessors must be able to clearly document their findings based on the HOSPI methodology. The supervisor (expert) should be an expert in HOSPI methodology and should be able to answer any questions from the assessors and resolve any discrepancies in assessors’ assessments. Assessors have the guidance of the methodology guide [34] and of the supervisor to clarify any doubt during the assessment.
The initial step in preparation for the data collection phase is to identify the complete list of Portuguese hospitals. This is retrieved from the health authorities’ websites and from national associations of private hospitals. The identification of each hospital website URL (link) is made simultaneously with this step. Another relevant aspect that is collected is the classification of each hospital as public or private and whether it is a research and teaching hospital. This particular information is available in the Health Ministry website [35].
After having all this information, the assessment starts through direct observation of the set of criteria, indicators, and sub-indicators. During the stage of data collection, value 1 is ascribed to the presence of the considered sub-indicator, value 0 to its absence, and the acronym ‘NA’ if it is not applicable.
4.2. Data Treatment
The assessment team supervisor is responsible for attributing a single value to each sub-indicator. In cases when the two assessors assign different values to a specific sub-indicator, this is signalled to them to be reassessed more thoroughly. In case the assessment discrepancy remains, the supervisor decides, upon website analysis, which value will be assigned to the sub-indicator. In cases when the assessors were not able to assess a sub-indicator or when only one assessor was able to make the assessment of a sub-indicator, the supervisor should decide, upon website observation, which value to attribute to that sub-indicator.
4.3. Index Calculation
After data collection and validation, the assessment index (iHOSPI) is computed, taking into consideration the value registered for each sub-indicator as well as the weights defined for each criterion and indicator as defined in [36]. A public report is then prepared, presenting the main global results, results per criterion, and other data segmentation analysis relevant for the Portuguese context [37].
5. Discussion and Conclusions
Assessment of hospital website services requires indicators that incorporate various types of information and aggregately measure the organization’s e-Health activity. The design of HOSPI accomplishes this aim: it provides to hospital management aggregated and at the same time scientifically sound information to advance their provided services, it offers to patients the possibility to compare and select health service provider, and finally, it assists health sector officials to monitor and further develop eHealth policy.
The instrument presented in this study advances current eHealth research on extending the knowledge of health sector websites assessment dimensions. It propels health sector e-Government research to develop guidelines for website development. More specifically the present work does the following:
- Contributes to e-Health and e-Government literature in exploring ways to further integrate more functionalities, provide more services, and improve usability and accessibility in health sector websites.
- Provides a comparison between hospital websites
- Raises awareness of the importance of the assessment methods to counter the perception that they are unduly expensive or time-consuming.
- Establishes performance benchmarks by which to measure the effectiveness and efficiency of health sector websites.
This study has significant potential to contribute to the policy and management of Portugal’s health service provision sector. Further studies can use the data coupled with patients’ needs, hospital objectives, and health sector policies to guide governments and health institutes in their policy-making decisions and action plan design.
The provided data and the values of the indicators can serve as a benchmark for gauging the efficiency of hospital e-health programs in Portugal. This procedure can be reproduced worldwide by altering the origin of hospitals. Furthermore, the HOSPI instrument could provide references for future research about designing relative assessment models and data collection processes.
Although our study covers a wide range of hospital websites, contributing to the representativeness of the data and its utility for future research, it has a few limitations:
- We have identified 147 hospitals in Portugal, and 102 websites were assessed. The missing data of 45 hospitals (because of hospital grouping or not functional website during the assessment) may contain information about specific institutes that were not included in the research scope, which may lead to some bias in the analysis and conclusions. In subsequent research, we will reconsider the missing hospitals to collect further relevant data. The 102 hospitals covered in this study are distributed across all Portuguese provinces, and the coverage of these data ensures the representativeness and reliability of the research results, providing a solid foundation for in-depth exploration of hospital website diversity in Portugal.
- The hospital website’s features will change due to patients’ changing needs and technological evolution. The assessment data fields we compiled should be updated in the future.
- Additional data fields should be included to cover a broader range of hospitals, particularly on a global scale, so that high-quality research can be developed using our data model.
- Language used to describe each sub-indicator is not always clear to the assessors and demands attention from the supervisor during the assessment to make sure assessors have the same understanding of each sub-indicator. This demands constant improvement of descriptions.
- Our data model and data store are a dynamic resource that will be continually updated with new data. We encourage researchers to stay engaged with our work by following the link to our repository. This will ensure that researchers have access to the most current information and can contribute to the ongoing research in this field.
Author Contributions
Conceptualization, D.S. (Dimitrios Sarantis), D.S. (Delfina Soares) and J.C.; methodology, D.S. (Dimitrios Sarantis), D.S. (Delfina Soares) and J.C.; validation, D.S. (Dimitrios Sarantis), D.S. (Delfina Soares) and J.C.; formal analysis, D.S. (Dimitrios Sarantis), D.S. (Delfina Soares) and J.C.; investigation, D.S. (Dimitrios Sarantis), D.S. (Delfina Soares) and J.C.; data curation, J.C.; writing—original draft preparation, J.C. and D.S. (Dimitrios Sarantis); writing—review and editing, J.C. and D.S. (Dimitrios Sarantis); supervision, D.S. (Delfina Soares); project administration, D.S. (Dimitrios Sarantis). All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The resulting dataset is available at https://doi.org/10.34622/datarepositorium/ENHJQZ (accessed on 18 November 2024) as per the CC0—“Public Domain Dedication” License.
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A
Table A1.
Metadata about criterion 1—Content.
Table A1.
Metadata about criterion 1—Content.
| Column Name | Description | Data Values | Condition to Be Different from NA |
|---|---|---|---|
| C1.i1 Health institution information available on the website | |||
| C1.i1.1 Institution name on the page header | Existence of the institution’s name on webpage header | List: [1,0] | |
| C1.i1.2 Institution logo on the page header | Existence of the institution’s logo on webpage header | List: [1,0] | |
| C1.i1.3 Welcome message | Existence of welcome message in the webpage | List: [1,0] | |
| C1.i1.4 Institution postal address | Existence of institution’s postal address in the webpage | List: [1,0] | |
| C1.i1.5 Institution telephone and/or fax number | Existence of the institution’s telephone and/or fax number in the webpage | List: [1,0] | |
| C1.i1.6 Institution e-mail address | Existence of the institution’s e-mail address in the webpage | List: [1,0] | |
| C1.i1.7 Institution VAT number | Existence of the institution’s VAT number in the webpage | List: [1,0] | |
| C1.i1.8 Map of the hospital area (including parking lots) | Existence of the map of the hospital area (including parking lots) in the webpage | List: [1,0] | |
| C1.i1.9 Ways of reaching the hospital: private and public transportation | Description of ways of reaching the institution location by car or public transportation in the webpage | List: [1,0] | |
| C1.i1.10 Complementary services: press, cafeteria, television, telephone, parking, religious service | Presentation of hospital’s complementary services and facilities (e.g., press, cafeteria, television, telephone, parking, religious service) in the webpage | List: [1,0] | |
| C1.i1.11 Phone directory | Existence of an institution’s phone directory in the webpage. | List: [1,0] | |
| C1.i1.12 Institution history | Presentation of the institution’s historical information on the webpage | List: [1,0] | |
| C1.i1.13 Statement of purpose (Mission, Vision, Values) | Presentation of the institution’s statement of purpose (Mission, Vision, Values) in the webpage. Note: If just one of the information elements (mission, vision, or values) is presented, consider “1” | List: [1,0] | |
| C1.i1.14 Area covered by the hospital (population served) | Presentation of the geographical area that the hospital serves | List: [1,0,NA] | public hospital |
| C1.i1.15 Quality Management Certification (e.g., ISO, EFQM) | Presentation of any quality management certification (e.g., ISO, EFQM) granted to the hospital. | List: [1,0] | |
| C1.i1.16 Management Reports (This item may include Activities Plan, Budgets, Activities Reports, Account Reports, etc.) | It might be just a seal of certification | List: [1,0,NA] | public hospital |
| C1.i1.17 Public procurement: announcement information on the website | Presentation of any hospital management reports | List: [1,0,NA] | public hospital |
| C1.i1.18 Public procurement: connection to national procurement portal | Connection to and use of national portal data in public procurement processes (e.g., base.gov.pt for Portugal, eaadhsy.gr for Greece etc.) | List: [1,0,NA] | public hospital |
| C1.i1.19 Emergency information | Existence of medical emergency information in the webpage, namely information regarding what to do in case of emergency, the national emergency number, or how to access/contact the hospital emergency room if there is one | List: [1,0] | |
| C1.i1.20 Home hospitalization information | Existence of home hospitalization information (health care provision in the patient’s home) in the webpage | List: [1,0,NA] | public hospital |
| C1.i1.21 Applied legislation to the health institutions context | Existence of information regarding applied legislation to the health institution’s context (e.g., law decrees, laws). It may be just a mention of the law decree or link to the legislation | List: [1,0] | |
| C1.i2 Quality Metrics | |||
| C1.i2.1 Number of institution beds disclosed | Presentation of the total number of institution’s beds on the webpage. The numbers may be provided per service, globally, or any other way | List: [1,0] | |
| C1.i2.2 Waiting time consultation | Presentation of hospital’s waiting time for first consultation | List: [1,0] | |
| C1.i2.3 Waiting time surgery | Presentation of hospital’s waiting time to surgery | List: [1,0] | |
| C1.i2.4 Waiting time to be seen in the emergency room | Presentation of hospital’s waiting time to be seen in the emergency room | List: [1,0,NA] | public hospital |
| C1.i2.5 Date of last monitoring of the waiting list disclosed | Presentation of the date of last monitoring of the waiting list | List: [1,0] | |
| C1.i2.6 Institution report of the number of admissions in the previous year | Presentation of institution report of the number of admissions in the previous year | List: [1,0] | |
| C1.i2.7 Institution quality indicator: nosocomial infection rate disclosed | Presentation of nosocomial infection rate in the website | List: [1,0] | |
| C1.i2.8 Institution quality indicator: inpatient mortality rate disclosed | Presentation of inpatient mortality rate in the website | List: [1,0] | |
| C1.i2.9 Institution quality indicator: surgical mortality rate disclosed | Presentation of surgical mortality rate in the website | List: [1,0] | |
| C1.i2.10 Institution quality indicator: others | Presentation of other institution’s quality indicators (values or reports) | List: [1,0] | |
| C1.i2.11 Information on births per year | Presentation of information relative to births per year in the institution | List: [1,0] | |
| C1.i2.12 The website provides open data regarding patients and hospital practices | Presentation of administrative open data regarding patients and hospital practices. Open data is considered data in any file format (doc, xls) | List: [1,0] | |
| C1.i2.13 Results of surveys regarding patient satisfaction are provided | Presentation of results of surveys regarding patient satisfaction | List: [1,0] | |
| C1.i2.14 Number of internships accepted by the hospital each year | Presentation of the number of medical internships the hospital accepted each year | List: [1,0,NA] | public hospital |
| C1.i2.15 The website provides clinical open data | Existence of open data regarding patients and hospital practices available for research purposes. Open data is considered data in any file format (doc, xls) | List: [1,0] | |
| C1.i3 Organisational Structure and Medical Information | |||
| C1.i3.1 Organisation chart (medical management, nursing management, institution management) | Presentation of the hospital’s organisation chart (medical management, nursing management, institution management) | List: [1,0] | |
| C1.i3.2 Services charter | Presentation of hospital’s services charter. The services charter is a statement that clarifies the responsibilities of the hospital. | List: [1,0] | |
| C1.i3.3 Listing of services available at the institution | Presentation of all the services available in the hospital. Services that require admission (e.g.,: maternity, complex surgery, rehabilitation) to the hospital and outpatient services (e.g.,: consultation, diagnostic services, minor surgery, routine exams) | List: [1,0] | |
| C1.i3.4 Departments or units providing user services: complete list | Presentation of a complete list of the departments or units providing user services in the hospital | List: [1,0] | |
| C1.i3.5 Departments or units providing user services: location | Presentation of the location of the departments or units providing user services in the hospital | List: [1,0] | |
| C1.i3.6 Departments or units providing user services: telephone and/or fax number and/or e-mail address | Presentation of the telephone and/or fax number and/or e-mail address of the departments or units providing user services in the hospital | List: [1,0] | |
| C1.i3.7 Departments or units providing user services: working hours | Presentation of the working hours of the departments or units providing user services in the hospital | List: [1,0] | |
| C1.i3.8 Personnel map | Presentation hospital’s employees map (information about the number of employees on each position, level of expertise, job descriptions, types of contracts, etc.—not only for health professionals) | List: [1,0] | |
| C1.i3.9 List of employed doctors | Presentation of the list of hospital’s employed doctors. The list can either be organized by alphabetical order or by specialization | List: [1,0] | |
| C1.i3.10 Doctors’ curricula/information | Presentation of doctors’ curricula/information | List: [1,0] | |
| C1.i3.11 Photos of the medical team (physicians, nurses) available | Presentation of at least one individual photo of the medical team (physician, nurse) | List: [1,0] | |
| C1.i3.12 Head of service | Information regarding the head of each service/medical specialty | List: [1,0] | |
| C1.i3.13 Possibility to read online or to download health-care booklets | Provision of healthcare booklets in electronic form (online or download form). Booklet can also be any small informative text that is easily shared online. | List: [1,0] | |
| C1.i3.14 Medical glossary or Conditions and Treatments available | Provision of medical glossary or conditions and treatments in electronic form (online or download form). A medical glossary is a set of concepts and respective definitions regarding medical terms, diseases, treatments, etc. | List: [1,0] | |
| C1.i3.15 Interest Conflict declaration or Declaration of non-conflict is shown | The presence of Interest Conflict declaration or Declaration of non-conflict is verified | List: [1,0] | |
| C1.i4 Patient Information | |||
| C1.i4.1 Information regarding patient privacy | Existence of information regarding patient privacy policy. A privacy policy is a statement or a legal document that discloses some or all of the ways a hospital gathers, uses, discloses, and manages a patient’s or client’s data | List: [1,0] | |
| C1.i4.2 Patient’s rights and obligations | Existence of a statement of patient’s rights and obligations | List: [1,0] | |
| C1.i4.3 Admission guide: different types of admission are disclosed | Presentation of different types of admissions | List: [1,0] | |
| C1.i4.4 Admission guide: information and rules to be followed on admission | Presentation of information and rules to be followed on admission | List: [1,0] | |
| C1.i4.5 Admission guide: information and rules to be followed during the stay at the institution | Presentation of information and rules to be followed during the stay at the institution | List: [1,0] | |
| C1.i4.6 Admission guide: information and rules to be followed on discharge | Presentation of information and rules to be followed on discharge | List: [1,0] | |
| C1.i4.7 Admission guide: information and rules to be followed by visitors | Presentation of information and rules to be followed by visitors | List: [1,0] | |
| C1.i4.8 Information and procedure for obtaining a copy of the medical documentation | Presentation of information and procedure for obtaining a copy of the medical documentation | List: [1,0] | |
| C1.i4.9 Patient care service or unit: location | Presentation of every patient care service or unit location. These are services that can be done without overnight stay | List: [1,0] | |
| C1.i4.10 Patient care service or unit: business hours | Presentation of every patient care service or unit business hours. These are services that can be done without overnight stay | List: [1,0] | |
| C1.i4.11 Patient care service or unit: telephone and/or fax | Presentation of every patient care service or unit telephone and/or fax. These are services that can be done without overnight stay | List: [1,0] | |
| C1.i4.12 Details of how to pay charges or fees | Presentation of details of how to pay charges or fees. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C1.i4.13 List of consultations/services with fees available | Presentation of the list of consultations/services with the relative fees or information about payment | List: [1,0] | |
| C1.i4.14 Information for foreigners | Presentation of information on how to act for foreigners | List: [1,0] | |
| C1.i4.15 Affiliated insurance companies and other entities | Presentation of information about insurance companies and other entities that have agreements with the hospital | List: [1,0] | |
| C1.i4.16 Mobile App available | There is information about the availability of a mobile app where the user can interact with the hospital | List: [1,0] | |
| C1.i5 Research and/or Teaching | |||
| C1.i5.1 Scientific studies that the institution promotes or is involved in | Presentation of scientific studies that the institution promotes or is involved in | List: [1,0,NA] | research and teaching hospital |
| C1.i5.2 Publications of the institution | Presentation of a list of scientific publications of the institution | List: [1,0,NA] | research and teaching hospital |
| C1.i5.3 Undergraduate or postgraduate courses that are held at the institution | Presentation of undergraduate or postgraduate courses that are held at the institution. If the hospital states on the website that it has/offers graduate and/or postgraduate courses, it should be assessed as a teaching hospital | List: [1,0,NA] | research and teaching hospital |
| C1.i5.4 Schedule of activities that take place at the institution: courses, congresses and conferences | Presentation of activities scheduled to take place at the institution (e.g., courses, congresses, conferences, etc.) | List: [1,0,NA] | research and teaching hospital |
| C1.i5.5 Existence of a library | Presentation of institute’s library | List: [1,0,NA] | research and teaching hospital |
| C1.i5.6 Library: address | Existence of institute’s library address | List: [1,0,NA] | research and teaching hospital |
| C1.i5.7 Library: business hours | Existence of institute’s library business hours | List: [1,0,NA] | research and teaching hospital |
| C1.i5.8 Library: publications catalogue | Existence of institute’s library publications catalogue | List: [1,0,NA] | research and teaching hospital |
| C1.i5.9 Library: services available, such as reading hours, loans, copies | Presentation of institute’s library available services (reading hours, loans, copies, etc.) | List: [1,0,NA] | research and teaching hospital |
Table A2.
Metadata about Criterion 2—Services.
Table A2.
Metadata about Criterion 2—Services.
| Column Name | Description | Data Values | Condition to Be Different from NA |
|---|---|---|---|
| C2.i1 Administration Procedures | |||
| C2.i1.1 Provision of online forms | Availability of forms to be filled online in the website | List: [1,0] | |
| C2.i1.2 Possibility of forms downloading | Availability of forms to be downloaded online in the website | List: [1,0] | |
| C2.i1.3 Possibility of filled forms uploading | Possibility to upload already filled forms to the website | List: [1,0] | |
| C2.i1.4 Online payment | Possibility of the user to make an electronic payment using any service that allows for online payment. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i2 Appointments | |||
| C2.i2.1 Manage medical examination via web | The website provides the possibility for the user to manage his/her medical examinations online. Management activities may include scheduling a medical examination, making a date/hour alteration, or consulting the date/hour of the appointment. This action may only be available within a private area or in the national health portal. Note: if the website states clearly that the option is available within a private area, “1” should be chosen. Note1: the value is “1” if it is possible to, at least, change an appointment, schedule an appointment, or consult and change the scheduled appointment. | List: [1,0] | |
| C2.i2.2 Manage admission via web | “The website provides the possibility for the user to manage his/her admission to the hospital online. Management activities may include scheduling an admission, making a date/hour alteration, or consulting the date/hour of the admission. This action may only be available within a private area or in the national health portal. “1”–if it exists “0”–if it does not exist. Note: if the website states clearly that the option is available within a private area, “1” should be chosen. Note1: the value is “1” only if it is possible to, at least, consult and to change the scheduled admission. | List: [1,0] | |
| C2.i2.3 Manage visits to outpatient consulting rooms via web | The website provides the possibility for the user to manage his/her visit to outpatient consulting rooms online. Management activities may include scheduling a consult, making a date/hour alteration, or consulting the date/hour of the appointment. This action may only be available within a private area or in the national health portal. Note: if the website states clearly that the option is available within a private area, “1” should be chosen. Note1: the value is “1” only if it is possible to, at least, consult and to change the scheduled consult. | List: [1,0] | |
| C2.i3 Patient Care | |||
| C2.i3.1 Asynchronous communication with the doctor via message exchange system | The website provides the possibility for the user to have an asynchronous communication with the doctor via message exchange system. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i3.2 Asynchronous communication with the doctor via e-mail | The website provides the possibility for the user to have an asynchronous communication with the doctor via email. It can be by making the doctors’ email address available or within a private area. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i3.3 Synchronous communication with interactive communication tool (chat with a doctor) | The website provides the possibility to the user of online synchronous communication with interactive communication tool with the doctor. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i3.4 Electronic directory with patient’s records | Provision of electronic directory with patient’s records. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i3.5 Provision of telemedicine (video-conference system) services | The website provides information about telemedicine (video-conference system) services. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i3.6 Patient telemonitoring (e.g., specific vital signs, blood glucose, peak flow rate, blood/urine chemistry) | The website provides information about patient telemonitoring services (e.g., specific vital signs, blood glucose, peak flow rate, blood/urine chemistry) is provided. Note: if the website states clearly that the option is available within a private area, “1” should be chosen | List: [1,0] | |
| C2.i3.7 Private area access: with login and password | The website provides the possibility to login in a private area using login and password | List: [1,0] | |
| C2.i3.8 Private area access: with Citizen Card or Mobile Digital Key | The website provides the possibility to login in a private area using a citizen card or a mobile digital key | List: [1,0] | |
| C2.i3.9 Possibility to require and/or obtain medical prescription | The website allows the user to request and/or obtain a medical prescription. | List: [1,0] | |
Table A3.
Metadata about Criterion 3—Community Information.
Table A3.
Metadata about Criterion 3—Community Information.
| Column Name | Description | Data Values | Condition to Be Different from NA |
|---|---|---|---|
| C3.i1 Participation | |||
| C3.i1.1 Suggestions via web | The possibility of suggestions submission via web is provided. It can be made either via email or web form | List: [1,0] | |
| C3.i1.2 Complaints via web | The possibility of complaints submission via web is provided. It can be made either via email or web form | List: [1,0] | |
| C3.i1.3 Information request via web | The possibility of information request via web is provided. It can be made either via email or web form | List: [1,0] | |
| C3.i1.4 Communication with the institution via chat | The possibility of communication with the institution via chat is provided | List: [1,0] | |
| C3.i1.5 Discussion forum or blog | Provision of discussion forum or blog | List: [1,0] | |
| C3.i1.6 If the website has a forum or a blog, it is possible to identify the administrator | The forum or blog administrator is identified | List: [1,0,NA] | blog or forum was identified in C3.i1.5 |
| C3.i1.7 Newsletter available | Provision of a hospital newsletter | List: [1,0] | |
| C3.i1.8 Opinion polls available | Provision of opinion polls | List: [1,0] | |
| C3.i1.9 FAQ | Existence of frequently asked questions section | List: [1,0] | |
| C3.i1.10 Associations that work at the institution: voluntary associations (social responsibility) | Provision of information about voluntary associations that work at the institution | List: [1,0] | |
| C3.i1.11 Associations that work at the institution: patient associations | Provision of information about patient associations that work at the institution | List: [1,0] | |
| C3.i1.12 Associations or offices at the institution for the defense of patients’ rights | Provision of information about associations or an office within the hospital for the defense of patients’ rights | List: [1,0] | |
| C3.i2 Media | |||
| C3.i2.1 The institution in the media: features news that appeared in press, radio, TV, social networks | Provision of institution’s news, that appeared in press, radio, TV, or social networks, in the website | List: [1,0] | |
| C3.i2.2 Website provides an up-to-date news/events schedule/newsletter | Provision of up-to-date (up to last year) news/events schedule/newsletter in the website | List: [1,0] | |
| C3.i2.3 Virtual visit to the institution | Existence of virtual visit to the institution | List: [1,0] | |
| C3.i2.4 Links to other websites of interest: hospitals, scientific societies, institutions | Existence of links to other websites of interest | List: [1,0] | |
| C3.i2.5 Public relations office: work hours | Provision of public relations office work hours | List: [1,0] | |
| C3.i2.6 Public relations office: location | Provision of public relations office location | List: [1,0] | |
| C3.i2.7 Public relations office: telephone and/or fax number | Provision of public relations office telephone and/or fax number | List: [1,0] | |
| C3.i2.8 Public relations office: e-mail address | Provision of public relations office e-mail address | List: [1,0] | |
| C3.i2.9 Institution news: new techniques used by the centre specialists, infrastructure improvement | Provision of institution news regarding new techniques used by the centre specialists, infrastructure improvement, etc. | List: [1,0] | |
| C3.i3 Advertising/Marketing | |||
| C3.i3.1 Website sponsors and investors are disclosed | Provision of website sponsors and investors. In the case of public hospitals, this may be just the logo of the National Health System. | List: [1,0] | |
| C3.i3.2 Advertising and contents are differentiated | Advertisements and contents are differentiated. The assessor must verify if it is clear what is advertisement and what is content provided by the hospital. Note: this item is NA if no advertisements exist | List: [1,0,NA] | page has advertising |
| C3.i3.3 Advertising is not contradictory with respect of the website contents | The assessor should check if the advertisement present in the website is not contradictory to the content provided by the hospital. Ex.: Advertisement of cigarettes | List: [1,0,NA] | page has advertising |
| C3.i3.4 Information on how to make a donation to the hospital | Provision of information on how to make a donation to the hospital | List: [1,0] | |
| C3.i3.5 Facebook link | Provision of Facebook link. | List: [1,0] | |
| C3.i3.6 Twitter link | Provision of Twitter link. | List: [1,0] | |
| C3.i3.7 LinkedIn link | Provision of LinkedIn link. | List: [1,0] | |
| C3.i3.8 Youtube link | Provision of Youtube link. | List: [1,0] | |
| C3.i3.9 Other social networks link (e.g., Instagram) | Provision of other social network tools. The assessor verifies if there exists a link to the hospital’s page on any other social media | List: [1,0] | |
| C3.i3.10 Information about job opportunities at the hospital | Provision of information about job opportunities at the hospital | List: [1,0] | |
Table A4.
Metadata about Criterion 4–Technology Features.
Table A4.
Metadata about Criterion 4–Technology Features.
| Column Name | Description | Data Values | Condition to Be Different from NA |
|---|---|---|---|
| C4.i1 Navigability | |||
| C4.i1.1 Website name appears on browser title bar | Appearance of website name on browser title bar | List: [1,0] | |
| C4.i1.2 Active part of the site appears on browser title bar | Appearance of the active part of the website in the current active tab | List: [1,0] | |
| C4.i1.3 Best browser version for the website is indicated | Indication of which is the best browser version for the website is present | List: [1,0] | |
| C4.i1.4 Interwebsite links are distinguished from intrawebsite links | There is a clear distinction (e.g., different colours) between interwebsite and intrawebsite links in the website | List: [1,0,NA] | inter or intra website links exist |
| C4.i1.5 Interwebsite links show a full description of the linked website | Existence of a full description of the linked website in the interwebsite links. The assessor should check if, when the mouse stops over a interwebsite link, it is possible to have a full description of the new page to be accessed through the link | List: [1,0,NA] | interwebsite links exist |
| C4.i1.6 Functioning intrawebsite links | Existence of functioning intrawebsite links (no broken links) | List: [1,0,NA] | intrawebsite links exist |
| C4.i1.7 Functioning interwebsite links | Existence of functioning interwebsite links (no broken links) | List: [1,0,NA] | interwebsite links exist |
| C4.i2 Accessibility | |||
| C4.i2.1 Compliance with level A WCAG 2.1 W3C | Compliance with level A of W3C WCAG 2.1 is tested. Assessors should access https://accessmonitor.acessibilidade.gov.pt/ (accessed on 20 November 2024); open the accessMonitor tool; and paste the hospital link. Then click “Validar”. The result will be presented as a table. If there are any errors “Não Aceitáveis” for level A, register “0”. Otherwise, register “1”. | List: [1,0] | |
| C4.i2.2 Compliance with level AA WCAG 2.1 W3C | Compliance with level AA of W3C WCAG 2.1 is tested. Assessors should access https://accessmonitor.acessibilidade.gov.pt/ (accessed on 20 November 2024); open the accessMonitor tool; and paste the hospital link. Then click “Validar”. The result will be presented as a table. If there are any errors “Não Aceitáveis” for level AA, register “0”. Otherwise, register “1”. | List: [1,0] | |
| C4.i2.3 Compliance with level AAA WCAG 2.1 W3C | Compliance with level AAA of W3C WCAG 2.1 is tested. Assessors should access https://accessmonitor.acessibilidade.gov.pt/ (accessed on 20 November 2024); open the accessMonitor tool; and paste the hospital link. Then click “Validar”. The result will be presented as a table. If there are any errors “Não Aceitáveis” for level AAA, register “0”. Otherwise, register “1”. | List: [1,0] | |
| C4.i2.4 Accessibility symbol or Accessibility Declaration available | Existence of official accessibility symbol on the main webpage or Accessibility Declaration. | List: [1,0] | |
| C4.i2.5 Website is validated through W3C CSS 3.0 Validation Service | Website is compliant with the CSS style sheet standards set by the World Wide Web Consortium (W3C). Assessors should access https://jigsaw.w3.org/css-validator/ (accessed on 20 November 2024) and paste the link of the institute using the default options. If any errors are found, the item receives value “0”. | List: [1,0] | |
| C4.i2.6 Website is validated through W3C Markup Validation Service (html5) | Website is compliant with the markup validity standards set by the World Wide Web Consortium (W3C). Assessors should access https://validator.w3.org/ (accessed on 20 November 2024) and paste the link of the hospital using the default options. If any errors are found, the item receives value “0”. | List: [1,0] | |
| C4.i2.7 Website listed on the first page of results after performing a Google search | Appearance of the website on the first page of results after performing a Google search with the institute name. The assessor should paste the full hospital name within quotation marks in the Google search bar and then verify if the correct and valid link for the hospital appears on the first page of results | List: [1,0] | |
| C4.i2.8 For individual sub-pages, there is a specific and meaningful description provided via the META/description tag | The webpage provides in the source code the meta description attribute that provides a brief summary of a web page. The assessor should click with the mouse right button and select “view source code”. Then try to find (ctrl+f) “meta name=”description””. Check the existence of the META/description tag in the source code of the webpage. | List: [1,0] | |
| C4.i2.9 Website is compatible with the 3 most used browsers in the country | The compatibility with the three most popular web browsers used in the country according to http://gs.statcounter.com/browser-market-share/all/portugal (accessed on 20 November 2024) is assessed. The assessors should test if the hospital’s website in each of these three web browsers is functioning. | List: [1,0] | |
| C4.i2.10 Website is compatible with the 3 most used mobile browsers in the country | The compatibility with the three most popular mobile web browsers used in the country according to http://gs.statcounter.com/browser-market-share/mobile/portugal (accessed on 20 November 2024) is assessed. The assessors should test if the hospital’s website in each of these three mobile browsers is functioning. | List: [1,0] | |
| C4.i3 Usability/Readability | |||
| C4.i3.1 Website map available | Existence of website map in the website | List: [1,0] | |
| C4.i3.2 Website search engine | Existence of a website search engine in the website | List: [1,0] | |
| C4.i3.3 Access to the website in foreign languages | Website content is provided in more than one language | List: [1,0] | |
| C4.i3.4 Website load time | The website loading time for the main page through the use of a tool (https://gtmetrix.com/ (accessed on 20 November 2024)) is assessed. Assessors should access the tool, paste the institute’s link, and click “test your site”. The tool will produce a report. Select “Performance” and check the value under “Time to Interactive”. Item receives value “1” if value is less or equal to 5 s | List: [1,0] | |
| C4.i3.5 Illustrations/pictures/photos accompany text to assist description | Provision of visual illustrations that help users with reading disabilities to understand difficult text that describes concepts or processes. The illustrations are provided in addition to the text. | List: [1,0] | |
| C4.i3.6 Graphics open conveniently (images/graphics are quick to load) | Convenience and speed in graphics opening is assessed. The assessor should check if images and graphics of the website open conveniently and at the same time as the page load time. | List: [1,0] | |
| C4.i3.7 Website pages can be printed | Verification if the website pages can be printed using a print button available on the website. | List: [1,0] | |
| C4.i3.8 Individual sub-pages have specific and meaningful titles | Verification if the individual sub-pages have specific and meaningful titles | List: [1,0] | |
| C4.i3.9 The layout of the website is responsive (i.e., does it adapt to varying screen sizes), or there is a separate version for mobile devices | Verification if the website adapts to different screen sizes and to different devices. | List: [1,0] | |
| C4.i3.10 The website offers means to adjust (increase) the text size without compromising the functionality of the website | It is verified if the website provides the possibility for the user to adjust (increase) the text size without compromising the functionality of the website | List: [1,0] | |
| C4.i3.11 The website offers means to adjust (increase) the contrast of textual information for visitors with visual impairments | It is verified if the website provides the possibility for the user to adjust (increase) the contrast of textual information for visitors with visual impairments | List: [1,0] | |
| C4.i3.12 The website provides a (consistently accessible) menu structure for navigating its sub-pages | The existence of a menu structure for navigating website’s sub-pages is available | List: [1,0] | |
| C4.i3.13 Website does not include pop-up advertising | The absence of pop-up advertising is assessed | List: [1,0] | |
| C4.i3.14 Website Technological Sophistication (universal services use via web services, APIs, widgets) | It is verified if the website provides services through web services, APIs, or widgets | List: [1,0] | |
| C4.i4 Credibility | |||
| C4.i4.1 The text is grammatically correct | It is verified if the provided text is grammatically correct. | List: [1,0] | |
| C4.i4.2 The text does not have spelling errors | It is verified if the provided text does not have spelling errors | List: [1,0] | |
| C4.i4.3 Date of last website update | The date of the last website update is available | List: [1,0] | |
| C4.i4.4 Pages have dates associated with them (There are indications of updates to materials) | It is verified if the internal pages have date indications of their updates | List: [1,0] | |
| C4.i4.5 Website has a hospital website certification/accreditation | It is verified if the website has any health information certification or accreditation. | List: [1,0] | |
| C4.i4.6 Copyright information available | The presence of copyright information is assessed. It can usually be found at the bottom of the webpage and might refer to webmaster characteristics, copyright, or ownership of the website. | List: [1,0] | |
| C4.i5 Privacy/Security | |||
| C4.i5.1 General disclaimers provided | Presence of general disclaimers is assessed | List: [1,0] | |
| C4.i5.2 The site is secure (encrypted) | It is verified if the website uses https protocol. The browser URL bar starts with https:// | List: [1,0] | |
| C4.i5.3 There is a website privacy policy | Existence of website privacy policy | List: [1,0] | |
| C4.i5.4 Cookie policy | Existence of a cookie policy | List: [1,0] | |
References
- Kelly, M.M.; Hoonakker, P.L.; Dean, S.M. Using an inpatient portal to engage families in pediatric hospital care. J. Am. Med. Inform. Assoc. 2017, 24, 153–161. [Google Scholar] [CrossRef] [PubMed]
- European Commission. eEurope, an Information Society for All, Communication on a Commission Initiative for the Special European Council of Lisbon, 23–24 March 2000; COM(1999) 687 Final; European Commission: Brussels, Belgium, 2000. [Google Scholar]
- European Commission. i2010—A European Information Society for Growth and Employment; COM (2005) 229 Final; European Commission: Brussels, Belgium, 2005. [Google Scholar]
- European Commission. Europe 2020. A European Strategy for Smart, Sustainable and Inclusive Growth; COM(2010) 2020 Final; European Commission: Brussels, Belgium, 2010. [Google Scholar]
- Dendere, R.; Slade, C.; Burton-Jones, A.; Sullivan, C.; Staib, A.; Janda, M. Patient portals facilitating engagement with inpatient electronic medical records: A systematic review. J. Med. Internet Res. 2019, 21, e12779. [Google Scholar] [CrossRef]
- Emmert, M.; Meszmer, N.; Schlesinger, M. A cross-sectional study assessing the association between online ratings and clinical quality of care measures for US hospitals: Results from an observational study. BMC Health Serv. Res. 2018, 18, 82. [Google Scholar] [CrossRef]
- Gole, I.; Sharma, T.; Misra, S.B. Role of ICT in healthcare sector: An empirical study of pune city. J. Manag. Public Policy 2017, 8, 23–32. [Google Scholar] [CrossRef]
- Alhuwail, D.; AlMeraj, Z.; Boujarwah, F. Evaluating hospital websites in Kuwait to improve consumer engagement and access to health information: A cross-sectional analytical study. BMC Med. Inform. Decis. Mak. 2018, 18, 82. [Google Scholar] [CrossRef] [PubMed]
- Westphal, M.; Yom-Tov, G.B.; Parush, A.; Carmeli, N.; Shaulov, A.; Shapira, C.; Rafaeli, A. A patient-centered information system (myED) for emergency care journeys: Design, development, initial adoption. JMIR Form. Res. 2020, 4, e16410. [Google Scholar] [CrossRef] [PubMed]
- Vijayasarathi, A.; Kharkar, R.; Salamon, N. Strategies for patient-Centered communication in the digital age. Curr. Probl. Diagn. Radiol. 2019, 48, 210–215. [Google Scholar] [CrossRef] [PubMed]
- Soares, D.; Sarantis, D.; Carvalho, J. HSWAI: Instrument Design and Validation Process; United Nations University (UNU-EGOV): Guimarães, Portugal, 2021. [Google Scholar]
- Llinas, G.; Rodriguez-Iiesta, D.; Mira, J.J.; Lorenzo, S.; Aibar, C. A Comparison of Websites from Spanish, American and British Hospitals. Methods Inf. Med. 2008, 47, 124–130. [Google Scholar] [CrossRef] [PubMed]
- Huerta, T.R.; Walker, D.M.; Ford, E.W. An Evaluation and Ranking of Children’s Hospital Websites in the United States. J. Med. Internet Res. 2016, 18, e228. [Google Scholar] [CrossRef]
- Randeree, E.; Rao, H.R. E-health and assurance: Curing hospital websites. Int. J. Electron. Healthc. 2004, 1, 33–46. [Google Scholar] [CrossRef]
- Moslehifar, M.A.; Noor, A.I.; Sandaran, S.C. Assessing the quality of trust features on website content of top hospitals for medical tourism consumers. Malays. J. Commun. 2016, 2, 469–489. [Google Scholar] [CrossRef]
- Patsioura, F.; Kitsiou, S.; Markos, A. Evaluation of Greek Public Hospital Websites. In Proceedings of the ICE-B—International Conference on e-Business, Milan, Italy, 7–10 July 2009; pp. 223–229. [Google Scholar]
- Lewiecki, E.M.; Rudolph, L.A.; Kiebzak, G.M.; Chavez, J.R.; Thorpe, B.M. Assessment of osteoporosis-website quality. Osteoporos. Int. 2006, 17, 741–752. [Google Scholar] [CrossRef]
- Guardiola-Wanden-Berghe, R.; Sanz-Valero, J.; Wanden-Berghe, C. Quality assessment of the Website for Eating Disorders: A systematic review of a pending challenge. Ciênc. Saúde Coletiva 2012, 17, 2489–2497. [Google Scholar] [CrossRef] [PubMed]
- Rezniczek, G.A.; Küppers, L.; Heuer, H.; Hefler, L.A.; Buerkle, B.; Tempfer, C.B. Quality of websites of obstetrics and gynecology departments: A cross-sectional study. BMC Pregnancy Childbirth 2015, 15, 103. [Google Scholar] [CrossRef] [PubMed]
- Moreno, J.M.; Del Castillo, J.M.; Porcel, C.; Herrera-Viedma, E. A quality evaluation methodology for health-related websites based on a 2-tuple fuzzy linguistic approach. Soft Comput. 2010, 14, 887–897. [Google Scholar] [CrossRef]
- Bilsel, R.U.; Büyüközkan, G.; Ruan, D. A fuzzy preference? Ranking model for a quality evaluation of hospital web sites. Int. J. Intell. Syst. 2006, 21, 1181–1197. [Google Scholar] [CrossRef]
- Maifredi, G.; Orizio, G.; Bressanelli, M.; Domenighini, S.; Gasparotti, C.; Perini, E.; Caimi, L.; Schulz, P.J.; Gelatti, U. Italian hospitals on the web: A cross-sectional analysis of official websites. BMC Med. Inform. Decis. Mak. 2010, 10, 17. [Google Scholar] [CrossRef] [PubMed]
- Norum, J. Evaluation of Norwegian cancer hospitals’ Web sites and explorative survey among cancer patients on their use of the Internet. J. Med. Internet Res. 2001, 3, E30. [Google Scholar] [CrossRef] [PubMed]
- Mira, J.J.; Llinás, G.; Tomás, O.; Pérez-Jover, V. Quality of websites in Spanish public hospitals. Med. Inform. Internet Med. 2006, 31, 23–44. [Google Scholar] [CrossRef] [PubMed]
- Calvo-Calvo, M.A. Quality and characteristics of websites of large Spanish hospitals. Rev. Esp. Doc. Cient. 2014, 37, e032. [Google Scholar] [CrossRef]
- Mancini, C.; Zedda, M.; Barbaro, A. Health information in Italian public health websites: Moving from inaccessibility to accessibility. Health Inf. Libr. J. 2005, 22, 276–285. [Google Scholar] [CrossRef]
- García-Lacalle, J.; Pina, V.; Royo, S. The unpromising quality and evolution of Spanish public hospital web sites. Online Inf. Rev. 2011, 35, 86–112. [Google Scholar] [CrossRef]
- Gruca, T.S.; Wakefield, D.S. Hospital web sites: Promise and progress. J. Bus. Res. 2004, 57, 1021–1025. [Google Scholar] [CrossRef]
- Liu, X.; Bao, Z.; Liu, H.; Wang, Z. The quality and characteristics of leading general hospitals’ Websites in China. J. Med. Syst. 2011, 35, 1553–1562. [Google Scholar] [CrossRef] [PubMed]
- Tsai, S.L.; Chai, S.K. Developing and validating a nursing website evaluation questionnaire. J. Adv. Nurs. 2005, 49, 406–413. [Google Scholar] [CrossRef] [PubMed]
- Carvalho, J.; Sá Soares, D.; Sarantis, D. HOSPI Application to Portuguese Hospitals 2023; Repositório de Dados da Universidade do Minho: Braga, Portugal, 2024; Available online: https://datarepositorium.uminho.pt/dataset.xhtml?persistentId=doi%3A10.34622%2Fdatarepositorium%2FENHJQZ (accessed on 18 November 2024).
- Sarantis, D.; Soares, D. Literature on Website Evaluation in Health Sector. In Proceedings of the 10th IADIS International Conference Information Systems, Budapest, Hungary, 10–12 April 2017. [Google Scholar]
- Alajarmeh, N. Evaluating the accessibility of public health websites: An exploratory cross-country study. Univers. Access Inf. Soc. 2022, 21, 771–789. [Google Scholar] [CrossRef]
- Sun, Y.; Zhang, Y.; Gwizdka, J.; Trace, C.B. Consumer evaluation of the quality of online health information: Systematic literature review of relevant criteria and indicators. J. Med. Internet Res. 2019, 21, e12522. [Google Scholar] [CrossRef]
- Soares, D.; Sarantis, D.; Carvalho, J. Health Sector Website Assessment Instrument: Notes on the Assessment Method; United Nations University (UNU-EGOV): Guimarães, Portugal, 2019. [Google Scholar]
- Serviço Nacional de Saúde. 2024. Available online: https://www.sns.gov.pt/institucional/entidades-de-saude/ (accessed on 20 November 2024).
- Soares, D.; Sarantis, D.; Carvalho, J. Assessment of the Portuguese Hospitals Websites Using HOSPI; United Nations University (UNU-EGOV): Guimarães, Portugal, 2024. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).