Validation of a Questionnaire Developed to Evaluate a Pediatric eHealth Website for Parents.

There is a need for health professionals to provide parents with not only evidence-based child health websites but also instruments to evaluate them. The main aim of this research was to develop a questionnaire for measuring users' evaluation of the usability, utility, confidence, the well-child section, and the accessibility of a Spanish pediatric eHealth website for parents. We further sought to evaluate the content validity and psychometric reliability of the instrument. A content validation study by expert review was performed, and the questionnaire was pilot tested. Psychometric analyses were used to establish scales through exploratory and confirmatory factor analyses. Reliability studies were performed using Cronbach's alpha and two split-half methods. The content validation of the questionnaire by experts was considered as excellent. The pilot web survey was completed by 516 participants. The exploratory factor analysis excluded 27 of the 41 qualitative initial items. The confirmatory factor analysis of the resultant 14-item questionnaire confirmed the five initial domains detected in the exploratory confirmatory analysis. The goodness of fit for the competing models was established through fit indices and confirmed the previously established domains. Adequate internal consistency was found for each of the subscales as well as the overall scale.


Institutional Review Board approval and informed consent process.
IRB approval.
This study used secondary data sources and did not collect personal information. IRB approval was not necessary.
Informed consent.
There was an informed consent at the beginning of the questionnaire. Participants were informed of the purpose of the study that no personal information was going to be collected. They were also informed of the length of the questionnaire.
Data protection.
The questionnaire did not collect personal information, so additional mechanisms to protect unauthorised access to those provided by the platform (Google Forms) were not needed.

Development and pre-testing.
Development and testing.
The questionnaire was developed by the five authors of the research, following the CHERRIES checklist and ensuring usability and technical functionality. For external validation of contents, the electronic questionnaire was reviewed by seven external expert reviewers. For internal validation, statistical analysis were performed.

Recruitment process and description of the sample having access to the questionnaire.
Open survey versus closed survey. The questionnaire was an open survey.
Contact mode.
The initial contact with the potential participants was made on the Internet, mainly in notodoespediatria.com.
Advertising the survey.
Advertising the survey. The survey was not announced or advertised in other webs, but users of notodoespediatria.com were invited to participate trough invitations posted via social media (Facebook, Twitter).

Survey administration.
Web/E-mail.
The e-survey was posted on the Web site that was going to be evaluated by the users (notodoespediatria.com). Context.
The survey was developed in Google Forms platform and posted in notodoespediatria.com, a web with paediatric resources for parents with 338 posts about symptoms, illnesses and well-child care advices, written by a paediatrician with evidence based information.
Mandatory/voluntary. Completing and submitting the survey was voluntary.

Incentives.
No incentives were offered. Timeframe.
The survey was open for 8 weeks.
Randomization of items or questionnaires. No randomization of items was performed.
No adaptive questioning was needed.
Number of Items. The questionnaire included 67 items: 26 demographical items and 41 qualitative items.

Number of screens (pages).
The questionnaire had 10 pages, including initial informed consent.
Completeness check.
All items of the questionnaire were stated as required. Therefore, to submit a questionnaire all items should be answered. By consensus, the authors concluded that only twenty one items of the questionnaire should include a «not applicable» or «rather not say» option.

Review step.
Respondents were able to review and change their answers through a Back button.

Response rates.
Unique site visitors. Notodoespediatria.com had 98,577 unique visitors during the eight weeks the survey was open.
View rate (Ratio unique site visitors/unique survey visitors).
Google Forms did not provide the number of unique visitors of the first page of the survey.
Participation rate (Ratio unique survey page visitors/agreed to participate). Google Forms did not provide this data.
Completion rate (Ratio agreed to participate/finished survey).
Of the 98,577 unique visitors of notodoespediatria.com during the eight weeks the survey was open, 516 questionnaires were submitted. Therefore, an estimation of the completion rate is 0.52% of the unique visitors of the web.
Preventing multiple entries from the same individual.

Cookies used.
No cookies were used to assign a unique user identifier to each client computer. As personal information was not collected, we could not prevent duplicates entries.
IP check.
The IP address of the client computer was not collected nor used to identify potential duplicate entries from the same user.
Log file analysis. Other techniques to analyse the log file for identification of multiple entries were not used.

Registration.
No registration was needed.

Analysis.
Handling of incomplete questionnaires. Only completed questionnaires could be submitted.
Questionnaires submitted with an atypical timestamp. Google Forms only provided a timestamp.
No statistical correction was needed.
English translation of the original Spanish questionnaire.
(Please consider that translation of the items into English does not replace a trans-cultural adaptation of the instrument). -Yes, in two or more occasions.

Observational items (26 items
-Yes, at least in one occasion.
-I don't know.

As a consequence of visiting [Web name blinded for review], have you ever avoided a visit to emergency?
-Yes, in two or more occasions.
-Yes, at least in one occasion.
-I don't know. 38. Do you think that similar webs, in the long term, can substitute physical consultations? -Very probable.
-It is possible.
-Not very probable. [1-5 points scale]. 57. Do you think [Web name blinded for review] is easy to use in a smartphone or a tablet? (1= I completely disagree; 5 = I absolutely agree).

Do you think [Web name blinded for review]
is written in an easy to understand language? (1= I completely disagree; 5 = I absolutely agree).
[1-5 points scale]. 59. Do you think [Web name blinded for review] has a wide variety of topics? (1= I completely disagree; 5 = I absolutely agree).
[1-5 points scale]. 60. Do you think that searching for information in [Web name blinded for review] is easy? (1= I completely disagree; 5 = I absolutely agree).
[1-5 points scale]. 62. How do you rate, from 1 to 5 points, that [Web name blinded for review] is an open access web? (1 = very bad; 5 = very good).