A Lay-User Assessment of Hepatitis C Virus Self-Testing Device Usability and Interpretation in Johannesburg, South Africa

Only 20% of people with hepatitis C virus (HCV) know their status. In low-income countries diagnosis is under 10%. Self-testing for HCV antibodies (HCVST) could expand the coverage of HCV testing services. Currently, there are no stringent regulatory authority (SRA) approved HCVSTs, therefore lay-user usability of three prototype kits was assessed. This was a cross-sectional observational study conducted with 171 (CareStart n = 60, Bioline n = 52, First Response n = 59) participants. Participants were given one of the three HCVST kits with only instructions for use (IFU) and asked to perform the test in front of a professional trained in rapid diagnostic tests (RDT). Usability indices were calculated based on the correctness of performing each step of the product-specific process followed by contrived results interpretation and a post-test interview. The usability index was 93.9% for CareStart, 90.7% for Bioline and 94.9% for First Response. Most errors were on incorrect handwashing, sample collection and transfer to the test device. An average of 93.1% of contrived results were correctly interpreted, with most errors related to interpreting invalid results. Most participants (n = 167) stated they would visit a clinic after a positive result. With negative results, nearly half (28/60 (46.7%)) stated they should condomize, while just over two-thirds of participants that used Bioline (35/52 (67.3%)) and First Response (38/59 (64.4%)) said they should re-test. Most participants (n = 162) found the devices easy to use. Participants liked that self-testing was fast, private and convenient, however there were some confusion with IFU steps and pictures, finger-pricking with the lancet, collecting blood after the finger-prick, and transferring the sample/buffer. Prototype HCVST kits exhibit high usability and result interpretation by lay-users, and should be considered for SRA approval.


Introduction
Liver inflammation is referred to as hepatitis, and it is commonly caused by viral infections. Viral hepatitis has been on the rise since the 1990s and accounts for 1.4 million global deaths per year, comparable to tuberculosis (1.41 million deaths per year) and much higher than HIV (770,000 deaths per year) or malaria (405,000 deaths per year) [1][2][3][4]. Hepatitis B and C contributes the majority of chronic viral hepatitis infections and mortalities.
In 2015, an estimated 71 million people were living with hepatitis C virus (HCV) disease, and direct percutaneous exposure with infected blood led to 1.75 million new infections annually, often via the sharing of infected needles for injection-drug use and unsafe healthcare procedures [5]. In 2016, the World Health Assembly approved a global strategy to eliminate viral hepatitis as a public health threat, with the specific targets of reducing new infections by 90% and reducing deaths by 65%, by 2030 [1,5]. There is currently no vaccine for HCV, however, direct-acting antivirals introduced in 2014 have transformed the treatment options for people living with HCV [6]. An 8-12-week

Study Design
This usability assessment was an observational cross-sectional study implemented from October 2019 to January 2020. Convenience sampling was used to recruit lay users from the general population of inner-city Johannesburg, South Africa. This evaluation of three prototype HCVST kits was directed by the protocols on self-testing which were developed based on a previously conducted usability assessment of HIVSTs in Johannesburg [35].
For each kit, a diverse sample size of at least 52 participants (170 in total) was recruited. Each of the HCVST kits were evaluated independently and in a systematic manner. Fingerprint scanning with a biometric enrolment system was used to ensure that participants were only enrolled for one device. Participants were included if they could provide written informed consent, were able to speak/read English, were 18 years of age or older, had no previous experience with self-testing, and were willing to provide fingerstick blood samples. People were excluded if they were practicing healthcare workers with prior experience using RDTs or had any condition (such as tremors, poor vision, or intoxication) that may have interfered with any of the study processes.

HCVST Kits
At the time of this evaluation, each of the HCVST kits were late-stage prototypes, packaged as intended for sale or distribution in South Africa. Each HCVST package included the manufacturer's instructions for use (IFU) and all kit components required for the self-test. Three HCVST fingerstick devices were assessed: CareStart EZ HCV (CareStart) (AccessBio, Inc., Somerset, NJ, USA), SD Bioline HCV (Bioline) (Abbott Rapid Diagnostics (Pty) Ltd., Jena, Germany) and First Response HCV Card Test (First Response) (Premier Medical Corporation Pvt Ltd., Gujarat, India). All three HCVST kits were single-use tests that are considered screening tests (i.e., all positive results would require a confirmatory HCV ribonucleic acid test).

Data Collection
After participants provided written informed consent, a study staff accompanied them to a clinical research room to simulate a private setting for conducting the HCVST. Participants were given a packaged and sealed HCVST kit, with no familiarization or demonstration, then asked to complete the self-test while observed by a healthcare worker trained in RDT use (observer). The observer monitored the testing procedure and documented the successes and errors of each participant using a product-specific semi-structured questionnaire. The observer provided no feedback or assistance during the test. After the self-test was completed, the HCVST kits were exchanged for devices that displayed contrived results for the participant to interpret. Participants did not interpret their own HCVST, as this study did not evaluate the performance of HCVSTs.
Usability is a component of WHO prequalification therefore there have been a variety of usability studies for HIVSTs, based on the WHO prequalification guidelines for HIVST [18][19][20][21][22][23][24]32]. There is no standardized validated questionnaire for conducting HCVST usability assessments, however a recent study in South Africa used product-specific semistructured questionnaires to conduct usability assessments on 7 HIVSTs with lay-users in the inner-city of Johannesburg [35]. Similar to this study, we have evaluated usability with a 3-part questionnaire that includes a usability index (based on a HCVST process checklist), the interpretation of contrived results and a post-test survey.

Usability Index
For each HCVST, the usability index was calculated with a product-specific yes/no checklist that corresponded to all of the HCVST steps as outlined in each device's IFU. The successful steps were presented as a frequency and percentage, and average of all successful steps was used to calculate the total usability index for each HCVST [35]. Some steps, such as Did the study participant have difficulty with twisting the sterile tab? had a negative inflection, so the 'No' response was used in the final usability index calculation. The final usability indices were presented as a percentage, from 0.0% (not usable) to 100.0% (very usable), with scores above 85% considered excellent [35][36][37]

Interpreting Contrived Results
Non-functional devices had been modified by the manufacturer to display the possible outcomes from their HCVST device and these contrived results were used to evaluate each participants' ability to correctly interpret each of the devices' possible results: (1) non-reactive/negative; (2) reactive/positive; (3) weak reactive/weak positive (Bioline and First Response only); (4) invalid (no control line) and (5) invalid (T-line). Trained study staff observed the interpretation of the contrived results and documented all of the correctly interpreted results.
All of the participants' HCVST devices were discarded right after the self-testing procedure and before the HCVST results appeared. Participants were not informed of their HCVST results, nor were these results interpreted or recorded anywhere. Participants that wanted to formally test for HCV were referred to the closest healthcare facilities where testing was available.

Post-Test Survey
After completing the HCVST and interpreting the contrived results, participants were asked what they should do following positive, negative or inconclusive results, as IFUs provided recommendations for each of the possible results. Participants were also asked about their perceptions of the HCVST and the testing experience, including their most favourite and least favourite aspects of the HCVST.

Data Analysis
For each HCVST, data from the product-specific semi-structured questionnaires were transferred to an Excel database by the research team. Data were cleaned and coded in Excel (Microsoft Corp., Redmond, WA, USA) then quantitative data was analysed with descriptive statistics.

Usability Index
The usability index for each HCVST is fully described below and in Table 2. Direct comparisons between the different HCVSTs should not be made, as each usability checklist applied product-specific steps to calculate each usability index. Total usability index 94.9 Abbreviations: n-number; %-percentage; IFU-information for use.

CareStart
The overall usability index of CareStart was 93.9%. All participants read and used the information sheet, while 56/60 (93.3%) selected a finger and firmly massaged it for 5-10 s and 51/60 (85.0%) cleaned the fingertip with the alcohol swab then let it air dry. None of the participants had difficulty removing the test content from the foil pouch, however one participant had difficulty twisting the sterile tab and another one had difficulty pulling the sterile Table All participants pushed firmly against the side of the finger, 58/60 (96.7%) participants squeezed behind the sample site to form a blood droplet and 48/60 (80.0%) managed to fill the tube with the correct volume of blood. Most participants (56/60 (93.3%)) placed the device on a flat surface and rotated the arm until it clicked to release the blood, 50/60 (83.3%) made sure that all of the blood was on the test strip and 59/60 (98.3%) participants used their thumb to push the button down firmly until it clicked to release the buffer.

Bioline
For Bioline, the overall usability index was 90.7%. All participants (52/52 (100.0%)) read and used the information sheet, none (0/52 (0.0%)) had difficulty removing the test content from the foil pouch and all participants (52/52 (100.0%)) successfully placed the kit materials on a flat surface, then opened all pouches and cap. Most (45/52 (86.5%)) participants washed their hands in warm water and dried them, then 51/52 (98.1%) participants correctly chose the ring or middle finger, massaged and warmed their hands, and cleaned the finger with an alcohol swab then let it dry. Most (45/52 (86.5%)) participants successfully pressed down firmly to prick their skin, 35/52 (67.3%) participants wiped away the first drop of blood, then rubbed to create a second large drop of blood, and 43/52 (82.7%) participants used the specimen dropper to collect the drop of blood. Forty-four out of fifty-two (84.6%) participants successfully dispensed the whole blood into the round specimen well of the device, 46/52 (88.5%) participants successfully applied the plaster and 46/52 (88.5%) participants were able to twist and pull the cap to open the assay diluent, then dispense all the assay diluent tube into the square well of the device.

First Response
The overall usability index was 94.9%. All participants (59/59 (100.0%)) read and used the information sheet and 49/59 (83.0%) participants washed their hands in warm water and soap then dried and checked the expiry date of the kit. Only 5/59 (8.5%) participants had difficulty removing the test content from the foil pouch, and 54/59 (91.5%) participants wiped their fingertip with an alcohol swab and allowed it to dry. All (59/59 (100.0%)) participants successfully removed the clear protective cap of the sterile safety lancet, successfully pressed down firmly to prick their skin, and gently squeezed their fingertip to obtain a large drop of blood. Most (51/59 (86.4%)) successfully wiped away the first drop of blood with the dry swab, 56/59 (94.9%) gently squeezed their fingertip to obtain a large second drop of blood and all (59/59 (100.0%)) participants used the specimen dropper to collect the drop of blood. Majority (53/59 (89.9%)) successfully dispensed the whole blood into the round specimen well of the device, all (59/59 (100.0%)) participants were able to twist and pull the cap to open the assay buffer. Lastly, almost all (57/59 (96.6%)) successfully added 2 drops of assay buffer into the specimen well.

First Response
For First Response, 97.3% of the contrived results were correctly interpreted by participants. All participants (59/59 (100.0%)) correctly interpreted the non-reactive/negative results and the reactive/positive results, while 58/59 (98.3%) correctly interpreted the weak reactive/weak positive and invalid (no control line) results, and 53/59 (89.8%) correctly interpreted the invalid (T-line) results.
These results are presented in Table 3.

CareStart
When CareStart participants were asked what to do after a negative HCVST result, 28/60 (46.7%) participants would condomize, while 8/60 (13.3%) did not know what to do and none said that they would retest at all (self-test or professional). Almost all (58/60 (96.7%)) participants said they would visit a clinic after a positive HCVST result, and if the test was invalid or did not work, 19/60 (31.7%) said they would visit a clinic, while 38/60 (63.3%) said they would retest.

Bioline
Bioline participants, when asked what to do after a negative HCVST result, 35/52 (67.3%) participants would retest, while 0/52 (0.0%) did not know what to do and none said that they would condomize. All (52/52 (100.0%)) participants stated that they would visit a clinic after a positive HCVST, and if the test was invalid or did not work, 24/52 (46.2%) said they would visit a clinic, while 28/52 (53.8%) said they would retest.

First Response
Of the First Response participants who were asked what to do after a negative HCVST result, 38/59 (64.4%) participants would retest, while all knew what to do and 4/59 (6.8%) said that they would condomize. Almost all (57/59 (96.6%)) participants stated that they would visit a clinic after a positive HCVST, and if the test was invalid or did not work, 13/59 (23.7%) would visit a clinic, while 45/59 (76.3%) said they would retest.

CareStart
All participants used the IFU, and most participants (58/60 (96.7%)) found them easy to follow. Most participants also found the device easy to use (58/60 (96.7%)) and were confident performing it on their own (59/60 (98.3%)). Almost everyone (59/60 (98.3%)) correctly identified 15 min as the amount of time you should wait before reading the results, and 56/59 (93.3%) participants found it easy to interpret the mock results. When asked about future testing, 50/60 (83.3%) participants said they would use this test again, 52/60 (86.7%) said they would prefer to use this HCVST at home instead of going to a clinic and 55/60 (91.7%) said they would recommend this test to someone else.
When asked about the advantages of HCVST, 52/60 (86.7%) participants liked that it was easy to use, 26/60 (43.3%) liked that it was fast, 29/60 (48.3%) liked that it was private and 10/60 (16.7%)) liked that it was convenient. Most participants (52/60 (86.7%)) had no complaints about the testing process, however 4/60 (6.7%) had difficulties pricking themselves with the lancet, 1/60 (2.7%) was confused about the IFU steps and pictures and 2/60 (3.3%) did not like the lack of information on what to do after the test.

Bioline
Almost all (51/52 (98.1%)) participants used the IFU, and 47/52 (90.4%) found it easy to follow. For the device, 45/52 (86.5%) participants found it easy to use and 46/52 (88.5%) participants were confident performing the test on their own. Almost everyone (51/52 (98.1%)) correctly identified the 15-min wait before reading the results and 48/52 (92.3%) found the contrived results easy to interpret. For future testing, 51/52 (98.1%) said they would use this test again, 42/52 (80.8%) said they would prefer to use the HCVST at home instead of going to a clinic and 51/52 (98.1%) said they would recommend this test to a sexual partner or friend.

First Response
All participants (59/59 (100.0%)) used the IFU, found the instructions easy to follow, and the device easy to use, while 56/59 (94.9%) participants were confident performing the test on their own. Most participants (58/59 (98.3%)) correctly identified the 15-min wait before reading the results and 55/58 (93.2%) found the mock results easy to interpret. When asked about future testing, 55/58 (93.2%) said they would use this test again, 49/59 (83.1%) said they would prefer to use this HCVST at home instead of testing at a clinic and 56/59 (94.9%) participants would recommend this test to a sexual partner or friend.

Discussion
This is the first study evaluating the usability of prototype blood based HCVST kits. At the time of the study, two evaluations of self-administered professional use oral fluid HCV RDTs [31,32] and one qualitative study on the acceptability had been published [33], however no reports had been published on HCVST usability to provide context for our findings. There is, however, a strong evidence base for the usability of HIVSTs, and their evaluations followed a very similar methodology. In early 2020, the HSTAR programme published an evaluation of 7 HIVSTs in the same region of Johannesburg, including 5 blood based HIVSTs [35]. These blood based HIVSTs had an average usability index of 93.6%, similar to the average usability index of the HCVST kits, which was 93.1%.
The high usability indices of the three prototype HCVST kits were supported by 86.5-100.0% of the participants that self-reported the tests were easy to use, which was consistent with the 96.3% of blood-based HIVST users from Johannesburg that also found their tests easy to use [35]. These results were also similar to other HIVST usability evaluations, including the 91% of Exacto HIVST users from the Central African Republic and the 94.4% of INSTI HIVST users in Kenya, who reported their respective HIVSTs were easy to use [38,39].
While participants reported that ease of use was the most important advantage of self-testing, participants from each prototype HCVST group liked that self-testing was fast, private and convenient. Similar attributes from HIVST studies have suggested that self-testing can remove barriers found in traditional clinic-based testing processes [14][15][16]. For HCV, barriers to testing include stigma and discrimination, which suggests that the privacy offered by HCVSTs may facilitate testing [9]. In South Africa specifically, a recent study among PWID identified barriers to accessing HCV treatment as fear of stigma, and long waiting periods [40], both barriers that can be facilitated by the speed, privacy and convenience offered by HCVST. HCVSTs have the potential to remove barriers linked with traditional HCV testing, so obtaining SRA approval for HCVSTs will help reach the 2030 targets [1,9,40], especially in key populations, like PWID.
Participants shared aspects of HCVST that they did not like, and these included unclear IFU steps and pictures, difficulties pricking themselves with the lancet, and difficulties transferring the specimen to the buffer. The lancet and specimen dropper/buffer difficulties were corroborated by the usability checklist, as steps concerning the collection of the specimen and the transfer of the specimen to the buffer were some of the only steps where more than 10% of participants made errors. The Johannesburg HIVST study identified difficulties collecting and transferring samples [35], as did an Atomo HIVST usability study in Cape Town, which reported lancing errors, the inability to collect a sufficient blood sample and transferring inadequate amounts of the sample [41]. Some of the HCV participants that cited difficulty with the IFU directly referenced the steps surrounding the collection and transfer of the specimen, which suggests that errors in specimen collection and transfer may be reduced by making clarifications/simplifications to the IFU.
On the HCVST checklists, almost 15% of participants did not wash and dry their hands according to the IFU, and over 10% of participants made mistakes wiping the first drop of blood away before collecting a sample. While these steps did not prevent the participant from completing the HCVST process, they may compromise the quality of the sample, possibly resulting in erroneous results. To see if errors in these steps would affect the accuracy of results and to determine the performance of HCVST, larger studies, similar to those for HIVST prequalification [23][24][25] should be conducted. While evaluation of the performance of HCVST prototypes was beyond the scope of the current study, we have assessed the ability of lay users to interpret the results of HCVST using contrived HCVST devices. For all three HCVST, contrived results were interpreted correctly by more than 90% of the participants. Invalid (T-line) results were interpreted correctly by 75-90% of the participants. The Johannesburg HIVST study did not have a contrived invalid (T-line) result, however the interpretation of contrived negative, positive, weak positive and invalid (no control) showed similarly high successful interpretation [35].
This study has shown high usability of prototype HCVST and has been useful in informing the improved development of blood based HCVST kits. While future usability studies on HCVST may lend from the methodologies described here, they should include more robust sample sizes from diverse populations and regions, especially in highprevalence settings and populations [27]. Furthermore, additional performance evaluations of HCVST kits will be needed primarily because high performance quality assured HCVST tools are crucial to enable further implementation research and policy development.
Lockdown restrictions to halt the spread of COVID-19, combined with over-burdened healthcare systems have led to the disruption of hepatitis screening programmes, with some countries completely halting hepatitis screening programmes [42]. Predictive models have shown that a 1-year delay in hepatitis interventions could lead to over 70,000 excess deaths by 2030, compared to the baseline scenario with no delay [43]. Currently there is no WHO recommendation on HCVST. The scaling up of HIVST during the pandemic, has been suggested to combat a similar drop in facility-based HIV screening [44], and the scaling up of HCVST could also fill the testing void due to COVID-19. It is therefore important for the WHO to take this into account as they consider updated guidelines on HCVST in early 2021 [45].

Limitations
The convenience sampling used in this study may have caused a sampling bias, as different proportions of sub-demographics were present between devices, specifically the participants' employment status. No key populations were involved in the study, so findings may be not applicable for these populations. The devices were evaluated in series to ensure that there was no cross-contamination, but the sampling of participants from the same community may have resulted in participants who enrolled later in the study becoming more aware of HCVST towards the end of the study. Furthermore, the participants from the same community have been previously sampled to participate in studies with HIVST. Although previous experience with any self-test excluded someone from participating in this study, they may have heard about self-testing from others in the community.
Although data collection was directed by previous HIVST usability assessments, there is no standardized usability test for HCVST, and at the time of data collection, there were no published studies of HCVST to compare methodologies with. The semi-structured productspecific checklists allowed for the quantification of usability through a usability index, however each device checklist was slightly different, which prohibited direct comparisons between devices. No diagnostic results were collected, so there was no way to determine whether user errors observed during study actually led to erroneous test results.

Conclusions
This HCVST study has shown that evaluated prototype blood-based HCVST kits exhibit high usability in the hands of untrained lay-users, with respect to their usability index, contrived results interpretation, and direct user feedback. Although there are no other published usability assessments of blood-based HCVST to give context to these results, our findings are in agreement with high usability scores for HIVSTs in similar settings [35,[38][39][40]. Given the high usability of these prototype HCVSTs, and their potential to remove barriers to traditional HCV testing, these findings are useful in informing the improved development of blood based HCVST kits towards SRA approval.