Auditing the Audits: A Systematic Review on Different Procedures in Telemedicine

Telemedicine is a process of delivering health care using information and communication technologies. Audit and feedback (A&F) constitute a systematic intervention that is aimed at collecting data, which are subsequently compared with reference standards and then returned to health care operators through feedback meetings. The aim of this review is to analyse different audit procedures on and by mean of telemedicine services and to identify a practice that is more effective than the others. Systematic searches were performed in three databases evaluating studies focusing on clinical audits performed on and by means of telemedicine systems. Twenty-five studies were included in the review. Most of them focused on telecounselling services with an audit and a maximum duration of one year. Recipients of the audit were telemedicine systems and service users (general practitioners, referring doctors, and patients). Data resulting from the audit were inherent to the telemedicine service. The overall data collected concerned the number of teleconsultations, service activity, reasons for referral, response times, follow-up, reasons why treatment was not completed, technical issues, and other information specific to each telemedicine service. Only two of the considered studies dealt with organizational aspects, and of these, only one analysed communicative aspects. The complexity and heterogeneity of the treatments and services provided meant that no index of uniformity could be identified. Certainly, some audits were performed in an overlapping manner in the different studies, and these show that although attention is often paid to workers’ opinions, needs, and issues, little interest was shown in communicative/organizational and team dynamics. Given the importance and influence that communication has in teamwork and care settings, an audit protocol that takes into account intra- and extra-team communication processes could be essential to improving the well-being of operators and the quality of the service provided.


Introduction
Telemedicine, Audit and Feedback The term "Telemedicine" (TM) refers to a mode of delivering healthcare services through the use of innovative technologies. When two or more individuals are not in the same location or are constrained in their movements, they can make use of telemedicine. This can occur both between health professionals and between the health professional and the patient. Since the process may also involve at-home patient self-care and self-monitoring regimens, the term TM is used interchangeably with "telecare" and "telehealth" [1]. The first case of telemedicine utilization dates back to 1906 when Dutch physiologist W. Einthoven carried out trials of remote electrocardiographic consultation using a telephone [2]. Due to the inadequate communication systems of that period, the experiment proved ineffective [3], but today TM is widely used. The rapid progress that has been made in the information technologies field [4] has allowed this health service to complement and support traditional medicine in four different modalities:

1.
Teleconsultation: This term can refer to two distinct processes.
• Remote counselling between health professionals from different areas of intervention. Consultation often occurs because of the absence of a medical specialist and concerns a patient's diagnosis or treatment [5]. • Remote real-time medical examination in which one or more physicians assess a patient's state of health using information technologies. Currently, there are two main options to deliver this kind of visit: purchasing a solution from a vendor and installing the study premises or buying video conferencing as a service from a vendor [6]. The first option allows easier management and better security control of devices, but the second option (the most popular one) involves:

2.
Remote cooperation: In emergencies or critical scenarios, one physician helps and cooperates with another online through real-time video and audio communication.
These scenarios could include both the recent COVID-19 pandemic [7] and operating room surgery [8,9]. In this latter case, the surgical team operates on the patient, while the remote participants can collaboratively work thanks to high-quality video-audio streams.

3.
Telemonitoring: The periodic biomedical remote monitoring of the patients from a tablet, smartphone, or computer web browser is connected with low-cost and basic medical devices such as blood pressure cuffs, thermometers, and digital scales [10].
These modalities make telemedicine capable of creating new opportunities and improving the health service delivered, bringing several benefits. In remote areas poorly connected to major cities, TM can greatly increase the equity of access and the availability of skilled health care [11] and facilitate patients' engagement in their own care [12,13]. Furthermore, telemonitoring can improve the quality of life in chronic patients through self-management solutions and improving self-care, patient education, treatment adherence, and survival [14]. Several studies suggest that TM seems to be cost-effective in that it reduces hospital admissions and improves the management of chronic disease and patient compliance [15].
Despite all these great benefits, TM is not without problems and limitations. Indeed, medical and health service researchers have begun to note that telemedicine initiatives frequently fail to achieve uptake or to demonstrate significant benefits to patients [16][17][18]. Factors such as the lack of integration and interoperability of computing systems and software, a failure to achieve cost-effectiveness, and the negative attitudes to telemedicine on the part of medical practitioners have been identified as barriers to successful telemedicine initiatives [16,18,19].
These can be traced back to both technical and human factors. To evaluate an entire TM process (even from different perspectives) and highlight any critical issues, an audit can be used.
An audit is a process of systematic data collection and comparison that seeks to improve the quality and outcome of different processes or practices through the modification of dysfunctional parameters [20].
This term is often used accompanied by attributes that indicate specific approaches to assess the quality of care or other aspects of the evaluated process: • Economic-managerial audit: aims to assess the appropriateness and efficiency of planning, the management of resources, and expenditure. • Organizational audit: evaluates organizational processes (e.g., information system, workload assessment, organisational procedures aimed at the acquisition and support of technologies). System audit: examines the main organisational aspects, practices, procedures, and controls that support the efficiency of an entire organisation, a hospital, or a department. Clinical audit: is an intervention aimed at collecting and comparing clinical data with reference standards [21,22]. It is fundamental to improving clinical practice and equity in different settings.
Beyond these approaches, an audit can be internal or external. In the former case, an organisation (e.g., a hospital) is reviewed by an internal group of experts who evaluate its performance to identify possible improvements. On the other hand, an external audit is performed by professionals who are not associated with the assessed service, facility, or treatment.
An audit multidisciplinary team provided a series of quality indicators on which data are collected and then compared with the reference standards. Subsequently, any resulting deviation from these standards is communicated to professionals in a structured phase called "feedback". This causes the audit to be conceived as a quality improvement process that aims to enhance patient care and outcomes.
The aim of this systematic review is to analyse different audit procedures on and through the means of telemedicine services.

Materials and Methods
A systematic review of currently published studies was performed following the standard guidelines. Online database searches were performed for articles published before 7 October 2022 on audits performed on and by telemedicine systems. The literature search was conducted via PubMed, Cochrane Central Register of Controlled trials, and Web of Sciences. It was carried out using the following search keywords and terms: ('telemedicine' OR 'telehealth' OR 'telecare' AND 'audit').
Inclusion criteria A study was included if it investigated a clinical audit performed on or through the means of telemedicine systems. Only articles written in English were included in the review.
Exclusion criteria A study was excluded if there was a lack of telemedicine systems used and if it was a dissertation, commentary, letter, or editorial. Systematic, integrative, or narrative reviews were also excluded, although their reference lists were checked and included if appropriate. No restriction due to the year of publication was adopted since the development of telemedicine is recent.

Results
The initial electronic data search yielded a total of 590 potentially relevant studies (327 results on PubMed, 211 on Web of Science, and 52 on Cochrane Library); of these, 159 were duplicates, and 10 were non-English language articles ( Figure 1).

Studies Data
Recipients of the audit performed by each study taken into account included telemedicine systems and service users. The latter category included general practitioners (GP), referring doctors, and patients.
The duration of the studies analysed by this review ranged from one month to eight years, but most of them (n = 14, 56%) took place within a maximum period of one year. Of course, all the studies focused their attention on data inherent in the telemedicine service. The overall data collected concerned: the number of teleconsultations, service activity, the reason for referral, response times, follow-up, reasons why treatment was not completed, technical issues, and other information specific to each telemedicine service.

Satisfaction
In five out of the twenty-five studies [23,24,41,42,44], the satisfaction of users taking part in services was assessed (Tables 3 and 4). They could be both patients and physicians. Almost all respondents were satisfied or highly satisfied with the service; most of them found it useful and said they would use it again. They found TM to be a process that:  General information was collected on every caller. Calls were divided into two main groups: a "registered" group and a "generic" one. Only the first group received specific clinical advice via a letter to their GP.

Service Users 1121 calls received
Demographics, relationship between caller and patient, consultation  Administrative and medical records from the TA service were retrospectively reviewed. Patients' records were taken from the electronic database of the "Salvatore Maugeri Foundation" TA Service in Lumezzane (Brescia). Administrative and medical records from the TA service were retrospectively reviewed.
Patients 396 consecutive CRF patients were analyzed.
Demographics, call characteristics, time of staff activity, salary cost evaluation, cost of the service.
Williams M. 2020 [41] An audit of a telehealth service for children and adolescents with type 1 diabetes mellitus at four rural sites. A feedback survey was administered to satisfaction with the telehealth service, with preference for in-person or telehealth consultation, and any concerns regarding deficits in care and suggestions for improvements.  A one-month audit of booked telehealth was completed, looking for anything that impacted on the delivery or billing of telehealth. Each clinician was also asked for feedback on the day of their consultation.

Clinicians 125 clinician feedback
Clinician feedback on telehealth appointments, Billing, Scheduling, and requesting telehealth.

Feedback
In one of the articles mentioned above [44], an A&F process was given via telemedicine. Specifically, in this article, participants received continuous feedback via a specific application. These healthcare professionals found this kind of feedback a valuable way to improve clinical practice; in fact, according to their opinion: In addition, it has emerged that it is not only important to communicate the error but also the timing and manner in which this occurs.

Organizational Aspects
The organizational aspects of telemedicine systems were evaluated in only two studies (8%). One of these two articles [40] took into account only the time of staff activity, while the other [31] specifically evaluated differences in the workflow of operators, training, changes in culture, attitude, and management. The latter was the only study in the sample to consider communicative aspects in the members of the healthcare team.
Data pertaining to the economic part were collected in 5 studies [26,31,39,40,46]. One of them included economic aspects as one of the indicators to be evaluated, but then no data were recorded because it was not in line with the aims of the study [31]. Another article [46] evaluated cost-effectiveness, but in reference to drug treatment. Using the telemedicine service shows a reduction in costs by 23% [39] and 39% [40]. Furthermore, avoided transfer and hospitalizations resulted in an overall total savings of AUD 1,892,584, or AUD 6460 per patient [26].
Within the sample, four studies were considered [28,34,36,38] as conducted during the COVID-19 pandemic. Of these two [28,38] was a confrontation between the TM service before and during the COVID-19 pandemic.

Models
Some of the studies analysed in this paper have referred to specific practical-theoretical models that are aimed at bringing about concrete and measurable change.
Owen et al. [36], in their study, adopted the RE-AIM framework: a planning and evaluation model that addresses five dimensions of individual and setting-level outcomes that are important to program impact and sustainability [48]. The above five dimensions are: reach the target population, effectiveness, adoption by target staff, settings, systems and communities, implementation costs and adaption, maintenance/sustainment of intervention effects in individuals, and settings over time (https://re-aim.org/learn/what-is-re-aim/, accessed on 22 November 2022).
Haydon et al. [31] used the model for assessment of telemedicine (MAST): an evaluation framework that focuses on the measurement of effectiveness and quality of care [49]. The MAST represents a multidisciplinary process, evaluating the medical, social, economic, and ethical aspects of telemedicine in a systematic, unbiased, robust manner [50].
The use of MAST includes three steps [51]. In the "preceding assessment" (Step 1), the maturity of TM technology and the organization using the device is assessed. If the maturity of the technology/organization using the TM service needs to be developed further, formative studies, including participatory design (PD)/usability/feasibility studies, must be carried out. Following, a "multidisciplinary assessment" (Step 2) could be carried out to analyse seven domains, including the identification of health problems and the characteristics of the application; safety; clinical effectiveness; patient perspectives; economic aspects; organisational aspects; and socio-cultural, ethical, and legal aspects. Finally, an assessment should be made of the transferability of the results (Step 3) reported in studies concerning the previous steps.

Discussion
The aim of this review was to identify an audit procedure that was more widely used or more effective than others. However, the complexity and heterogeneity of the treatments and services provided and examined by each study did not allow for the identification of a uniformity index. Certainly, some audits are performed in an overlapping manner in the different studies, and these show that although attention is often paid to workers' opinions, needs, and issues, little interest was shown in communicative/organizational and team dynamics. Kapoor et al. [52] found that human factors are most responsible for the successful outcome of their telemedicine project. In fact, supporting a telemedicine program can require a unique set of skills compared to a standard consultation. Several studies also elicit the importance of communication within teamwork, especially when this occurs within a healthcare setting [53][54][55]. There is a need for someone who can interface with the clinical staff without being out of their depth: this person will obviously need to be an excellent communicator and should have the ability to speak in a plain, nontechnical language that end users can understand [6]. Communication and interpersonal skills turn out to be crucial, especially in highly time-dependent contexts such as emergency-urgency departments [56].
Furthermore, in a study that analysed TM from an organizational perspective [57], authors found that a dearth of perceived central leadership, a lack of information about the technology itself, poorly designed and cumbersome means for scheduling and utilizing telemedicine technology, the absence of explicit strategic goals, and poor communication in introducing this technology to medical personnel and to the public would undermine the telemedicine process.
Then, it might be useful and interesting to identify a standardized audit procedure that takes into account some cross-disciplinary and psychological aspects, such as communication. In this regard, Hargie and Tourish [58] in 1993 proposed an audit methodology for communication. First, since performing this process on a very large organization or telemedicine service can raise ethical and economic difficulties that may even interfere with the daily functioning of most organizations, they suggest a full and comprehensive audit, but for a small section of an organization or of a TM service. In general, they refer to 'the use of certain survey instruments'. For example, they suggest validated questionnaires and/or semi-structured interviews to evaluate current communication practices, identify areas of strengths and weakness, and suggest how improvements could be affected [59,60]. Subsequently, audio or video recordings of interpersonal encounters could be analysed and evaluated, as well as live observations of such episodes in situ. Another simple and readily available instrument is diary analysis, through which it is possible to obtain information concerning communication contacts over a set period of time.

Conclusions
Audit and feedback constitute a fundamental process for controlling and improving the quality of service. Data collection and comparison with reference standards take place in depth, especially with regard to the treatment process and the results obtained. However, elements that have a strong impact on the process and, consequently, on its results are often overlooked. These elements include communication, motivation, organisation, workrelated stress, and operators' coping skills. This is why it would be useful to broaden the concept of "quality" by delving more deeply into the psychological aspects related to the staff and their work.
Future studies might include a wider range of keywords in order to explore further aspects of this challenging topic. Moreover, an effective and universally recognized set of targeted audit protocols could be developed based on future research focused on refining, updating, and standardizing existing procedures to evaluate organizational aspects such as communication.