Efficacy of Handheld Ultrasound in Medical Education: A Comprehensive Systematic Review and Narrative Analysis

Miniaturization has made ultrasound (US) technology ultraportable and, in association with their relatively low cost, made handheld devices attractive for medical education training programs. However, performing an ultrasound requires complex skills, and it is unclear whether handheld devices are suitable for the training of novices. Our study aimed to identify to what extent handheld US devices can be employed in medical undergraduates’ and residents’ education. We selected studies that evaluate the results obtained by students and/or residents who have participated in ultrasound training programs using handheld devices. The studies were included if they reported post-test (pre-test optional) achievements or a comparison with a control group (a group of experts or novices who underwent a different intervention). Twenty-six studies were selected, and their characteristics were summarized. Handheld ultrasound devices were used in training programs to learn echocardiography, abdominal, and/or musculoskeletal ultrasound. Statistically significant progress was noted in the ability of naïve participants to capture and interpret ultrasound images, but training duration influenced the outcomes. While ultrasound training using handheld devices has proven to be feasible for various body regions and purposes (e.g., better understanding of anatomy, clinical applications, etc.), the long-term impacts of handheld education interventions must be considered in addition to the short-term results to outline guidelines for targeted educational needs.


Introduction
Reducing the size of medical equipment has become the norm, and size decreases are usually accompanied by a decrease in price [1].Ultrasound (US) probes can be carried in a lab coat and connected to smartphones and tablets, all for a fraction of their larger counterparts' costs [2].Their increased accessibility and high portability make them attractive for medical education and general practice alike [3,4].No recommended set dimensions and weights existed for ultraportable US devices until 2012, when the World Health Organization provided an expected dimension (13 × 7 × 1 cm), weight (300 g), and price range (between USD 5000 and USD 10,000) [5].However, the prices of handheld US devices ranges from around USD 3000 (USD 2699 for one probe + USD 420/yr membership for Butterfly iQ [6]) to around USD 9300 (Kosmos Torso-One [7]).The brands available on the market exhibit specific features and are superior in terms of specific characteristics (e.g., Lumify™ has the highest image quality, and Vscan Air™ is the best in terms of ease of use) [8].
Handheld ultrasound devices can be vital in developing countries and other resourcelimited settings, often being the only available imaging devices.Ultraportable devices have already been used in emergency medicine [9,10], remote areas [11], disaster responses (e.g., the New Mexico Disaster Medical Assistance Team used handheld ultrasound machines to aid patient care, and this proved effective in making management decisions [12]), and military situations (wherein they are easy to use, valuable in discerning a diagnosis [13]), or they can act as an auxiliary to clinical examination at point-of-care examination [14][15][16][17].
Ultrasound is a complex imaging modality, and image capturing and image interpretation require training [18,19], but there are still no standardized approaches regarding the content of training programs, their duration, or required trainer qualifications [20].Considering that ultrasound examinations are user-dependent, the results reported in the scientific literature support the accuracy in measurements and lower image quality when compared to conventional US cardiac examinations [21][22][23].
Handheld ultrasound devices are finding a place in medical students' pockets.Irvine Medical School (University of California) offered all first-year students a Butterfly iQ ultrasound probe [24].Handheld ultrasound technology has proven to be a valuable adjunct to physical examinations in experienced hands, despite the lower resolution of the images involved [25].For an inexperienced student or resident, lower image quality is more likely to impact their ability to make a correct diagnosis.On the other hand, the increased accessibility of handheld devices could make up for their downsides.In 2017, Galusko et al. [26] reported on the use of handheld ultrasound devices in medical education and concluded that data are scarce and incomplete.Considering technological developments, in this study, we aim to provide an update on the state of the art regarding the use of handheld US training in medical education, both for undergraduate medical students and residents.

Materials and Methods
Our systematic review was performed following the PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [27].The protocol of the review was registered on PROSPERO (CRD42021249070, first search).

Search Strategy
The search strategy was constructed to address the following:

•
Problem: medical students and residents lack of competence in performing ultrasounds.

•
Outcome: skills in acquiring and interpreting ultrasound images.

•
Study design: any exceptional case reports.
Our initial search of the literature was carried out on 15 May 2021.We performed a literature search on PubMed, ScienceDirect, and PMC using the following search string: (portable OR handheld OR pocket-size) AND ultrasound AND (education OR training OR students OR residents).
A second search was conducted with the same search string on 23 November 2023 to cover the latest studies reported in the scientific literature.
We selected studies that used a truly handheld ultrasound device in an ultrasound training program aimed at medical students and/or residents.We excluded studies that used any other portable ultrasound devices (e.g., laptop-size devices); studies involving training medical specialists instead of students and residents; secondary research papers (e.g., reviews, systematic reviews, meta-analyses); letters to editors; and case reports.
We collected information regarding the aim of the study, the type of ultraportable device that was used, the anatomical structures or pathologies that were assessed, the number of students or residents involved, the type of training program and its duration, the number of patients that were scanned, and data reflecting the performance of the educational intervention.

Article Selection
The selection of the articles involved a two-step screening process.The screening process was carried out after the elimination of duplicates using the Duplicate values . . .feature in Conditional Formatting (Excel, Microsoft Office 365), applied to title and/or identification numbers (DOI, or PMID, or PMCID).Two independent researchers screened all articles' titles and abstracts for relevancy to handheld devices (first step).The full texts were retrieved in the second step, and two independent researchers assessed the studies for inclusion.Consensus was mediated by a senior researcher to settle any disagreements.

Data Extraction
A self-developed extraction form was used for data extraction.The following information was collected independently by two researchers: (1) study settings (e.g., when?, where?, duration?, etc.); (2) study design (e.g., type, target condition, study aim, portable device used, participants); (3) study results (e.g., evaluated subjects, protocol and period of training, parameters used for the evaluation of trainees).

Quality of Reporting Assessment
The MERSQI tool [28], developed to appraise methodological quality in medical education research, was used to evaluate the quality of the studies.

Presentation of the Findings
A narrative method was used to synthesize the results, including a description of the study selection process, a summary of the characteristics of the included studies, an overview of the various study strategies employed, and conclusions.

Results
The search yielded 1525 articles after the removal of duplicates.A total of 42 full-text studies were assessed for eligibility, and 26 were selected and summarized (Figure 1).

•
A total of 94.74% of residents strongly acknowledge the benefits of HHUS.

•
A total of 137% mean improvement in image interpretation (3rd year).

•
Self-directed learning is not as efficient as supervised training sessions for gaining image acquisition skills.
Andersen et al., 2014 [33] • Nine hours of theoretical and practical training, as well as hands-on sessions by specialists.

•
Students used their own HHUS devices and performed at least 75 exams.• Six hours of echocardiography training using parasternal long-and short-axis, apical four-chamber, and subcostal views in B-and color Doppler-mode to detect significant* valvular lesions.
• US was superior to PE: significant MS, Ao stenosis and TR.

•
No improvement in physical examination (PE).• Four weeks of clinical traineeship for up to four people (HHUS for RUSH protocol).
• ~90% of students described the use of HHUS devices as helpful in improving their examination skills.

•
Students were able to identify the liver (60%) and spleen (40%) without help; they identified the major branches of the Ao (30%), only two branches of the Ao (40%), the portal vein (30%), and the bile duct (30%) with guidance.

•
A total of 30% could not locate any branches of the Ao, and all students needed help to locate the gall bladder and pancreas.

•
Goodman and Kruskal's gamma was applied to assess the students' abilities and their self-reported confidence levels.
• Generally, self-confidence levels were low.

•
Overall, 13-20% of students felt "very confident" about performing an ultrasound.• The participants were randomized into a video lecture or self-training group and completed preand post-tests.

•
The written post-vs.pre-test total scores were significantly higher.

•
The combination of the video lecture and telepresence instructions was perceived as effective preparation for the post-test.

Kaiser et al., 2022 [43]
• Ultrasound practical course with integrated hands-on activity using Vscan Air™, including duplex mode.The quality of the ultrasound images from previously selected organs was evaluated.

•
The rated image quality (18) never fell below a mean of 3 for the examined organs (liver, spleen, kidneys, rectovesical space, portal vein, pancreas, hepatic artery, and veins).• Five ultrasound-teaching sessions-anatomy education.

•
Five-point Likert questionnaire on self-perception.

•
Overall, 97% enjoyed the sessions, and 95% believed that US should be in the curriculum.

•
A total of 50 echocardiograms and 20 interpretations.

•
High agreement across all aspects: LV and RV structure and function and presence of effusion.

•
Three blinded assessors rated the image (max 10 points).
• Medium-term skill retention was reported.

•
The curriculum was sufficient for subcostal four-chamber and subcostal inferior vena cava skill retention but inadequate for parasternal long axis, parasternal short axis, and apical four-chamber.

Slader et al., 2022 [45]
• Two days of instruction on core ultrasound fundamentals on two groups: handheld devices vs. standard US.

•
Qualitative survey regarding personal experience.

•
Overall, 604 images were obtained and graded by two emergency medicine physicians.

•
No statistically significant difference in participants' perceived difficulty of ultrasound learning, comfort level, or self-estimated capability to perform ultrasounds in the future.

•
The median quality score of the images obtained by the standard group was eight compared to seven in the handheld group (p < 0.01).

•
One-hour Zoom POCUS education program and in-person training (urology), and two weeks independent practice.

•
Assessment of ultrasound competency and participant satisfaction.

•
The written exam scores were 3.36/5 in the training phase.

•
Students expressed satisfaction with the program.

Training
The training programs ranged from roughly 30 min [30] to up to 8 weeks [32], and all programs offered hands-on training.The minimum amount of time dedicated to practical training per student was 10-15 min [30], while other programs offered unlimited access to simulators/handheld devices for the entire duration of the training (up to 4 weeks) [32].Two studies provided unlimited access to learning resources, implying a partly self-directed framework [32,50].In some studies, the training was completely self-directed [41,50].

Outcomes
The heterogeneity of the measured outcomes is a characteristic of the evaluated studies, but most of them targeted or included the cardiac system (Table 3).Some studies reported a significant increase in diagnostic accuracy when handheld ultrasound technology was used in concert with a physical examination [35,39,53].Some contradictory results have been noted in the evaluated articles.For example, two articles reported adequate imaging and measurements of the abdominal aorta diameter by students in more than 92% of cases [31,33], with no learning curve noted over time [31], while only 50% of residents adequately imaged the abdominal aorta [49].
One study reported a higher rate of false positives after training, explaining that a short training duration is most probably insufficient [48].Self-directed learning is reported to be effective in acquiring a theoretical knowledge of ultrasounds, whereas practical skills appear to be more effectively gained with the help of an expert sonographer [32].
Almost 90% of students consider handheld ultrasound devices helpful for improving clinical examination skills [39], 98% would like to see the continuation of ultrasound training programs in their subsequent university years [36], and 94.74% of residents agreed that access to a handheld ultrasound device improves the ability to perform imaging exams [50].When given a choice, 96% of the students stated that they would prefer to use conventional ultrasound technology over handheld ultrasound technology due to the higher image quality and more intuitive user interfaces involved [29], while the medical image quality scores proved significantly lower in HHUS group [45].The majority of participants felt comfortable learning the new technology (95.3% [46]), enjoyed learning abut anatomy (94.4%), and felt they gained a better understanding of the clinical relevance of learning about anatomy (97.2%) [46].Students also reported that "it would be useful to have more ultrasound session" [42], and the majority believed that US training should be in the curriculum [46].
Edwards et al. [46], on their study on using handheld US technology in anatomy training, reported a thematic analysis of the three dimensions of the barriers in US training: image interpretation (lack of familiarity, small screen, hard to achieve a clear image, etc.), future enhancement (more background knowledge, longer or more sessions, larger screen, independent learning, etc.), and religious barriers (contact with the opposite sex and modesty concerns).

Medical Education Research Study Quality Assessment
Most of the evaluated studies applied a pre-test and post-test study design, objectively measuring the participants' knowledge and going beyond statistical analysis, but none described the validity of the applied evaluation instruments (Table 4).

Discussion
Handheld ultrasound training has proven feasible for various body regions and purposes (e.g., by providing a better understanding of anatomy, increased diagnostic accuracy when used in concert with a physical examination, etc.) with or without assistance in the identification of organs.However, when available, a conventional high-end ultrasound device was preferred, while the displaying of US images on a small screen was reported as a barrier of handheld ultrasound examinations.

Rethinking Medical Education
The development of ultrasound technologies changes diagnoses (including ultrasoundguided punctures) and treatment in healthcare (e.g., interventional ultrasound).Ultrasound devices come in different sizes (from handheld devices to high-end devices), and prices (starting from USD 3000 in the case of handheld devices [6]) and features (e.g., image quality, ease of use, portability, costs, and probes) vary [8].Nielson et al. [55] highlighted the need to define the "minimum requirements for equipment, education, training, and maintenance of skills" for all medical specialists and medical students.Given the limited number of high-end US devices, handheld US devices could be an alternative for entry-level medical US training.It has already been demonstrated that short-term (a few hours to a few days) handheld ultrasound training can significantly improve students' and residents' abilities to capture and interpret specific ultrasound images accurately [31,[33][34][35]41,42,44]. Limiting the number of trainees in a group and increasing the number of procedures per trainee improve performance [38].Short-duration courses might increase the number of false positive diagnoses, highlighting the importance of sufficient practice [48].
Regardless of the dimension, the US examination had some advantages compared to other imaging techniques, such as lack of radiation, low cost, reproducibility, and the possibility to examine patients at point-of-care [56].The trainees preferred a conventional high-end ultrasound device when available [29], while generally, the measurements were concordant [23,45,57], and the difficulty of US learning and comfort levels were similar (p > 0.15) [45].
Training to acquire appropriate images and interpret the images is a must, but the optimal duration and frequency of continuous hands-on sessions still need to be determined.It may be unrealistic to expect long-lasting progress after a few hours of participation in training programs.The performances of the students in terms of identifying anatomical structures via ultrasound significantly drop at medium follow-up (6 months) in the absence of repetitions [44,58].In ultrasound training, the consistency of the educational programs is essential, especially as their benefit has been shown to wane over time as ultrasound skills can diminish due to a lack of practice.For example, the expected number of (conventional) US examinations and interpretations in the case of focused cardiac ultrasound and transthoracic echocardiography needed to obtain accreditation depends on the level of certificate (e.g., having performed from 20 to 250 exams and up to 300 interpretations [59]).
Hellmann et al. [60] evaluated the learning curve of thirty medical residents participating in an echocardiography training program and reported an overall technical proficiency skill improvement of 0.79 points (95% confidence interval [CI] 0.53-1.04,0-3 assessment scale) per 10 scans completed.Interpretation accuracy also improves with an increase in the number of examinations (1.01 points, 95% CI 0.69-1.39per 10 scans, considering a 0-3 accuracy index) [60].Considering that ultrasound training programs for specialists take about eight weeks of intense training for the main systems (e.g., cardiac, vascular, abdominal, musculoskeletal), four weeks of basic ultrasound learning and four weeks of advanced ultrasound learning, with practical hands-on sessions lasting for several hours a day (i.e., a more extensive training program) would likely be more effective.
Radiologists' views regarding ultrasound teaching might differ from the opinions of clinicians.Webb et al. [19] highlight that teaching students a few ultrasound basics would instill a false sense of confidence in their skills, considering the extensive training required to perform an ultrasound at the level of a board-certified radiologist.Webb et al. [19] also noted that no established systems ensure minimum competency or examination quality, even though students and medical administrators demand the inclusion of ultrasound training in medical school curricula [42,46].
Compared to physical examination alone, handheld ultrasound has been repeatedly proven to increase physicians' ability to establish correct diagnoses [51,61,62].A continuous approach to training should be employed to prevent the loss of knowledge and improve image acquisition and interpretation skills.At Loma Linda University, ultrasound education programs start in the first year of study and continue throughout the four years of the undergraduate medical curricula [63].Dinh et al. [64] showed that ultrasound teaching during the first year of medical school significantly improves student skills and emphasizes the importance of early implementation during medical education when comparing the performances of US-trained students with those of untrained students.
Potential problems that can arise concerning ultrasound training programs are tutors having insufficient experience, a lack of established structure for the training programs, time-limited instructions, and an insufficient number of ultrasound devices (either handheld or conventional).The use of handheld devices for training novices (e.g., students and inexperienced physicians) can be more challenging due to the lower quality of the images involved [28,46,48].The most important characteristics of handheld ultrasound devices are image quality, ease of use, portability, total costs, and the availability of different probes [8].Image quality differences exist between different handheld probes and targeted examinations [8,65].It should be noted that the differences in image resolution, detail, and quality vary according to the conventional US device used and user experience [66][67][68].

Handheld Ultrasound Devices in the Future
Rapid technological advancements and innovations will probably lead to an improvement in the quality of the images associated with handheld ultrasound devices, eventually causing the quality of these images to be close to that of the images obtained using high-end US devices.This expectation is supported by the appearance of ultrasound-on-a-chip-and silicone-chip-based handheld ultrasound probes [69], with an increase in imaging quality being expected roughly every two years.Augmented reality and extended reality algorithms are expected to be implemented to assist the user in patient positioning and guide the acquisition of images as support for both training and clinical examinations [69,70].Furthermore, diagnostic algorithms, along with light and affordable handheld devices, could make ultrasound examinations available to more patients [71], while internet connection could allow physicians to work from locations far from hospital wards (i.e., working remotely) [72].Murray et al. [73] already reported the feasibility of virtually supervising POCUS handheld ultrasound examinations carried out by 3rd-year undergraduate medical students in four domains, namely image generation, image optimization, clinical integration, and knowledge of indications, showing the effectiveness of digital-assisted evaluation.
Ultrasound training is expected to be included in the undergraduate medical curriculum.As US technology evolves, the emergence of smaller and more affordable handheld probes with the ability to store examination data in a cloud-based open-data system is expected, facilitating real-time data analytics.

Limitations
Our review was limited to studies published in English.The evaluated studies only sometimes reported the methods used to decide which students and residents should be included in their study, and mainly, participant recruitment was based on voluntary participation.Voluntary participation induces a selection bias because the participants might have a particular interest in the technology under study.Consequently, the reported results do not reflect the target population represented by undergraduate students and residents and might only apply to some students and junior doctors.The evaluated studies had distinctly different training programs and evaluation methods, and this did not allow for the quantification of the optimal duration of instruction in relation to the trainees' performances.Furthermore, the selected studies were indexed in some specific bibliographic databases from 2010 to 2023, reflecting the temporary stalling of hands-on medical education programs.The evaluated studies' heterogeneity in terms of training duration and targeted organs did not allow us to perform a quantitative analysis, so the presented results are only descriptive and have limited applicability.

Conclusions
Scientific data regarding the use of handheld ultrasound training involving undergraduate students and residents is limited and heterogeneous in regard to curricula, duration of training, and targeted organs and pathologies (most frequently cardiac).The students in the evaluated studies proved they can accurately examine specific organs, but the long-term impacts of handheld education interventions must be considered in addition to students' short-term performances to outline guidelines for targeted educational needs.These guidelines must be established to ensure that trainees retain proficiency skills.Furthermore, the financial burn and associated effects must also be evaluated.

Table 1 .
Summary of studies with residents and others as the target population, as well as what was studied, which anatomical structures or pathologies were investigated, the number of participants who underwent training, and who the US investigations were carried out on.

Table 2 .
Summary of studies with students as the target population, as well as what was studied, which anatomical structures or pathologies were investigated, the number of participants who underwent training, and who the US investigations were carried out on.
HHUS = handheld ultrasound; US = ultrasound; std = students; LV = left ventricle; RV = right ventricle; LA = left atrium; Ao = aorta; n/a = not available.* non-medical students completing a course with a focus on anatomy and physiology.a FAST (Focused Assessment with Sonography in Trauma): Morrison's pouch, splenorenal recess, four-chamber view of the heart, bladder and pelvis; b muscles, blood vessels, tendon, nerves, bones, menisci; c Ao stenosis/regurgitation, mitral stenosis/regurgitation, tricuspid regurgitation.

Table 3 .
Description of training and presentation of key findings.

Table 4 .
Quality of reporting assessment with MERSQI (Medical Education Research Study Quality Instrument).