Abstract
Background: Spectral Doppler ultrasound examination of pedal arteries is one of the most frequently used noninvasive assessment methods by health-care professionals for the diagnosis and ongoing monitoring of people at risk for or living with peripheral arterial disease. The aim of this study was to determine the interrater reliability of the interpretation of spectral Doppler waveform analysis. Methods: An interrater reliability study was conducted among five experienced podiatric physicians at the University of Malta Research Laboratory (Msida, Malta). A researcher who was not a rater in this study randomly selected 229 printed spectral Doppler waveforms from a database held at the University of Malta. Each rater independently rated the qualitative spectral waveforms. Results: Interrater reliability of the spectral Doppler waveform interpretation was excellent among the five experienced podiatric physicians (α = 0.98). The intraclass correlation coefficient showed a high degree of correlation in waveform interpretation across raters (P < .001). Conclusions: This study demonstrates high interrater reliability in visual spectral Doppler interpretation among experienced clinicians. The current foot screening guidelines do not refer to spectral Doppler waveform analysis in their recommendations, which has been shown in studies to be an important modality for the diagnosis of peripheral arterial disease when ankle-brachial pressure indexes are falsely elevated in calcified arteries. If interpreted correctly, the information obtained can provide an indication of the presence of peripheral arterial disease and facilitate early management of this condition.
Although diagnostic and therapeutic decisions in patients with vascular disease are guided primarily by the medical history and physical examination, the use of noninvasive investigations has increased significantly in recent years, mainly as a result of technological advances in ultrasonography.[1] Ultrasonic spectral Doppler devices are widely used by health-care professionals for the assessment of peripheral arterial disease (PAD).[2] Early detection of peripheral vascular disease is imperative because PAD is a significant public health problem; however, as many as 50% of patients living with PAD are asymptomatic.[3] It has been estimated that only approximately 25% of patients living with PAD receive treatment because most people are not aware that they are living with this condition.[4] Despite the lack of symptoms, such patients still share the same prognosis as symptomatic patients,[5] showing a risk profile comparable with that of patients with symptomatic lower-extremity PAD or with chronic heart disease.[4]
Regular screening of all patients at risk for peripheral vascular disease, especially people living with diabetes, is essential to ensure early identification of asymptomatic individuals who are at high risk for the disease or its complications through appropriate screening tests.
Several countries and organizations, such as the World Health Organization and the International Diabetes Federation, have set goals to reduce the rate of amputations by up to 50%. Implementing a foot screening and protection program for patients with risk factors for ulceration reduces morbidity and is cost-effective[6]; however, a diabetic foot screening tool needs to be evidence based and relevant to the characteristics of the target population.[7] Furthermore, inaccurate interpretation of spectral Doppler ultrasound outputs by clinicians could contribute to the suboptimal management of PAD or conversely subject patients to further unnecessary investigation or misdiagnosis.[8]
Auscultation and palpation of the proximal lower-limb pulses can give an indication of the condition of the underlying vessels.[9] Handheld continuous wave spectral Doppler ultrasound has long been considered a reliable tool and has been validated and accepted into routine medical practice.[10] Spectral Doppler ultrasound examination of pedal arteries is one of the most frequently used noninvasive assessment methods by health-care professionals for the diagnosis and ongoing monitoring of people at risk for or living with PAD.[11]
In the foot, the two most frequently examined arteries are the dorsalis pedis artery and the posterior tibial artery. Both audio and visual analysis of spectral Doppler waveforms could be performed by clinicians[12] and are usually classified as triphasic, biphasic, and monophasic. However, the current literature has shown that because there is considerable variability among sonography professionals and educators in defining and classifying peripheral arterial waveforms, the issue of spectral Doppler waveform classification, particularly pathologic waveforms, deserves higher priority in research.[11]
A variety of diabetes foot screening guidelines and expert consensus documents, including those by the International Working Group on the Diabetic Foot, have been formulated during the past decade by various organizations—or experts—in the field, recommending both palpation of pulses and measurement of the ankle-brachial pressure index (ABPI) as the recommended screening tools for PAD in the diabetic population.[13] However, the literature highlights major limitations in ABPI measurements in people with diabetes, and spectral Doppler has been shown to be a more accurate and suitable screening tool in this population, which is known to present with calcification of arteries. Note that in the evaluation of the individual patient there may be errors and that the reliability of any diagnostic test depends on the previous probability of disease (the Bayes theorem). This is mainly a result of poorly compressible arteries due to the presence of medial arterial calcification due to smoking or underlying disease, thus rendering the diagnosis of PAD by ABPI alone less reliable and inaccurate due to artifactually raised occlusion pressures, especially in patients with diabetes mellitus and end-stage renal failure.[14]
However, a clinical assessment tool is only of value if the interpretation of the results is correct and repeatability has been clinically established.[15] This prompted us to conduct this study to further evaluate and determine the interrater reliability of the interpretation of this testing method among experienced clinicians. This is essential to determine the clinical efficacy of using this technique for early diagnosis of PAD and ongoing peripheral vascular screening and monitoring.
Methods
An interrater reliability study was conducted among five experienced podiatric physicians at the University of Malta Research Laboratory (Msida, Malta). A prospective, single-blind, comparative study design was used. This study was approved by the University of Malta Research Ethics Committee. All investigations were performed in accordance with the principles of the Declaration of Helsinki as revised in 2000.
Raters
Five experienced podiatric physicians (ie, the raters) with previous hospital experience with spectral Doppler assessments in people with diabetes were randomly selected and invited to participate in this study. These podiatric physicians had all received undergraduate training during their respective courses from a specific university. Written informed consent was obtained from each participating podiatric physician. All of the raters had previous extensive experience (>5 years) with the use and interpretation of spectral Doppler ultrasound for lower-limb vascular assessment.
Procedure
A researcher at the University of Malta who was not a rater in this study used simple random sampling techniques to select 229 printed spectral Doppler waveforms of the posterior tibial artery and the dorsalis pedis artery from a diabetes spectral Doppler waveform database held at the same university. This method of random sampling is the most popular method for choosing a sample. It has been stated that the logic behind this sampling is that it removes bias from the selection procedure and results in the representative samples.[16] A power calculation was not performed for the purpose of the study owing to financial constraints.
Each waveform had previously been acquired by an experienced clinician using a Dopplex Assist color spectral Doppler device (Huntleigh Healthcare, Cardiff, Wales) under standard procedures, with an 8-MHz probe held at 60° against the blood flow of the dorsalis pedis and tibialis posterior arteries of patients living with type 2 diabetes. Each waveform was consequently transferred to proprietary AR2 software, which is a database that allows both storage and postanalysis of waveforms.
Each rater (n = 5) independently rated the qualitative spectral waveforms and interpreted these as triphasic, biphasic, monophasic, or monophasic continuous (Fig. 1) using standard medical definitions.[17-20] Triphasic patterns are the result of the combination of ventricular systole, elasticity of the blood vessels, and the backflow caused by the closing of the semilunar valves; biphasic patterns occur in more distal blood vessels as the result of ventricular systole and the elasticity of the blood vessels; monophasic patterns occur when the flow of blood is no longer pulsatile; and more severe disease is represented by monophasic continuous waveforms with attenuated amplitude and continuous flow.
Figure 1.
Spectral Doppler waveforms as visualized on the proprietary AR2 software. Waveforms are depicted as triphasic (A), biphasic (B), monophasic (C), and monophasic continuous (D).
Data Analysis
Interrater reliability of the qualitative spectral waveforms among the five podiatric physicians was calculated by determining the level of agreement between measures using the Cronbach alpha and the intraclass correlation coefficient (ICC). The Cronbach alpha is a measure of internal consistency, that is, how closely related items are as a group. It is considered to be a measure of scale reliability. This test was performed to measure the coefficient of reliability (consistency). The ICC[2,1] was used for the assessment of consistency of measurements made by the five different raters measuring the same variable.
Results
There was complete agreement in spectral Doppler waveform interpretation across the five raters in 142 (62%) of the 229 waveforms presented for analysis. Of the remaining 87 waveforms (38%), 20 were triphasic, 22 were biphasic, 22 were monophasic, and 23 were monophasic continuous as classified by at least 3 raters. Using IBM SPSS Statistics for Windows, Version 21.0 (IBM Corp, Armonk, New York), internal consistency was measured using the Cronbach alpha test, which registered excellent internal consistency (α = 0.980). Furthermore, interrater reliability of the spectral Doppler waveform interpretation was found to be excellent among the five experienced podiatric physicians, with significant (P < .001) ICCs of 0.904 for single measures and 0.979 for average measures with the two-way mixed effects model (Table 1).
Table 1.
Intraclass Correlation Coefficients (ICCs)
Discussion
The aim of this study was to determine the interrater reliability of the interpretation of qualitative spectral waveform analysis among experienced clinicians. The results of this study suggest that podiatric physicians had a high skill level in the interpretation of visual spectral waveform analysis. Correct identification of the four types of blood flow patterns was observed to be in agreement among the five raters participating in this study. These results are similar to those observed in a recent study[11] in which private and public podiatric physicians obtained interrater and intrarater reliability of κ = 0.9 and κ = 0.77, respectively, when interpreting visual waveform analysis. In the present study, however, we felt the need to further subdivide pathologic waveform classification into monophasic and monophasic continuous because the severity of PAD and the treatment regimen may vary substantially between these two clinical presentations.
Regarding clinicians from different professions, the literature reports conflicting results related to interrater reliability of the interpretation of spectral waveforms among clinicians having different levels of expertise in spectral Doppler assessment and interpretation. Review of the literature shows that poor training, inexperience, and time constraints may result in errors when interpreting spectral Doppler ultrasound outputs[8,15,18] and that interobserver and intraobserver reliability is variable and depends on experience and technique.[9] Studies have found that qualified staff, trained in vascular assessment, are more accurate than junior staff of their respective professions in Doppler sound interpretation.[8,21] This study confirms this finding and concludes that when spectral Doppler waveform reports are interpreted by experienced clinicians in the field, accurate interpretation of the patients' lower-limb arterial status is accomplished.
The level of experience in interpreting spectral waveform analysis is of maximum importance if this method of screening/assessment for arterial perfusion is to be recommended in the high-risk foot. We recommend that peripheral arterial perfusion be assessed using spectral Doppler waveforms. Diabetes foot screening guidelines do not refer in any way to spectral Doppler waveform analysis in their recommendations,[22,23] which has been shown in recent studies to be an important modality for the diagnosis of PAD, especially when ABPIs are falsely elevated in calcified arteries.[22] The spectral Doppler is a versatile, inexpensive, and portable piece of equipment that allows rapid assessment of limb perfusion. If interpreted correctly, the information obtained can detect lower-limb arterial disease.
This study highlights the importance of spectral Doppler waveform analysis by clinicians experienced in the interpretation of these waveforms for early detection of possible pathologic disorders to allow initiation of appropriate and timely risk factor control measures. The authors recommend that reports of vascular screening in high-risk patients should be scrutinized by experienced clinicians to ensure correct diagnosis. Early detection of abnormal blood flow in the lower limb may lead to early treatment and advice. It will also aid in timely preventive measures to avoid deterioration and development of serious complications, such as lower-limb ulceration, amputation and, consequently, even death. Recommendations about physiologic testing of peripheral perfusion using qualitative spectral waveform analysis should be considered to facilitate the early identification of asymptomatic PAD in the high-risk foot.
Because this study investigated spectral Doppler waveform analysis, results cannot be generalized to other forms of Doppler assessment. Furthermore, this testing method is qualitative in nature, operator dependent, and should never be used in isolation to determine a patient's ongoing management plan.
Conclusions
This study demonstrates high reliability in spectral Doppler interpretation among experienced podiatric clinicians. Current foot screening guidelines do not refer to spectral Doppler waveform analysis in their recommendations, which has been shown in studies to be an important modality for the diagnosis of PAD when ABPIs are falsely elevated in calcified arteries. The spectral Doppler device is a relatively inexpensive and portable clinical tool that allows rapid assessment of limb perfusion. If interpreted correctly by experienced podiatric clinicians, the information obtained can give an indication of the presence of PAD and facilitate early management of this condition.
Financial Disclosure: None reported.
Conflict of Interest: None reported.
References
- Donnelly R, Hinwood D, London NJM: Non-invasive methods of arterial and venous assessment. BMJ320: 698, 2000.
- Evans DH, Barrie WW, Asher MJ, et al: The relationship between ultrasonic pulsatility index and proximal arterial stenosis in a canine model. Circ Res46: 470, 1980.
- Norgren L, Hiatt WR, Dormandy JA, et al: Inter-society consensus for the management of peripheral arterial diseases (TASC II). Eur J Endovasc Surg 33 (Suppl 1): S1, Epub2006November29.
- Hooi JD, Kester ADM, Stoffers HEJH, et al: Incidence of and risk factors for asymptomatic peripheral arterial occlusive disease: a longitudinal study. Am J Epidemiol153: 666, 2001.
- Hirsch AT, Haskal ZJ, Hertzer NR, et al: ACC/AHA 2005 Practice Guidelines for the management of patients with peripheral arterial disease (lower extremity, renal, mesenteric, and abdominal aortic). Circulation113: e463, 2006.
- National Institute for Health and Care Excellence: Type 2 diabetes foot problems: prevention and management of foot problems. Available at: http://guidance.nice.org.uk/cg10. Accessed December 29, 2014.
- Kuhnke J, Botros M, Elliot J, et al: The case for diabetic foot screening. Diabetic Canada1: 8, 2013.
- Ray SA, Srodon PD, Taylor RS, et al: Reliability of ankle: brachial pressure index measurements by junior doctors. Br J Surg81: 188, 1994.
- Vowden K, Vowden P: Hand-held Doppler Ultrasound: The Assessment of Lower Limb Arterial and Venous Disease, Huntleigh Healthcare, Cardiff, Wales, 2004.
- Carter SA: Clinical measurement of systolic pressures in limbs with arterial occlusive disease. JAMA207: 1869, 1969.
- Tehan PE, Chuter VH: Use of hand-held Doppler ultrasound examination by podiatrists: a reliability study. J Foot Ankle Res8: 36, 2015.
- Hass SM, AbuRahma AF: “Segmental Doppler Pressures and Doppler Waveform Analysis in Peripheral Vascular Disease of the Lower Extremities,” inNoninvasive Vascular Diagnosis: A Practical Guide to Therapy, edited byAFAbuRahma, DFBandyk, p 287,Springer London, London, 2013.
- Bakker K, Schaper NC, International Working Group on Diabetic Foot Editorial Board: The development of global consensus guidelines on the management and prevention of the diabetic foot 2001. Diabetes Metab Res Rev 28 (Suppl 1): 116, 2012; doi: 10.1002/dmrr.2254.
- Caruana L, Formosa C, Cassar K: Prediction of wound healing after minor amputations of the diabetic foot. J Diabetes Complications29: 834, 2015.
- Caruana MF, Bradbury AW, Adam DJ: The validity, reliability, reproducibility and extended utility of ankle to brachial pressure index in current vascular surgical practice. Eur J Vasc Endovasc Surg29: 443, 2005.
- Gravetter FJ, Forzano LB: “Simple Random Sampling,”inResearch Methods for the Behavioural Sciences, 4th Ed, p146, Wadsworth Cengage Learning, Belmont, CA, 2011.
- Scissons R: Characterizing triphasic, biphasic, and monophasic Doppler waveforms. J Diagn Med Sonogr24: 269, 2008.
- Scissons R: Traditional peripheral arterial Doppler waveform descriptors: are they still worth teaching?J Diagn Med Sonogr28: 51, 2012.
- Scissons R, Comerota A: Confusion of peripheral arterial Doppler waveform terminology. J Diagn Med Sonogr25: 185, 2009.
- Jaff MR: Lower extremity arterial disease diagnostic aspects. Cardiol Clin20: 491, 2002.
- Kaiser V, Kester AM, Staffers HE, et al: The influence of experience on the reproducibility of the ankle-brachial systolic pressure ratio in peripheral arterial occlusive disease. Eur J Vasc Surg18: 25, 1999.
- Formosa C, Gatt A, Chockalingam N: A critical evaluation of existing diabetic foot screening guidelines. Rev Diabet Stud13: 158, 2016.
- Formosa C, Cassar K, Gatt A, et al: Hidden dangers revealed by misdiagnosed peripheral arterial disease using ABPI measurement. Diabetes Res Clin Pract102: 112, 2013.
© 2018 American Podiatric Medical Association