A Study on the Performance Evaluation Criteria and Methods of Abdominal Ultrasound Devices Based on International Standards

Abstract

The medical diagnostic ultrasound market has grown steadily in recent years, thanks to the increasing popularity of minimally invasive surgery among patients and active investments in the medical ultrasound market. In addition, the diagnostic ultrasound market is expected to continue to grow in the global market, given the importance of ultrasound and ongoing technological developments. In the case of the gallbladder, cases of gallstones and nonalcoholic fatty liver disease, all of which can be diagnosed with abdominal ultrasound, the number of domestic patients continues to increase every year. The safety of abdominal ultrasound has also been improved due to the national health insurance expanding its coverage to include gallstones and diseases such as those of the liver and gallbladder. However, no standards by which the performance of abdominal ultrasound can be evaluated have been explicitly established. Therefore, we aimed to assess the performance of abdominal ultrasound using criteria such as sound output level, operating frequency accuracy, resolution, maximum display depth, distance accuracy, time accuracy, and blood flow velocity accuracy. These criteria were derived by investigating the principles of ultrasound and the characteristics and structure of transducers, as well as by analyzing the specifications of abdominal ultrasound set by the International Electrotechnical Commission (IEC) 60601-2-37.

1. Introduction

The global market for medical ultrasound was $6.3 billion in 2018 and is expected to reach $8.4 billion in 2023. The average annual growth rate is 5.9%, owing to the increasing popularity of minimally invasive surgery among patients, continuous technological development, and increased medical facilities and investment. Furthermore, interest in new ultrasound technology is expected to further expand its share of the market. The increased incidence of various cancers has led to technological advancement in ultrasound-based diagnosis and treatment [1].

According to the statistics for public interest diseases in Korea, the number of patients with gallstones or conditions of the gallbladder increased by 63% from 96,304 in 2015 to 152,052 in 2019. In addition, the number of patients with nonalcoholic fatty liver disease increased by approximately 28% from 28,368 in 2015 to 99,616 in 2019. In the case of kidney cancer, the number of patients increased by about 24% from 26,138 in 2016 to 32,502 in 2019, and it shows a steady increase every year [2].

In Korea, the national health insurance expanded its coverage for upper and lower abdominal ultrasound in the first half of 2018 and the first half of 2019, respectively. In 2018, upper abdominal ultrasound became fully covered, and in 2019, epigastric ultrasound and lower abdominal ultrasound were included in the coverage. The national health insurance now covers ultrasound diagnosis of the most frequently addressed body parts, such as the liver, gallbladder, and biliary tract [3].

The organs that can be measured by abdominal ultrasound include the liver, gallbladder, biliary tract, and kidneys. Skills, experience, and knowledge are very important to completely examine organs. Understanding medical devices is also important, and the quality of the examination is directly related to the safety of the patient and should be approached carefully. The accuracy of medical devices that support this is also one of the most important aspects [4].

The number of patients with diseases that can be diagnosed by abdominal ultrasound is increasing every year. As the national health insurance continues to expand its coverage of such illnesses, the role of abdominal ultrasound is becoming critical to more patients. However, since there are no explicit performance evaluation criteria for abdominal ultrasound, it is necessary to establish not only performance evaluation criteria, but also diagnosis methods that apply them.

In the current study, we aimed to establish performance evaluation criteria and diagnosis methods based on the International Electrotechnical Commission (IEC) 60601-2-37 by examining domestic and international specifications and guidelines of abdominal ultrasound devices.

2. Physical Properties of Ultrasound and Performance Evaluation Criteria and Methods

2.1. Physical Properties of Ultrasound

Sound is transmitted through media such as solid, liquid, and gas, but it cannot travel through a vacuum. Given that the body is mainly composed of water, ultrasound, a type of sound, travels inside the body with a speed similar to that of an object in water. The role of medical ultrasound devices varies according to frequency intensity, which causes ultrasound to exert different effects in the body. High-frequency ultrasound can be used in the body, because it is able to travel in a straight line without spreading to the surroundings [5].

A sound source refers to an oscillating body that produces sound; typically, a sound source has a certain frequency and propagates in the form of a wave. Frequencies between 20 and 20,000 Hz are considered to be within the audible range, whereas frequencies above the audible range are categorized as ultrasound. The frequency range used in medical devices corresponds to 1–30 MHz, and the frequency used for an ultrasound test is in the order of MHz [6]. However, although these high frequencies transmit well in liquid or solid media, they are transmitted poorly in the air; therefore, ultrasound propagates better in abdominal organs or soft tissues than in the lungs or digestive organs. The frequency band of 3–5 MHz is mainly used for abdominal ultrasound [7].

It is established that ultrasound has a constant speed in the human body, depending on the type of medium that it travels through. Unlike light, the speed of ultrasound transmission depends on the characteristics of the medium that the ultrasound propagates through. Since most of the human body is made of water, the propagation speed within the human body is similar to that of water (1540 m/s). Factors affecting the propagation speed of sound waves include the density, hardness, and volume of the medium.

Table 1. Transmission rate of ultrasound waves depending on the medium.

Material	Speed (m/s)
Air	330
Fat	1450
Water	1540
Liver	1549
Blood	1570
Muscle	1585
Cortical Bone	4080

shows that the propagation speed varies according to the medium [5].

2.2. Properties of the Transducer

Convex Transducer

A transducer is composed of a piezoelectric element, an acoustic matching layer, and a sound-absorbing layer. Direct piezoelectric effect is the transformation of mechanical energy into electrical energy. Converse piezoelectric effect is the transformation of electrical energy into mechanical energy [8]. A piezoelectric element is a material that generates a voltage when pressure is applied and induces mechanical changes when electrical energy is used. A piezoelectric element has particularly high conversion efficiency, and it performs an interconversion between electrical and mechanical vibration energies. The piezoelectric effect is a compression caused by the repulsion of internal charges when the power of the positive pole is applied to the upper part, and the power of the negative pole is applied to the lower part of the piezoelectric element. On the contrary, when the negative pole is applied to the upper part, and the positive pole is applied to the lower part, the piezoelectric element expands. Upon applying alternating current voltage, ultrasound energy corresponding to the frequency is radiated, and the piezoelectric element of an ultrasound transducer makes use of these properties. Ultrasound transducers mainly use converse piezoelectric effects [9].

The use of medical ultrasound is determined by the frequency bandwidth, which can be widened or narrowed via the length adjustment of the ultrasound wave in the sound-absorbing layer. Medical ultrasounds with a large amplitude and narrow bandwidth are used for treatment, whereas those with a low amplitude and wide bandwidth are used for diagnosis [10].

In the sound convergence layer, the impedance value of the skin that comes into contact with the transducer and the transducer itself is different, which helps to alleviate this. It is a device that transmits elastic waves generated by piezoelectric elements and receives elastic waves reflected or transmitted again [11].

A convex transducer has a convex arrangement of the probe elements, and the electric flux angle is about 70° to 100°. A convex transducer is mainly used for abdominal ultrasound, given that it is easy to emit under or between ribs [12].

2.3. Four Categories of Ultrasound Image Display Methods

Ultrasound image display methods can be largely classified into four categories [12].

2.3.1. A-Mode

Amplitude mode (A-mode) is expressed in terms of intensity over time and is indicated as the amplitude height in an image. A-mode is a one-dimensional display method used in the early stages of ultrasound diagnosis. An area with a strong reflected wave has a high amplitude, whereas areas with a weak or no reflected wave have a low or no amplitude.

2.3.2. M-Mode

Motion mode (M-mode) is used for moving organs such as the heart; it expresses the distance of a moving reflector as a change over time.

2.3.3. B-Mode

Brightness mode (B-mode) receives reflected waves and displays them as brightness. B-mode is currently used in most ultrasound diagnostic equipment and can be used to display the movement of an organ in real-time. B-mode is expressed as the brightness of a point, which is proportional to the amplitude of the reflected signal.

2.3.4. D-Mode

Doppler mode (D-mode) requires prior knowledge of the Doppler principle; specifically, that vibration increases in inverse proportion to the distance of an object. D-mode acquires information about blood flow in the body using various applications of the Doppler principle, such as pulse wave Doppler, color Doppler, output Doppler, and harmonic Doppler methods.

2.4. Performance Measurement Equipment

The performance measurement equipment was an ultrasound measurement system that consisted of an ultrasound medium (water tank), a hydrophone, a digital oscilloscope, an electric positioning device (alignment), and ultrasound characteristic analysis software. After aligning the transducer and the hydrophone of the diagnostic ultrasound device in the water tank, ultrasound waves were generated by the diagnostic ultrasound device, and the frequency of the ultrasound received by the hydrophone was measured as an electrical signal using an oscilloscope. The ultrasound measuring system could measure both frequency accuracy and temporal accuracy, while the multipurpose ultrasound phantom was a device that could measure the resolution, maximum display depth, and distance accuracy. The reflector in the phantom was radiated by the transducer of abdominal ultrasound to distinguish the reflector, test the depth of the ultrasound, and measure the accuracy of its volume or distance. The accuracy for blood flow velocity was measured using a flow velocity simulator for blood flow measurement with an ultrasound Doppler device [13].

2.4.1. Sound Output Level Test Criteria

Ultrasound medical devices have a reference value of not more than 720 mW, as they are abdominal ultrasound devices.

Test Equipment

Ultrasound water tank, ultrasound measurement system, hydrophone, digital oscilloscope.

Test Method

The ultrasound transducer and the hydrophone in the ultrasound water tank were arranged horizontally. The maximum output waveform was measured using an oscilloscope. The measured maximum output waveform was converted to a sound output level value using the ultrasound software system. Then, the measured value was checked.

2.4.2. Operating Frequency Accuracy Criteria

In general, operating frequencies can be set differently depending on the type of ultrasound transducer in sound level output tests. In this study, the abdominal ultrasound transducer was set at the manufacturer’s standard of 3 to 5 MHz.

Test Equipment

Ultrasound water tank, ultrasound software system, hydrophone, digital oscilloscope.

Test Method

The ultrasound transducer and the hydrophone in the ultrasound water tank were arranged horizontally. The maximum output waveform was measured using an oscilloscope. The measured output waveform was converted to operating frequency using the ultrasound software system. Then, the measured value was checked.

2.4.3. Resolution Criteria

A resolution test is a test that uses ultrasound to distinguish objects in a similar position. The vertical resolution shall be less than 2 mm, the reference given by the manufacturer. The horizontal resolution shall be less than 3 mm, or the reference given by the manufacturer shall be satisfied.

Test Equipment

Ultrasound Phantom for Versatile Measurement.

Test Method

The abdominal ultrasound diagnostic medical device was set to B-mode. The abdominal ultrasound transducer was placed in contact with the surface of the ultrasound phantom. The target was then scanned by adjusting the position and angle of the abdominal ultrasound transducer. Then, the measured value was checked.

2.4.4. Maximum Expression Criteria

The maximum indicator that meets the criteria presented by the manufacturer shall be checked on the screen.

Test Equipment

Ultrasound Phantom for Versatile Measurement.

Test Method

The abdominal ultrasound diagnosis medical device was set to B-mode. Then, the abdominal ultrasound transducer was brought into contact with the surface of the ultrasound phantom. It then adjusted the position and angle of the abdominal ultrasound transducer to scan the image to the deepest target inside the ultrasound phantom. Then, the measured value was checked.

2.4.5. Distance Accuracy Criteria

The vertical distance accuracy and horizontal distance accuracy shall be within 5% of the reference value or within 1 mm.

Test Equipment

Ultrasound Phantom for Versatile Measurement.

Test Method

The abdominal ultrasound diagnosis medical device was set to M-mode. Then, the abdominal ultrasound transducer was brought into contact with the surface of the ultrasound phantom. The position and angle of the abdominal ultrasound transducer were then adjusted to select two points on the vertical and horizontal lines inside the ultrasound phantom to scan the image. Then, the measured value was checked.

2.4.6. Time Accuracy Criteria

The time accuracy should be within 3% of the reference value.

Test Equipment

Ultrasound Phantom for Versatile Measurement.

Test Method

The abdominal ultrasound diagnostic medical device was set to M-mode. Afterwards, the abdominal ultrasound transducer was placed in contact with the surface of the ultrasound phantom. The test was performed by adjusting the position and angle of the abdominal ultrasound transducer and selecting the time measured inside the ultrasound phantom. Then, the measured value was checked.

2.4.7. Blood Flow Velocity Accuracy Criteria

The blood flow velocity accuracy shall not exceed 15% of the reference value.

Test Equipment

Flow velocity simulator.

Test Method

The abdominal ultrasound diagnosis medical device was set to D-mode. The flow rate of the flow velocity simulator was then tested by setting the flow velocity value within the flow velocity measurement range. Then, the measured value was checked.

2.5. Domestic and International Standards and Classifications for Medical Ultrasound Devices

According to the Regulation on Medical Devices and their Classifications, published by the Ministry of Food and Drug Safety in Korea, abdominal ultrasound devices are classified as Class 2 and A26000, used for internal function tests.

Table 2. Classification and class of ultrasound diagnostic equipment.

Classification Number	Class	Item Name	Definition
A26380.01	2	Universal Ultrasound Wave Imaging Diagnostic Device	A general ultrasound imaging diagnostic device that transmits ultrasound energy to the affected area for diagnosis, and receives imaging reflective signals.
A26380.03	2	Mobile-type ultrasonography device	Ultrasound imaging diagnostic device used by attaching it to vehicles, such as vans and trailers, for use in various places

shows the classification.

The international standards for abdominal ultrasound devices are IEC standards, which are recognized in Europe as well as the UK, and include IEC 60601 Part 1 and IEC 60601-2-37, as shown in

Table 3. International standards related to ultrasound.

Country	Standard Number	Standard Name
International Standard	IEC 60601 Part 1	To regulate for the safe design and manufacture of all medical equipment
	IEC 60601-2-37	The basic safety and essential performance of ultrasound medical diagnostic and monitoring equipment

. These standards provide detailed information, such as recommended output, safety, and performance evaluation criteria, for the performance evaluation of diagnostic ultrasound.

With regards to the general requirements for the safety of electronic medical devices, IEC 60601 Part 1 contains requirements such as the safe design of components and electrical and thermal safety, whereas IEC 60601-2-37 deals with the criteria for evaluating the performance of diagnostic ultrasound devices in more detail. We aim to derive essential items for the performance evaluation criteria of abdominal ultrasound in accordance with the international standard provided by IEC 60601-2-37 [14,15].

2.6. Performance Evaluation Criteria

Major performance evaluation criteria were selected according to IEC 60601-2-37. The main performance criteria comprised the sound output level, the accuracy of operating frequency, resolution, maximum display depth, distance accuracy, time accuracy, and blood flow velocity accuracy [15].

2.6.1. Reference Value for Sound Output Level

The reference values for the I_SPTA.a (spatial-peak temporal-average intensity), I_SPTA.a (spatial-peak pulse-average intensity), and MI (mechanical index) are used. In case there is a Doppler effect, reference values for I_SPTA.a and MI are used. The values above are based on

Table 4. Standard value of sound output level.

Category		ISPTA.a (mW/cm2)	ISPTA.a (W/cm2)	MI
When mechanical index (MI) and TI values are not indicated on the display	For ophthalmology	17	28	0.23
	Fetus or others	94	190	1.9
	For heart	430	190	1.9
	For peripheral blood vessel	720	190	1.9
When MI and TI values are indicated on the display or when there is a fetus Doppler function	For ophthalmology	50	-	0.23
	For peripheral blood vessel, heart, fetus or others	720	-	1.9

.

2.6.2. Operating Frequency Accuracy

The operating frequency accuracy should be within ±15% of the reference value.

2.6.3. Resolution

The vertical resolution should be less than 2 mm or satisfy the value suggested by the manufacturer.

The horizontal resolution should be 3 mm or less, or satisfy the value suggested by the manufacturer.

2.6.4. Maximum Display Depth

More than the suggested maximum display depth should be checked on the screen.

2.6.5. Distance Accuracy

Vertical distance accuracy and horizontal distance accuracy should be within ±5% of the reference value or within 1 mm.

2.6.6. Time Accuracy

Time accuracy should be within ±3% of the reference value.

2.6.7. Blood Flow Rate Accuracy

Blood flow rate accuracy should be within ±15% of the reference value.

3. Result

3.1. Performance Evaluation of Abdominal Ultrasound According to Performance Evaluation Criteria

The performance evaluation of the transducer of abdominal ultrasound was conducted, based on the reference value (the value suggested by the manufacturer) and the performance evaluation criteria of the international standard IEC 60601-2-37.

3.1.1. Sound Output Level Test

The sound output test of the abdominal ultrasound transducer checks whether the level of sound output carries harmful effects to the human body. After switching on the abdominal ultrasound transducer, the ultrasound signal received by the hydrophone is amplified into an electrical signal that is measured with a digital oscilloscope. The performance level should fall within the reference values for I_SPTA.a (spatial-peak time-average intensity), I_SPTA.a (spatial-peak pulse-average intensity), and MI (mechanical index). With regard to abdominal ultrasound, which includes the Doppler effect, the performance level should match values close to the reference values for I_SPTA.a and MI. In accordance with IEC 60601-2-37: 201.7.9.3.101, Figure 1 demonstrates that the performance level in our study satisfied the values with I_SPTA.a (mW/cm²) = 538.6, MI = 1.102.

3.1.2. Operating Frequency Accuracy Test

The accuracy test for the operation frequency of the abdominal ultrasound transducer checks for the frequency accuracy of ultrasound to evaluate the performance of the device. After aligning the transducer and hydrophone of the diagnostic ultrasound device in the water tank, ultrasound is generated by operating the ultrasound device, and the ultrasound received by the hydrophone is measured as an electrical signal using an oscilloscope. The electrical signal frequency is then measured. The range of reference frequency values for abdominal ultrasound is between 3 and 5 MHz. Figure 2 shows that the values measured in our study satisfied the range of the reference values.

3.1.3. Resolution Test

The resolution test examines the spatial resolution of the ultrasound transducer by checking whether two distinct reflectors laying in the direction of ultrasound can be distinguished by the naked eye. Vertical resolution refers to the ability to distinguish between two reflectors that lay in front of and behind each other in the direction of ultrasound propagation, whereas horizontal resolution refers to the ability to distinguish between two reflectors that lay next to each other. A group of nine reflectors are placed in pairs 5, 4, 3, 2, and 1 mm apart, where those close to the transducer have a wider separation (Figure 3). All reflectors must be clearly distinguished. The vertical and horizontal resolution should be less than or equal to 2 mm or 3 mm, respectively, or satisfy the values suggested by the manufacturer. Our results show that both the vertical and horizontal resolutions were approximately 1 mm, which satisfied the range of the reference values.

3.1.4. Maximum Display Depth Test

The maximum display depth is also known as sensitivity and is used to evaluate how deeply a medium can be penetrated when ultrasound propagates. The performance of the ultrasound device is evaluated by testing the penetration depth in an anechoic structure. Figure 4 shows the penetration depth of an abdominal ultrasound. The vertical axis of Figure 4 is in centimeters. In our study, it was larger than the reference value (80 mm).

3.1.5. Distance Accuracy Test

The distance accuracy test measures the vertical and horizontal target distance to the beam axis and checks whether a device accurately measures the size and volume of a given structure. Ultrasound images are acquired by adjusting the position and angle of the transducer during a diagnostic ultrasound and selecting two points of the vertical and horizontal distance inside the phantom. The vertical and horizontal distances are measured using the diagnostic ultrasound measuring equipment. In order to pass the test, the distances should be within ±5% of the reference value or within 1 mm. Figure 5 demonstrates that the measured value in our study matched the reference value (10 mm).

3.1.6. Time Accuracy Test

The time accuracy test is performed when the abdominal ultrasound has the M-mode function. The time accuracy test aims to test the accuracy of periodic ultrasound signals generated by the device. The vertical axis indicates frequency amplitude, whereas the horizontal axis indicates the time (s). The frequency period should be within ± 3% of the reference value. As shown in Figure 6, time is measured periodically every 2 s. The reference value for the period is 0.5 Hz, and it was 1/2 = 0.5 Hz in our test, which satisfied the reference value range.

3.1.7. Blood Flow Accuracy Test

The formula for obtaining blood flow velocity is Δf = 2·v·f0·cos α/c, each of which means as follows [16]:

• Δf: Doppler frequency shift; Hz
• v: Blood flow velocity
• f0: Source frequency; Hz
• cos α: Ultrasound projection angle for blood flow direction
• c: Ultrasound wave velocity in tissue

The accuracy test for blood flow velocity is performed if an ultrasound device has the D-mode function. The transducer in diagnostic ultrasound is brought into contact with the flow rate simulator to measure the flow velocity using the D-mode. The vertical axis represents velocity (cm/s). The accuracy test should be performed within 15% of the reference value (60 cm/s). As shown in Figure 7, we tested with a reference value for flow velocity of 60 cm/s and obtained a blood flow velocity of 62.1 cm/s with an error rate of less than 15%.

3.1.8. Discussion of the Experimental Results

As diseases of organs such as liver, gallbladder, gallstones, and kidneys increase, the importance of abdominal ultrasound diagnosis medical devices is increasing. The usability of abdominal ultrasound diagnosis medical devices has increased, and the safety of medical devices has become more important. Therefore, related performance evaluation experiments were conducted. The experiment was set up according to the international standard IEC 60601-2-37. The medical device used in all experiments is the same medical device. The experiment was conducted according to established standards and it was confirmed that all the results were within the range of the error rate. Therefore, it was proved that the performance of the abdominal ultrasound medical device is safe.

4. Discussion

In imaging diagnosis, biometric diagnosis is beneficial for treatment. Accurate diagnosis is essential because the diagnosis classification of normal or abnormal, as well as the consequent treatment, depends solely on the diagnostic ultrasound results [17]. Since a subjective element of ultrasound diagnosis plays a significant role in evaluating the diagnosis of a lesion, it is necessary to understand the disease of interest and the diagnostic ultrasound, as well as the anatomical structure of the patient for an accurate diagnosis [18]. Ultrasound diagnosis can be used not only for the abdomen but also for various other body parts. Patients visit the emergency room with a wide variety of diseases, the diagnosis of which is often difficult to make based exclusively on the symptoms. More recently, studies have evaluated how accurately pathology can be diagnosed in patients with acute scrotal pain [19]. In addition, other related fields have been continuously investigated from the 1990s to the current year of 2020. Therefore, it is necessary to stress the importance and safety of abdominal ultrasound transducers, as well as various other diagnostic ultrasounds.

5. Conclusions

In the current study, we aimed to establish the safety of abdominal ultrasound and performance evaluation indices. The abdominal ultrasound is one of the most widely used medical devices, the safety of which is important. In this study, we established a firm understanding of the basic principles of ultrasound, and elaborated on the composition and principles of abdominal ultrasound transducers to improve understanding of medical ultrasound. In addition, we show that the speed of ultrasound varies according to the medium inside the body, and describe how the different types of imaging modes in diagnostic ultrasound can be used.

We derived test categories and performance evaluation criteria based on the international standard IEC 60601-2-37. We conducted an experiment on the following test categories: sound output level, operating frequency accuracy, resolution, maximum display depth, spatial accuracy, temporal accuracy, and blood flow velocity accuracy. For each test category, we used the following criteria to evaluate the performance of the ultrasound device: (1) for sound output level, I_SPTA.a(mW/cm²) should be 720 or less and MI should be 1.9 or less; (2) for operating frequency, the accuracy should be maintained within 15% of the error rate of the reference value; (3) for resolution, the vertical and horizontal resolutions should be either 2 mm or less, or 3 mm or less, respectively, or satisfy the values suggested by the manufacturer; (4) for maximum display depth, a value greater than the reference value must be displayed on the screen; (5) for spatial accuracy, the vertical and horizontal distance accuracies should have values within a 5% error rate of the reference value or values less than 1 mm; (6) for temporal accuracy, the values must be within a 3% error rate of the reference value; and (7) for blood flow velocity accuracy, the abdominal ultrasound with Doppler effect should have values within 15% of the error rate of the reference value.

Performance evaluations of abdominal ultrasound diagnostic devices were performed based on IEC international specifications and the safety and effectiveness of medical devices were verified. We hoped that the proven results will enhance the safety and effectiveness of abdominal ultrasound diagnostic devices and contribute to the industrial development of ultrasound medical devices.