Petsiuk, Aliaksei L.; Pearce, Joshua M. Low-Cost Open Source Ultrasound-Sensing Based Navigational Support for the Visually Impaired

Nineteen million Americans have significant vision loss. Over 70% of these are not employed full-time, and more than a quarter live below the poverty line. Globally, there are 36 million blind people, but less than half use white canes or more costly commercial sensory substitutions. The quality of life for visually impaired people is hampered by the resultant lack of independence. To help alleviate these challenges this study reports on the development of a low-cost, open-source ultrasound-based navigational support system in the form of a wearable bracelet to allow people with the lost vision to navigate, orient themselves in their surroundings and avoid obstacles when moving. The system can be largely made with digitally distributed manufacturing using low-cost 3-D printing/milling. It conveys point-distance information by utilizing the natural active sensing approach and modulates measurements into haptic feedback with various vibration patterns within the four-meter range. It does not require complex calibrations and training, consists of the small number of available and inexpensive components, and can be used as an independent addition to traditional tools. Sighted blindfolded participants successfully demonstrated the device for nine primary everyday navigation and guidance tasks including indoor and outdoor navigation and avoiding collisions with other pedestrians.


Introduction
According to the World Health Organization (WHO) [1,2], approximately 1.3 billion people live with some form of vision impairment, and 36 million of them are totally blind. The vast majority of the world's blind population live in developing countries [3]. In addition, this challenge is falling on the elderly at an increasing rate, with the group of visually impaired people over 65 years of age growing with a per-decade increase of up to 2 million persons, which is faster than the overall population with visual impairments [3]. However, even in developed countries like the U.S. this is becoming an increasing problem because of several factors. First, the U.S. is aging: Americans 65 and older is projected to more than double from 46 million today to over 98 million by 2060, and their share of the total population will rise to nearly a quarter from 15% [4]. Second, the elderly in the U.S. are increasing financially vulnerable [5]. According to American Foundation for the Blind [6] and National Federation of the Blind [7], more than 19 million American adults between the ages of 18 and 64 report experiencing significant vision loss. For working age adults reporting significant vision loss, over 70% system [40] utilizes two separate ultrasound sensors for near (2 to 40 cm) and far (40 to 180 cm) distance ranges respectively. Tudor et al. [40] use simple linear pulse width modulation dependency for both vibration motors. Most of these projects allow to navigate within the distance range of 4 m, but suffer from drawbacks related to cost and complexity and thus accessibility to the world's population of visually impaired poor people. One approach recently gaining acceptance for lowering the costs of hardware-based products is the combination of open source development [41][42][43] with distributed digital manufacturing technologies [44,45]. This is clearly seen in the development of the open source self-replicating rapid prototyper (RepRap) 3-D printer project [46][47][48], which radically reduced the cost of additive manufacturing (AM) machines [49] as well as products that can be manufactured using them [50][51][52] including scientific tools [42,[53][54][55][56][57], consumer goods [58][59][60][61][62][63], and adaptive aids [64]. In general, these economic savings are greater for the higher percentage of the components able to be 3-D printed [65,66].
In this study, a low-cost, open-source navigational support system using ultrasonic sensors is developed. It utilizes one ultrasound sensor with one or two vibration motors (depend on the designed model) in a 3-D printed case to make the system compact and easily fixed on the wrist as a bracelet. The developed Arduino software performs distance measurements in the full sensor range from 2 cm to 4 m according to the ultrasound sensor specifications and implements an algorithm that uses the motors as low-and high-frequency vibration sources. The distance modulation technique consists of splitting the entire four-meter measurement range into four bands as described in Section 2.4, and of implementing the non-linear duty cycle modulation for the most significant middle band of 35 cm to 150 cm. The system can be largely digitally manufactured including both the electronics and mechanical parts with conventional low-cost RepRap-class PCB milling [67] and 3-D printing. The system is quantified for range and accuracy to help visually-impaired people in distance measurement and obstacle avoidance including the minimal size of the object. According to [24,68], the proposed system partially fulfills the Electronic Travel Aid (ETA) requirements, which consist of providing tactile and/or audible feedback on environmental information that is not available using traditional means such as white cane, guide dog, etc. Sighted blindfolded participants tested the device for primary everyday navigation and guidance tasks including: (a) walk along the corridor with an unknown obstacle, (b) bypass several corners indoors, (c) walk through the staircase, (d) wall following, (e) detect the open door, (f) detect an obstacle on the street, (g) bypass an obstacle on the street, (h) avoid collisions with pedestrians, (i) interact with known objects.

Design
An open-source navigational support with the 3-D printable case components was developed in form of two models of wearable bracelet with one and two vibration motors respectively ( Figure 1) to help visually impaired people in distance measurement and obstacle avoidance. The system is based on a 5-volt HC-SR04 ultrasonic sensor [69], which uses SONAR (originally an acronym for sound navigation ranging) to determine the distance to an object in the range of 0.02-4 m with a measuring angle of 15 degrees. It detects obstacles in front of the user's body from the ground to the head and above, and provides haptic feedback using a 10 mm flat vibration motor [70], which generates oscillations with variable frequency and amplitude depending on the distance to the obstacle. The microcontroller unit allows us to coordinate the operation of the motor and the sensor based on the developed algorithm, uploaded as a C-code to Arduino.
The device can be placed on the right or left hand, and it does not prevent the use of the hand for other tasks. It conveys point-distance information and could be used as a part of an assembly of assistive devices or as an augmentation to a regular white cane. In that way, the active sensing approach [15] was utilized, in which a person constantly scans the ambient environment. This method allows a user to achieve better spatial perception and accuracy [15] due to the similarity to natural sensory processes [71,72].

Bill of Materials
The system was prototyped for people with no engineering skills and the lack of available materials, so they can finish the assembly with the minimal toolkit. The bill of materials is summarized in Table 1. A 5V DC-DC boost step-up module can be considered as an optional component that can be used for battery life extension.

Bill of Materials
The system was prototyped for people with no engineering skills and the lack of available materials, so they can finish the assembly with the minimal toolkit. The bill of materials is summarized in Table 1. A 5V DC-DC boost step-up module can be considered as an optional component that can be used for battery life extension.

Assembly
After 3-D printing all the necessary components, electronic parts should be soldered together following Figure 2 and assembled to a sensor core with the vibration motor.

Assembly
After 3-D printing all the necessary components, electronic parts should be soldered together following Figure 2 and assembled to a sensor core with the vibration motor.

Figure 2. Electrical circuit.
The Arduino Nano board should be programmed with the code available at [73], and the electronic core assembly should be placed in the 3-D printed case to finish the whole assembly ( Figure  1k). All the CAD models and STL files are available online under an open-source CC BY-SA 3.0 license (Creative Commons -Attribution -Share Alike) [74]. The hand bracelet ( Figure 1j) has an online option for customization [75], so a person with no experience with complicated 3-D modeling software could print the part after adjusting it to their hand size. For Arduino programming, it is necessary to download free open source Arduino IDE [76].

Operational Principles
The ultrasonic sensor emits acoustic waves at the frequency of 40 kHz, which travel through the air and reflects from objects within the working zone. Every measurement cycle it sends a 10 µ s trigger input pulse and an 8-cycle sonic burst at the speed of sound and its reflection from an object is received by an echo sounder [69]. It receives a 150 µ s -25 ms (38 ms in case of no obstacle) output echo pulse, Techo, which width linearly depends on the distance to the detected obstacle: where Techo is the output echo pulse width [µ s], D is the measured distance to an obstacle [cm], k = 58 (µ s/cm) is the conversion factor given by the sensor datasheet. The measurement cycle, Tmeasure, specified in the Arduino program and represents a constant time duration declared by a developer, but according to the sensor datasheet, this time duration should be no less than 60 ms [69]. The distance to the object is measured by the time delay between sending and receiving sonic impulses in the Arduino program.
A single exponential filter [77] was used to smooth noisy sensor measurements. It processes the signal with the desired smoothing factor without using a significant amount of memory. Every time a new measured value yt is provided, the exponential filter updates a smoothed observation, St: The Arduino Nano board should be programmed with the code available at [73], and the electronic core assembly should be placed in the 3-D printed case to finish the whole assembly ( Figure 1k). All the CAD models and STL files are available online under an open-source CC BY-SA 3.0 license (Creative Commons -Attribution -Share Alike) [74]. The hand bracelet (Figure 1j) has an online option for customization [75], so a person with no experience with complicated 3-D modeling software could print the part after adjusting it to their hand size. For Arduino programming, it is necessary to download free open source Arduino IDE [76]. The final assembly of the sensor core and 3-D printed case components is demonstrated in the supplementary material (Video S1).

Operational Principles
The ultrasonic sensor emits acoustic waves at the frequency of 40 kHz, which travel through the air and reflects from objects within the working zone. Every measurement cycle it sends a 10 µs trigger input pulse and an 8-cycle sonic burst at the speed of sound and its reflection from an object is received by an echo sounder [69]. It receives a 150 µs -25 ms (38 ms in case of no obstacle) output echo pulse, T echo , which width linearly depends on the distance to the detected obstacle: where T echo is the output echo pulse width [µs], D is the measured distance to an obstacle [cm], k = 58 (µs/cm) is the conversion factor given by the sensor datasheet. The measurement cycle, T measure , specified in the Arduino program and represents a constant time duration declared by a developer, but according to the sensor datasheet, this time duration should be no less than 60 ms [69]. The distance to the object is measured by the time delay between sending and receiving sonic impulses in the Arduino program. A single exponential filter [77] was used to smooth noisy sensor measurements. It processes the signal with the desired smoothing factor without using a significant amount of memory. Every time a new measured value y t is provided, the exponential filter updates a smoothed observation, S t : where S t−1 -is the previous output value of the filter, y t is a new measured value, α = 0.5 is the smoothing constant. Total measurement time consists of the traveling time caused by the finite speed of sound and the delay necessary for measurements. The time delay caused by the finite speed of sound, T max travel , is: where, D max is the maximum measured distance to an obstacle and V sound is the speed of sound in air. The measured distance is modulated with vibration amplitude and translated in real-time as a duty cycle parameter from the Arduino board ( Figure 3). Distances up to 35 cm are characterized by single vibration pulses with a relatively high periodicity. Distances from 150 to 250 cm are characterized by single pulses with low periodicity, and distances above 250 cm are modulated with two-pulse beats.
where, Dmax is the maximum measured distance to an obstacle and Vsound is the speed of sound in air.The measured distance is modulated with vibration amplitude and translated in real-time as a duty cycle parameter from the Arduino board ( Figure 3). Distances up to 35 cm are characterized by single vibration pulses with a relatively high periodicity. Distances from 150 to 250 cm are characterized by single pulses with low periodicity, and distances above 250 cm are modulated with two-pulse beats.
(a) (b) where m = 127, k = 70 and b = 35 are the calibrated parameters, and D is the measured distance in the range of 35 cm to 150 cm. This modulation law is based on hyperbolic tangent function, tanh, (Figure  3c), which is close to the inverse of pain sensitization in its shape [78,79] and demonstrated the best efficiency in most common tasks (Figure 4). According to [80,81], there are four major types of tactile mechanoreceptors in human skin: 1) Merkel's disks, 2) Meissner's corpuscles, 3) Ruffini endings, and 4) Pacinian corpuscles. Meissner's corpuscles respond to high amplitude incentives with low frequency and Pacinian corpuscles, in turn, respond to low amplitude incentives with high frequency. Thus, varying amplitude and frequency of vibrations, it is possible to activate these mechanoreceptors separately, which increases the working range of sensitivity levels (Figure 5a). The vibration pad (Figure 1g) with the vibration motor are in contact with the skin on the outer area of the wrist. As the nominal frequency characteristics of the motor (Figure 5b) do not cover the full range of human threshold for detection of vibrotactile stimulation, switching the motor on and off at different intervals allows us to simulate low-frequency pulsations. An optimal duty cycle equation (Figure 3c) for the most common distance range of 35-150 cm was found during experiments and calibrations (Figure 3b). The generated duty cycle for the Arduino output, M DC is: where m = 127, k = 70 and b = 35 are the calibrated parameters, and D is the measured distance in the range of 35 cm to 150 cm. This modulation law is based on hyperbolic tangent function, tanh, (Figure 3c), which is close to the inverse of pain sensitization in its shape [78,79] and demonstrated the best efficiency in most common tasks (Figure 4).  Estimated current for the whole device is at the level of 50 mA assuming that the vibration motor works 40% of the time. According to this, a 400 mAh battery will provide us with 8 hours of autonomous work, which is an efficient amount of time for test purposes as well as for general use if a blind person was walking throughout the entire working day.
Finally, the cost saving in percent (P) of the device was determined by: where c is the commercial cost of an equivalent device and m is the cost in materials to fabricate the open source device. All economic values are given in U.S. dollars.

Testing of the Device
Since there are no well-established tests for sensory substitution devices, the experimental setup was based on previous experience. García et al. [26] conducted an experiment with eight blind volunteers and evaluate the results in form of quiz, where participants noted the efficiency in obstacle According to [80,81], there are four major types of tactile mechanoreceptors in human skin: (1) Merkel's disks, (2) Meissner's corpuscles, (3) Ruffini endings, and (4) Pacinian corpuscles. Meissner's corpuscles respond to high amplitude incentives with low frequency and Pacinian corpuscles, in turn, respond to low amplitude incentives with high frequency. Thus, varying amplitude and frequency of vibrations, it is possible to activate these mechanoreceptors separately, which increases the working range of sensitivity levels (Figure 5a). The vibration pad (Figure 1g) with the vibration motor are in contact with the skin on the outer area of the wrist. As the nominal frequency characteristics of the motor (Figure 5b) do not cover the full range of human threshold for detection of vibrotactile stimulation, switching the motor on and off at different intervals allows us to simulate low-frequency pulsations.  Estimated current for the whole device is at the level of 50 mA assuming that the vibration motor works 40% of the time. According to this, a 400 mAh battery will provide us with 8 hours of autonomous work, which is an efficient amount of time for test purposes as well as for general use if a blind person was walking throughout the entire working day.
Finally, the cost saving in percent (P) of the device was determined by: where c is the commercial cost of an equivalent device and m is the cost in materials to fabricate the Estimated current for the whole device is at the level of 50 mA assuming that the vibration motor works 40% of the time. According to this, a 400 mAh battery will provide us with 8 hours of autonomous work, which is an efficient amount of time for test purposes as well as for general use if a blind person was walking throughout the entire working day.
Finally, the cost saving in percent (P) of the device was determined by: where c is the commercial cost of an equivalent device and m is the cost in materials to fabricate the open source device. All economic values are given in U.S. dollars.

Testing of the Device
Since there are no well-established tests for sensory substitution devices, the experimental setup was based on previous experience. García et al. [26] conducted an experiment with eight blind volunteers and evaluate the results in form of quiz, where participants noted the efficiency in obstacle detection above the waistline. Pereira et al. [12] evaluated their prototype on both blind and sighted participants in five different scenarios to simulate the real-world conditions, including head-, chest-, foot-level obstacles, and stairs. Maidenbaum et al. [15] performed a set of three experiments with 43 participants (38 of them are sighted blindfolded) to evaluate their prototype on basic everyday tasks, including distance estimation, navigation, and obstacle detection. Nau et al. [83] proposed an indoor, portable, standardized course for assessment of visual function that can be used to evaluate obstacle avoidance among people with low or artificial vision.
Summarizing the experience of previous researchers, the set of experiments used to test the devices here consists of indoor and outdoor, structural and natural environment in order to explore the intuitiveness of the developed device and its capabilities in everyday human tasks.
Five sighted blindfolded lab researchers took part in a series of tests, the main purpose of which was to collect the necessary information about adaptation pace, usability, and performance of the developed system. The experiments were conducted in a familiar indoor and outdoor environment for the users.
Participants were assigned to the following nine tests ( Each participant had a 10-minute training before nine tests with two attempts, one attempt using the Model 1 (the white device, Figure 1c) and one attempt using the Model 2 (the blue device, Figure 1d). The tasks were to complete a distance of approximately 10 meters (20 steps) with obstacles (pedestrian, road sign, etc.) or with landmarks (wall, door, steps, etc.). Success was counted if the participant has finished the path relying only on the sensing device. Any collision with an obstacle or difficulty in passing the test path was considered a failure. The time spent on passing the test was not taken into account. Results of the experiments presented in Table 2. Some of the experiments were recorded and included in the supplementary material (Video S1).
the Model 1 (the white device, Figure 1c) and one attempt using the Model 2 (the blue device, Figure  1d). The tasks were to complete a distance of approximately 10 meters (20 steps) with obstacles (pedestrian, road sign, etc.) or with landmarks (wall, door, steps, etc.). Success was counted if the participant has finished the path relying only on the sensing device. Any collision with an obstacle or difficulty in passing the test path was considered a failure. The time spent on passing the test was not taken into account. Results of the experiments presented in Table 2.

Results and Discussion
All versions of the device were built for less than $24 USD each in readily available purchased components and 3-D printed parts. The economic savings over generally inferior commercial products ranged from 80.2-97.8% for the base system to 73.5-97.0% for the optional module system.
The devices were tested to demonstrate that it has intuitive haptic feedback as outlined above. The device range and accuracy was found to allow a person with a lost eyesight to detect objects with the area of 0.5 m 2 or more within the distance range of up to 4 m. To omit head-level obstacles the user must sweep the head area along the direction they are moving. During the experiments was also found that it is possible to track objects with the moving speed of up to 0.5 m/s within the one-meter distance range.
As can be seen from Table 2, the failure rate is 6.7%, with the majority of failures occurring in experiments with Model 2 (the blue device with two vibration motors, Figure 1d). Some participants

Results and Discussion
All versions of the device were built for less than $24 USD each in readily available purchased components and 3-D printed parts. The economic savings over generally inferior commercial products ranged from 80.2-97.8% for the base system to 73.5-97.0% for the optional module system.
The devices were tested to demonstrate that it has intuitive haptic feedback as outlined above. The device range and accuracy was found to allow a person with a lost eyesight to detect objects with the area of 0.5 m 2 or more within the distance range of up to 4 m. To omit head-level obstacles the user must sweep the head area along the direction they are moving. During the experiments was also found that it is possible to track objects with the moving speed of up to 0.5 m/s within the one-meter distance range.
As can be seen from Table 2, the failure rate is 6.7%, with the majority of failures occurring in experiments with Model 2 (the blue device with two vibration motors, Figure 1d). Some participants experienced difficulty navigating a staircase, as well as finding and avoiding unfamiliar obstacles indoors and outdoors.
The preliminary testing of the device was determined to be a success based on the majority of the participants being able to complete the nine tasks outlined in the methods section. All participants during the experiments noted the effectiveness of the haptic interface, the intuitiveness of learning and adaptation processes, and the usability of the device. The system produces fast response and allows a person to detect objects that are moving. It naturally complements primary sensory perception of a person and allows one to detect moving and static objects.
The system has several limitations. First, for the developed system, it is necessary to note a narrow scanning angle and a limited response rate, which is expressed in ignoring the danger posed by small and fast-moving objects. Second, the low spatial resolution of the system is also noted. Thus, in the conditions of an outdoor street environment, it was difficult for the experiment participants to track road curbs and determine the change in the level of the road surface. Third, indoors, soft fabrics, such as furniture and soft curtains, as well as indoor plants, can cause problems with distance estimation caused by acoustic wave absorption. In open outdoor areas, determining the distance can be difficult on lawns with high grass and areas with sand. In addition, given the increase in the threshold of sensitivity with age [84], the performed experiments do not cover the diversity of the entire population of people with visual impairments. This is a preliminary study on the technical specifications of the new device and a much more complete study is needed on human subjects including blind subjects. Future work is needed for further behavioral experimentation to improve data acquisition methods, obtain more data, and perform a comprehensive statistical analysis of the developed system performance. This will allow designers to utilize achievements in haptic technologies [85] and to improve the efficiency of its tactile feedback, since the alternation of patterns of high-frequency vibrations, low-frequency impulses and beats of different periodicity can significantly expand the range of sensory perception. Similarly, improved sensors could expand range and improved electronics could increase the speed at which objects could be detected. Minor improvements can also be made to the mechanical design to further reduce the size, alter the detector angle to allow for more natural hand movement, and improved customizable design to allow for individual comfort settings as well as aesthetics.

Conclusions
The developed low-cost (<$24 USD), open-source navigational support system allows people with lost vision to solve the primary tasks of navigation, orientation, and obstacle detection (>0.5 m 2 stationary within the distance range of up to 4 m and moving up to 0.5 m/s within the distance range of up to 1 m) to ensure their safety and mobility. The devices demonstrated intuitive haptic feedback, which becomes easier to use with short practice. It can be largely digitally manufactured as an independent device or as a complementary part to the available means of sensory augmentation (e.g., a white cane). The device operates in similar distance ranges as most of the observed commercial products, and it can be replicated by a person without high technical qualification. Since the prices for available commercial products vary from $90-800 USD, the cost savings ranged from a minimum of 73.5% to over 97%.