Synthetic Aperture Computation as the Head is Turned in Binaural Direction Finding

Abstract

Binaural systems measure instantaneous time/level differences between acoustic signals received at the ears to determine angles $\mathsf{\lambda }$ between the auditory axis and directions to acoustic sources. An angle $\mathsf{\lambda }$ locates a source on a small circle of colatitude (a lamda circle) on a sphere symmetric about the auditory axis. As the head is turned while listening to a sound, acoustic energy over successive instantaneous lamda circles is integrated in a virtual/subconscious field of audition. The directions in azimuth and elevation to maxima in integrated acoustic energy, or to points of intersection of lamda circles, are the directions to acoustic sources. This process in a robotic system, or in nature in a neural implementation equivalent to it, delivers its solutions to the aurally informed worldview. The process is analogous to migration applied to seismic profiler data, and to that in synthetic aperture radar/sonar systems. A slanting auditory axis, e.g., possessed by species of owl, leads to the auditory axis sweeping the surface of a cone as the head is turned about a single axis. Thus, the plane in which the auditory axis turns continuously changes, enabling robustly unambiguous directions to acoustic sources to be determined.

1. Introduction

This article proposes a biologically inspired solution to the binaural location of directions to single or multiple acoustic sources in both azimuth and elevation. Wallach [1], based on observations of human behavior, inferred that humans locate directions to acoustic sources by “dynamically” integrating information received at the ears as the head is turned while listening to a sound. A synthetic aperture computation analogous to those performed in the geophysical process of migration applied to seismic profiler data, and in synthetic aperture sonar/radar systems, can explain Wallach’s observations, constituting the “dynamic” process alluded to. The solution can readily be implemented in a binaural robotic system measuring interaural time differences as the head is turned. This might inform the development of hypotheses on biological acoustic localization as well as being of considerable interest in the field of robotics in its own right.

Sound received at the human ear is perceived by our conscious mental view of the world as a continuously updated collection of tones. The human auditory system detects sound over a broad range of frequencies from ~20 Hz to 20,000 Hz, ~10 octaves, which for a transmission velocity in air of 330 ms⁻¹ corresponds to wavelengths of 16.5 m to 0.0165 m (1.65 cm). For comparison, a concert piano accommodates a tonal range of just over seven octaves. Sound with a wavelength the distance between the ears for a human (nominally ~0.15 m) has a frequency of ~2200 Hz.

The human auditory system characterizes sound from the shape of amplitude/power spectra and recognizes sound sources with reference to memorized associations between spectral characteristics and sources. This function of the human auditory system has inspired the development of methods applied to marine geophysical sonar data to characterize and classify sea-beds based on features describing the morphologies of sidescan sonar trace power spectra [2,3].

The human visual system in contrast to the auditory system perceives a little less than a single octave of the optical part of the electromagnetic (e-m) spectrum in frequency bands centered on just three frequencies. The human mind constructs a high resolution color worldview of the environment from images formed on the retina at the backs of the eyes, focused at the center of the field of view.

Auditory systems operate on signal having considerably greater wavelength than visible e-m radiation with correspondingly lower capacity for resolution, estimated to be 1.0$°$ (Mills [4]) to 1.5$°$ (Brughera et al. [5]) in the direction the head is facing. For comparison the face of the Moon subtends an angle of ~0.5$°$ at the Earth’s surface. Nevertheless, similar to the human visual system, the auditory system provides us with spatial information on the location of energy sources. Humans do not perceive sound as images in the way we perceive objects illuminated by light, though there is no reason in principle why this is so, nor why other animals, or robotic systems, should not. With an index finger pointing at arm’s length in the direction of an acoustic source within our field of vision, a human can impose aurally derived information on its visual system demonstrating that despite the absence of an aurally derived image, we nevertheless have an aurally informed conscious worldview either incorporated into or existing in parallel with and augmenting our visually informed one.

The human visual system is binocular. Two eyes confer no advantage over one for direction finding, but extend the field of view to a little less than a hemisphere (a solid angle a little less than 2π steradians), and more significantly provide a perception of range. The closer an object, the more accurate is the estimate of range. The amount of convergence applied to the axes of the eyes to achieve a single focused image, and the amount of compression applied to the eyes’ lenses constitute measures of the distance to an object in our visually informed worldview.

With two ears we are able to extract information on the direction to an acoustic source over a spherical field of audition (a solid angle of 4π steradians) based on differences in arrival times of sound at the ears (interaural time difference, ITD). We might expect the limiting acoustic frequency for measurement of ITD to be that corresponding to the wavelength approximately equal to the distance between the ears (frequencies less than ~2200 Hz), however the limiting frequency is found to be somewhat less at approximately 1500 Hz (e.g., Wightman and Kistler [6], Brughera et al. [5]). Measurement of arrival time difference might be made by applying a short time-base cross-correlation process to sounds received at the ears (Sayers and Cherry [7]) or by a functionally equivalent process (e.g., Jeffress [8], Colburn [9], Kock [10], Durlach [11], Licklider [12]). For acoustic signal at higher frequencies locating acoustic sources is dominated by the use of interaural level difference (ILD) [5,6].

With two ears there is a difficulty in determining the direction to an acoustic source. A single estimate of an angle between the acoustic source and the auditory axis$\mathsf{\lambda }$, whether by an ITD or ILD ambiguously determines the acoustic source to lie on the surface of a cone and does not unambiguously determine the direction to an acoustic source in azimuth (longitude) and elevation (latitude) with respect to the direction the head is facing. Some species of animal possess ears with independently orientable external ears, pinnae, for example, cats. These animals can explore sounds by rotating their pinnae without turning their heads. In this way, they appear to be able to determine the direction to a source of sound with a single ear and, in principle with a pair of ears even be able to estimate range by triangulation. However, many species of animal including humans cannot orientate their pinnae and some other mechanism must apply for binaural direction finding, and it is with this that this article is concerned. Wallach [1] found that the ambiguity inherent in finding the direction to an acoustic source with a pair of ears in humans is overcome as the head is actively turned to explore a sound (also [13,14,15,16,17]), which he posited is achieved through a “dynamic” integration of information received.

Other aural information might also be integrated into an interpretation of the direction to an acoustic source. A nodding rotation of the head about the auditory axis might provide information on the elevation of a sound source due to the effect on the spectral content of the signal arriving at the ears of the shape of the pinnae and the head around which sound has to diffract; the so called head related transfer function (HRTF; Roffler and Butler [18], Batteau [19], Middlebrooks et al. [20], Rodemann et al. [21], also Norberg [22,23,24] for owls). However, Wallach [1] observed that nodding is ineffective for locating a sound source. Nevertheless, a HRTF effect might provide supplementary information in aural direction finding.

It is proposed in this article that the dominant process by which the directions to acoustic sources are located by the human and other animals’ binaural systems, and one which is readily implementable in a robotic binaural system, employs a synthetic aperture computation acting on a stream of estimates of the angle $\lambda$ as the head is turned while listening to a sound. The synthetic aperture computation process locates directions in both azimuth and elevation for single or multiple sources. There is no front–back ambiguity. There is a below–above horizon ambiguity for turning the head about a single vertical axis for a pair of listening sensors arranged without vertical offset, but this ambiguity is overcome by introducing a vertical offset (a slope on the auditory axis). The method is described in terms of geometrical mathematical manipulation and is readily implementable in a robotic system. In fact, the detail of how the method might be implemented in nature in terms of biological/neural components performing functions equivalent to mathematical operations is currently shrouded in rather more mystery.

The synthetic aperture computation process is an integration that can be imagined as being performed in a virtual field of audition, implemented in a robot as a 2D array of azimuth, elevation (longitude, latitude) positions, or in a natural auditory system by a subconscious representation of the field of audition existing in parallel with the one of which we are aware. The synthetic aperture computation has the effect of promoting the direction finding capability of two ears to that of a large two-dimensional array of stationary ears. The process is analogous to those applied in the anthropic technologies of migration in seismic data processing and in synthetic aperture side-looking radar, and sidescan sonar systems (Appendix C).

It is only relatively recently that binaural sensing in robotic systems has developed sufficiently for the deployment of processes such as finding directions to acoustic sources [25,26,27,28,29,30,31,32,33,34,35,36]. Acoustic source localization has been largely restricted to estimating azimuth [26,27,28,29,30,31,32,33] on the assumption of zero elevation, except where audition has been fused with vision for estimates also of elevation [34,35,37,38]. Information gathered as the head is turned has been exploited either to locate the azimuth at which ITD reduces to zero thereby determining the azimuthal direction to a source, or to resolve the front–back ambiguity associated with estimating only azimuth [28,29,30,31,32,33,34,39,40]. Recently, the application of Kalman filters acting on a changing ITD has produced promising results [41,42,43].

2. Angle $\lambda$ (Lamda) between the Auditory Axis and the Direction to an Acoustic Source

A straight line simplification of the relationship between arrival time difference and angle to acoustic source is illustrated in Figure 1. In fact, sound must diffract around the head to reach the more distant ear particularly for large values of$\left|\lambda -90°\right|$ rather than travel the straight line paths shown [39,44]. The relationship is rendered more complicated still where differences in signal level at the ears, as well as differences in arrival time, have an effect on the estimate of$\lambda$. However, the relationship would be accurate for a simple robotic head designed for the purpose of acoustic localization experiments, consisting of little more than a pair of microphones, and for which acoustic signal dominated by wavelengths greater than the distance between the microphones is utilized.

From Figure 1, the relationship between angle to source and path length difference for straight line and far-field approximations is:

$$\cos \lambda =fd/d=f$$

(1)

$$\lambda =\mathrm{acos}\left(f\right)$$

(2)

where

$\lambda$	is the angle between the auditory axis and the direction to the acoustic source;
$d$	is the distance between the ears (the length of the line LR); and
$fd$	is the difference in the acoustic ray path lengths from the source to the left and right ears as a proportion of the length $d$ ($-1\le f\le 1$).

The distance $fd$ is related to the difference in arrival times at the ears measured by the auditory system by:

$$fd=c\Delta t$$

(3)

where

$c$	is the acoustic transmission velocity in air (e.g., 330 ms−1); and
$\Delta t$	is the difference in arrival times of sound received at the ears.

A machine computes an angle to source by performing the calculation in Equation (2). In nature the mind subconsciously computes the angle by a functionally equivalent process.

An estimate for $\lambda$ does not uniquely determine the direction to an acoustic source but ambiguously determines the location of the source on the surface of a cone situated to one side of the head, rotationally symmetric about the auditory axis, with its apex at the auditory center midway between the ears. The surface of the cone projected from the auditory center onto a spherical surface centered at the auditory center, maps onto the sphere’s small circle of colatitude for angle $\lambda$ (a lamda circle). With respect to the direction the head is facing, the circle in the field of audition has an elliptical aspect. In the most extreme case, in which the acoustic source is equidistant from the ears ($\lambda =90°$), the cone reduces to a plane coincident with the median plane, and the corresponding circle in the field of audition appears rotated by 90$°$ to an ellipse with zero width i.e., a line.

One way to uniquely determine the direction to an acoustic source would be to align the auditory axis with the direction to an acoustic source by turning the head to maximize the time delay between the ears. The estimate of angle would be compromised by the need for sound to diffract around the head to the more distant ear, but more importantly, the uncertainty in estimates of angle from the difference in arrival times becomes large as $\lambda$ approaches 0° (or 180°).

The uncertainty in the angle $\lambda$ between the auditory axis and the sound source is:

$${\sigma }_{\lambda }=\frac{d\lambda }{d\Delta t}{\sigma }_{\Delta t}$$

(4)

where

${\mathsf{\sigma }}_{\Delta \mathrm{t}}$	is the uncertainty in the estimate of arrival time difference $\Delta t$; and
$d\lambda /d\Delta t$	is the rate of change of $\lambda$ with respect to $\Delta t$.

Differentiating$\lambda$ with respect to$\Delta t$ Equations (1) and (3) gives:

$$\frac{d\lambda }{d\Delta t}=\frac{-c}{d}/sqrt\left(1-{\left(\frac{c\Delta t}{d}\right)}^2\right)$$

(5)

The quantity $\left(c\Delta t/d\right)$ tends to unity as $\lambda$ approaches 0° or 180°, and the rate of change $d\lambda /d\Delta t$ tends to$-\infty$, and so the estimate of $\lambda$ is maximally inaccurate when the acoustic source is on the auditory axis. By the same token,$d\lambda /d\Delta t$ is smallest when$\Delta t$ is zero, and the estimate of$\lambda$ is therefore most accurate, hence in principle the most accurate estimate of $\lambda$ is made for $\lambda$ = 90° (the auditory central fovea [21,45]). In practice, humans in exploring a sound do not turn the head to align the auditory axis with the direction to an acoustic source in fact quite emphatically we tend to turn our face to it (aligning the auditory axis normal to it).

The need for an accurate determination of angles at which sounds arriving at the ears maximally correlate for$\Delta t$ ≈ 0 suggests that in humans/animals the data upon which the cross-correlation process acts are not the power spectra of which we are consciously aware, for phase information in waves is lost in computing amplitude/power spectra. To correlate two series of continuously updated power spectra as a two-dimensional cross-correlation would require power spectra to be updated at a very high rate, and require a varying signal to be present at the highest frequencies discernible to the human ear. In fact, we can accurately determine the null position $\Delta t$ = 0, when high frequency signal is absent. These considerations suggest that the cross-correlation process acts upon a subconscious transduction of the raw pressure signal received at the ears [7].

3. Synthetic Aperture Audition

It is now considered how information from a series of estimates of values for $\lambda$ determined as a binaural head is turned while listening to a sound can be integrated for estimation of the direction to an acoustic source in azimuth and elevation. This is done with reference to simulated data computed for an acoustic source at an azimuth or longitudinal position $\theta$ = 0°, and at an elevation or latitude of $\phi$ = −30° (i.e., below the horizontal).

3.1. Horizontal Auditory Axis

The integration of data acquired by a binaural system as the head is turned, constituting a synthetic aperture computation, is illustrated for simulated data in Figure 2 and Figure 3. Figure 2 shows a chart of a collection of lamda small circles of colatitude on the surface of a sphere in a virtual field of audition in Mercator projection for five instantaneous lateral (longitudinal) angles $\theta$ to the right of the direction the head is facing to a single acoustic source. The direction to the acoustic source is used as the datum against which measurements of $\theta$ are made. In fact any direction could be chosen for this purpose. The discrete values of $\theta$ in Figure 2 vary from 90° to 0° in intervals $\Delta \theta$ (−22.5°) as the head is turned. Figure 3 illustrates the relationship between the orientation of the head and the orientation of the lamda circles for each of the discrete positions of the head for the same data shown in Figure 2.

In a practical robotic implementation, and in nature, the number of lamda circles as the head is rotated through 90° would be many. Just five are shown in Figure 2 and Figure 3 for the sake of clarity for the purpose of illustrating and describing the synthetic aperture process. Simulated values of $\lambda$ as a function of $\theta$ and $\phi$ in Figure 2 are computed (Appendix A) from:

$$\lambda =\mathrm{acos}\left(\cos \theta \cos \phi \right)$$

(6)

The implementation details of how values for $\lambda$ and $\theta$, e.g., as shown in Figure 2, are rendered as lamda circles for display in a chart, are provided in Appendix B.

The vertical center of Figure 2 represents the vertical position of the auditory center. With the head facing left ($\theta$ = 90°) the auditory axis is perpendicular to the page and aligned with the lateral position of the acoustic source. The source is ambiguously located on the small circle of colatitude labeled 1 (red), and in a synthetic aperture computation, acoustic energy is integrated over the circle into the virtual field of audition (Figure 2 and Figure 3, top left). As the head is turned through an angle $\Delta \theta$ = −22.5$°$, the mind/vestibular system or a robotic analogue, compensates by rotating the spatial information in the worldview with respect to the position of the head, by −$\Delta \theta$ = 22.5$°$. Thus, the representation of objects in the worldview and data pertaining to them (e.g., the integrated acoustic energy over lamda circles in a virtual field of audition), do not move relative to objects in the real world as the head is turned. With the observer’s head turned one quarter of the way towards the acoustic source ($\theta$ = 67.5°), the updated instantaneous value for $\lambda$ will locate the source on the cone represented by the circle numbered 2 (orange), and again in a synthetic aperture computation, acoustic energy is integrated over the circle into the virtual field of audition (Figure 2 and Figure 3 top middle). This continues for successive values of $\theta$ until the observer is facing the source of sound ($\theta$ = 0°) and the acoustic source is located on a cone that is reduced to a plane coincident with the median plane, represented in the observer’s worldview by the great circle of colatitude for $\lambda$ = 90° numbered 5 (blue) appearing as a line, and in a synthetic aperture computation acoustic energy is integrated along the line into the virtual field of audition (Figure 2 and Figure 3 bottom middle).

As the head is turned, acoustic energy (alternatively, the amplitude of a peak in a short time-base cross correlation function between acoustic amplitudes received at the ears) over multiple instantaneous lamda circles is integrated in the virtual/subconscious field of audition. The direction to the acoustic source is given by the direction to the maxima in the integrated acoustic energy in the acoustic image in the virtual field of audition. This corresponds to the direction to the intersections of the lamda circles as they would appear in the virtual field of audition. In this way, the ambiguity associated with directions to a multiplicity of points on individual lamda circles collapses to an unambiguous (or at least, less ambiguous) direction to points of intersection of the circles.

By the time we have turned our head through a few, to a few tens, of degrees we have usually unambiguously located the direction to an acoustic source both azimuthally (in longitude) and in elevation (latitude) [1,13,14,15,16,17] having generated estimates for $\theta$ and$\phi$ with respect to the direction the head is facing, by the mind subconsciously performing in real time a computation functionally equivalent to an integration of energy over numerous instantaneous lamda circles. Note that whilst Figure 2 and Figure 3 (and similarly Figure 4) show collected lamda circles pertaining to the time when the head is turned such that $\lambda$ = 90° and for a large change in$\theta$, in fact a solution to the direction to the acoustic source requires only a sufficient change in$\theta$ for a solution to emerge, and does not require the head to turn to face the source. The result of the synthetic aperture computation is presented to the aurally informed worldview as the (in nature consciously perceived) location of an acoustic source along a line. The perceived directions to acoustic sources may be refreshed as required by repeating the application of the synthetic aperture computation process by turning the head.

In the synthetic aperture computation, the auditory system integrates geometrically coherent data received by a pair of ears as the head is turned, to endow it with the functionality of a large static two-dimensional (2D) array of hearing sensors.

3.2. The effect of an Inclined Auditory Axis

Rotating the auditory axis within a single plane leads to the ambiguity of two possible locations for an acoustic source in Figure 2 ($\phi$ = ±30°). To uniquely determine the direction to an acoustic source, the head could be turned such that the auditory axis is swept within more than a single plane. Wallach [1] noted based on experimental observations made, “We have found a number of different movements of the head to be effective in sound localization. The most frequent natural head movement is a turning of the head upon which a tilting to the side is gradually superimposed as the motion approaches the end of the excursion”, (italics Wallach’s). It was also noted that “a side-to-side motion is very effective but unnatural” [1].

The effective use of a lateral tilt of the auditory axis for direction finding during head rotation is exploited in an evolutionary adaptation by species of owl which have ears vertically offset on an auditory axis slanting at an angle [22,23,24], e.g., at approximately 20°. Thus, as a slanting auditory axis is rotated about a single axis, the auditory axis sweeps over the surface of a cone and in this way the plane in which the auditory axis sweeps continuously changes. Therefore the integration of acoustic energy over lamda circles in a synthetic aperture computation as the head is turned for various values of $\theta$ yields a robustly unique direction to the source of sound.

Angles$\lambda$ in the table in Figure 4 are computed for values of$\theta$,$\phi$ and$i$ (Appendix A) from:

$$\lambda =\mathrm{acos}\left(\sin \theta \cos i\cos \phi +\sin i\sin \phi \right)$$

(7)

where$i$ is the inclination of the auditory axis to the right across the head.

The angle$\theta$ for $\lambda =90°$ in which an inclined median plane intersects the source of sound (the line labelled 6 in Figure 4) (Appendix A) is:

$$\theta =\mathrm{asin}\left(\tan i·\tan \phi \right)$$

(8)

It is seen in Figure 4 that an unambiguous solution for the direction ($\theta$, $\phi$) to an acoustic source is generated for ears with a vertical offset and slanting auditory axis, rather than the ambiguous ones for a horizontal auditory axis (Figure 2).

It is often stated that the slant of the auditory axis of owls endows an advantage to hearing performance (e.g., in ornithological guide books), however, apart from a use in synthetic aperture computation, this adaptation would serve no obvious advantageous purpose.

4. Remarks

Synthetic aperture computation during the turning of the head delivers a solution to the direction to acoustic sources both azimuthally (longitudinally) and in elevation (latitudinally) with respect to the direction the head is facing.

The computation in synthetic aperture audition (SAA) is strikingly similar to the process of migration applied to seismic profiler 2D or 3D images. This is described and illustrated in Appendix C. In the SAA process, the location of data distended over circles generated for instantaneous determinations of acoustic source direction, reduces to points after the data are integrated in the virtual field of audition in which the SAA process is performed. Similarly, in migration, data that appear in pre-migrated distance-time sections to be located over hyperbolae, are actually ambiguously located on circles. When the data are subsequently distended over the circles and integrated in the migrated distance-distance section, hyperbolae in the raw pre-migrated section collapse to points in the migrated section [46,47,48,49,50]. Similarly, SAA computation is analogous to synthetic aperture computation performed in synthetic aperture radar (SAR) and sonar (SAS) systems to compute precisely located positions of targets on high resolution distance-distance radio/sonograms from poorly resolved linear elongated features spread over multiple traces on unprocessed distance-time images [46,51,52,53,54,55].

These anthropic computer programmed image processing methods involve computationally intensive (and often very time consuming) calculation. Migration and SAR/SAS computations require accurately navigated data in order to take advantage of the geometrical coherence inherent in the raw data. Similarly, SAA computation must incorporate accurately measured (in nature vestibular) attitudinal data for the head as it is turned, continuously and in real time, to re-orient the aural worldview to appropriately realign compound integrated data in the virtual field of audition, in readiness for integrating acoustic energy over the current instantaneous lamda circle.

An interesting aspect of SAA in humans and undoubtedly other animals too is that sound from multiple sources arriving at the ear from different directions can be disentangled and sources located simultaneously. The synthetic aperture computation approach is naturally extensible for multiple acoustic sources. Multiple sources leading to multiple events (peaks) in short time-base cross correlation signals can all be mapped to lamda circles. Multiple sources then lead to multiple sets of lamda circles with intersections at correspondingly multiple points. Accumulations of acoustic energy at multiple points in the virtual field of audition will allow the directions to multiple sources to be simultaneously determined. Spurious events in short time-base cross correlation signals not associated with primary acoustic sources but with secondary effects, will be less likely to produce lamda circles coherently intersecting at points and therefore be unlikely to register and be identified as primary sources in the virtual field of audition.

It should in principle be possible to estimate range as well as direction in SAA computations from near-field deviations from far-field behavior. By relaxing the far-range approximation and computing lamda circles at multiple spherical shells with radii equal to distances $s$, it should in principle be possible to estimate range by optimising the distance $s$ to that for which the corresponding lamda circles best converge/focus to a point. This would add a dimension to the synthetic aperture calculation, and acoustic energy maxima would be sought in a three dimensional volume over a virtual field of audition, rather than over a two-dimensional surface. Whether humans are capable of estimating range in this or an equivalent way is questionable but it is likely that animals with more highly developed auditory systems are capable of this. Bats hardly need to since they measure distance to target more directly using an active source sonar system [56], but owls’ hearing is passive and yet they are known to be able to catch prey in total darkness suggesting they are equipped to measure distances to acoustic sources.

Bats follow convoluted flight paths, much more so than swallows, swifts and martins in pursuit of the same kind of prey. The purpose of this might be to perform SAS type calculations in generating a worldview informed almost exclusively by sonar in the near absence of visually derived information.

We subconsciously move our heads to update our worldview on our environment, integrating both visually and aurally derived information. In the absence of visual cues, for example in complete darkness, we tend to turn our heads in a quite exaggerated way to explore aural signal. This suggests that visual and aural directional cues are integrated by a top level worldview manager incorporating information from both (plus other) types of sensory system, and this approach has been exploited in robotic systems (e.g., [35]).

Acoustic source direction finding is achieved in nature by formidable feats of acoustic signal and image processing: first, to estimate values for$\lambda$ from the results of a short time-base cross-correlation of acoustic signal received at the ears [7]; and second, to integrate acoustic intensity over lamda circles (or an equivalent process) in a virtual field of audition in performing a synthetic aperture computation. Some processes in Nature were not recognized until after processes developed for anthropic technologies suggested they might exist, e.g., the development of anthropic sonar technologies (Chesterman et al. [57]) suggested echo-location in bats (Griffin [56]) and toothed whales, e.g., dolphins (Au [58], Au and Simmons [59]). The same may apply to synthetic aperture processing. These computationally intensive calculations dramatically, and seemingly to the initiate almost magically, find degrees of resolution in processed geophysical images very much absent in the raw data images, and it now appears that analogous processes have been operational all day every day in natural auditory systems including our own for epochs.

It seems possible even likely that animals having highly developed auditory systems such as owls, bats and dolphins experience aurally informed mental images akin to those associated with the visual system in humans, possibly in some cases even incorporating color. There is no reason in principle why this should not be so. The worldview imaging capability of nocturnal animals in particular would otherwise be underutilized in low light conditions [60,61]. An option to display acoustic images on monitors could be provided for a robotic system for the human visualization of the results of the stages of processing acoustic data. The option could be exercised for the purpose of system development though it need not necessarily be exercised in subsequent routine use. In this way the workings and results of intermediate calculations in a robotic system would be rendered visible. It is an unfortunate fact that humans are aware and conscious of the result of some astounding feats of acoustic data processing, but we are quite unaware of the workings out along the way performed sub-consciously.

Methods for performing SAA computations, and hypotheses on how SAA computations are carried out in natural auditory systems, could be developed and explored by experimenting with robotic auditory systems to perform the tasks achieved by natural audition. An engineered system could incorporate a monitor for visualizing the content of the virtual field of audition as aural data are subjected to various stages of processing, and for displaying a summary of inferences made in a visualization of the aurally informed worldview.

5. Summary

An approach has been developed to uniquely determine directions to acoustic sources using a pair of omnidirectional listening devices (e.g., ears without independently orientable pinnae) based on measurement of arrival time differences in signals received at the ears, and on the integration of information in a virtual, and in nature subconscious, field of audition as the head is turned. At any instantaneous position of the head, a sound is ambiguously located on a small circle of colatitude of a sphere centered at the auditory center. As the head is turned, the ambiguity collapses to the point of intersection of multiple small circles of colatitude in the virtual/subconscious field of audition in which the positions of the circles are continuously updated by a direction measurement sensor or in nature the vestibular system, as the head is turned. This process constitutes a synthetic aperture computation promoting the direction finding capability of a pair of listening sensors to that of a large two-dimensional array of sensors, and is remarkably similar to those performed in migration applied to seismic data and in synthetic aperture radar and sonar systems. The method can elegantly account for the observations of human acoustic localization by Wallach [1] and might constitute the “dynamic” process he posited whereby data acquired as the head is turned are integrated for determining directions to acoustic sources. The method is readily implementable in robotic systems capable of determining angles between the auditory axis and an acoustic source by measuring interaural time differences (or by some other method), and with an appropriate motion sensor for measuring head orientation, in emulation of natural audition, and in-so-doing will: enhance robotic auditory capability; provide a powerful basis for exploring developing hypotheses on the operations of auditory systems in nature; and enable a comparison of robotic auditory performance with those of natural auditory systems.