Abstract

Advancements in embedded computer platforms have allowed data to be collected and shared between objects—or smart devices—in a network. While this has resulted in highly functional outcomes in fields such as automation and monitoring, there are also implications for artistic and expressive systems. In this paper we present a pluralistic approach to incorporating environmental factors within the field of site-responsive sonic art using embedded audio and data processing techniques. In particular, we focus on the role of such systems within an ecosystemic framework, both in terms of incorporating systems of living organisms, as well as sonic interaction design. We describe the implementation of such a system within a large-scale site-responsive sonic art installation that took place in the subtropical desert climate of Arizona in 2017.

1. Introduction

In this paper we discuss an approach to incorporating both acoustic and environmental factors within a large-scale multichannel sound installation. This research makes use of developments in portable, embeddable sensor, and microcomputer technologies. Sounding Out Spaces is an ongoing project that concerns context-based live electronic music and sonic art. Specifically, it aims to develop performances and installations that occur in response to a particular location or space. The sites involved range from retired industrial structures and visually stimulating landscapes, to architecture with unique acoustic properties, natural environments, and places of cultural or historical significance. Common to the collection of practices that has been developed is the theme that sound is produced in response to certain perceived or measured attributes of a particular site. These features may be acoustic, environmental, historic, and, perhaps, even imagined.

Sounding Out Spaces explores how we can use digital means to participate directly in our environment, leading to novel and inclusive experiences. These ideas are developed through iterative practice. The series began in 2014 with a focus on guerrilla performance using portable analogue technologies, and has evolved to involve, most recently, large-scale public installations. While the theoretical framework of this creative practice has been thoroughly discussed elsewhere (see [1]), the most recent work in this series, which will be addressed in what follows, focuses on developing transferable techniques and methodologies for both spontaneous and planned works. In this latest iteration we explore the implications of an ecosystemic approach. We describe a methodology where the resulting sound is contingent not only on all agents involved and their organizational relationships, but also the environment and the effects of its perturbations [2] on such a system.

Overview of Garden Ecologies

In April 2017, the latest iteration of the series was developed as a mutlichannel sonic art installation. Sounding Out Spaces: Garden Ecologies involved numerous audio and sensor processing platforms that were embedded throughout a community garden. These programmable digital–physical interfaces took readings from plants, vegetation, and the environment. Several new systems were developed in order to analyze and process biophysical information from the plants, soil, wind, water, and light. Along with the acoustic information from the site picked up by microphones, this data was used to affect various musical parameters of the digital signal processing (DSP) on the embedded microcomputers. The audio output of these was sent to multiple loudspeakers placed throughout the garden (see Figure 1 and Figure 2) bringing forth an emergent, continuously evolving musical world.

Various community groups were invited to take part in the project through a series of sound-based workshops based at the Clark Park Community Garden, Tempe, and the Children First Leadership Academy, Phoenix, Arizona. In these workshops, children from one of our partner organizations, Free Arts for Abused Children of Arizona, learned about listening to the environment, sound recording, and using biophysical properties of organic matter, such as plants and vegetation, along with simple electronic circuits to create new musical instruments. This culminated in a public site-responsive performance. The material produced by the children was also incorporated into the sound installation, weaving together the acoustic characteristics of the site with environmental and biophysical information from the garden and plants into an evolving and dynamic sonic ecosystem. The installation ran over two consecutive days and included an evening preview, in addition to the daytime presentations.

2. Background

In this section we outline several areas of research within the field of sound and music computing that have informed this work.

2.1. Sound and the Environment

Over the last few decades there has been a growing body of work being undertaken within the areas of soundscape composition [3], environmental sound art [4,5], and acoustic ecology [6,7]. The latter of these associated areas of research is an interdisciplinary field that “studies the social, cultural, and ecological aspects of our environment through sound” [8]. Currently, the majority of this research makes extensive use of field recordings, and collects, analyzes, and archives large amounts of sound material from the environment. Using the concept of acoustic indices [9], researchers can evaluate the long-term changes in the biodiversity of both terrestrial and marine ecosystems. Such estimations, which may include amplitude or intensity data, are important because of the volume of audio recordings that can accumulate over time, this often being more than can be reviewed and interpreted by human labor alone. On the other hand, much can be garnered about the conditions of a place simply by listening. Through its unfolding, sound can reveal complex, dynamic, and often otherwise hidden activity. The ambiances of a place can meander through the mundane repetitive processes of movement and growth, interrupted by sudden punctuations which disrupt the passing of time. With portable recording technology, we are able to make high-quality audio recordings of the soundscapes (see [10] for a critique of the notion of soundscape in its objectification of sound) of natural environments. Later in the studio, these sounds can be “humanly organized” [11] into elaborate arrangements for reception via loudspeakers within a concert setting, or through headphones when on the move. However, by bringing the outside in, through the careful capturing of the sounds that are sampled, we may lose a crucial element within the process of sensemaking [12] as these recordings are severed from their original contexts. Sound is always situated socially, culturally, and perceived within a particular space.

2.2. Ecological Systems in Sound

Concurrently, the environment has featured heavily within discussions of live electronic music in its role within the co-constitution of the body–instrument world that takes place during musical performance (see [13] for a detailed discussion of this). Out of this has grown a steadily increasing collection of theoretical developments and practices examining the ways in which ecological systems can be used to understand how musicians and audiences make sense of musical activity (see, for example, [14,15]). Additionally, this paradigm can be used to design new technological frameworks. Of these, Agostino Di Scipio’s work is notable in his proposition that emergent and dynamic behavior can arise out of the structural coupling of digital processes with the space in which they are situated [16]. For Di Scipio, “Eco-systems are systems whose structure and development cannot exist (let alone be observed or modeled) except in its permanent contact with a medium” [16] (p. 271). Ecological systems—or ecosystems—typically comprise biological communities, yet the paradigm can be extended to include human society or cultural networks. It can also be used to describe the very nature of the structural relationships within certain computational compositional domains. This particular application is the focus of project that will be discussed in this paper.

Di Scipio’s work challenges the often unquestioning adherence to the commonly used model of interactivity, where a musician can create, manipulate, or organize sound in time. In the digital–physical domain, this is facilitated through a digital musical instrument (DMI). DMIs are technologies which, in general, comprise a control interface (hardware) and a sound generator (software) [17]. In the ubiquitous interactivity model, which often views engagement with DMIs through the lens of human–computer interaction (HCI), the performer is in fact necessary in order to initiate—through some sort of physical or gestural manipulation—some changes of state within the instrument’s DSP. Rather, Di Scipio proposes an ecosystemic model in which an evolving, self-regulating sonic entity is brought forth by “composing interactions” [16] (p. 270). In this approach, the role of the environment is integral to—and inseparable from—this process. He suggests that such ecosystems “are autonomous (i.e., literally, self-regulating) as their process reflects their own peculiar internal structure. Yet they cannot be isolated from the external world, and cannot achieve their own autonomous function except in close conjunction with a source of information (or energy). To isolate them from the medium is to kill them” [16] (p. 271). While this does not preclude a performer from being part of the ongoing negotiations and adjustments that take place within the system, in this model their presence is not required for musical activity.

One key approach behind Di Scipio’s work involves setting up the potential for acoustic feedback to manifest from within systems of microphones and loudspeakers, which are often digitally mediated. This establishes sonic entities which self-regulate rather than ones which are the result of human organization of sound into particular aesthetic or systematic arrangements [18]. In collaboration with Dario Sanfilippo, some of the most recent work in this area focuses on techniques for incorporating the environment within these systems [19]. The environment in this work refers to the sonic ambiance of the space in which sound is being produced, along with any significant structural characteristics, and the effect that these may have on the audible feedback loops. Explicitly, the environment here is the “performance space” [19] (p. 23).

In their work, feedback occurs due to resonances both within instruments—which may comprise computers, loudspeakers, software, and other technology—as well as spaces. The instrument is said to be coupled with the environment because “they operate in a condition where they mutually affect each other” [20] (p. 31). However, in an open space it can be more difficult to find resonances, such as those formed by a the tube of an acoustic instrument such as a flute, or the four walls of a small room. An example of a solution to this problem can be found in Chris Kallmyer’s piece This Distance Makes Us Feel Closer (2013), which uses sound to create echo and resonance where there are neither any structures nor land features to do this naturally. These are orchestrated by the artist who creates autonomous desert fog horns, which suggests the presence of reflected sound waves—despite being positioned one mile apart—in the expanse of the New Mexico desert [21]. Nevertheless, in both these examples, the sonic result is uniquely dependent upon the situatedness of the work itself—it is necessarily site-specific.

2.3. Responsiveness to Site

The term site-specific—its origins grounded within the visual arts and minimalist sculpture [22] has become ambiguous through its usage as an umbrella term for describing art works that are either held outdoors (hosted outside of cultural institutions) [23], or works that would no longer make sense if moved to a new location [24]. In this broad view, site-specificity could be said to describe works which are constructed through a relational process, where the artist works with certain properties of a particular site. However, Miwon Kwon has noted that even the definition of site can be interpreted in multiple ways: as “phenomenological” [24] (p. 3), concerning the experienced physical qualities of a space; as addressing “social/institutional” [24] (p. 3) power structures; or as a site for “discursive” [24] (p. 3) activity surrounding political or economic issues.

A site-responsive practice [1] might be a more helpful way of describing the types of creative activities that engage with the environmental, sonic, cultural, or historical identities of a particular location—or the communities connected to that site—with careful reflection and sensitivity. It sidesteps the contradictions that arise from notions of site-specificity and focuses instead on developing a new set of methodologies to address its goals. These types of works connect “people with the space they occupy by bringing awareness to how their presence affects their environment” [25] (p. 231). Previous iterations of Sounding Out Spaces have addressed notions of multimodal participation that a site-responsive practice can foster [1]. In such cases, the audience is not directed to experience the work in a particular way, but is invited to explore their own trajectory through a variety of access points. Such approaches can result is unexpected behaviors. Daichi Misawa, while reflecting on his piece Transparent Sculpture (2012) which followed this embodied approach and premiered at Ars Electronica in 2012, observed that this can be affective “to such an extent that [the] audience started to walk, dance and scream” [26] (p. 390). This has implications not only for the accessibility of the work, but also moves away from musical experiences that Bennett Hogg describes as relying on a “framing of sounds within a stereo frame to be listened to under concert conditions [which] seems something of a betrayal of sound’s potentially ecosystemic properties” [27].

3. Methodology

In this section we describe our methodology for realizing this project. We do this both by outlining the design of the technological and biological ecosystems, as well as detailing the artistic and compositional decisions that informed the use of the site. We also discuss the choice of sonic material, sensors, and the relationships between these elements. While the resulting installation was very much site-responsive, it should be noted that the aim of this work was to develop a set of techniques that could be implemented in different locations, and respond effectively and uniquely in each scenario.

3.1. Surveying the Site

The selected community garden was favorable as the location for the installation both due to the authors’ familiarity with the site—a local farmers’ market is hosted there each week—and its close proximity to the university area. After establishing initial links with the grassroots organization that stewards and develops the site, we discovered that the Tempe Community Action Agency (TCAA) not only facilitates opportunities for locals to build community through their gardens, but also works to serve the underprivileged by providing food and shelter [28]. In addition to pragmatic factors, such as the site being a desirable size for the scale of this iteration, the diverse range of communities involved with the garden would enable us to utilize multiple perspectives as input and feedback during development. As Christabel Stirling has observed, this is crucial in order to determine the actual social impact of public sonic art beyond the implications reported by the artists themselves [29]. Furthermore, having regular interactions with volunteers and those who were growing produce at the site provided us with knowledge about the types of vegetation and wildlife that inhabited the garden.

The project began with a survey of the location through repeated visits. This involved spending a significant amount of time listening at different parts of the site to learn about, and become familiar with, its sonic characteristics. In addition to focused, selective listening techniques [30], we also recorded on-site audio from static points around the site, and as moving trajectories. The microphone was employed as a way to amplify, or examine in detail, potentially interesting sonorities that might remain otherwise unheard. As Diane Willow—a media artist and researcher working with technology and materials—asks, “What happens when we amplify and make tangible our perception of the subtle, the ephemeral and the seemingly silent?” [31] (p. 333). This initial sampled sound was not used in the final installation itself, but was analyzed and inspected off-site. This provided a quantitative measurement of the audible frequencies and characteristics of sounds within the various regions of the site, which complemented the observations made through repeated visits. We considered:

• which aspects of the environment could be used structurally or compositionally?
• what in the environment is moving, could make sound through movement, or is already audible?
• which elements of the environment are static, and which are dynamic/changing?
• which aspects are consistent/inconsistent?

After this process of surveying, observation, listening, and analysis, the site was divided up into four regions that would each be used to experiment with a different approach to site-responsiveness.

3.2. Re-Sounding Material

While the main thrust of this project involved autonomous, self-organizing systems, we were keen to incorporate some established HCI models within the site, particularly to engage the children that we were hosting through our workshops at the garden. Building on prior research examining embodied approaches to sound and technology education [32,33,34], these workshops involved developing listening practices and exploring the garden in order to discover potentially interesting sounds through field recording techniques using hand-held audio recorders (https://www.zoom-na.com/products/field-video-recording/field-recording/zoom-h1-handy-recorder). We appropriated an existing art installation comprising five colorfully-painted bicycles mounted vertically on one of the walls of a structure in the garden (see Figure 3).

Our work in the garden utilized several Bela [35]-embedded microcomputing systems. Bela is a low-latency audio and sensor platform developed by Augmented Instruments Laboratory at the Centre for Digital Music (C4DM) at Queen Mary University of London. Its ability to run software built in Pure Data (Pd), a commonly used programming environment for sound synthesis and design, allowed for the rapid prototyping, development, and composition of sonic interactions to be done on-site and in the surrounding environment. Comprising low-power processors, these systems were powered using portable Universal Serial Bus (USB) battery packs. Given the the subtropical desert climate, we also experimented with solar power banks. These certainly would have been a sensible option had we run the installation over a longer period of time.

By attaching reed sensors to the bicycle forks, and a magnet on one of the spokes of each bicycle wheel, the children were able to perform the sounds that they had recorded from the garden back into the site itself. For this section of the installation, a single Bela was used to run Pd. Within the software, the speed of the rotation of each of the wheels was calculated from the time intervals detected by the reed sensors. The speed of the freewheeling was mapped in a nonlinear manner to the speed of playback of the samples. The recorded audio samples were stored in lookup tables and played back using both the digital sample position and low-frequency sawtooth waveforms. In the first case, which was audible when the wheels were spun quickly—taken as when the sensor was triggered at a frequency >1 Hz—a timbral effect was produced. The latter case—spin rate <1 Hz—allowed the samples to loop rhythmically, and then eventually slow down, and cease to sound. The children explored ways to play and manipulate their samples through the physical interfaces of the bicycles, using gestures similar to scratching records with the wheels, as well as using physical objects pushed against the spokes to give audible added resistance to the spin.

We also worked with MakeyMakey [36]—a low-cost electronics invention kit—connected to various plants and vegetation in the garden. By building simple connections, the plants and humans became part of a circuit. The MakeyMakey was connected to a laptop running software which, again, triggered various samples recorded by the children when the human–plant circuit was closed (see Figure 4). Through an exploratory approach, the children often connected different areas of vegetation in the garden and played these new instruments collaboratively. In some cases, more than one person had to be involved in order to complete the circuit. Sounds from the site were used by participants as improvisational material, and they were able to work sounds taken from the garden back into the space. In this way, the garden became a reconfigurable and constantly evolving digital–physical instrument.

3.3. Ecological Garden-Raised Beds

3.3.1. Environmental Sensing

As discussed in Section 2.2, within Di Scipio and Sanfilippo’s approach, the environment is generally considered in terms of its ability to disturb and affect actual—or potential—acoustic phenomena. We build on their work by expanding the role of the environment in its ability to perturb aspects of the various autonomous systems that function within it by collecting data that is not restricted to the sonic domain. This was explored in the most technically challenging part of the installation: an ecosystemic network of 18 loudspeakers, each positioned in 18 raised garden beds among vegetables and plants. The beds were separated into three groups, each outfitted with its own system of microphones, speakers, and environmental sensors. In total, three Belas were used to run a variety of sensors in this area of the garden (see

Table 1. Placement of sensors, microphones, and loudspeakers within the raised beds.

Equipment	Description	Bed Location
Mighty Dwarf	portable 10 W speaker	A1, B1
Minirig	portable 15 W speaker	C1, B5
X-Mini	low-cost 2.5 W portable speaker	all remaining beds
Zoom H1 audio recorder	used for powered X-Y stereo microphones	A1, B1, C1
LOLLETTE anemometer	wind speed sensor	B1
Adafruit accelerometer	used to detect movement	B1
cadmium sulphide (CdS) photoresistor	used to detect movement via fluctuations in light	A1
Sonbest temperature/humidity sensor	used to detect slow changes in soil state	C1

and Figure 5 for details).

In addition to some high-quality portable loudspeakers (http://www.mightydwarf.com/product/blueii) (https://minirigs.co.uk/portable-speaker), it was possible to use low-cost loudspeakers (https://www.cnet.com/products/x-mini-uno-speaker-for-portable-use-xam14pu/specs/) for the majority of the beds due to the fact that they were battery powered and powerful enough to create a significant sonic impact when sounding simultaneously. In fact, it was crucial that the electronic sound did not overpower the sonic activity of the space, which was different depending on the time of day and what was happening at the site. Instead, the system was designed to fold any detectable sound into itself, and weave its own output back into the present sonic state.

Using custom software written in Pd and loosely based on Di Scipio’s work on autonomous feedback systems and emergent sonic structures [18], self-adaptive feedback loops were created between the microphones and nearby loudspeakers. Essentially, these comprised a dynamic bandpass filter which would allow more or less frequencies to pass through depending on the amplitude of sound being received through the microphones. Importantly, no sound other than what was picked up by the microphones was used as source material.

The Larsen effect [20] was induced by placing the microphones proximally to the loudspeakers. Tamed by the adaptive filter, this sound was further processed using delay lines, granular synthesis, comb filters, and bandpass filters. In additional to the audible processed feedback, sounds from both humans and animals entered into the network, including significant activity from birds and crickets. The environmental sensors, which included measures for soil, wind, movement, and light, were used to inject additional disturbances into the feedback computation (see Figure 6). The resulting sound was the result of several self-organizing systems coexisting within the same environment.

3.3.2. Composed Relations

Being the result of the coupling of autonomous systems, the work itself was not composed, though compositional decisions could be said to reside in the mappings and relationships between sensor input and computational processes. Following several visits to and observations in the gardens, sensors were placed within the beds primarily on the basis of which sensor was the best fit for the bed’s conditions. This might depend on the location of the bed within the site, or the density and type of vegetation within it. The data from the environmental sensors was then analyzed in real-time and normalized, scaled, and mapped to control various parameters of DSP on the Belas. The result was a curated collection of simple relationships between organic matter, computational processes, and the environment, combined to produce complex emergent behaviors. The aesthetic and experiential considerations were determined through repeated and ongoing listening and observing within the site on multiples days, and at different times. Below are a few examples of such composed relations:

Wind: Along with the sea, wind is arguably one of the only naturally continuous excitations [37]. Its continuous nature made it a useful resource to harness within this work. Wind speed readings from an anemometer (https://www.adafruit.com/product/1733) were used to control how quickly sounds were spatialized throughout the garden, creating a poetic connection between the movement of the wind, and the diffusion of sound within space. Minimum and maximum wind speeds were calculated continuously over the duration of the installation. When a new input value rose above a previous maximum (or below a minimum), the respective input parameter was adjusted, and the wind speed was calibrated accordingly. Due to the design of the anemometer—comprising three hemispherical cups mounted via arms onto a vertical shaft—wind speed is always measured as an average. This provided sufficiently useful wind-related data, which did not require further smoothing.

The relationship between the wind speed and sound spatialization was directly correlated: the faster the speed of the wind, the faster and more chaotic the movement of sounds. Slower wind speeds resulted in a stillness in sound content and diffusion. The spatialization was implemented using constant power panning, which was controlled by the wind speed. Timbral changes were achieved by mapping the rate of change of the scaled wind speed value to the density parameter of the granulator. This varied the number of grains of sound being played back within the software, leading to changes in perceived sound density.

Movement: In the same bed, an accelerometer (https://www.adafruit.com/product/163) was used to measure the movement of a swaying burlap sunshade. The three-dimensional data was normalized based on the minimum and maximum of values received over a sliding time-window. The method of rescaling the data was again used to account for the changing environmental conditions. In this case, the movement data was used to vary timbral aspects of the sound by modulating the speed of certain control waveforms within the granulator. The audible result was almost gestural, visually linked to the movements of the cloth, which allowed spectators to gain some insight into the processes at work.

Temperature: In another portion of the system, a soil sensor (https://cdn-shop.adafruit.com/datasheets/SLHT5.pdf) was used to measure the temperature and humidity of the soil. The initial soil state was read at the launch of the system, and subsequent fluctuations in these values modulated the overall tuning of several bandpass filters within the feedback system. The center frequencies of these bandpass filters were initially tuned to another audible source within the garden (see Section 3.4). The soil sensor values were scaled and mapping to these center frequencies. As they slowly changed over time, the resulting shifts in harmonicity led to an almost indiscernible process of detuning and retuning. Of course, in a climate more susceptible to rain—or if the bed had been watered—these changes would become more immediately perceptible to observers.

Ambient Light: Located in number 19 of the raised garden beds, a further system was developed, which involved several sunflowers. These were embedded with photoresistors (https://www.adafruit.com/product/161) to measure levels of ambient light (see Figure 7). The photoresistors were used as an affordable, lightweight solution to measure the amount of movement of individual flowers. This was accomplished by measuring the change in light over each subsequent frame. The sensor inputs of the Bela are sampled at audio rates, which allows for smooth alignment with audio data [38]. To avoid encumbering the movement of the plants, the photoresistors were strewn together using conductive thread.

Following the movement of each flower, fluctuations in light gathered by the photoresistors were mapped to the amplitude of several subtractive synthesis engines. These comprised white noise passed through eight resonant bandpass audio filters. Each filter was tuned to a different harmonic (positive integer multiple) of the audible drone of a nearby dairy factory (see Section 3.4 for details). As with the nonlinear nature of audible feedback systems, the movement of each flower is also complex, being influenced by environmental forces and its physical structure, as well the movements of adjacent branches. By giving several flowers their own voice, this imitation [39] allows for further insight into the complexity of their movements, bringing attention to subtleties of the environment by emphasizing the collective rhythm of moving plants.

3.4. Highlighting Acoustic Phenomena-Tires

As mentioned above, one feature of the site that we wanted to work with was the audible drone from a local dairy factory that could be heard for several kilometers within the surrounding neighborhood. The drone was always present in the garden, although its intensity varied sporadically at different times of day.

We mimicked this drone by using three sinewave oscillators tuned to 237 Hz, 284 Hz, and 363 Hz, at amplitudes of 0.5, 0.25, and 0.4 respectively. We played this synthesized chord back through large scrap tires that had been abandoned in the space (see Figure 8). In this way, the sound of the site was reinforced, and a found acoustic phenomenon was appropriated as a key sonic element within the sound design. We were able to play on the disparity between what was included in the site—both sonically and materially—and what was omitted.

As with the more complex ecological systems described above, this part of the installation was also semi-autonomous and emerged over time through a very simple negative feedback system. Minirig speakers were placed inside the tires, along with a single omnidirectional condenser lavalier microphone (http://www.audio-technica.com/cms/wired_mics/9c6eca17168eef6f/index.html), and a Bela microcomputer. In this case, the amplitude values of the sound picked up by the microphone were fed into an accumulator with a leak value. This was fixed at a rate that would allow the most audible dynamic behavior to arise from the system, based on both self-generating feedback within the tires, as well as sounds picked up from the garden. Again, this was controlled using an adaptive filter.

3.5. Historical Narratives

While visiting the garden, we met many volunteers and community members who rented the raised beds in order to grow produce. Through these relationships we learned more about the history of the site. The garden was created in 2014 at the location of a former municipal pool. While this pool had recently been filled with soil and was used as a growing area, parts of its existence were still visible. We amplified the sound of water filtration systems in the garden using two omnidirectional microphones set up as boundary microphones (http://www.dpamicrophones.com/microphones/dscreet/stereo-microphone-kit-with-sc4060), and spatialized this over four high-quality loudspeakers (https://www.genelec.com/8010) to transform the seated gathering area into an auditory water bath.

This was the only part of the installation that did not use battery-powered equipment, although all elements could have been powered by a small stand-alone power source, such as a generator. As with the other self-adapting parts of the installation, this system changed over time, responding and adjusting to the level of measured sound. The sound of water picked up by the microphones was processed through a spectral filter, two different live sampling audio engines, and delay effects. These processes would become more or less audible depending on the density of the captured sound—this was again calculated using a leaky accumulator.

4. Conclusions and Future Work

As a site-responsive installation, Garden Ecologies aimed to engage with the sonic, cultural, historical, and environmental factors of the Clark Park Community Garden. The work involved the development of an ecosystemic framework, which included several autonomous systems coexisting in the same place, unique in output, yet responding to perturbations from the same environmental cues, and each other. We built upon prior research into musical systems that are structurally coupled to the environment by incorporating environmental information that is extra-sonic. We have developed some techniques for working with organic vegetation and changes in weather states. While this post-digital approach successfully bridges the physical and digital worlds, in future iterations we hope to explore in more depth techniques that have been established within the bio art communities, such as Laura Cinti’s radical work on plant neurobiology, which engages with the perceptual and cognitive capacities of plants [40].

Technologically, this work has made significant progress in the design and creation of a portable, low-cost toolkit for site-responsive projects that can be implemented at and will be responsive to a range of locations. For example, at the time of writing, the technology had just been used in the sub-arctic tundra during the 2017 Ars Bioarctica Residency in Kilpisjärvi, Finland, a location with a drastically different environmental profile to the Sonoran Desert of Arizona.

The synchronicity between audio and sensor input afforded by the Bela platform, along with its ease of use, has certainly opened up the possibilities of working out “in the field” rather than in a studio, or gallery space. Nevertheless, despite the calibration and adaptation techniques described in this paper, significant time at the site was required to evaluate the efficacy of a particular approach or algorithm. Differences in hardware used, such as microphone type, would have a drastic impact on the sonic outcomes, as with the placement of such devices. Care was also required to select sensors that would provide meaningful data in various conditions. For example, we had to test photocells with different resistance ranges to determine which would be most useful both in bright daylight, as well as under the floodlights which lit the garden in the evenings.

We have designed the software systems as a modular framework, including tools for the scaling and mapping of data, as well as instruments for sound synthesis and feedback control. We plan to adopt a similar modular approach to the hardware, with the design of custom, robust boards that can facilitate the addition of a variety of sensors quickly and easily, and which can also powered by solar energy. Through further iterations at multiple sites we hope to begin to understand the efficacy of such an approach as an alternative to the sampling and analysis techniques that are used within acoustic ecology, as well as offering a more dynamic approach to sonification through nonlinear techniques.