Waves to Waveforms: Performing the Thresholds of Sensors and Sense-Making in the Anthropocene

Abstract

This paper details the technical and conceptual background for the developing art project Waveform. This project is a creative-critical meditation on the role of digital sensors in monitoring and representing environmental change. It explores the origins and functioning of the global sensory architectures used to detect and assess these changes, deconstructing the connotations of omniscience, abstraction, and control associated with the ‘top-down’, data-driven mappings they generate. In so doing, Waveform enacts a speculative instance of how digital sensors can highlight the ambiguities and tensions of life in an increasingly damaged ecology. This experimental aspect involves capturing images of coastal shorelines using an airborne camera drone, and then analysing these using software that maps the outlines of incoming waves. The resulting data is then processed by software that generates text resembling free-verse poetry. These steps are not autonomous, and are subject to human intervention at each stage, with the generated poems being curated so as to engage themes concerning coast, a changing climate, and scientific knowledge-making. The outcome is an assemblage of artefacts, processes, and representations that can suggest alternative narratives of sensing and sense-making, so as to better apprehend the complexities of the present moment.

1. Introduction

Commencing in the spring of 2017 Waveform is a speculative redeployment of varied technologies—airborne drones, machine vision, and automatic text generation—in search of alternative narratives of environmental sensing and sense-making (see Carter 2018). In this project, a camera drone is used to capture images of incoming ocean waves above remote coastal shorelines. These images are then analysed using a program that traces the nebulous threshold between land and ocean. The coordinates that plot this wavering boundary act as a source of variables for another program that generates text resembling free-verse poetry. The source vocabulary for this generator is curated to evoke different themes concerning the coastal environment, a changing climate, or the practices of measurement and classification in a scientific context. Each stage of this process yields distinctive visuals that are placed together in sequence, and it is this which constitutes the project’s primary output.

The primary goal of Waveform is to generate a creative assemblage that both meditates and speculates on the role of digital sensors in environmental monitoring and representation. At one level, the project draws attention to the perceptual ‘thresholds’ of these systems and the outputs they generate. That is, how varied phenomena become observable and expressible as data, through the convergence of specific sites of interest, technologies of sensing, and contexts and techniques of interpretation. This is a depiction of the ‘observable’ not as the straightforward detection and recording of latent facts and measurements, but as emerging through a dialogue between multiple actors, both human and nonhuman. It is through this dialogue that intelligibility and significance are defined, interpretations become established, and taxonomic boundaries are drawn.

It is at this point that the project’s more speculative aspects come into play. In using quantitative data to generate a form of poetic output, as opposed to a numerical graph, Waveform redeploys digital sensors as a framing device for modes of sense-making that fall outside their established thresholds of detection. In other words, Waveform adapts systems premised on sensing and representing the world in terms of the ocular, the abstract, and the quantifiable (digital cameras and machine vision software) in order to highlight their limitations, and so hinting at other, less privileged, modes of experience. These include the myriad uncertainties, critical aporia, and affective tensions of living amidst ecological crisis.

The aim of this gesture is not to suggest that these different modes of experience and sense-making are diametrically opposed, but that placing both into dialogue can help assemble narratives that are better able to capture the complexities of the present moment. That is, to recognise the multiplicity of perspectives at play, and the entanglements that exist between them. Ultimately, the aim of Waveform, as a creative endeavour, is to present a set of provocative artefacts to think with, whose respective histories, as will be discussed in this paper, give cause for reflection on the origins of contemporary ecological challenges, and how they might be represented.

At the time of writing, Waveform is still at a prototypical stage in its technical and conceptual development, and is still subject to further improvements and revisions. Consequently, this paper is presented not as a final summary, but to situate Waveform with regard to related art works and theoretical debates. This academic background is an integral component of the project’s outcomes, in its attempt at bringing into dialogue different modalities of thought, technology, and practice.

2. Technology

Prior to detailing the conceptual aspects of Waveform, a further word on the curatorial decisions and the technology behind the project, as these have been developed so far.

2.1. Location

The first stage of the creative process involves using a commercial camera drone to orbit at low altitudes above a coastal shoreline, taking still images of incoming waves from a top-down perspective (see Figure 1).

The coastal environment itself, as a site of artistic enquiry and intervention, is a key point of reference for the Waveform project. In the context of contemporary climate science, the coast represents a vital domain for observing, monitoring, and experiencing the salient markers of rapid ecological transformation, e.g., rising sea levels, turbulent weather patterns, shoreline contamination, or cliffside erosion. Consequently, it represents a dynamic, liminal environment—a threshold not simply between land and ocean, but between the forces of climate, ecology, energy, technology, consumption, jurisdiction, and trade. The coast is thus a vivid site in which to observe processes that both shape and threaten the contemporary world.

All the initial photography for this project has taken place at a single location: a remote, Atlantic-facing beach in far southwest of the United Kingdom, known as ‘The Strangles’. The origins of this sinister name are undocumented, but it is attributed locally to its bordering by jagged rocks and the presence of strong currents and riptides in the vicinity, making it a dangerous location for people and shipping alike.

The selection criteria behind this initial site was largely pragmatic, as dictated by flight safety requirements. The UK Civil Aviation Authority has published a ‘drone code’, which establishes legal minimum distances and maximum altitudes by which a drone can operate in the vicinity of people and infrastructure. Following these guidelines mean that only sparsely occupied areas can be flown over legally by unlicensed drone operators, and so ruling out many potential beach sites by default. The project itself thus required a suitably isolated location, which the challenging access conditions of ‘The Strangles’ provided. Nevertheless, this site also had additional attributes that resonated with certain themes of the Waveform project, and so further justifying its initial selection.

Just like many coastal sites facing into large oceanic currents, shoals of human generated detritus wash ashore daily at this location, and can build up substantially. The remoteness of ‘The Strangles’ ensures that large piles of debris are a frequent occurrence on the shoreline, offering evidence enough of the inescapable impact of human activity. Moreover, its high plastic content is a vivid indicator of the petrochemical industries that drive an energy-intensive global economy, and the excessive consumption, wastage, and pollution this generates.

A more intriguing aspect of ‘The Strangles’ is that it is located very near the landing sites where key submarine data cables arrive from across the Atlantic. These cables are an essential component of internet infrastructure, and their configuration traces the histories of colonisation, conflict, and multinational capitalism that underpinned its emergence in a Cold War context (see Starosielski 2015). These same histories are implicated in present ecological challenges, sitting alongside the fact the internet is now integral to the global sensory architectures which monitor them. While this site cannot claim any especial significance here, it is one of the closest such beaches in the region that can be flown over by a drone. Consequently, as an initial setting for an act of speculative sense-making, emerging out of the concerns outlined above, ‘The Strangles’ afforded an apposite space in which to prototype the concept.

Nevertheless, it should be stressed that the future intention of this project is to document multiple sites of interest, with the aim of engaging not just the relatively high-level concerns identified here, but to interrogate sites whose social geographies provide more localised accounts of current ecological stresses.

2.2. Photography

The capturing and selection of the source imagery for Waveform is dictated significantly by the levels of contrast present in the scene—between the white of the incoming wave crests and the dark sands of the shoreline. This is crucial, because it is in detecting this contrast that the machine vision software attempts to delineate the threshold between land and ocean.

To explain, the image analysis software (developed using the open source Processing toolkit) divides a grayscale source image into a grid of 64 sampling fields. These fields are then mapped according to their overall brightness values. The modal brightness value of the sampling field is then calculated, and matched according to a predefined index pallet (16 colour, grayscale), so as to facilitate comparison with the other fields.

The comparison process involves assessing the contrast levels between a given sampling field and its surrounding neighbours. Those passing a certain threshold along a linear path are then designated as marking the edge of the incoming wave. The coordinate values generated are then stored in a two-dimensional array which is used to draw a wave outline upon the original source image (see Figure 2).

This outlining process is uncertain by design. In cases where the source image exhibits a high level of contrast between wave and shore, it is often tightly conformal. However, in cases where the shoreline is interrupted by the presence of rockfalls and patches of sea foam, the excess contrasts can make the wave outline oscillate considerably. A simple smoothing routine attempts to eliminate the worst of these outliers.

The coordinate values generated in the process of drawing the wave outline are then passed on to another program (also developed with Processing) that generates the final textual outputs. The relationship between these values and the texts resulting is that they constitute a supply of random variables for the generative program, providing the dynamo for its linguistic constructions.

2.3. Text Generation

The generative program itself is built around the principle of a Markov-chain. A source text is scanned to detect particular frequencies of word groupings within it, n-grams, before constructing a table recording the relative probability by which they occur in relation to one another. Selecting an arbitrary n-gram will suggest those most likely to follow, and so determining the other probable options for the next link in the linguistic chain. Making random selections from across these interlinked possibilities, weighted in accordance with their probability distribution, can result in outputs that echo the coherency of the source, but yield novel semantic juxtapositions.

It is this schema that enables the arbitrary values of wave coordinates to be used for generating coherent textual outputs. Once the initial n-gram to build the linguistic chain is chosen arbitrarily by the system, the rest are determined by the coordinate values. Nevertheless, depending on the initial choice, myriad different strings can be assembled with each execution of the program, and it is left to the artist’s discretion which are finally associated with the source image. The criteria in these instances are whether the output is grammatically coherent, and whether it resonates at some level with the themes of the project (e.g., maritime, scientific, or environmental). The generated string is divided automatically into short lines of between three to six syllables each, and then overlaid finally onto the source image—along with the latter’s geographic coordinates of latitude and longitude (see Figure 3).

Given the potential for many different output strings, and the consequent demands of curating only one for final display, care is necessary when selecting the source texts supplying the vocabularies used by the generative program. The choices made thus far encompass a mixture of fictive and factual works with themes relevant to those of Waveform, and thus likely to yield appropriate output. These texts are: R. M. Ballantyne’s The Ocean and its Wonders (1874), which is a lyrical maritime exposition; Paul Edwards A Vast Machine (2003), reflecting on the evolution and politics of climate modelling; Alfred North Whitehead’s The Concept of Nature (1920), and Annie Dillard’s Teaching A Stone to Talk (2003), a poetic meditation on how the natural environment is variously captured in the fields of science and literature.

When generating a text to accompany each image, these literary sources are sequenced in-turn, exploring their varied possibilities of expression. Out of the above listed, those which have yielded the most coherent outputs so far, and from which the examples in Figure 3 are derived, are Ballentyne’s Ocean, and Whitehead’s Concept. With its frequent references to varied maritime phenomena, Ballentyne’s writing style ensures that any text generated will often resonate with its accompanying coastal imagery. Moreover, being written at a time of growing interest in investigating the polar regions, Ocean mixes descriptions of danger and uncertainty in the face of the unknown with its framing using the tools and vocabulary of the scientific method. Whitehead’s Concept is an extended meditation on the latter, interrogating the relationship between perception and knowledge, and thus yielding a vocabulary centred on acts of measurement and classification. However, whereas Ballantyne’s text is characterised by its straightforward acceptance of scientific knowledge, Whitehead critiques its very foundations. Notably, Whitehead rejects the philosophical ‘bifurcation of nature’ into (subjective) sensed experience and an (objective) cause of these experiences. In its place, Whitehead posits sensing and knowing as an event in time—as concurrent, embedded, and contingent, rather than a reflection of the absolute.

Although Ocean and Concept represent very different texts for different audiences, they nevertheless bracket a period in history when the Enlightenment vision of an objectively knowable, progressive, rational universe was being complicated by new scientific discoveries and, later, the cultural disillusionment of the First World War. In this regard, the texts in Figure 3 echo the sentiments of their differing sources, with the presumed superficiality of sensed experience (3a) giving way to a recognition of the arbitrariness in separating the fundamental and perceived attributes of the world (3b).

As will be discussed in the next section, contemporary discourses surrounding climate and ecology are characterised by a similar recognition of the entanglements of scientific knowledge-making in the very phenomena under study. Such critical reflexivity takes on especial importance in this context, for the latest predictions derive from mapping events that far exceed the spatial and durational thresholds of the human senses. Whitehead’s Concept represents an antecedent to the discussions of the present moment, and both are responding to the historical mindset seen in Ballantyne’s Ocean. While these aspects will not be immediately present to the viewer of Waveform, they do, nevertheless, contribute to its critical background, and provide an apposite point of reference as it interrogates the future of sensing and sense-making in the contemporary environment.

3. Sensing the Anthropocene

The Waveform project first emerged in response to the varied implications of the ‘Anthropocene’ hypothesis developed by Crutzen and Stoermer (2000), in which the Earth is characterised as entering a new geological epoch. Here, the impact of human activity on the material ecology of the planet is seen as being irreversible and fundamental, in the form of changing climactic patterns, rapid losses in biodiversity, and the residues of nuclear technology.

Although the Anthropocene has yet to be formally ratified by international bodies (as of the time of writing) it has become a term deployed with increasing frequency in the critical and creative arts, calling for a reimagining and, indeed, a remaking of human relations with the material ecology of the planet (see, e.g., Davis and Turpin 2015). In these contexts, however, the term ‘Anthropocene’ is far from uncontested. Moore (2016, p. 6) is one longstanding critic, and, amongst other objections, designates the Anthropocene as a label for symptoms in isolation of their causes, suggesting in its place ‘Capitaloecene’: ‘as a way of organising nature—as a multispecies, situated, capitalist world-ecology’. Haraway (2016) notes similarly that the conditions of the Anthropocene have emerged within the specific political, social, and technical contexts of a globalised petro-capitalism. Nevertheless, Haraway goes further in her critique, objecting to both the Anthropocene’s and Capitalocene’s depiction of a unitary human condition, exceptional in its agency above all other phenomena, and the consequent apocalypticism of an epoch premised on humanity rendering its environment unliveable. Haraway’s alternative suggestion here is ‘Cthulucene’, a deliberately monstrous label, highlighting the seething interrelations that have always existed across species and processes, as well as the myriad stories and ways of seeing that emerge from these exchanges.

Although it is beyond the scope of this paper to contend for which of these labels are most useful, they reveal a desire to unearth a multiplicity of more nuanced outlooks, as compared to the sweeping narratives implied by an epochal marker. Even within the technoscientific framings that Haraway associates with Anthropocene discourse, there is a growing enquiry concerning the primacy of electronic sensors in defining its key parameters, with the consequence of bracketing anything that exceeds their thresholds of detection. In this regard, Bolen et al. (2016) note the formative role these systems have played in characterising the Anthropocene:

[I]deas of the Anthropocene have been shaped by a technospheric net of innumerable satellites, cameras, and detectors, resulting in an aesthetic regime composed of data that has been used to narrate profound changes to climate, landscape, and biodiversity over the past 400 years.

The critical questions arising here go beyond surveying the basic functioning and deployment of environmental sensors, and include the epistemic assumptions they encode, as well as the representations they produce subsequently. It is here Bolen et al. (2016) enquire:

If quantification, abstraction, and the logic of evidential traces have been the means by which we’ve largely come to recognize our purported Anthropocene condition, then the question becomes how we might proceed so that our “sensing” is less “remote,” and forge aesthetics that incorporate not only the representational, but also the lived and affective experiences of various anthropo-scenes.

The chief contention here is that sensory systems, as they have been used to map epochal changes in climate and ecology, have generated representations of the world primarily in terms of the purely quantifiable—as statistical trend lines on graphs, rather than the lived experiences through which they concretely manifest. Although the utility of such actions for the modern scientific method are not disputable, the implication is that to craft a mode of ‘sensing’ that draws attention to these wider experiences can facilitate a more responsive ecological praxis.

To illustrate the kinds of immanent perspectives that can be missed by this data-driven episteme, Schuppli (2014, p. 60) recounts how indigenous peoples living in the Canadian Arctic have begun observing the sun setting further into the West, and the alignments of stars appearing to change:

Sunlight is behaving differently in this part of the world as the warming Arctic air causes temperature inversions and throws the setting sun off kilter. Light is bending and deceiving eyes that have tracked the position of the sun for generations, using it as an index of place and a marker for direction. The crystalline structures of ice and snow are twisting and morphing, producing a new optical regime borne out of climate change and indigenous observations.

These stories were captured in a series of interviews with Inuit elders in the documentary film Inuit Knowledge and Climate Change (2010), by Zacharias Kunuk and Ian Mauro. However, Schuppli notes that it was met with some hostility by parts of the scientific community, who rejected the credibility of these indigenous accounts on the basis that they had misidentified their cause—suggesting the Earth had tilted on its axis (2014, p. 63). While such views might indeed be scientifically inaccurate, the experiences leading up to them remain cogent for understanding (and, crucially, empathising) with the far-reaching impacts of human collective agency on geophysical processes.

Cosgrove (2001) documents exhaustively the varied drives behind perspectives that emphasise the global and the absolute, over and above the kinds of uncertain, varying accounts associated with more local experiences and relations. Cosgrove (2001, p. 243) notes especially how, in a post-war environment:

The sense of closed global space that had disturbed the strategic thinking of so many Westerners at the beginning of the twentieth century found expression in a global geopolitics of competitive imperial struggle as the frontiers of Western empires converged at the ends of the earth. Mapping the globes climates and physiography into “natural” regions could naturalize patterns of control and strategy among the imperial powers. Geopolitics suggested that unalterable geographic “realities”—the distribution of lands and seas, of landforms, natural resources, or “races”—had to be exploited if a state was to survive, compete, and prosper.

Cosgrove observes how these attitudes were facilitated by infrastructures of sensing and data collation that present a totalising, top-down view of the world—such as provided by orbiting reconnaissance satellites, and the military drive towards establishing an omniscient ‘God’s Eye View’ over the battlespace environment. The resulting energy and resource intensive economics of geopolitical competition are seen in the key ecological and climactic markers of the Anthropocene (or, indeed, the Capitalocene), which thus becomes in-part a consequence of measuring and representing the world purely in terms of political and capitalist exchange.

It is in response to such developments that Müller-Hansen (2016) asks how we might engage in the ‘process of making possible a new sensorium, one that is better adapted to the world as it is and behaves?’ He notes subsequently that ‘Sensing has its own politics’ and that ‘how we construct, train, apply, and critically engage with this new sensorium’ will be vital when intervening in the structures of power that are most implicated in key ecological stressors. For Müller-Hansen, the Anthropocene thesis represents ‘a call to re-forge our sensory‒aesthetic practices, so that we sharpen our powers of judgment with respect to the epochal transformations currently underway […] a call to re-forge our sensibility toward the Earth’.

4. Airborne Sensing

Evocative though such rhetoric of a ‘new sensorium’ may be, questions arise immediately as to what this might resemble in practice. The basic suggestion is that it will document life within a material ecology subject to accelerating change and disruption—and perhaps be reflexively aware of its own role in these transformations—but how this might be achieved is left open to speculation.

One possibility implicit within the very conception of a ‘new sensorium’ is the creative redeployment of existing sensory systems and platforms. That is, to reorient their established utility, the representations they generate, and the types of knowledge they crystallise.

4.1. Aerial Activism

An example of this can be found in the Aerocene Foundation of artist Tomás Saraceno (working in collaboration with many others). ‘Aerocene’ is a label for a speculative new epoch, a conscious response to the Anthropocene that replaces its bleak prognosis with a more hopeful array of imaginaries and practices. Concurrently artistic, activist, and scientific, this project seeks to overcome ‘extractive’ attitudes towards the circulation of energy and resources, facilitating instead a more collaborative, ethical relationship towards atmosphere and environment. This is achieved through freely distributing open source ‘Aerocene Explorer’ kits, enabling participants to construct floating sculptures whose buoyancy is achieved through entirely passive means. As described, these kits are ‘[d]esigned to engage participants in thinking-through-making’, educating them in varied fields of physics and meteorological science, as well as inspiring their creative agency in declaring ‘independence’ from fossil fuels (Aerocene 2018).

Each Explorer carries multiple onboard sensors for documenting the ensuing voyage, including detectors for altitude, temperature, pressure, humidity, as well as motion trackers and cameras. It is these latter devices especially that facilitate a more creative, speculative engagement with atmospheric sensing and exploration. The motion trackers, for instance, are described as enabling digital visualisations of an Explorer’s airborne trajectory—passive ‘signatures’ that contrast with the gridded striations of modern air travel, and the energy paradigms it embodies (Aerocene 2018). Likewise, the shaky, wind-blown output of the onboard cameras become records of the singular character of each voyage, and thus evocative of a new environmental imaginary, one that is more idiosyncratic, nuanced, and contingent upon the interplay of both human and nonhuman actors.

While there is a diverse body of critical literature surrounding the Aerocene Foundation, in the context of this discussion it can be framed as a response to the consumptive impulses and extractive infrastructures associated with the Capitalocene paradigm especially. Just as sensory architectures are integral to latter, they are equally so with Aerocene, but here become instruments of diverse communities of practice, yielding a distributed matrix of perspectives. This stands in opposition to the centralised infrastructures and ‘top-down’ mappings of military and commercial systems, as documented by Cosgrove.

Another sensory project conducted on a similar premise to Aerocene was PigeonBlog (2006) by Beatriz da Costa. PigeonBlog was billed as a grassroots scientific initiative, collecting localised air pollution data across the southern Californian region. The project involved fitting homing pigeons with GPS trackers and miniaturised pollution sensors, before having them fly regional circuits. The data collected was then visualised in real-time, being plotted over Google Maps for the public to observe (see Da Costa 2017).

There were multiple goals sought by the PigeonBlog project, but of particular interest to this discussion was Da Costa’s (2017) performative approach towards airborne sensing:

With homing pigeons serving as the “reporters” of current air pollution levels, Pigeonblog attempted to create a spectacle provocative enough to spark people’s imagination and interests in the types of action that could be taken in order to reverse this situation. Activists’ pursuits can often have a normalizing effect rather than one that inspires social change. Circulating information on “how bad things are” can easily be lost in our daily information overload. It seems that artists are in the perfect position to invent new ways in which information is conveyed and participation inspired. The pigeons became my communicative objects in this project and “collaborators” in the co-production of knowledge.

PigeonBlog thus staged an act of speculative sensing, one that entered into a collaboration with nonhuman partners so as to inspire new ways of seeing and thinking amongst its human observers. On this point, Haraway (2016, pp. 23–24) reads PigeonBlog as a striking instance of the exploratory kinships that are of particular import for life in the Cthulucene: ‘Perhaps it is precisely in the realm of play, outside the dictates of teleology, settled categories, and function, that serious worldliness and recuperation become possible’.

4.2. Drone Art

Projects such as the Aerocene Foundation and PigeonBlog appear a world away from the sensory architectures built by militarism and capitalism. Such contrasts designate the latter as an especially poor basis on which to realise a ‘new sensorium’. Nevertheless, it was within the conceptual spaces opened up by these art projects that Waveform has sought to redeploy the offspring of more established architectures. A recurrent figure of reflection here has been the ‘drone’: a totemic artefact that links a number of discourses surrounding machine sensing, and so offers a suggestive, if not unproblematic vehicle for creative intervention.

A cursory glance through current reports on airborne drones will yield myriad vignettes concerning their efficacy as sensory platforms—whether for filming and photography, for scientific research, or, most prominently, for surveillance, reconnaissance, and intelligence gathering. In a Western military context, the exponential growth of drone forces has been key to orienting doctrinal violence around the persistent, ‘unblinking’ surveillance of designated targets of interest, and so becoming a vital and seductive contributor to the aforementioned ‘God’s Eye View’ over the contingent spaces of modern conflict. Integral to this Apollonian perspective is a presumption concerning (and a desire to pre-empt) the threatening vectors of that which is surveilled. This is demonstrated vividly through the use of drones in attacking distant figures that are constituted as ‘immanent threats’ in the eyes of this sensory and political assemblage (see, e.g., All-Party Parliamentary Group on Drones 2018).

Given these activities, it is unsurprising that much academic and creative work on drones interrogates their ethical and political consequences in military and policing contexts. Exemplifying such enquiry is the work of the Forensic Architecture group, which has conducted highly detailed investigations into the circumstances and consequences of specific drone strikes, using this as a means of critiquing their ultimate legality.

Turning to drone-based art specifically—in which the drone, its technologies, imagery, and impacts, feature as an irreducible aspect—this martial spectre looms large in the context of many works, which are strongly activist in focus. The photography and video art of Trevor Paglen is just such an instance here, seeking to make visible the often highly clandestine world of military drone operations, and meditating especially on the drone’s sensory aspects in these contexts.

In Drone Vision (2010), a sequence of intercepted drone video feeds is shown in the raw, with no contextualising information or commentary. The film is grainy and uncertain, with the drone’s camera turret roving across the sky around and the landscape below according to no discernible pattern or motive.

Most evident, on first viewing, is the striking presentation of a transient, uncertain drone’s-eye-view, as contrasted against the smooth, unwavering focus of the popular imaginary and press release footage, which emphasises a lethal ‘locked on’ gaze. Amidst the arbitrary flow of imagery, Vandenburg (2016) reads into this piece a repositioning of the audience’s own gaze, splicing into the unceasing streams of data that characterise the Western military episteme:

We find ourselves on the side of the drone and its pilot, desperate to understand this torrent of collected images, the better to control and dominate—and, we realize, destroy—what and who lies below. The unmediated flow of visual and spatial data that passes through drone eyes collapses the distance between device and operator, between American air base and Middle Eastern valley, into a single moment of seeing.

Nevertheless, an important dichotomy can be noted here, in that the very act of sensing and ‘seeing’ does not translate automatically into knowing. For all the myriad images presented, no clear picture emerges as to the activities depicted, either regarding the drone itself or the unwitting targets of its gaze.

A similar dynamic plays out in a related photographic series by Paglen. In untitled (Drones) (2010), the viewer is confronted by a selection of immense skyscapes, their subtle palettes reminiscent of the paintings of Agnes Martin or Joseph Turner. Only after careful scrutiny will the viewer be able to pick out a tiny fleck of shadow amidst the shifting colours. This fleck betrays the presence of a single military drone, photographed from many miles distance.

In searching for the drone, the viewer is confronted with how the entire sky can function as part of a new infrastructure of observation. The imbalance of power facing the objects of its attention is captured in the sheer challenge of observing the drone itself at work. Just as Drone Vision offers no clear understanding as to its actual contents or contexts, the untitled (Drone) series gives no sense as to what the photographed drones are either looking at, or indeed, for.

Even artworks that seek to detail the specifics of military drone activities are haunted by this asymmetry, as in James Bridle’s Dronestagram (2012–2015). As the name suggests, this project appropriates the social media platform Instagram in order to document (and help make visible) the distant sites of known drone strikes, posting satellite imagery and short reports detailing each attack. Nevertheless, as Vandenburg (2016) points out: ‘Instagram’s very name denotes the real-time transmission of events as they happen, but nothing is ever happening in Bridle’s images: they document what occurred earlier, elsewhere, far below the satellite through which we see’. In this sense, the imagery of Dronestagram exhibits an unsettling emptiness. The sprawling complexes of highly remote settlements become homogenised signifiers of distant violence, inaccessible to outsiders, and barely registered in the gaze of orbital infrastructures.

Uniting the work of Paglen and Bridle, then, is a depiction of the limits of sensory technologies to deliver information about the world. Inferences can be gathered, but definitive conclusions are curiously absent, and so undercutting narratives of an omniscient ‘view from above’.

As a final illustration of this dynamic in drone art, and to draw this discussion back to its role in the Waveform project, it is worth turning to a pioneering artwork made in the late 1990s by the Bureau of Inverse Technology (the artists Natalie Jeremijenko and Kate Rich). BIT Plane (1997) took the form of a radio-controlled model aircraft fitted with a transmitting video sensor, and which was then flown over the no-camera zones protecting the commercial campuses of key Silicon Valley entities, including various defence contractors. The resulting footage was gathered into a video installation whose sparse, officious captions mimicked an advertisement for military hardware.

The core premise of BIT Plane was not, in-fact, to straightforwardly surveil the corporate nerve centres of Silicon Valley, defying their efforts at safeguarding their intellectual property, but to demonstrate the very absurdity of this idea. Unquestionably, the video footage gathered from BIT Plane was so grainy and low-resolution as to be useless for meaningful intelligence gathering, but in a later interview, Jeremijenko (2013) made the point that a more sophisticated conception of ‘information’ precludes such naïve undertakings in the first instance:

Information is a property of people and communities and discussions, and actual work, it’s not something you can just take a picture of and steal. But that was the paradigm, and it actually remains the prevailing paradigm, that information is property, that it can be stolen with cameras. So flying the BIT Plane through these no-camera zones was part of the exploration of what you could actually just see. What could you actually see? What information could you take from the plane? Of course, the answer is not much (as contemporary drones have so aptly demonstrated)—lots of images but not much actual trustable information.

Jeremijenko’s observation here is that information is not an abstract, purely cognitive phenomenon, but is expressed through, and sustained by, multiple intersecting actors and environments. A ‘snapshot’ video from a drone traces more the act of sensing itself, being unable to capture the enactments and exchanges that give rise to the scene being recorded—or, indeed, of any impacts caused by the drone’s presence, such as the complaints of television signal interference generated by its overflights.

It is in this respect that Jeremijenko and Rich pioneered the kind of drone art practiced by Paglen and his contemporaries, revealing the ‘view from above’ to be not as all-encompassing as popular mythology may suggest, and demonstrating how techno-aesthetic tropes of ever-increasing sensory resolution do not translate into greater knowledge of the scene under observation.

The critical dialogue of artists towards drone technology, and the conceptions of a ‘new sensorium’ outlined by Anthropocene scholars, begin to converge on this point. Both share concerns regarding any implied distance and objectivity associated with sensory systems, and emphasise how they construct particular views that make sense only within particular contexts. In this regard, they both critique the limitations of viewing the world in terms of the ocular and the abstract alone, bracketing off the lived relations that enact its daily being.

It is here that Waveform developed into a vision of deploying an airborne drone as the basis for an alternative sensory platform. At first, it would invoke multiple fields of reference with regards to the origins and impacts of its constituent technologies, and of the ‘view from above’ more broadly. In this sense, it would act as both a node and an emblem of the myriad networked systems that articulate views of a global environment. Going further, it would then serve as provocative vehicle for their subsequent interrogation, highlighting the limitations and contingencies that are inherent to their operation.

5. Vision Machines

5.1. Signal Processing

Exactly how to reframe the drone as a sensory platform represented the definitive challenge for the Waveform project. A recurrent area of consideration was how electronic sensors are designed to minimise extraneous noise, while also separating out particular types of signal return. This filtering of signals across various thresholds of detection are foundational to the depictions of the world they generate subsequently. This suggested a point on which the varied concerns outlined above might turn, and it was here that one of the more prominent early instances of electronic sensing, radar, provided an important case study in defining the eventual shape of Waveform.

In Virilio’s (1994) terms, radar, as with drone technology, is a species of ‘vision machine’—an architecture of detection and interpretation. Here, the aim is to generate an instantaneous, noise-free depiction of the battlespace environment, and so clearly predict and engage that which crosses the thresholds from object to threat to target. Nevertheless, it is here that Virilio observes an unsettling vector, for human interpretative agency and authority becomes subject to ‘a splitting of viewpoint, the sharing of perception of the environment between the animate (the living subject) and the inanimate (the object, the seeing machine)’ (emphasis original). In other words, the emergence of seeing machines, which perceive the world in ways that the human senses cannot, challenges the idea of a coherent human subject, surveying an objectively knowable world, as the preeminent basis for measuring and characterising observable reality.

The story of radar marks the moment when this ‘splitting of viewpoint’ became fully apparent. During the Second World War, the evolution of radar was driven by the need to more accurately determine the status of airborne contacts. This spurred the development of algebraic signal processing, which separated out moving targets from unwanted interference, such as atmospheric noise, weather formations, or terrain echoes. As Link (2016, p. 74) observes, this had a transformative impact on conceptions of what technological modes of sensing would represent, both during and after the conflict:

The rays received from the external world allowed for the precise algebraic processing of successive waveforms, which visualised objects that had previously been imperceptible to the naked eye. […] Photography and television were touted as technologies that faithfully recorded reality. Radar, however, broke the apparent unity of reality and its representation apart, because it programmatically manipulated the image. The pictures were not a faithful record of the rays received; they merely represented the initial data for filtering, that is, the algebraic calculation of the image. Slowly but surely, algorithms were beginning to determine what was considered as real.

Here lies the origin of the abstract data mappings that would later indicate a growing ecological crisis. Radar itself played a part here, in being a key enabler of the military actors engaged in a global geopolitical competition, and the cycles of excess consumption and pollution this generated.

Link goes on to observe that the arbitrary manipulation of electrical signals gave rise to the conception of the ‘waveform’ in scientific and engineering theory. Moreover, these processes of filtering were what allowed the pristine, discrete units of digital signals to be expressed within the confines of electronic circuitry, and so enabling the practical realisation of digital computing more broadly.

It is at this point in his account of these historical developments that Link provides an unusual vignette, describing how one of the earliest, fully electronic, programmable digital computers, the UK Ferranti Mark I, was used by mathematician and engineer Christopher Strachey for an experimental text generation program in 1952, producing a series of simple, comic love letters. Here is a typical output from this program:

Darling Sweetheart

You are my avid fellow feeling. My affection curiously clings to your passionate wish. My liking yearns for your heart. You are my wistful sympathy: my tender liking.

Yours beautifully,

M.U.C.

(Strachey 1954, p. 26)

Despite its crude operation and evident whimsy, one of the key innovations of Strachey’s generator was its use of a random number algorithm to select arbitrary words from a source vocabulary—a characteristic of many subsequent works of digital art, in their mixing of predefined primitives. Such algorithms have been subject to intensive enquiry in the computer sciences, for they are, ultimately, pseudorandom in nature: algebraic emulations that can exhibit significant levels of recursion. The need for such algorithms represents a by-product of the signal processing that is foundational to all digital computing, in that it forecloses the presence of arbitrary noise in the system. To reincorporate the latter for the purposes of calculation necessitates using a sensor that detects random signals in the ambient environment, such as atmospheric noise, and then deploying this as a source of variables.

The importance of randomness for Strachey’s generator is significant not simply for the textual effects it yielded, but for the very contexts in which they were achieved. In so harnessing a digital computer, then still a highly militarised technology, to generate random variables for the assembly of non-quantifiable outputs, Strachey’s generator involved the deliberate fostering of the emergent and the ambiguous back into a technical and operational context that otherwise strove for its elimination.

While Strachey (1954) developed his generator as an exercise in creative programming, his account of its functioning is part a greater reflection on the potential for computing machinery to yield just these kinds of unpredictable effects—to appear to act spontaneously, and so draw attention to its status as a distinctive agent. In this sense, his generator can be read as a creative inversion of the deterministic, martial impulses encoded within the interlinked genealogies of radar, signal processing, and digital computing. Such gestures serve to make these impulses all the more visible on reflection, and so inspire enquiry concerning their effects. Strachey’s generator can thusly be regarded as a pioneering instance of digital art, revealing the expressive and self-reflexive potentialities within devices that were then closely associated, both functionally and operationally, with tasks of the gravest utility for the state.

5.2. Experimental Aesthetics

Strachey’s generator was not programmed explicitly in the name of art, or to critique the troubling legacies of the Second World War, but it resonates with a longstanding tradition of using experimental aesthetics as a mode of creative resistance to modern conflicts and anxieties. Of particular note here was the Dadaist movement that emerged midway through the First World War. Dadaist practice was fuelled by its opposition to the conflict, most especially in terms of its propagandistic appeals to the defence of higher ideals. For its cofounder, Tristan Tzara, such invocations of ‘‘truth’ and ‘intelligence’ were part of the rhetoric that had brought about the war and continued to fuel it … In the process, they had lost every shred of meaning’ (Buelens 2015, n.p). For this end, much of the art and poetry of Dada was driven by the ‘deliberate annihilation of ordinary language use and prevailing aesthetic standards’ (Buelens 2015, n.p).

In pursuing disruptive new modalities of expression, such as assemblage, textual collage, and photomontage, Dadaist critique was enabled using the very techniques it placed into question, drawing on the fragmentary media landscape of newspapers, films, and advertisements. Tzara’s own cut-up poems, taken from newspapers, are striking in their use of contingency to evoke the visceral chaos of machine-generated violence, and its rending apart of once coherent, believable narratives of Enlightened progress. A similar theme is captured in Raoul Hausmann’s post-war assemblage The Spirit of Our Time—Mechanical Head (1919). Here, a crudely carved wooden head is studded with various measuring instruments, such as a watch, a tape measure, and a ruler. This assemblage suggests the naivety of the Enlightenment vision of the autonomous, rational subject, revealing instead its embedding within the attitudes and epistemes of its social environment, and its consequent vulnerability.

It is here, then, that Strachey’s generator can be read in light of these earlier efforts at appropriating the tools and materials of a damaged world—sometimes themselves implicated in the disaster—as a catalyst for reflection. The work of the artists already mentioned, Saraceno, Da Costa, Paglen, Bridle, Jeremijenko and Rich, operate in a similar vein, with Saraceno and Da Costa drawing attention not just to the ecological consequences of technologies rooted in warfare and competition, but of the possibilities they might yet afford. Although such creative gestures may be limited in their ultimate impacts—Dada, after all, did not preclude the myriad conflicts that followed the Armistice of 1918—they serve the task of keeping alive a resistant, alternative imaginary to the present. Waveform itself, as a deliberately strange, provocative assemblage, is seeking to provide a similar legacy.

6. Waves to Waveforms

It is at this point that the varied threads of this discussion can be drawn together. Stated in concise terms, Waveform’s use of algorithmically processed images to generate poems represents an updating of Strachey’s original creative gesture. Specifically, it meditates on the global sensory infrastructures that radar, algebraic signal processing, and digital computers gave rise to—and of which drone technology represents the latest iteration. Following Cosgrove, these infrastructures have emerged from, and are implicated within, the geopolitical struggles of recent decades, whose planetary impacts they have only detected subsequently, and thus embedding and evidencing this sense of a new ecological epoch.

By way of deconstructing the impulses driving this cycle, Waveform takes the form of a sensory assemblage attuned to noisy patterns in the ambient environment, using this as a source of arbitrary values for the presentation not of any clear, quantifiable data, but of poetic text. The intention here is to interrogate and unsettle the normative representations made of the world using digital sensors, to inspire enquiry as to the assumptions underpinning these practices, and to stage the kinds of reflexive alternatives to existing sensory paradigms being speculated on by various scholars.

As detailed in the opening section of this discussion, the sensory aspects of Waveform are supplied by an airborne camera drone, engaging directly with the top-down perspectives that have played a crucial role in constructing an image of a closed global space, ready-to-hand for human demands. In its close association with the military ‘policing’ of this space, in the name of establishing a secure environment for market expansion and trade, the drone serves as an especially charged instance of a sensory agent—a device that is implicated profoundly in the very changes it observes. The coastal environment it orbits, although seemingly benign from even a relatively low altitude, is being increasingly damaged by the detritus of human activity on a global scale, with rising sea levels being an implicit, and ever-growing threat.

Cumulatively, the result is a project that creatively appropriates varied technologies and practices whose genealogy is closely tied to the production of scopic regimes that deliver views of a ready-to-hand world. In so doing, the aim is to show that far from being systems of absolute precision and imposition, they operate instead as complex assemblages within an interpretive ecology, subjecting incoming signals to varied processes, and reading them across a variety of perceptual thresholds, both human and machinic. The very act of drawing a precise line where the edge of a wave might reside is ultimately nonsensical as an act of measurement, but it reveals consequently the system’s functioning as it grapples with so uncertain a target. In this respect, the presence of breakdowns arising within this assemblage, where the components fail to gel, is a welcome development. Perfect wave outlines and pristine poetry are not in-fact the primary goal, for any failings expose the functional contingency of the assemblage as a whole.

This point made, however, the poems generated through this process are still curated in order to engage themes appropriate to a critical interrogation of digital sensors—although the meaning of each poem, and any relationship to the source imagery, is left open to the viewer’s discretion. Nevertheless, one goal here is to draw attention to how the meanings assigned to all forms of digital sensory data, however it is rendered, arises through an intersection between human and machine, with both coming together as sensing and interpreting agents. In using devices that perceive the world in ways very different from a human observer, such data only becomes fully meaningful and actionable when understood in relation to the structures and processes through which it was derived. In cases where this is obscure or ambiguous, as in Waveform, the observer is tasked with attempting to establish these links for themselves. The interpretative aporia caused by the parsing of the visual into the poetic is thus an effort at challenging any sense of transparency to data representations—that they are a direct manifestation of the phenomena under study, rather than being a mediated and continually mediating vision. This in-turn shows how these aspects are still at work in more established cases, where the technologies and practices involved have been rendered normative, unremarkable, and uninterrogated, such as satellite photography or automatic image analysis.

The eventual outcome of the Waveform project will be an extended series of hybrid images. At the time of writing, the first set of these images are being compiled into an artists’ book. This format has been chosen so that the turning of the pages will illustrate the ‘transition’ from image to text in each case, as well as offering space for notes concerning the vocabularies and algorithms behind their curation.

As stated in the introduction, Waveform is an ongoing project that is still at an experimental stage of its development. Nevertheless, it brings together diverse strands of artistic and theoretical endeavour, so as to assess diffractively their interrelations and potentialities. The goal is not to produce a definitive assessment of any one aspect involved, but to generate through practice novel vectors of enquiry, and thus bringing to light new ways of thinking through the ecological challenges of the present.