The Emergent Rhythms of a Robot Vacuum Cleaner—An Empirically Grounded Account of Agential Realism

Abstract

This article builds on the argument that design for complex interactive systems should shift from creating linear transactional interactions toward organizing relational complexity. Grounded in Karen Barad’s agential realism, we argue that a designer’s role can benefit from not predefining interactions but from curating the material-discursive conditions under which meaningful relations can emerge. To explore the empirical and temporal dimensions of this practice, we conducted an exploratory workshop setting the conditions for emergent gameplay dynamics and discussions on agential realist anticipation. Participants utilized a custom-designed game and built their own physical controllers to anticipate and adapt to shifting gameplay conditions. Our results demonstrate how alterations in relational constraints, rather than explicit pre-programmed goals, drove the emergence of non-predefined gameplay rhythms. The findings provide empirical grounding for an agential realist understanding of anticipation, showing that an interactive system’s identity lies in its unfolding processual patterns rather than a static final state. Based on these findings, we propose three design principles for further exploration: Design for Relational Emergence, Design for Re-membering, and Design for Emergent Patterns. Consequently, we conclude by outlining a conceptual approach for non-linear computational architectures, drawing on principles from Enactive AI and reservoir computing.

1. Introduction

Our work is situated at the intersection of agential realism, emergent gameplay, dynamic interactive systems, and complexity theory.

In the design of complex interactive systems, traditional approaches often rely on linear causality and predefined goals. The prevailing logic suggests that an interaction is a transaction between independent entities, where A leads to B in a predictable chain of events. The interaction is thought of as between distinct entities, where one (the cause) modifies and leaves its mark on the other (the effect).

From an agential realist perspective, causality is not between isolated entities. The temporality of causality is within phenomena, where the determination of cause and effect is made in a temporary contextual unfolding. Agential intra-actions are the causal enactments in which marks are left on the agencies of observation (the effect) that constitute and reveal specific features of an entity (the cause). Causality is thus not a linear transaction but a specific materialization of the world, resulting from “agential cuts” (contextual reconfigurations) [1]. This also means that agency is not a property possessed by an agent but an enactment that emerges from the material-discursive arrangement.

what is important about causal intra-actions is that “marks are left on bodies”: bodies differentially materialize as particular patterns of the world as a result of the specific cuts and reconfigurings that are enacted. Cause and effect emerge through intra-actions.

(p. 176, [2])

Within HCI (Human–Computer Interaction), the Cartesian divide between human subjects and material objects has been criticized before. Paul Dourish’s definition of embodied interaction is “the creation, manipulation, and sharing of meaning through engaged interaction with artifacts.” (p. 126, [3]), is well established and highlights the fact that meaning-making is not a purely cognitive activity. A fine example of trying to rethink relations between humans and things is Morse Things by Wakkary et al. [4]. Their work emphasizes the entanglement of these entities, suggesting that things “are conditioned by humans and in turn shape what it means to be human” (p. 505, [4]). Another example is Gemeinboeck’s framework for relational-performative aesthetics in human–robot interaction, in which the design focus shifts from designing an agent to exploring the dynamics through which social agencies can emerge between humans and robots [5].

As systems become increasingly entangled with human agency and material conditions, the transactional view falls short. It fails to capture the emergent dynamics where the result of an interaction is not a fixed output but a continuous process of becoming.

The main argument in our previous work is that interaction is not something that happens between independent entities but a process that constitutes the entities themselves. This perspective, taken from Karen Barad’s agential realism [2,6], implies that the designer’s task is not to define an interaction but to create the material-discursive conditions under which both expected and unexpected relations can unfold and interaction can emerge. We call this organizing relational complexity [7]. Our use of the term organization is based on Edgar Morin’s work on complexity. Morin suggests a rationality for complexity that resonates deeply with agential realism. From this perspective, organization is a “continually generative and regenerative activity at all levels based on computation, strategic planning, communication, and dialogue.” (p. 128, [8]).

1.1. Background: Toward Non-Causal Computation and Anticipation

This study is part of a broader effort to pursue a non-Cartesian and non-causal approach to computational design, complex interactive systems, and artificial intelligence. Traditional research and computing traditions are heavily reliant on linear causality. A computer is fundamentally conceptualized as a Turing machine, manipulating symbols based on predefined rules along an infinite linear time strip. Likewise, AI models and autonomous decision algorithms often utilize the logic of Markov processes, where state changes create a linear chain of events and the probability of moving to the next state depends solely on the current state.

The apparent gap between the relational, co-constitutive nature of reality described by agential realism and the linear causality of our computing machines presents a deeply intriguing field to explore. Under agential realism, causality is not a direct state change transmitted from a cause to a separate entity. Instead, causal structures are enacted through specific intra-actions. As Scholz argues in [9], agential realism challenges traditional quantitative scientific concepts and measurements, suggesting we must assume an indeterminacy within a certain range rather than uncertainty about one true pre-existing score.

In terms of anticipation, we have actively taken part in the emerging interdisciplinary field of anticipation studies [10]. As Mihai Nadin, one of the most prominent researchers in anticipation studies, points out, anticipation is frequently and mistakenly confused with prediction [11,12]. Nadin argues that non-living systems may be deterministic and predictable, whereas anticipation is a fundamental ontological entity definitive of the living. An agential realist perspective challenges this sharp Cartesian boundary. Under agential realism, the distinction between living and non-living is not an a priori given in terms of anticipation. Humans, for example, are never isolated biological entities; we are always already entangled, situated subjects, co-constituted by our material-discursive environment. Consequently, anticipation is not an internal cognitive property restricted solely to human biology but a relational force enacted by an entire socio-technical assemblage. In robotics, researchers such as Williams and Gärdenfors [13] reflect a similarly expanded view, arguing that anticipation should not rely on an underlying causal model. They argue that it should instead be viewed as a strategy, a subjective position, and a pattern of behavior that emerges in competitive contexts.

Furthermore, this tension between causal and non-causal frameworks is actively being explored in the philosophy of science. For instance, Alisa Bokulich [14] distinguishes between causal explanations that model specific linear physical processes (like wind forming desert sand ripples) and non-causal explanations such as the “defect dynamics model”. This non-causal model explains ripple formation by revealing an organizational pattern independent of its specific causal realization, successfully explaining both underwater and wind-blown ripples. The distinction directly emphasizes the paradigm shift that this paper advocates. Traditional HCI focuses on transactional user–computer exchanges, attempting to design specific predetermined causal interactions. Our proposed approach of organizing relational complexity aligns with non-causal models. The goal is not to design the specific causal steps of an interaction but to curate the material-discursive conditions where relations can self-organize.

In theoretical computer science, the work of Baumeler and Wolf [15] suggests that this can be achieved computationally. They propose a model of “non-causal circuits” that removes the assumption of a fixed linear causal order, replacing it with the requirement of mere logical consistency. In their computational model, information can loop back from the “future” to the “past”. Although largely theoretical, this demonstrates that computing need not be conceptually bound to linear causality.

It is also important to clarify our terminology here to avoid philosophical misalignment. When fields like theoretical computer science or physics refer to “non-causal” models, they are specifically rejecting linear Cartesian causality. Cartesian causality is the assumption that pre-existing entities act upon one another in a strict, chronological sequence. However, adopting these models does not mean abandoning causality altogether. In agential realism, causality remains highly potent, but it is not a linear transaction; it is a specific materialization of the world resulting from agential cuts. Cause and effect do not pre-exist their interaction; rather, they emerge dynamically through intra-actions. Therefore, when we look toward “non-causal” computing architectures, we are actually looking for systems that support intra-active causality, systems that allow cause, effect, subject, and object to be continually and temporarily co-constituted within the unfolding context.

1.2. Aim and Research Questions

By introducing agential realist anticipation into interactive design, we aim for new relational encounters. For example, instead of an AI functioning as a tool with a fixed input-output interface, it could function as a co-creative participant responding to the rhythms, pauses, or material-discursive capacities of a specific situation. Fewer pre-made agential cuts will be made, blurring the distinction between agent and tool. The value of such a system lies in the richness of unfolding relations, not in the optimization and efficiency of predefined tasks.

The key concepts that we bring from agential realism and our previous analysis to this paper include: how anticipation can be seen as a tensional force that emergently shapes our disposition [16] and re-membering as the process in which the past is not recalled but actively reconnected and reshaped in the present [11,17]. As anticipation is not a purely cognitive prediction, we argue that it should also be used in the design of dynamic interactive systems and autonomous agents. While our previous work offered a framing of anticipation in terms of agential realism in the context of game design, the empirical and temporal dimensions of designing for such emergent dynamics remain largely underexplored. To bridge this gap, this study asks the following research questions.

• How do changing material-discursive constraints (such as physical hardware and gameplay rules) reconfigure anticipation and enact emergent agency within a complex interactive system?
• How can an agential realist understanding of these emergent dynamics be translated into actionable design principles that encourage relational complexity rather than transactional interaction?
• What conceptual properties are required for a computational architecture to support and value non-linear, emergent relational rhythms rather than static goal optimization?

To address these questions, we draw on a workshop in which participants constructed custom physical controllers to take on the role of robot vacuum cleaners. The results from the workshop have provided material for a qualitative analysis of how a system’s identity and goals can dynamically shift. Particular focus is put on how a system designed for “cleaning” emergently transformed into a system of “fighting” based on relational reconfigurations. The article also distills our findings into three actionable design principles for further exploring organizing relational complexity: design for relational emergence, design for re-membering, and design for emergent patterns. Lastly, the article outlines conceptual properties for computational architectures (proposing a shift from homeostasis to homeorhesis) capable of supporting non-linear emergent dynamics.

Table 1. Core concepts, their theoretical definitions, and their empirical focus in this study.

Core Concept	Theoretical Definition	Empirical Focus
Intra-action	The premise that interaction does not happen between independent, pre-existing entities; rather, it is a process that constitutes the entities themselves.	Observing how human players, physical custom controllers, and digital game rules co-constitute one another to form an inseparable emergent assemblage during play.
Agential Cut	Contextual reconfigurations that temporarily determine specific boundaries and properties of an entity, enacting a specific causal structure.	Analyzing linguistic and behavioral shifts to see how participants dynamically draw, or blur, boundaries between themselves (subject) and the technology (object).
Anticipation	An active, future-creating, embodied, and situated force operating in an indeterminate world, distinct from a cognitive prediction of a fixed future.	Examining how anticipation operates as a continuous, situated force—a shifting disposition of an entangled human-technology subject that dynamically reconfigures in response to the evolving relational context of the game.
Re-membering	The ongoing process where the past is not passively recalled, but actively reconnected and dynamically reshaped in the present.	Tracing how static physical hardware is dynamically given new meaning, purpose, and utility as the relational context of the game evolves over time.
Emergent Rhythms	The unfolding patterns of state change that characterize a system’s ongoing becoming and form its true identity.	Identifying distinct, unprogrammed phases of gameplay that arise organically from relational constraints rather than from explicit game rules or goals.
Homeorhesis	A system’s ability to maintain a stable trajectory or rhythm of change, contrasting with homeostasis (maintaining a static internal state).	Observing the system’s capacity to seamlessly stabilize into entirely new trajectories of play when rules change, rather than forcing a return to the originally intended task.

summarizes the core concepts in the study, their theoretical definition, and how they are empirically framed in the context of the workshop and analysis.

2. Materials and Methods

As an empirical basis for our arguments, we will report on and present the results of a workshop that was arranged at a conference (GEM 2025 (IEEE Games, entertainment, and Media Conference 2025—https://ieeexplore.ieee.org/xpl/conhome/11155221/proceeding (accessed on 19 January 2026))) in conjunction with one of our previous papers [16]. The workshop was arranged to explore anticipation from an agential realist perspective. In that paper, we made a crucial distinction between prediction and anticipation. Unlike prediction, which relies on a causal model to predict an uncertain outcome, anticipation is described as a future-creating, embodied, and situated force operating in a world that is indeterminate, not uncertain. Anticipation is not a passive expectation but an active affective force of the present that opens up some futures and excludes others. This resonates with Nadin’s vast work on anticipation [12] and approaches in cognitive science such as Varela et al.’s Embodied Mind [18].

Detailed descriptions of both the workshop plan and the actual execution are included to provide a comprehensive view of the socio-technical environment. We contend that the results can only be fully understood through deep engagement with the context from which they emerged.

2.1. Participants

We invited conference attendees to the workshop beforehand. The invitation was sent to conference attendees with a statement: “How agential reality shapes our gaming experience”, and a question: “What does it take to become a top-tier robot vacuum cleaner?”. We set a limit of 20 participants to be able to cope with the preparation of the material for the workshop and to have a reasonable number of people sharing the same room, having discussions among them, and allowing them to play a game together on a shared screen. To claim a seat, participants could register in advance via an online form. Registered participants received a confirmation and a link to our related article. We only stored the number of registrations to be able to close it if all seats were filled. In the end, 10 participants joined the workshop.

2.2. Data Collection

The primary empirical material consists of qualitative observations, documented through written field notes and photographs taken during the preparation, play, and discussion phases. To ground these observations, we cross-referenced our qualitative documentation with system recordings: all physical inputs and digital actions taken by the participants were recorded by the Unity game application. This allowed us to play back each session afterward to correlate our field notes regarding participant utterances and behaviors with the corresponding state of the digital environment. We tried to capture as much detail as possible about the whole context with our documentation. Apart from that, we decided beforehand to specifically focus on two things in how the participants expressed themselves verbally and by action:

1.

Subjects and agency: how subject/object relations unfolded and how agency was distributed.

2.

Emergent dynamics: how reconfigured relational constraints gave rise to emergent gameplay.

2.3. Interpretive Analysis and Alternative Readings

The material was analyzed using an iterative and theoretically driven interpretive process. We collaboratively discussed our respective observations, reviewed field notes, photographs, and digital playbacks of the gameplay sessions. To analyze shifting agency and anticipation, we specifically tracked the verbs participants used during play (e.g., the shift from “pressing” and “cleaning” to “blocking” and “attacking”). Because agential realist anticipation is an active, situated disposition rather than a static prediction, verbs serve as vital material-discursive markers. They articulate not only a participant’s immediate intentions and expectations but also how they continuously negotiate and enact their relational boundaries within the unfolding context of the game.

We acknowledge that emergent shifts (e.g., from “cleaning” to “fighting”) could certainly be analyzed through other established frameworks commonly applied in HCI, such as Activity Theory (AT) [19], Distributed Cognition (DC) [20], Actor-Network Theory (ANT) [21], or Gibsonian affordances [22], while these frameworks offer valuable insights, they generally operate on an epistemology of interaction, relying on a priori ontological distinctions that treat humans, tools, and environments as pre-existing entities that subsequently connect. AT treats the human as a pre-existing subject acting upon an object via a distinct tool; DC maps cognitive load across structurally separate nodes; and affordances, while highlighting the relation between agent and environment, preserve a pre-existing boundary between the two. Even ANT, which treats human and non-human actors symmetrically, methodologically traces networks of distinct, interacting entities.

In contrast, we deliberately applied an agential realist focus to prioritize intra-action—the premise that relata do not pre-exist their relations. Agential realism was uniquely suited for this study because it provided the vocabulary to capture the fluid ontological shifts we observed in the data. For example, when participants linguistically merged with their hardware by stating, “I ran out of battery”, they were not simply describing a tool failure, but enacting a momentary agential cut where the boundary of the ‘subject’ expanded to include the device. An agential realist approach allowed us to analyze these phenomena as continuous reconfigurations of reality, rather than forcing them into pre-existing analytical categories.

2.4. The Workshop Plan

The workshop involved a custom-designed game. We aimed to capture the essence of agential realism by staging a game where participants became involved in a socio-technical arrangement. The entire context should in some way influence the course of events, such as how the participants were organized in the room, interactions with the digital game world, and the participants’ interactions and discussions with each other. In the game, the workshop participants took on the roles of robot vacuum cleaners, and they had to anticipate and adapt to changing relational constraints, leading to distinct emergent gameplay rhythms that were neither explicitly programmed nor strictly predictable.

The digital part of the game was developed using Unity (Unity Technologies, San Francisco, CA, USA, version 6000.0.40f1), designed to be projected onto a large enough screen so that all participants could simultaneously play together in the same room. The digital game world consists of three Unity scenes that depict different rooms from a top-down perspective. To the right in the game view is a square play area, and to the left is an overview panel. In the play area, participants control their respective vacuum cleaners, navigate the room, and collect dust. In the overview panel, information about each vacuum cleaner is displayed. Figure 1 shows the three rooms and the overview panels, with robot vacuum cleaners in their starting positions.

The overview panel shows each participant’s vacuum cleaner together with icons representing how much battery is left and how much space is left in the dustbin. Each vacuum cleaner can be identified by different colors as well as their name displayed below them. The naming convention is the following: “Vacuum01”, “Vacuum02”, …, “VacuumNN”. The ending number (NN) of the name is also displayed on top of each vacuum cleaner in the play area. The names and colors of the different vacuum cleaners stayed the same throughout the workshop. The only exception to this was in Teamplay, where only two colors were used based on how the participants were divided into two teams. Below each vacuum cleaner, a counter is displayed showing the current amount of dust collected. At the bottom of the overview panel, a timer counts down, showing how much time is left to play in the current room. In Teamplay, the total score as the accumulated dust for each team member is displayed at the bottom of the overview panel.

In addition, custom-designed circuit boards were created so that participants could assemble their own game controllers from a collection of parts. A main board housing an ESP32 microcontroller (model LCKFB-ESP32S3R8N8, LCSC, Shenzhen, China) was designed and programmed using the Arduino library. Figure 2 shows the main board. Six different smaller input boards were created to allow different modes of physical input. Figure 3 shows the input boards. The main board had a limit of four inputs, numbered from 1 to 4.

The main board is powered via USB-C, and the smaller boards were powered by connection to it. OSC (Open Sound Control) messages were sent as UDP packets from the main board to a computer running the Unity application. On each participant’s controller, a number was printed; the same number that was used in the previously mentioned naming of the robot vacuums.

The input boards were designed to allow for different types of physical input. Input sensors among the six input boards were three push buttons in three different colors (red, green, and blue), one flip switch, one twisting potentiometer, and one sliding potentiometer.

An overview of the technical setup is shown in Figure 4.

3. Results

In this section, we present the workshop results chronologically, concluding with a summary that bridges our observations back to the core theoretical framework. Although the sessions generated numerous verbal and behavioral notes, we highlight the most representative and recurring examples. Following our stated methodology, we focus on how participants expressed themselves through both speech and action by specifically tracing the evolution of subject/object relations and how reconfigured relational constraints gave rise to emergent gameplay.

The workshop was divided into two phases: a preparation phase and a play phase. To establish the initial socio-material conditions of the workshop, it is important to note the physical and social arrangement of the space. The physical room was small enough to foster a sense of a single cohesive group, allowing seamless discussions among all participants. However, within this shared space, participants naturally divided themselves across different tables into smaller, observable subgroups. Although there was a baseline level of general familiarity among all attendees due to the conference history, participants clustered by choosing to sit physically next to individuals they knew better personally and by name. This pre-existing physical and social arrangement formed the initial relational baseline from which the subsequent gameplay dynamics would emerge and eventually diverge.

3.1. Preparation

In the first phase, the purpose of the workshop was introduced using our paper and its arguments. Since the workshop was arranged before our paper was presented at the conference, we gave a summary of our research study and the arguments in the paper. After that, the participants were introduced to the material we had prepared and the basic principles of the game.

The participants were initially informed:

• You are about to take on the role of a robot vacuum cleaner.
• You have a controller board—this is the brain of your vacuum. There is a number on the back of the board; remember your number.
• You will select four out of six input types. These inputs give you the following abilities, but you will not know until the game starts which input number is tied to what action.

  –

  Drive forward;

  –

  Turn left;

  –

  Turn right;

  –

  Vacuum.

• You can view the status of your vacuum cleaner in the overview panel on the left. You have a battery that you need to manage. Using all inputs at once will drain your battery faster.
• You have a dust bag that will fill up after a while when vacuuming.
• To charge your battery and empty your dust bag, you need to go to a charging station (Note: only one vacuum cleaner can be charged at a time).
• If you run out of battery, your vacuum cleaner will stop. Then, you will need to be pushed to a charging station.

The participants then chose four input boards and connected them to their main board. They were also provided with cardboard, scissors, adhesive tape, and double-sided adhesive tape to build a controller. They could cut and put together their controller to their liking. They were free to attach the mainboard and the input boards to the cardboard in whatever arrangement they wanted, restricted only by the length of the connection wires. Figure 5 shows two examples of built controllers.

Before continuing, we held a discussion on the participants’ anticipations so far. Anticipations at this stage were articulated with verbs in relation to their newly built controllers or to the presented robot actions, such as twist, turn, vacuum, and press. The participants also expressed an eagerness to try their controllers. At this point, the on-screen robot vacuums were not referred to as a subject. The subjects were the participants themselves, expressing agency in relation to their newly built controllers as objects.

3.2. Play

In the play phase, the participants played the game together in different configurations. First, they got to play the Lobby room, then the Living room, and last the Garage room (see Figure 1). In between each play session, a discussion was held to elaborate on experiences and how anticipations changed. In the following part of the section, we present both our observations and our interpretations chronologically based on the game room played. Lastly, we summarize the results by bridging back to our key theoretical concepts.

The Lobby room: The play phase started by letting the participants get to know their controllers and vacuum cleaners in the Lobby room. After a short period of confusion about things like what input does what action, how to identify one’s vacuum, and how to charge, the participants got to know their controller and how to go around the room vacuuming. They also found out that running out of battery really ended up in their vacuum turning lifeless. At this point, they started pushing each other to a charging station. We spent about 15–20 min in the Lobby room to really allow everyone to get a feel for the basics of the game and their controllers.

Before continuing, we had another short discussion on anticipations. At this point, we observed how anticipations were expressed by new verbs such as “exploring”, “pushing” (others to help), “searching” (for one’s robot), and “charging”.

Based on the verbs used, the physical controllers had started to become (at least partly) embodied. One participant said, “I think the fact that I built my own controller made this more immersive”.

The Living room: When we changed to the living room, the participants were told to compete against each other to see who could collect the most dust in 5 min. We likened this to FFA play (Free-For-All), a term often used for competitions open to everyone without restrictions. Since this room added furniture that could be bumped into and even go under, losing sight of the on-screen robot vacuum, we let the participants try out the new room for about 10 min.

Again, a short discussion was held. Anticipations were expressed by yet new verbs, such as “emptying”, “dusting”, and “queuing” (to charge). During this play mode, participants had started to refer to their vacuum cleaners as “I”, now with quotes as “I ran out of battery” and “can someone please help to push me to a charging station”.

The expression of subjects and agency was now twofold. The participants were mostly attributing the agency to themselves with their robot vacuum as objects with quotes such as “which is my robot”, “my robot keeps spinning left”, or “my robot ran out of battery”. They were the cleaners with an external entity (the robot vacuum) as a tool to help them. However, there were a few examples of referring to oneself with “I ran out of battery”. Referring to the other players was done, at points, based on their physical location in the room, asking a neighbor for help. This was met with quotes such as “I will help to push you to a charging station. Where are you? What color?”. Agency, thus, was partly attributed to an entangled subject including both a human and an on-screen robot vacuum. The physical controller became a bridge for the entanglement. Although not all utterances aligned uniformly, such shifts in agency attribution recurred among the participants and across play sessions, indicating a pattern rather than isolated linguistic events.

The social dynamic of pushing each other to a charging station was not an explicit game rule; there was no programmed reward for it. The more participants played, the more consistently they expressed themselves as entangled subjects. The agencies that performed the actions became increasingly an inseparable assemblage of human, on-screen robot, and controller. Emergent gameplay was observed and articulated, such as having to queue to charge. And, to our surprise, although competitive, there were several helpful participants that pushed competitors to a charging station.

The Garage room: In the final room played, the participants were randomly divided into two teams. This meant that although the team’s players were grouped together in the digital world, they were not necessarily so in the physical room. A participant sitting next to you was not necessarily a teammate, as we did not change physical seats based on the random division into teams.

The teams were told to compete, team by team, still with the goal of collecting the most dust in 5 min. The teams scores (joint accumulated dust) were displayed and updated in real-time so that the participants could track which team was in the lead. After a first round of 5 min, the participants wanted to go again for another 5 min of team play.

A final discussion was held. Several interesting topics were raised. No one referred to their vacuum as a separate entity anymore. Even more verbs were used, out of which blocking, hindering, and attacking stood out. The teams tried to hinder each other from getting onto a charging station or blocking by staying in one. This made the teams try to attack to push away the opponents to free a charging station. A large part of the room was never visited.

Observing the social dynamic among the participants, it had now shifted from looking and talking to the neighbor, to more loud talking at the screen with quotes like “Number 9! Try to block the other charging station”. After all, they only knew some of their teammates based on the robot vacuum name on the screen.

In the end, one of the participants concluded, “Dust suddenly had no meaning anymore, we forgot about dusting”. What had emerged was not only a new social structure and a new strategy but also a new competitive goal; it was all about hindering, blocking, and attacking. The participants had now become entangled with the context to a degree where they talked to each other through the screen with the name of their respective robot vacuum.

3.3. Summary of the Play Sessions

By mapping these room-by-room observations back to the core concepts of our agential realist framework, the relational shifts of the workshop become visible. The premise of intra-action was evident throughout. The human player, the physical custom controller, and the digital game rules did not act as independent entities, but continually co-constituted one another to form an inseparable, emergent assemblage. This entanglement was punctuated by clear agential cuts, most notably the linguistic and behavioral shift where participants stopped treating the robot as an external object (“my robot ran out of battery”) and began embodying it as an entangled subject (“I ran out of battery” or “I am number 9”).

Furthermore, anticipation operated not merely as a cognitive prediction, but as an active, situated force. Participants physically structured their future gameplay possibilities by building their controllers based on early expectations of “cleaning”, but this anticipation dynamically shifted as the relational context evolved. As the context shifted to the Garage room, we observed the ongoing process of re-membering; while the physical materiality of the controllers remained constant, their meaning was dynamically reconfigured. A controller built and anticipated for “vacuuming” was re-membered and embodied as an instrument for “attacking”.

Finally, the progression through the rooms illustrated the system’s emergent rhythms and capacity for homeorhesis. Rather than a single static gameplay loop, distinct unprogrammed rhythms emerged from the shifting relational constraints: the exploratory rhythm of the Lobby, the helpful/gathering rhythm of the FFA mode, and the aggressive/blocking rhythm of the Team mode. Within these broader rhythms, localized rhythms also materialized, such as the cyclical pattern of venturing out to vacuum and returning to recharge. Crucially, when an individual player’s battery depleted, this rhythm was not hard-coded to automatically reset; instead, it was at times maintained through the emergent social dynamic of competitors pushing lifeless vacuums to the charging station. When the constraints shifted in the final room, the game did not break, nor did it force the players back to the original goal of homeostasis (dust collection). Instead, the system exhibited homeorhesis, seamlessly stabilizing into an entirely new, deeply entangled trajectory of play.

4. Discussion

We deliberately divided the workshop into two distinct phases, inviting participants to actively participate in the creation of the initial conditions. As we argued in our previous work, using Pelle Ehn’s “Design Things” [23], our objective was to create a performative assemblage. Instead of creating a finished object or pre-thought experience for a participant, we organized the initial conditions for relational complexity, a Thing where relations and interactions can emerge. The aim of such a design goal was to create the potential for a plethora of possible unfolding relations during gameplay.

Findings from the workshop imply an extension to our framework by exploring how design can curate the conditions for agential cuts, rather than defining rigid transactional interaction. A design practice that adheres to this also needs a novel temporal logic, and the system it designs must likewise use a novel temporality in contrast to linear time moving from $A\to B\to C$. Therefore, we propose a different approach, grounded in agential realism and Edgar Morin’s complexity theory.

Traditional project models are linear and teleological as they move towards a predefined goal. Based on agential realism and the Morinian rationale for organization as a constantly generative and regenerative activity, the object and evaluation of design change. The focus shifts from creating ready-made artifacts for specific users and use cases to shaping the dynamic and material conditions of an entire socio-technical arrangement. The goal is not to predetermine and control outcomes, but to cultivate a rich scope of evolving relations.

In game design theory, emergence is typically understood as complex behaviors that arise from the interaction of formal rules (e.g., [24,25]). Although these theories acknowledge that designers cannot completely control the final experience, they often treat the system as a closed loop of logic. In addition, Gaver et al.’s concept of ambiguity [26] points to the fact that a lack of prescriptive meaning allows users to construct their own narratives.

However, we argue that emergence is not just about the lack of full control of a user experience or about interpretation. Emergence is an ontological shift of a system’s components and their relations. The workshop showed how a game controller can transform from a tool of cleaning to an instrument of aggression based on the unfolding history of the system. This happened when participants after a while embodied an entangled subject/object agency that before was attributed to an external subject (the robot vacuum cleaner). Much like Leibovitz’s profound realization after playing Legend of Zelda: Twilight Princess [27] for several hours:

I was not truly a subject, at least not in the pure Cartesian sense; I was no thought and all extension (p. 45, [28]). […] Finally, two hours after a muscle memory of sorts first became apparent in my play, I used the one word to describe Link that I had previously consciously avoided: I.

(p. 49, [28])

4.1. Why an Agential Realist Interpretation?

The emergent shift observed in the Team mode (where participants abandoned the goal of collecting dust) could arguably be analyzed through established HCI frameworks such as Activity Theory (AT), Distributed Cognition (DC), Actor–Network Theory (ANT), or Gibsonian affordances. However, we argue that these frameworks are insufficient for capturing the profound ontological shifts that occurred during gameplay, making an agential realist interpretation uniquely valuable.

An AT lens would view the participants as pre-existing, stable subjects using the controllers and on-screen vacuums as mediating tools to act upon an object (the goal of the game). Under AT, the shift from cleaning to fighting would simply be explained as a conscious change in the object of the activity, triggered by the introduction of the team-play rule. Because AT maintains a strict separation between the subject and the tool, it fails to account for the linguistic and behavioral shifts where players declared, “I ran out of battery”, or how the physical controller was recursively given new meaning. The participants did not merely change their goal; their very subject-agency was dynamically reconfigured in entanglement with the hardware and software.

A DC perspective would analyze how the cognitive load of the new “fighting” strategy was distributed across a network of humans, screens, and custom controllers. While DC excels at mapping information flow, it still treats these nodes (human, sensor, screen) as fundamentally distinct, pre-existing entities that simply interact. Similarly, an ecological psychology perspective using Gibsonian affordances might argue that the new game rules simply highlighted different physical affordances of custom controllers. However, the concept of affordances still preserves a fundamental, pre-existing boundary between the perceiving agent and the environment, failing to capture the deep ontological entanglement where the participant, the controller, and the digital on-screen vacuum merged into a single acting subject.

Even Actor–Network Theory (ANT), which treats human and non-human actors symmetrically, approaches this relationality differently. While ANT excels at tracing how actors are shaped through their associations, methodologically it relies on identifying discrete actants to map those translations. In contrast, our findings highlight the mechanism that comes before the network: the agential cuts that temporarily materialize what constitutes an actor or an object in the first place.

Agential realism posits that relata (subjects and objects) do not pre-exist their relations. The workshop demonstrated that the players, the custom-built controllers, and the game rules did not interact as separate entities; their salience and meaning emerged by intra-action, enacting temporary causal structures. The physical controller did not remain a static tool that was simply used differently; its meaning, salience, and material impact were ontologically reconfigured by the unfolding history of the system.

Although other frameworks are highly useful for their intended purposes, they are not equipped to explain a system whose identity is not fixed in its discrete components or predefined goals (e.g., gathering dust), but lies entirely in its ongoing process of becoming and its emergent rhythms. We argue that agential realism is uniquely equipped to explain this profound relational complexity.

The results of the workshop provide empirical support for an agential realist interpretation. It shows that even a minimal change in relational conditions can cause a cascade of changes that completely change the purpose and meaning of a system. Specifying a goal (e.g., gathering dust) is no guarantee that that goal will be relevant to the agencies that actually unfold, and thus make up the system. Adopting agential realism compels a fundamental re-evaluation of what constitutes a successful interactive system. Instead of measuring efficiency, accuracy, or task completion, this framework encourages the development of methods to value the relational complexity of emergent rhythms a system produces.

4.2. Limitations and Generalization

Although the workshop provided empirical grounding for our theoretical claims, this exploratory study has several methodological limitations. The sample size was small ($N=10$), and participants were drawn from a conference focused on gaming, entertainment, and media. The familiarity with interactive systems among participants was advantageous in rapidly engaging with the custom hardware and complex dynamics of the game. However, this also means that the observed behaviors may not fully represent how the public would interact within the system. Furthermore, the empirical setup was confined to a multiplayer game environment, which naturally encourages role-play and competitive behavior.

The primary contribution of this study lies in its theoretical and methodological framework. Despite limitations, demonstrating how to apply an agential realist reading provides a valuable lens for the HCI community. We highlight that we are not tracking the cognitive intentions or subjective choices of ten individuals to prove statistical invariance across a population. We are demonstrating the processual consistency of agential cuts within this specific socio-technical assemblage. As agential cuts are not deliberate choices made by pre-existing agents, the agential cuts are contextual reconfigurations enacted by the entire socio-technical assemblage. The observed shifts in language and behavior are treated as ‘marks on bodies’ [2], empirical evidence of how the phenomenon materializes specific boundaries and agencies in real-time.

The shift from transactional interactions to relational complexity was tracked by documenting the recurrence of specific ‘agential cuts’ across the participant group, such as the linguistic transition from third-person tool description (‘my robot’) to first-person entangled subject (‘I ran out of battery’). By correlating these verbal markers with recorded gameplay sessions, we observed the manifestation of underlying processual rhythms. By documenting these rhythms through the workshop, we observed how the identity of the system was processually enacted. This qualitative analysis allows us to translate observed micro-shifts in agency into broader design principles that prioritize the system’s ongoing ‘becoming’ rather than static task completion.

5. Conclusions

The emergent rhythms of gameplay observed in our workshop reveal a gap in transactional interaction design and linear computational models. By analyzing how these patterns emerge, we lay the foundation for a novel approach to design systems not to solve tasks but to embody the rhythmic, re-membering nature of agential reality. Whether explicit state machines or goal-oriented optimization algorithms, they rely on linear causality and predefined objectives. In such models, an interaction is treated as a transaction in which an input A is processed to achieve a goal B. If we were to model our workshop using this logic, the emergent shift from “cleaning” to “fighting” would be classified as a failure. A system optimized for dust collection would view the participants’ behavior in Team mode not as a meaningful relational reconfiguration, but as noise or error deviating from the ground truth. But, our analysis shows that this deviation was the moment when the system became the most salient to the participants, even making them forget the original goal. The emergent behavior was not a bug; it was a new stable rhythm of becoming.

5.1. Design Principles for Organizing Relational Complexity

To move beyond transactional interaction models, a framework built on agential realism requires a novel perspective on temporality and system dynamics. In line with agential realism, the ‘agential cuts’, such as the shift from vacuuming to attacking, are not deliberate human decisions but are enacted by the entire socio-technical assemblage. The tracking of these shifts, viewed as material-discursive enactments rather than cognitive human choices, provides a grounding for how the identity of a system is constituted processually. This grounding serves as a foundation for the following design principles, intended for designers to explore emergent system dynamics rather than optimizing for pre-defined goals:

5.1.1. Design for Relational Emergence (Embracing Non-Linear Dynamics)

Traditional interactive systems often rely on linear causality, where a system’s state is caused solely by the immediately preceding state or where a user’s cognitive choice leads to a predictable chain of events ($A\to B$). Instead, we propose designing for non-linear dynamics, where the state of the system is an expression of the complete, current organization of its relations.

• Observation: This became dramatically evident when the conditions of the game changed. In Team mode, a simple rearrangement of relational constraints caused the previous social structure to collapse, and new antagonistic relations (“block”, “hinder”, or “attack”) emerged. The system’s identity shifted non-linearly, and the original competitive goal was entirely forgotten.
  The emergent dynamics observed did not rely on the subjective intentions of ten individuals, nor are they the result of pre-programmed rules within the system. The participants did not decide to feel like a vacuum, nor did the software force an ‘attack mode’. The agential cut was drawn by the intra-action of the human, physical controller, the digital rules, and the social context.
• Design Implication: The object of design must shift from scripting a system’s final form to organizing the initial material-discursive constraints. Importantly, designing for emergence does not imply the absence of rules or prior instruction; rather, it focuses on how the salience, meaning, and enacted relevance of those rules can shift over time through the unfolding relational configuration of the system. The different room configurations (Figure 1), together with the physical social arrangement, functioned as contextual ‘stages’ that allowed the identity of the system to shift. When the relational constraints shifted in the Garage room (Figure 1c), the initial goal collapsed and new antagonistic relations emerged.

5.1.2. Design for Re-Membering (Fostering Recursive Reconfiguration)

In a transactional system, time moves sequentially, and memory is treated as a static data log. In contrast, an agential realist system operates through recursive reconfiguration, where time moves in loops: the past is not fixed, but is actively reshaped and given new meaning by current intra-actions.

• Observation: When participants first built their controllers, it created an irrevocable historicity. Initially, players used a vocabulary tied strictly to the physical materiality of their hardware (“press”, “push”, “twist”, “turn”, or “slide”). However, as the history of the system progressed into Team mode, this material history became recursively reconfigured; the controller was re-membered and embodied as an instrument to “attack” and “block”.
  The temporal reconfiguration was not a deliberate cognitive choice where participants decided to change the meaning of their tools, nor was it a pre-programmed state transition within the system’s software. The past intra-actions reverberated into the present, where the utility of the controllers was co-constituted by the entire material-discursive arrangement.
• Design Implication: Systems should be designed so that their past actively shapes their future organization through structural coupling rather than static retrieval. By allowing a system to re-member, designers can create architectures where the meaning of tools is continually and temporarily co-constituted within the unfolding context. The choice of input boards (Figure 3) and assembled controllers (Figure 5) created an irrevocable historicity; while initially built for vacuuming, these physical artifacts were re-membered—embodied as instruments for attacking as the system’s history progressed.

5.1.3. Design for Emergent Patterns (Valuing Processual Emergence)

In traditional design, success is measured by achieving an optimal predefined state. However, under agential realism, the value of a process is not a final stable state, but the ongoing process itself. The identity of the system lies in its patterns of change—its rhythms.

• Observation: The workshop revealed distinct rhythms that were never explicitly programmed. The initial rhythm was based on a relation with the on-screen robot as a separate entity. This transitioned into the FFA rhythm with a diffuse separation, where the system emerged as a busy socio-technical arrangement. Finally, the impassioned Team rhythm took over. These rhythms are an observation of a system’s ability for homeorhesis, which means the ability to temporarily stabilize on a new trajectory of change.
  The rhythms were not chosen by the participants. They were the result of the entanglement of human, controller, and social context. The rhythms were not hard-coded into the game rules. They resulted from the entire socio-technical assemblage temporarily stabilizing in different rhythms.
• Design Implication: Designing for emergent patterns means expressing dynamics through temporal patterns across the entire socio-technical assemblage. The true value in design and participant experience lies not in optimizing for a specified goal (e.g., collecting dust), but in curating the rich and unexpected variety of emergent socio-material dynamics. The technical and physical setup (Figure 4) facilitated emergent rhythms, such as the helpful social dynamic of pushing competitors to a charging station. These rhythms were never explicitly programmed. These patterns represent the system’s true identity as a continuous process of becoming.

5.2. Implications for Computational Architectures: From Homeostasis to Homeorhesis

Adopting these design principles has implications for how we understand and construct the computational architectures that underpin interactive systems. Our results demonstrated that the meaning of the gameplay was not found in reaching a static, predefined state (e.g., maximizing the amount of dust collected), but in the system’s capacity to support shifting relational rhythms (e.g., moving from dusting to exploring to attacking).

To build systems that organize relational complexity, we argue for a shift in computational values from homeostasis to homeorhesis. While homeostasis refers to the ability of a system to correct deviations to maintain a static internal state, homeorhesis is the ability to maintain a stable trajectory or rhythm of change. When our participants abandoned dust collection for antagonistic team play, the system did not break; rather, it underwent a homeorhetic shift, settling into a new, stable rhythm of becoming. We thus argue that dynamic interactive systems should not just correct deviations to return to a status quo (homeostasis). Instead, they must be capable of an affective attunement, resonating with the changing relational context to temporarily stabilize within new emerging trajectories (homeorhesis).

Translating our findings into a computational architecture is a provisional hypothesis that requires further empirical and technical validation. However, this approach strongly aligns with the tradition of Enactive AI and dynamical systems theory in cognitive science. As Varela et al. argue [18], cognition is not the processing of pre-given information, but the enaction of a world through embodied action. From this perspective, an agent does not passively reflect an independent environment, but brings a meaningful world into existence through its structural coupling.

To realize this computationally, we must move away from simulating agency through rigid behavioral rules. Froese and Ziemke [29] clarify this by distinguishing between behavioral and constitutive autonomy, arguing that true enaction requires a system to actively generate the conditions for its own unfolding. In our continuing work, we are exploring Continuous-Time Recurrent Neural Networks (CTRNNs) and Reservoir Computing (RC) [30,31] as promising directions that fulfill these conditions. As Beer demonstrated in [32], minimally cognitive behavior can emerge from the coupled dynamics of a CTRNN and its environment without requiring explicit internal representations. Similarly, RC models operate on principles of high-dimensional dynamic memory and non-linear mapping. Because models like this utilize a ‘fading memory’, where previous inputs continuously reverberate and influence current states, they mathematically embody the re-membering and rhythmic nature of agential reality far better than strictly algorithmic logic. By combining such models with the concept of non-causal circuits, we aim to develop interactive systems that do not merely execute deterministic interactions, but genuinely become one with their human participants.