Computational Streetscapes

Abstract

Streetscapes have presented a long-standing interest in many fields. Recently, there has been a resurgence of attention on streetscape issues, catalyzed in large part by computing. Because of computing, there is more understanding, vistas, data, and analysis of and on streetscape phenomena than ever before. This diversity of lenses trained on streetscapes permits us to address long-standing questions, such as how people use information while mobile, how interactions with people and things occur on streets, how we might safeguard crowds, how we can design services to assist pedestrians, and how we could better support special populations as they traverse cities. Amid each of these avenues of inquiry, computing is facilitating new ways of posing these questions, particularly by expanding the scope of what-if exploration that is possible. With assistance from computing, consideration of streetscapes now reaches across scales, from the neurological interactions that form among place cells in the brain up to informatics that afford real-time views of activity over whole urban spaces. For some streetscape phenomena, computing allows us to build realistic but synthetic facsimiles in computation, which can function as artificial laboratories for testing ideas. In this paper, I review the domain science for studying streetscapes from vantages in physics, urban studies, animation and the visual arts, psychology, biology, and behavioral geography. I also review the computational developments shaping streetscape science, with particular emphasis on modeling and simulation as informed by data acquisition and generation, data models, path-planning heuristics, artificial intelligence for navigation and way-finding, timing, synthetic vision, steering routines, kinematics, and geometrical treatment of collision detection and avoidance. I also discuss the implications that the advances in computing streetscapes might have on emerging developments in cyber-physical systems and new developments in urban computing and mobile computing.

1. Introduction

In this paper, I will review the state-of-the-art in streetscape science, as supported by computation. This review is, perhaps, quite timely. There is a recent flourish of interest in streetscape phenomena and in developing computer technologies that rely on streetscape dynamics as a medium for services that might be delivered on streets or that might support people and things as they course through the phenomena that streets support [1,2]. Concurrently, our understanding of how people behave on streetscapes is open to investigation in ways that have never presented as paths for inquiry before. Here, again, computation provides a medium for new forms of exploration, via big data in particular [3], and via the ambient intelligence that mass instrumentation now casts from our everyday activities [4]. Along other frontiers, burgeoning advances in computational biology and imaging science have opened up entirely new vistas of possible exploration of the neurological basis for our decision-making as we move through places, spaces, and tasks [5]. Taken together, computation is providing a catalyst for innovative investigations of streetscape science, with the future promise that we might be able to piece together explanatory pathways for streetscape phenomena that stretch from the sources of cognitive structures at epigenomic level to the very building-blocks of behavior, among the actions of individuals and the motifs of massively interactive phenomena at the built-human interface. The set of potential applications for this science is broad, ranging from understanding processes of panic and evacuation in critical scenarios [6], to building more entertaining mobile games for augmented reality platforms [2,7].

Interest in streetscape science is not new. People’s behavior while they traverse streets has always been of broad importance across a variety of interests, whether this behavior manifests as a physical relationship between humans and the built fabric of the streetscape [8]; as cognitive phenomena that form as people negotiate their own goals and thoughts relative to what they see or do [9]; as social dynamics produced by the interplay between inter-personal factors and the tangible contexts of built places [10]; or as the cyberspace that interactions between things, people, and data produce as they dance between physical and virtual realms [11,12]. Understanding (rather than simply observing or measuring) the interdependency between streetscape entities, events, and processes is crucial in explaining how the aforementioned phenomena might work, and in managing the processes that enliven them in cities. However, building understanding of streetscape dynamics is tricky. Even as a fundamental concern, it is, for example, not always clear where one should look to find answers to questions regarding streetscape phenomena. After all, the processes that underpin streetscape dynamics have explanations at the urban scale, which includes the everyday rhythms of movement as downtown populations wax and wane between the beginning and end of the workday [13,14,15,16], and as groups take to sidewalks to shop and play [17]. Other factors that determine streetscape dynamics may be found at the population scale. For example, the walkability of streets, the socio-spatial determinants of movement and interaction, and the place-making they support are of growing relevance in understanding population-wide activity levels with implications for community health [18] and geographies of inclusion and exclusion [19]. Similarly, the timing and accessibility of streets to particular population groups has always been of significance in explaining bias and equity [20,21,22], but some of these influences are fleeting, while others build up over wide swaths of history. The explanations for streetscape behavior are experienced by us all, as individuals that acquire ambient information about our place in streetscape surroundings as we walk and interact [23,24,25,26]. In this sense, the “atoms” (or spatially non-modifiable units or entities) of streetscape phenomena would seem to reside at the scale of individual people. However, there is growing evidence that these behaviors are sourced at the neurological level even before they take hold in our cognition, as the handiwork of dedicated place cells in the brain that work to store information specifically for navigation and wayfinding [5,27,28,29]. Research in neuroscience thus indicates an even finer resolution (and perhaps even a finer-scale realm) for many streetscape phenomena. One must also consider that the relationship between walking and place is not static with respect to space and time. Therefore, explaining streetscape phenomena often requires vistas that bend with temporal focus. For example, during certain periods of the year, streets and crowds shift phase to form carnivals, parades, and mass parties [30], and the phenomena that result often present with cultural and historical subtleties that shape people and streets in intricate ways [31,32]. In emergency scenarios, streets and the crowds that course through and over them become critical conduits for escape and access; critical streetscape dynamics may form suddenly, surprisingly, and in quite extraordinary ways in these situations [33].

While communities of scholars have been studying streetscape phenomena for a very long time, much remains to be learned. Alas, building a knowledge base for streetscape science can be terrifically difficult. Unlike other scientific endeavors, in which phenomena of interest can be probed and played with, physical experimentation with real people and real streets is often infeasible. Streetscape phenomena also involve so many attributes acting in complex interactions that as soon as we think that we have made headway in one area of our understanding, our perspective is expanded by discovery in other domains. Moreover, streetscape phenomena are continually in flux, and building knowledge of their functioning is very much a moving target. Nevertheless, building steadfast understanding and reliable what-if plans and policies for streetscape phenomena is often critical. For example, when the conduits that provide resilience and safety in the interplay between people and built and natural environments lose resiliency or fail, the results can be catastrophic [34,35,36,37,38,39].

In everyday management of streetscapes, a set of best practices and codes have evolved to assist civil engineers and planners in designing and building systems and environments that are safe and efficient [40]. Generally, regulations and design goals that emerge from planning efforts are focused on the level of service that streets might (or ought to) provide [41], on safety features to separate people and vehicles [42], and on plans for emergency procedures [43]. (There is also an interest in planning for social goals on streetscapes, particularly for equity in access to streets, as spaces and places, as well as to activities and opportunities that access might afford [44], but social goals can be difficult to codify.) In supporting planning and design objectives, an emphasis is usually placed on issues of street geometry, lighting, street materials, and street furniture [40]. However, design standards often provide little insight into other what-if questions, such as how and why people may behave on a streetscape, what activities might transpire, and what it feels like to traverse a street with a given design. These types of issues have customarily fallen to theoretical models for exploration, but that theory is often difficult to ground in fact. While there is a rich tradition of observation of people and streets to support theoretical work [45,46], streetscape observation is always challenging to accommodate, because of the size, scope, and subtlety of the phenomena that streetscapes produce. As a result, one generally must turn to models and simulations to explore what-if issues, with models serving as synthetic media that can be experimented with.

To be useful in supporting planning and decision-making, models and simulations of streetscape phenomena often need to be authentic and frequently need to be detailed, down to the resolution of individual people and streetscape features. Indeed, the need for detail is perhaps more important now that many pedestrians carry location-aware and location-enabled devices that serve as sensors as they move [47] (and now that the limits of our understanding of how streetscape behavior forms reaches in some instances to the neurological scale): the proliferation of these technologies has raised new questions at new sizes and scales of streetscape dynamics that potentially can capture phenomena as they unfold in fleeting moments of space and time, up to whole swaths of a city and its population. The quest for fidelity brings us back, full circle, to the long-standing conundrum that we often have little physical evidence with which to parameterize and evaluate theoretical models. New data-sets and new suites of analysis that they support could potentially bring streetscape science into the contact realm of the computational and data sciences (CDS) [48,49,50]. Indeed, computing and data science allow simulation to become the platform (perhaps also the medium and tool) for representation and experimentation, replacing or supplementing theory, observation, and measurement in ways that open up new lines of inquiry and exploration [48,51,52,53]. In CDS, the “digital fabric” of the simulation replaces the tangible realm of the physical, and the algorithmic dynamics of model processes are used as substitutes for counterpart behaviors and phenomena in the real world. However, the substitution of the virtual for the physical in CDS has traditionally been more feasible for well-understood and well-bounded systems and phenomena in the physical and natural sciences than it is for the messy complexity of the socio-behavioral realm.

The extension of experimentation with streetscape phenomena to the virtual world of CDS has also benefitted research into the relationship between cyberspaces and cyber-places [54], where streetscape phenomena are built and studied as information spaces. In other words, the mutability between bits and atoms in virtual and physical realms is embraced, and accepted as given; the investigation of processes and phenomena turns to issues of how streetscape phenomena function across the cyber-barrier separating worlds. How phenomena take shape across cyber-barriers has implications in a range of computing interests, such as wearable computing [55], urban computing [4], and social computing [56,57], which rely on brokering dynamics across real and virtual communities of pedestrians and streetscapes to “work”. For example, many telecommunications systems rely on individuals and the networked devices that they carry along streets as mobile points of presence in massively complex webs of interactive data transfer [58,59]. In this sense, streets may operate as cyber-physical systems [60], and there is wide interest in understanding how people move so that services or content can be delivered to those systems in the right places, time, and context. The “location-aware computing” [61] that works as the secret sauce behind location-based services [2,62,63] is perhaps a popular example of the computing that is emerging to handle street-level cyber-physical systems.

Models and simulations of streetscape phenomena are also of direct interest (often as computational media rather than as scientific experiments) to people working in computer graphics, gaming, and special effects. These communities have long been interested in building believable and entertaining street dynamics into their movies, character interactions, and behaviors [64]. Because most people experience walking on streets and in crowds as a matter of course in their everyday experience, gamers’ and audiences’ expectations for realism can be demanding, and in response the graphics for models of streetscape phenomena have advanced reasonably close to the state-of-the-art in research simulation and theoretical modeling [65] (in other words, the lines distinguishing graphics as a medium that is distinct from a scientific simulation are blurring). Indeed, some of the work being developed in computer graphics is now merging with real, tangible, physically built spaces. For example, with the relatively recent emergence of operational augmented reality technologies [7] has come a renewed interest in details of how people sense and make sense of streetscapes as they follow directions and as they find their way, using information gleaned from real geography and virtual geography in combination [66].

Again and again, the limitation of generally being unable to choreograph real people and unable to mold real streets for research questions, coupled with the challenge of faithfully representing messily complex streetscape phenomena in computer models presents across diverse areas of inquiry and interest. How the gap between the real and the computationally synthetic is bridged is often critical in governing the science, policy, plans, and theories that research questions in streetscape science produce. In this paper, I pursue the origins and outlook for these challenges. I consider how computing has contributed to a newly emergent focus on sensing, modeling, and experimenting with streetscape phenomena in silico (i.e., in the silicon substrate of computer processors), and how the resulting science that is developing has been sourced in diverse disciplinary origins, and how it is being actuated with diverse methodologies. In the text that follows, I review the variety of computing approaches that have been used to model streetscape phenomena in different disciplines and in service of a range of problem-sets. This work includes models and applications primarily developed in physics, urban studies, animation, psychology, and behavioral geography, although applied topics in retailing, public health, human abilities, and sociology weave connections between them. This paper is focused on computing, and I review a wide set of techniques for building and running computer models of streetscapes, whether as simulations or services, including acquiring and generating streetscape data, forging data models, heuristics and methods for path-planning, navigation and way-finding routines, the use of timing to govern action and interaction in rule-based models, synthetic vision, steering for interaction and avoidance, geometry for kinematics and effort, and collision detection and avoidance schemes. The science of modeling streetscapes is, however, continually in a state of flux and improvement, especially now when several technologies are beginning to emerge into popular practical use and creative fusion across big data, deep data, automation, and model-based support systems is taking place. To conclude the paper, I will review the prospects for the future of computing streetscapes relative to these fluctuating influences.

2. Domain-Specific Approaches to Streetscape Modeling

In much of the debate about how and why streetscapes yield the dynamics that we experience, there is a neat division between streets, as the physical substrate for activity, and people as the fabric for social interaction atop them. Interest in one or another dimension of streetscape science often follows academic disciplines: architects’, civil engineers’, and transport engineers’ interests are weighted toward streets as tangible, built spaces; psychologists, sociologists, and urbanists are more likely to study the people that traverse those spaces. Biologists are increasingly interested in the patterns and behaviors that form as people move across streetscapes as analogs to foraging and group movement in animal and insect collectives [67,68,69], or as counterparts to emerging neuroscience studies of the role of place cell firing patterns in the access and use of mental maps [5,27,28,29]. Geographers’ attention lies somewhere in between, as they traditionally focus on human-environment interactions that encompass both streets and people [8,9,10,65,70,71,72,73]. Not surprisingly, methods for exploring people and streets have also followed in this dichotomy between the physicality of streetscapes and the humanity of their use, sometimes with the consequence that the fruits of knowledge produced by one method are rather incommensurate with others. For example, level-of-service schemes from civil engineering [41] are reasonably divorced from issues of affordance that built spaces provide as environments that people emote and reason about [74], even though in some instances we have data about both, and in other arenas (retail stores, malls, and casinos, for example) the connection between the physical and behavioral-emotional are examined meticulously [75,76,77]. In what follows, I will explore the literature in a few domains that are central to advancing the explanation of streetscape phenomena. In that discussion, I will focus in particular on domains that have adopted computational approaches to studying streetscapes, primarily through the lens of modeling and simulation.

2.1. Physics

It is perhaps most common that so-advertised “behaviors” in pedestrian movement and interaction models are derived from equations originally tasked with accounting for the physics of particles in flow [65]. In many of these cases, realistic-looking (rather than realistically-driven) patterns of crowds, in aggregate, are the end-goal for the application of physics equations, and behavioral fidelity is not always a main concern. In many examples, physics models are used to simplify the explanation for streetscape phenomena, although the dynamics that the models produce may be quite complex or complicated. Indeed, physics-based equations have found popular use in models of streetscape phenomena because physics is a well-studied and well-appreciated driver of other complex interacting flow processes. In these cases, physics is invoked in ways that draw inspiration from systems dynamics and mechanics of physical or natural phenomena that are well-represented by some elegant equations, such as the Navier-Stokes [78,79]. In these cases, the practice of modeling crowds simply with physics equations speaks to the parsimony of the approach, rather than to the fidelity of the schemes used.

Physics-based models of streetscape phenomena often function on a premise that people, under certain emergency conditions (where escape through a space is of utmost concern) or in tightly-bounded architectures (corridors), generate collective movement patterns that exhibit (statistical) signatures shared with molecules in a gas, grains in a moving granular mass [80], or fluid in flow [81]. For example, physics models of crowds have been based around Maxwell-Boltzmann distributions on the basis that the distribution can represent transitions in collective behavior as if they were phase changes between liquids and gases [82,83]. A triple of physical processes—particle interaction as flow, forces of attraction and repulsion, and phase change—feature rather ubiquitously in pedestrian models, often as the rules for agent behavior in simulation. For example, the triple is used to produce continuum mechanics for crowd aggregates [84], many-particle interaction within crowd flows [85,86], ebb and flow over vector-fields [87], and the physical or social forces of compression and expansion over repulsive-attractive fields [84,88,89].

While there is some evidence to show that the patterns of movement of pedestrian crowds exhibit dynamics that look similar to physical phenomena of gaseous or granular flow [90], there is usually not evidence that real people actually behave in the ways that the equations have been ascribed, particularly outside the specialized scenarios of packed crowds or crowd flow through corridors (see Torrens [65] for a detailed discussion). Certainly, some interactions that pedestrians engage in are physical, but most are not. For example, physics approaches to treating streetscape dynamics may invoke considerations of distance and personal space in collision, which are relevant in examining Newtonian-type forces between pedestrians [88,91], whether through tangible collisions, or as human-perceived repulsion and attraction effects [73,92,93,94,95,96]. However, decades of socio-behavioral science provide alternative cognitive and social explanations for much of the dynamics presented in physics-based models [45]. The computational elegance of many physics-based models is an understandable attraction of the approach. However, an assumption of primacy in the physical attributes and drivers for pedestrian movement may actually limit the explorative power of computer models. For example, a focus on physics directs attention to large-scale issues of flow and turbulence (about which most pedestrians on a street are unlikely to care or sense) and small-scale issues of collision (which the majority of pedestrians avoid in any case). Nevertheless, physics-based models have been particularly useful in focusing the community on solid empirical approaches for describing and explaining movement, particularly across the spectrum that connects microscopic behavior of individuals to macroscopic phenomena in crowds. This is a big advantage of the physics approach, although much work remains in building socio-behavioral schemes that can explain and generate those dynamics in models with greater realism.

2.2. Urban Studies

Models of street-level dynamics in urban studies (and here I will deploy the term urban studies to conveniently collate urban planning, urban geography, urban design, and elements of civil and transport engineering) are generally focused on built settings as facilitators of movement, and decisions about movement [97]. Traditional investigative themes include examining the sources, sinks, and paths that pedestrians use during trip-making [17,71,98,99,100,101,102]; how pedestrians acquire and use the built environment to structure movement decisions and to evaluate choices [10,22,44,45,72,97,103,104,105,106,107,108,109,110]; crossing behavior where sidewalks meet roads [73,111,112,113,114]; and the quality of traversal that streets provide [40].

Models of trip-making for pedestrians usually follow the determination of origins, destinations, and shortest paths that are widely used in routing and transport modeling more generally [101]. This makes sense, as walking is often considered as one of a set of travel modes, where pedestrian trips account for a portion of the modal split in travel [115], and where the choice of walking among available modes of travel is dependent upon the available infrastructure for trip-making, cost, historical factors, and other parameters of accessibility [116].

Trip-planning on streetscapes is usually governed by structural attributes of origins, destinations, and paths. The size of populations exiting transit stations, the occupancy of a work-site, and the physical length of sections of sidewalks are examples of the attributes of paths that we might consider and settle on as pedestrians. Invariably, we must weight street attributes before voyaging on streetscapes. Consequently, there is a long-standing interest, among urbanists, in exploring how people use streetscapes, and their features, to frame and to evaluate choices from available options and how people make decisions that are informed by their surroundings [99,100,103,108,117,118]. Pedestrian choice is often quite different (and more individually-founded) than other choosing schemes in trip-making. For example, pedestrians have many more paths to choose from than drivers of cars might be afforded, and pedestrians can flit between paths with little relative effort [119]. Urban studies can also tell us a lot about the features of streetscapes and how they might serve as pedestrian generators/attractors [118,120] and the accessibility that streetscapes afford [22,44], whether that accessibility pertains to broadly-considered urban settings or design, or whether it relates to the accessibility of particular (social, natural, historical) environments or those that cater to special pedestrian activities such as shopping [17,121,122]. Much of the existing work on how pedestrians make choices on streetscapes has been well-explored through operations research [123] or discrete choice studies [70,124]. However, I am unaware of instances in which streetscape choice dynamics have been robustly incorporated into the sorts of computational models of streetscape phenomena that we consider here, although the work by Borgers, Kurose, and Timmermans [71,98,122] is close.

Another strand of work in urban studies is tethered to the idea that movement is determined by how far people can see [125]. This shares some features of traditional terrain analysis in computer graphics research [126], and draws influence from theories of visual flow in psychology [68,127,128]. The visual approach to streetscape dynamics engenders, in particular, exploration of how the viewsheds that streets afford from particular locations might influence movement [129], and how trips might be parsimoniously mapped as journeys through sequences of connecting views [130].

The question of how many people streets can accommodate is raised commonly in pedestrian models for urban studies [41], particularly for applications tasked with supporting design or engineering decisions [107,109,131]. This question may focus on the relative flow offered by particular components of streets: junctions with roads [105,114], conduits formed by the built environment [104], or bottlenecks that form around architectural constructs, for example [90,132,133,134,135]. Questions regarding the occupancy that streets afford also lead to examination of the patterns of crowd congestion or dynamics such as queuing when density gets high [136]. For example, work by Fruin [137] pioneered a class of models that use pedestrian movement to classify the level of service, such as flow and density [138], that streetscapes support.

2.3. Animation

Traditionally, animators manually painted positions of characters, by hand, to populate streetscape scenes in moving image media [139]. As animation embraced a digital age, these methods were translated into the new medium and the workflows it supported (see Porter [140] for example), yielding computer-assisted automation (particularly in movie-making). By the 1970s [141], many in the animation community could see that reliance on computers and computing, then in the infancy of their use in animation, could beget fully-fledged information systems that would assist in managing the expanding intricacy of animation pipelines. From the 1980s to 2000s, parsimonious real-time computing of motion was of central concern for animation studios. As a result, techniques pioneered during that time frequently placed a premium on computational efficiency, which often resulted in use of shortcuts to produce realistic-looking animation for streetscape scenes in film and games, rather than behaviorally-faithful techniques in situations in which one was not attainable without the other. More recently, as computing barriers were overcome and as the techniques matured, both efficient computing and high-fidelity realism could often be achieved.

One significant catalyst in these developments toward high-fidelity streetscape scenes has been the introduction of high-level control schemes for articulating the movement of characters, and low-level kinematics and collision routines for anchoring animation to realistic locomotion. A focus on control and kinematics in animation came about as a function of the transition from analog media to digital animation media. Moreover, as animation appeared with greater significance in computer games, and users began to interact naturally with objects on-screen, the movement of characters against urban backdrops emerged as a significant component of gameplay and story [142]. In movies and computer generated imagery (CGI), if digitally-animated extras move in an implausible fashion, it may distract the audience from the narrative that the director is trying to create, and so significant effort is often put into building realistic-behaving content in special effects [140,143,144,145,146]. Similarly, when non-player characters (NPCs) in computer games present in the wrong time and wrong place, or when they interact with player-avatars in unconvincing ways, it may interrupt players’ engagement with the game, and streetscape scenes have therefore been developed in many games with significant user-interaction content [142,147,148,149,150,151,152,153,154,155,156,157], perhaps because gamers generally have a lot of experience participating in and thinking about streetscape dynamics in their everyday lives.

Approaches to modeling streetscape scenes and crowds in CGI are diverse. Of particular relevance to this paper are efforts in automating kinematics for character rigs and motion control of individually-behaving characters and groups. Kinematics for mobile characters on streetscape scenes were traditionally settled per frame (or keyframe) and “tweening” was used to resolve the interstitial transitions that happened between those frames [158]. As characters began to be animated in geometrical settings, a need for occlusion and collision in animated sequences [159] presented. To address this need, animators had to develop realistic body models for characters, which shifted to kinematic and physics-based routines for resolving tweening [141,160,161]. In recent years, these schemes have been supplemented by motion capture approaches that blend very high-resolution tracked performance data with kinematic solvers [162,163]. These developments were incredibly useful in animation, as they facilitated the calibration of low-level movement directly to real data [164,165,166], and they allowed for significant detail of ambulation, gait, and body language to be encoded in characters [167,168,169], while also solving for the collisions and steering that are usually required while traversing busy street scenes.

Hand-animating characters is time-consuming, even digitally, and the task has long been supplemented (sometimes replaced) by computational models to control movement [170]. Initially, motion controllers [171] for characters emerged from motor control in engineering (and, later, in robotics) that used virtual actuators on virtual joints (which matched to nodes on digital character rigs) [172,173]. Other motion control schemes were developed as finite state machines [174], which allowed animators to develop procedural rules for characters [175]. Also, control schemes within origins outside computing have been woven into computing (see Pelechano et al. [64] for an excellent overview): physics models are popularly used, as are psychology-like approaches [176,177], cognitive schemes [170,178,179], group traits derived from collective human and animal behavior [180,181,182], machine-learning models that build control functions from trajectory data of real people [183,184,185], and schemes that afford computational efficiency in control across complex solution spaces [186,187]. Of particular relevance is the tradition of using the built environment (urban morphology, road networks, naturalistic paths, and implied movement effort) to impose hierarchies or abstractions that might ease look-up schemes in model databases, balance rendering loads in animation, and scale crowds to large populations [188,189,190,191,192,193,194,195]. There is also a recent trend in supplementing fine-scale detail of gestures and ambulation to characters atop these schemes, and in some instances they are integrated into the control pipeline so that they match with activities such as directional gaze when changing course [196] or expressions that correspond to state cycles [197]. Also noteworthy are attempts to build character motion from artificial intelligence [198]. This is exemplified by Craig Reynolds’s introduction of the Actor/Scripter Animation System, which used what he then called actors (what we would now call agents), as blocks of customizable behavioral code with message-passing [199]. In this case, the characters became self-driven, to some degree.

2.4. Psychology

The last point above, about the utility of modeling the autonomy and intelligence of characters in animated streetscapes, touches on some of the interoperability between animation and psychology, in particular work on environmental cognition [200]. To some extent, environmental cognition turns the sort of attention that urban studies place on the built environment as a placeholder for people’s activity on streetscapes inwards [201], to introspective/egocentric questions [202] of how people use the information that they acquire from streetscape phenomena to engage in tasks and shape their behaviors, and how they might be embodied [203] within that context. Aspects of these emotive-focused theories have appeared in streetscape models in computer graphics where they are used as the basis for belief-desire-intent, perception-action, and personality-lookup schemes to determine motion control [176,204,205], or for endowing characters with synthetic emotions that correspond to their environmental setting [206]. Interestingly, psychologists often make use of virtual reality environments and game engines as proxy settings for their experiments, although these are generally limited to avatar representation of interacting participants for task-performance, rather than representation of the behaviors for characters within those environments [207,208,209]. Much of the perception (awareness) and cognition (understanding) that takes place in streetscapes is well-explored in the psychology of spatial behavior and movement [93,210], and recently this work has extended to cognitive neuroscience [211]. At least anecdotally, aspects of both observational [9] and clinical [5,27] studies seem to have some synergy with computational methods used in streetscape modeling (see Torrens [65] for a discussion).

2.5. Behavioral Geography

Behavioral geography [23] presents something of a bridge between the ego-centric and allocentric attributes of cognition, and the built context views of streetscape dynamics, while also melding environmental and human considerations in a unified theoretical framework. At its most essential, behavioral geography is concerned with how people acquire geographic information introspectively and from their surroundings (via cognition, perception, bias, communication, etc.); how they map that information to their own goals; how they ascribe it to the intent of others around them; and how they use that information to act, react, interact, and transact in space and time (via spatial skills, learning, spatial reasoning, spatial choice, etc.).

Behavioral geographers have traditionally had a keen interest in navigation and way-finding along streetscapes, as this often presents as a sandwich layer between microscopic (individual awareness and motives) and macroscopic (urban geography) scales for phenomena that present in the study of behavioral geography [212,213], and because there is a great deal of individual heterogeneity in how, why, when, and where people use these behaviors relative to the context of the information that they garner. Models of behavioral geography on streetscapes have largely been built using agent-based methods, with careful treatment of way-finding rules for particular types of people, movement tasks, and environmental constructs [214,215,216]. There is considerable breadth in the range of theoretical approaches that are considered when modeling navigation and way-finding in these traditions. For example, models have been used to examine the economy of movement in retail streetscapes relative to particular points of interest [217], way-finding in emergency contexts with unfamiliarity and stress [218], the influence of errors and misjudgment on agents’ way-finding [219], hierarchy in how information is stored and prioritized [220,221,222], and the role of affordances in weighting decisions [223]. Work in formalizing the concepts that these behaviors share as ontology is particularly well-developed [224,225]. Models of mental maps [226,227] that are generated as people reason about their surroundings are also well-explored [228]. Recently, there has been interest in developing a more unified platform that would dock behavioral geography in simulation pipelines that also accommodate methods and interests from physics, animation, and urban studies [65], and that would intertwine sequences of behavioral processes in a unified scheme [229].

2.6. Biology

For a long time, researchers have discussed the comparisons between patterns of movement that form among crowds of people on streets relative to those that form in mobile groups of insects [230] and animals [231,232]. In many ways, the order that can emerge when collections of moving people, animals, and insects course over a landscape is analogous [233], at least when cast through the lens of the statistical regularities that form through correlated walks, fractal geometry of paths traversed, distance-displacement from trip origins, and so on. The notion that groups of animals (as herds, packs, prides, and so on) or insects (as swarms and colonies) might develop coordinated movement either deliberately, by happenstance, or through some interactive synergy [67], and that this coordination might help us better understand how to build streetscape efficiencies in similarly-complex human systems [234], has also attracted considerable investigative interest. Indeed, many modeling platforms (agent-based modeling, in particular) have developed in tandem in the biology, urban studies, and transport communities [235,236,237], at least initially, along these shared lines of inquiry and methodology.

More recently, there has been a tight-coupling of interests between the biological sciences and streetscape science in two key areas. The first relates to gaze and to how people and animals train their (often limited) attention on features of their surroundings that are in flux, and how they extract or estimate signals of change in those surroundings over fleeting moments of space and time within which they might have to prepare a response. In particular, work on visual aspects of streetscape behavior is finding considerable interest in biology (and computational biology), where it has proven useful in relating gaze and the acquisition of environmental information to ideas about how humans and animals use visual flow to process environmental stimuli [68,69]. Thanks in large part to the development of motion capture technology alongside eye-tracking systems, new work is also being performed in the intellectual space between vision and locomotion that connects action and response to issues of neurological pathways [92,93,210,238], timing, and ordering in cognition [239,240,241], to motor impairment as a function of memory loss [242], and other related issues.

Behavioral geography has long been closely intertwined with research in psychology (and vice-versa), but the atomic units of those analyses have often been the individual and their decision units (for example, whether to turn left or right to avoid a collision [243,244], whether to speed-up to resolve a path through space and time relative to an agenda revision [217], which feature of the streetscape to prioritize in attention [245]). More recently, breakthroughs in neuroscience have extended the limits of our understanding to entirely new scales, down to the firing of neurological cells in the brain, and there is growing evidence that these firing patterns might form the basis for how we perceive, measure, store, and organize information that we collect when way-finding and navigating in particular. Much of this work has developed from the examination of how animals structure stimuli relative to the burgeoning mental maps that we have long thought to form in cognition [210], but that we now also think may form in neurological activity [246]. Recent work has gone as far as to suggest that these cells may play a role in encoding experiences that shape movement behavior in animals [247].

3. Dataware for Streetscape Models

A variety of approaches are used to infuse streetscape models with informational ingredients, and these include qualitative and quantitative information from the theoretical themes and explanations discussed above, as well as digital data cast from real-world activity or gleaned from sensors, and representative schemes that mimic how real people build informational awareness as they move on streets. Collectively, we might consider these ingredients as dataware that work together to inform and fuel models: an interlocking mélange of data, data access schemes, and organizational structures that provide the informational scaffold and substrate for streetscape dynamics.

3.1. Acquiring and Generating Peoplescape Data

Data for parameterizing and calibrating components of streetscape models are a critical concern for many applications. The topic of building artificial or life-like built environments for streets and populations of walkers atop them is well-covered in the computer graphics and urban design literature [248,249], and recent developments in this area have, among other things, focused on allying synthetic cities to real-world modeling pipelines that can treat the physics of natural and built dynamics within them [250,251,252], often as modeled characters pulse along their simulated sidewalks and buildings [229,253]. In applied simulation, it is often important to benchmark patterns and processes in run-time relative to real-world analogs [254,255,256,257]. In animation, data are often used to reduce the workload in model-building and simulation and to imbue life-like attributes to otherwise scripted behavior. In each instance, individuality is often a significant concern. Similarly, building model streetscapes that look and behave like their counterparts in reality requires that humans and environments be matched to the considerable heterogeneity and detail that real life affords [20,258].

A variety of methods have developed to acquire streetscape data from the real world and for porting it to models. In animation, there is a long tradition of using the artistic talents of animators to map their ideas about streetscapes to scenes. Animators traditionally used their imaginations and creative skills to hand-script movement and interaction sequences for characters on streetscape scenes based on artists’ intuitive abilities in deciphering subtleties of body language, emotion, choreography, story, and so forth [259]. This tradition continues, for example, in libraries of motion capture and animation sequences [167], that can be invoked in game engines or CGI packages that hold digital facsimiles of streets, often with programmable characters that use finite state machines or scripting languages to accept events in-simulation as triggers for recalling motion data as animation [185,192,194,260,261,262,263]. For fully digital animations, streetscape data are introduced to model representations of the phenomena being simulated via several channels. Within the motion capture community, there exists advanced schemes for fitting motion data at very detailed resolution, accounting for collisions and other physical constraints in simulation, and these may be docked directly with kinematic solvers [162,169]. Recently, new schemes for building walking paths for streetscape phenomena have been applied from computer vision research [264,265,266,267], and these have been developed further as machine-learning models for extracting basic rules from assemblies of path [183] and trajectory data [184].

Similarly, a suite of metrics has been used to extract information from streetscape simulations during run-time for the purpose of benchmarking model performance in scientific applications of computational streetscapes. For example, several metrics have been developed in Geographic Information Science (GIS) for measuring and comparing the shape, scale, and complexity of movement sequences and paths on streetscapes [165,254], and other metrics of relevance to the efficiency of computation and rendering in crowd simulations have been produced in computer graphics research [268]. Many of these metrics are now massively efficient over the huge volumes of data that streetscape simulations often produce [256,269,270,271].

3.2. Big Data

Big data continue to play an increasingly important role as a source of information for streetscape models, and the technology behind big data are also beginning to influence how we build model systems for acquiring and using data in simulations. Big data are “big” with regard to streetscape science in a few key ways. First, many data sources for streetscape dynamics now present vistas that can often cover almost entire populations of streets, and in some instances the data represent populations of walkers spanning several blocks [272,273,274,275]. Data that are “big” in this way allow model-builders (at least theoretically) to move away from sampling small windows of streetscape activity and away from coarse statistical modeling of likely or probable populations of walkers and activities. In place of statistical sampling, researchers are now approaching scenarios in which streetscapes can be “reality-mined” [276] (i.e., the data that are produced are sufficient for understanding the activities being observed at their characteristic spacing and timing). To date, much of this work has been proven on small populations [277], although research in big data analytics of cell-phone signals [278], Wi-Fi positioning [59], and social media messaging [279] is beginning to provide very useful data over very large populations: these include data on the origins and destinations of trips [280,281], activity profiles of districts within cities [282], and several motifs of mobility [283], often in real-time or near-real-time resolution for many thousands of people [278]. During a short time of use in research, these big data sources are bolstering (some would argue that they are supplanting) many traditional “small data” sources. In transportation work, for example, big data sources have almost replaced the necessity for model-builders to construct synthetic trips from spatial interaction models and expensive longitudinal survey data [284]. However, growing concern over the quality of many big data sources (relative to organized survey data in particular) is growing [285].

3.3. Data Models

The idea of encapsulating digital and mathematical representations of humans, built matter, the relationships between them, and the processes that animate them as data models is common across many implementations of streetscape simulation. The built components of streetscapes perhaps naturally follow an object-based model with classes, objects, and methods organized in hierarchies that conform to architectural or urban design concepts [249]. Grammar-based rules have long been used in computer graphics to procedurally generate built form, and shape grammars in particular have been usefully employed as methods for street objects [248]. Objects have also been used to represent humans on streetscapes and some innovative work has been done in swapping-out different representative schemes on objects to speed rendering when simulations are congested with high-resolution polygons [286,287], and in building toolkits that permit assembly of characters with different properties and abilities [288,289]. Grammars have also been used for the generation of human actions in streetscape scenes [175]. For example, object models have been developed to seed simulated streetscape scenes with context-aware synthetic crowds as “patches” that function as block-objects in simulation and that guide the activity of the digital humans that traverse them [191,290,291].

Roads and sidewalks have been well-represented in models by graph structures that allow for encoding of waypoints, landmarks, activity sources and sinks, obstacles, and so on as vertices, and the weighted connectivity between them as edges [292]. This approach is widely used in path-planning for streetscape dynamics [293]. Graphs have also been used to represent mobile humans as vertices, and the relationships between them as polygons bounded by edges (for example, as adaptive and probabilistic road-maps [294]): these graphs can be useful in building real-time path-planning for streetscape models, in particular [295,296,297].

Automata, in various forms, are also widely used in streetscape modeling. For example, automata states can be used to encode attributes of built form and humans in models. Automata may also flexibly leverage neighborhoods to facilitate the exchange of information, and transition rules to ascribe processes to entities in models [298]. Automata may be implemented in different forms in streetscape models. Finite state machines [299] often use events as triggers for transition between states [300,301]. Agent automata [302,303] can ascribe human agency to agent-characters [302,304,305] or functional agency to urban forms [306]. Multi-agent systems [307] are useful when assigning different motives and behaviors to classes of agents based on their role in streetscape dynamics [308,309]. Cellular automata [310] are commonly employed to tessellate streets into automata units and transition the flow of individuals or crowds across the lattice that they form in streetscape models [138,311,312]. Geographic automata [313] make use of GIS to flexibly specify movement and neighborhood relations [65] across the built and human substrate of synthetic streetscape phenomena.

While the choice of data model used in streetscape simulations can invoke a panoply of motivations, there are situations in which the data model matches particular behaviors or processes with real-world analogs from real streets and real crowds. For example, the hierarchy implied in many finite-state automata approaches can be used to compartmentalize different movement actions or informational acquisition strategies for decision-making, which has significant basis in behavioral geography and neuroscience [222,314,315]. There is also recent evidence that the human brain stores spatial information in place-based cells as metric data [5,27,93,210], which in essence store acquired geographic information in the brain as states that can be relied uopn for processing decisions.

Recently, there has been a movement to build significant geographic realism into the virtual environments that play host to streetscape simulations. Much of this work is focused on building data models with one-to-one representative fidelity to the real-world systems that they are seeking to model. The development of data models that interact to form “virtual geographic environments” (VGEs), in particular, is of relevance in this regard [252,316]. VGEs serve as immersive settings, often built as virtual realities, but with the added distinction of featuring well-referenced spaces and catalogs of objects and entities that match real-world settings [250]. Lately, research in VGE development has also been pursuing the inclusion of life-like process models that handle physical and environmental forces [317,318], as well as human-based phenomena on streetscapes [229,319].

4. Prevailing Methods for Streetscape Simulation

There is a vast array of processes and phenomena that interplay to produce streetscape dynamics. And so, understandably, a varied range of methodologies have been explored to simulate streetscapes in (and using) computers. Many of these methodologies rely on or invoke computing as the algorithmic engine for model processes, as artificial intelligence for mimicking streetscape behavior, or as the system substrate for organizing streetscape dynamics in run-time. In what follows, I will discuss the predominant types of streetscape models that are being developed with computing, including path-planning, navigation and way-finding, timing, vision, steering, kinematics and effort, and collision detection and avoidance.

4.1. Path-Planning

While progressing along real streets, walkers often structure their traversal paths deliberatively relative to the social and environmental surroundings [217] using a variety of heuristics. Commonly, pedestrians opt to pursue a shortest course between activity sources and sinks [24,99]. Walkers may also pursue a given (space-time) agenda for their activity [320,321], meanwhile preserving straight paths and eschewing diversions [20,322]. There is also some evidence that people continually update and reassess this information while walking [323].

These path-planning behaviors are well-handled in computer models using graph structures (which I discussed above), and are well-accommodated through the application of path-planning heuristics [293,324]. For example, models of streetscape dynamics are often built with characters that plan paths to provide the least physical effort and preserve straight-line viewsheds [179,325]. (The viewshed idea has some basis in urban design, as I already mentioned.) In other applications, blended heuristics have been employed, trading off multiple objectives [326] or introducing adaptive strategies that run in real-time [297]. Planning in three-dimensions, within complex multi-platform streetscapes [327] or over varied terrain [328] requires special consideration. Similarly, traversing very large graphs (representing large networks of streets or whole downtown areas, for example) often requires some careful treatment in models, particularly regarding computational efficiency [329], as the look-up requirements can grow large if not well-clustered [330]. Path-planning in unknown settings presents a recent twist on the path-planning problem [331], in which uncertainty encountered in streetscape dynamics must also be considered. Path-planning can also be nested with motion-planning schemes that also account for feasibility of kinematic motion and collision detection along the way [332].

4.2. Navigation and Way-Finding

How people engage in way-finding [24,25,110,222,225,333,334,335,336,337,338,339,340,341] and navigation [321,342,343,344] is a critical consideration when seeking to understand and to model many streetscape phenomena, as it is one of the central factors that determines how people interact with the built environment. There is a long tradition of survey and observational work that has examined and measured the role of an array of factors that govern way-finding, particularly in addressing individual differences in way-finding and navigation routines [345,346,347]. Computational streetscape models have been employed to explore, in simulation, several themes around the agency invoked in navigation and way-finding dynamics, for example, what impact the complexity of tasks might have on those dynamics [348], as well as the role that distance and direction heuristics might play in how navigation and way-finding behaviors are actuated on streetscapes [336,338,349]. The topic of how people way-find in unfamiliar settings [215,335] and over large spaces [341] is also well-pursued in simulation.

How people use information while navigating and way-finding is of broader interest across many practical domains, and questions about pedestrians’ information use on streetscapes are well-suited to simulation where the flow of information itself can be studied and modeled, including the role of dynamic updating during way-finding [323], the influence of biased perception [110], the nature of feature recognition and significance of salience while way-finding [340,343,344,350], and the accuracy of human way-finding under various informational criteria [219]. There has also been considerable investigation into navigation and way-finding for particular environmental contexts, and these environmental factors can often be built into models and evaluated in simulations. There has, for example, been particular interest paid to modeling shopping malls, where the built setting is often designed to specifically invoke way-finding of particular kinds [351] and for which models can be used as what-if tools for examining customer flow, marketing the impact of anchor stores, and evaluating the relative positioning of facilities.

There are a few recurring methods by which aspects of navigation and way-finding are translated from our observational and theoretical understanding into streetscape models. Traditionally, modeling has focused on (directly) providing navigation information to characters in streetscape models via navigation graphs that they traverse, for example [193,297,328,352,353,354,355,356]. Navigation graphs, as they are used in agent-based models, have much in common with the path-planning schemes that we discussed earlier, but may also add aspects of the decision-making process (time-to-task, landmark salience, visibility, activity schedule) as actionable information on graph vertices or edges. In this way, they mimic some of the creative processes that real people engage in when using ambient geographic information to build their navigation behaviors. Additionally, the navigation graphs that are used tend to be locally-focused in the view of the streetscape that they afford, compared to trip graphs, which invariably need to provide a more holistic vantage.

There is also a strong tradition from robot motion planning in formulating navigation and way-finding in models, in which characters use their (synthetic) senses to figure out where they are in a model streetscape or space, focusing in particular on goals, way-points, and distances as metrics by which they prioritize their movement. Variants of this are used for machine-learning approaches that employ additional components of the navigation decision (angles of approach and departure, bearing, facing choices, shapes of landmarks, etc.) to extract movement rules from path data of real movement [184].

In some instances, characters in streetscape models are tasked with composing their own maps of environments that they sense and traverse. For cognitive agents, for example, the model built surroundings can be used to create “mental maps” (often realized in spatial databases of the information acquired while mobile), which simulated characters can use as information for navigation strategies. In some instances, these can be developed on ontological schemes [357]. In others, characters’ agency is used to produce life-like cognition and awareness [229].

4.3. Timing

In real situations, timing is a critical factor in the decision-making and interactions that shape streetscape activity. In simulation, timing is equally important in conveying that sense of reality and in faithfully representing real processes, but timing must also be addressed in computation, where often huge amounts of information and processing need to be exchanged, queued, updated, and reconciled, commonly within the rhythms of diverse data models and data streams.

The movement of people is strongly influenced by their perception of the space and time available for activities [358]. In reality, people use a variety of time-scales for action, reaction, and interaction [359]. We know that people in the real world organize their activity around space-time anchors [320,360]: for example, they may need to leave a place at a given time and, based on those constraints, they will allot space and time to reach a destination or chain activities [361]. Some sources and sinks for activity might be planned over the course of a day (decisions about when to go to work, when to schedule recreation and errands, when to return home to rest). Others, such as when to avoid collisions on a street or road, may take place on the order of seconds [73,239]. Additionally, people may have some pliability in the rigidity of their timing in some instances (when to go grocery shopping, for example), but not in others (when to begin work).

Timing is handled with a rather large variety of techniques in streetscape models. Some models rely on a universal clock that artificially freezes all entities in simulation and updates their information between each tick in the clock’s tempo. This is particularly common in automata-based schemes, although pure automata should be parallel in their update and this causes complications with access to data among interacting automata when serial and spatial autocorrelation effects play out between state updates [362]. Other model schemes allow characters a short look-up time to collect information and resolve the conditions that the information presents to them [363]. (This conceptualization of look-up time actually fits well with the emerging idea that human brains may simulate the physics of interaction and timing in the visual scenes they detect before decisions are made [364]) Still other schemes decouple timing, segmenting action between long-run tasks such as trip-planning, and short-term needs such as collision avoidance [65]. Approaches developed in urban studies typically follow an activity space approach, whereby models begin a simulation by releasing modeled people in particular places and times to engage in various behaviors appropriate to that time geography [365,366,367]. Some really innovative work has been done to accomplish this for historical streetscapes [368,369], to put characters in the right place, time, and context for ancient Roman environments, for example [370].

4.4. Vision

People’s perception of their surroundings is a critical component of their behavior on streetscapes and much of this perception is visual in origin and in form [93,129]. For example, people use their sight to gauge distances [93] and to determine relationships between objects [210]. Pedestrians, in particular, may employ a fleeting “look-ahead” packet of space and time to check for collisions [93], which they will then model internally in their mind, resolve, and prioritize. Aspects of the weighting that occurs in estimating collision potential involves spacing and timing of movement [371], and the affordances that individual people associate with the things that they see around them [20], including the information conveyed by gait and body language [372]. Much of this information is inaccurate in human sensing, however, as are people’s estimates of the relative influence of conflicting factors [240], and inaccuracies may grow more cumbersome as we age [373,374], or when we encounter things that are unfamiliar [375,376]. Visual information is also often highly dependent upon the state and position of the person doing the looking, and in this regard gaze has been shown to be of particular importance [238,371,377]; indeed, this component of vision is now readily testable using eye tracking technology and has therefore received considerable attention [9].

Although we know quite a lot about how people focus their attention when walking and judging collisions, infusing that knowledge into models in a realistic way is challenging. The most common approach relies on cellular automata, using fixed, symmetric raster neighborhoods as a very simplified proxy for vision [138,365]. This has some serious limitations and agent-cells in such simulations can often behave unrealistically, with the ability to see behind them as they walk, for example [254]. The information that is communicated via the neighborhood search in pedestrian-based automata models tends also to be relatively straightforward, including details such as the presence of agents in the neighborhood filter [378]. The approach is, however, reasonably efficient in computation as it is well-handled natively in the cellular automata scheme [379]. It can also be varied for cellular-like functionality when, for example, a sector of vision is used as the neighborhood filter representing the field of view [65], following pioneering work by Terzopoulous et al. [380]. More recently, Ondřej et al. [381] introduced a quite novel adaptation, using overlapping filters that capture different vision perspectives.

Using ray-casting as a surrogate for vision is also popular in streetscape models, particularly for collision detection. In these instances, agents cast a ray at a fixed distance in the direction of travel (or a set of ray-like whiskers) [382]. In other instances, rays may be cast from different parts of the agent’s body to control collisions of different types, e.g., rays from the knees for collisions with fixed objects, and rays from the eyes to determine attentive collisions [229]. If the ray intersects with an object, the identity and distance to the object is returned. This can also be efficient, as the algorithms for ray-casting are well-known and well-optimized, in particular on graphics processing hardware [383,384]. The number, direction, and length of the rays can also be focused using scene culling [385], masking [179], proactive vision that estimates likely trajectories [178,181], or equivalents [386].

Much work in architecture and urban design has yielded schemes for vision that account for the influence that street geometry [10,387,388] and layout [116] have in drawing attention to features of built environments [103]. Invariably, this invokes geometry approaches that rely on central lines of tendency in the arrangement of urban morphology [130,389]. This work is in some senses adapted from the viewshed analysis of terrain, where afforded views can be interpreted by “isovists” (lines of equal vista) [125,390]. These schemes can also assume computational efficiency, particularly when based on computational geometry [391].

Work in computer vision has long sought to directly model the neural and cognitive actions involved in vision [392,393] and this work has appeared in “animat” (animal-like robots) research [394], whereby artificial intelligence of animals is treated directly in the model, and often tied to their movement [380]. Some of this work makes use of advances in dynamic camera placement in 3D scenes [395] to extend the idea of “virtual vision” [396] or “embodied vision” [397] to “human animats” [398,399].

4.5. Steering to Avoid and Avail of Interaction

Walkers on streetscapes often couple their vision of things in their surroundings with decisions about whether and how to act upon those things, react to them, or interact or transact with them. In its most straightforward treatment in streetscape models, these factors are usually represented and coupled using steering, collision detection, and avoidance.

A holistic perspective on movement atop streetscapes might reveal that walkers rely on a shortest-path search (of some form or another) to budget the use of space and time when walking [100], but the actual space-time path that they pursue through streets, opportunity spaces, crowds, etc. is refined using a combination of this global information and the conditions that they encounter locally [110]. These local negotiations, whether deliberated internally or worked-out with inter-personal choreography, are mediated by what people see (the vision described above), but also by what gets in their way [371], and by their spatio-cognitive relationships to those objects as people judge and weight them on their own terms [400]. Reciprocity between potential colliders can also play a role in determining how, when, and by how much people will steer to avoid a collision [190]. Olivier et al. [96] have shown that roles may be important in determining this over small bundles of space and time, and that these take on signaling schemes that intertwine with the positional data that are briefly exchanged and estimated as a person determines steering. Reciprocity may assume varying form, depending on different strategies, spacing, and timing when steering for animate and inanimate colliders [240].

Following Reynolds’ [363,401] pioneering demonstration of simple steering heuristics for agent-based mobile entities in computer models, much work has been done to design “steering behaviors” for streetscape models. Generally, steering behaviors couple small motion (really angular and linear acceleration) decisions relative to known or estimated positional and rotation data that characters or agents in streetscape models compute from information that they poll within the model environment; those computational “decisions” on acquired information are then composited into larger model maneuvers that mimic real-life behavior, such as avoiding, fleeing, evading, pursuing, following, milling, etc. [402]. Model steering behaviors can be brokered by individual entities or negotiated among groups [181,229,268,354,363,381,402,403].

In real life, walkers on streetscapes often consult their own personal information, preferences, and abilities when calculating steering. Recently, streetscape models have begun to treat those behaviors in simulation more directly. Guy et al. [189], for example, have developed a sophisticated model of crowd movement that incorporates the effort involved in walking as a determining heuristic for path-planning. Similarly, Basili et al. [92] have incorporated the smoothness in movement as a control scheme. Much work remains to build these faculties into models and this will likely require close engagement with theory for the portions of the science and knowledge base that remain elusive. For example, walker-estimated time to collision has long featured in models of steering [241], but the cognitive drivers behind even basic movement choreography remain unresolved, e.g., tendency to move to the right or left when stepping aside [243,244], clockwise and counter-clockwise steering [404], or side of collision when bumping into things [405]. From the literature, the lack of consensus would seem at least partially to result from the myriad scales of movement information that are involved and that need to be reconciled when engaged in locomotion. This presents as an area of theory that could leverage streetscape models as a tool to think with.

4.6. Kinematics and Effort

Pedestrians’ determination of the paths that they consider as available is often influenced by path-planning, vision, and steering, each of which must be reconciled when modeled, and then be actuated for virtual walkers to convert it into a guide for their locomotion. In simulation run-time, locomotion is usually achieved using kinematic- and collision-based motion planning and motion control [332], often matched to skeletons and to range of motion parameters derived from motion capture of real people moving in real situations. The current state-of-the-art in handling locomotion in model streetscapes most commonly sees locomotion realized through the inverse kinematic (IK) [406] and forward kinematic (FK) [407] solution of the task of mapping a character’s four-dimensional skeletal movement (positioning and rotation of joints in three dimensions of space and one of time) to a three-dimensional track of their progression through a scene (usually through two dimensions of space and one of time). The IK/FK solver thus articulates characters’ skeletons along the path provided by the higher-level movement schemes and this is accomplished in computation by working out where various effectors on the skeleton should fall within that bundle, and what the subsequent displacement should be for the rest of the joints in the body [161,406,407,408,409,410].

4.7. Collision Detection and Avoidance

Collision detection is usually handled in two general ways in streetscape models. First, in most settings, simulated walkers are programmed to avoid colliding with each other and with static objects in their near (close, but not too close) surroundings. This is generally handled using steering maneuvers, e.g., avoidance routines that steer a walker free from a wall, or vector-estimation that plots future trajectories relative to mobile objects and returns a collision-free point that is closest to the planned path [190]. Second, via IK/FK, walkers endeavor to not collide with themselves (by avoiding colliding limbs, for example) and with other things that come into (very) close contact. Here, various schemes are used to “back-out” collisions produced during kinematics to collision-free points and these may make use of motion-planning graphs, for example [159]. The use of approximations of pedestrian bounding volumes (as rigid bodies, using cylinders or other neat geometries, or by testing for the collision of joint-mounted spheres or boxes) is often used as a quick-test evaluation of whether a collision is likely [411,412,413]. If a collision is estimated as a result of those routines, more in-depth testing may be performed, but only if and when necessary, to reduce computation. This is particularly relevant when real-time motion is desired [414]. Collision detection and resolution are areas in which the computational prowess of streetscape models can be incredibly useful because of their ability to efficiently traverse the solution-spaces of potential collisions, and to resolve them relative to many impinging factors.

5. Conclusions

In this paper, I have endeavored to present an overview of the many motivations, justifications, techniques, and applications for computing streetscape dynamics, and my focus has largely been upon ways that computation can assist in modeling streetscapes. The range of potential topics that one might cover in such a review is broad and I have perhaps merely touched the surface of what is a very large community of interests, with the intention of highlighting the diverse range of problem-sets that are examined when modeling streetscapes, the bag of tools and methods that have developed within various domains and across domains, and growing points of fusion across disciplines, particularly at the boundaries between visually realistic representations of streets and people in simulation and behaviorally authentic theoretical underpinnings that might map representative detail or nuance to real-world details. Of course, much of the fusion across different vistas on streetscape models is now vanishing as computing for streetscape applications and services has begun to augment the physical reality of behaving in and around streetscapes with the virtuality that computing can provide through access to data, algorithms, and hardware [7,415].

Some recent developments perhaps point to growing potential synergy in our ability to examine streetscape phenomena from the traditionally diverse vantages of the computing sciences, the visual and cinematic arts, and the socio-behavioral sciences. These include breakthroughs in domain science, in sensing technologies, and in data science. While the domain science pertaining to streetscape dynamics continues to advance all the time, breakthroughs in geography, public health, kinesiology, and psychology have been particularly noteworthy in addressing the “meso-scale” of streetscape science, as the dynamics that occur between the micro-scale of individual movement and the interaction atop the built-human interface, and the macro-scale of broader activity patterns and rhythms of life that land-use and urban geography afford. It is at the meso-scale that much of the future investigative work on streetscape phenomena and patterns may well focus, and models need to be able to support innovative ways to facilitate that exploration.

The increased use of high-resolution and high-precision sensing technologies, such as motion capture and high-speed video, as well as related analytics for making sense of the data that they yield, have allowed communities of scholars in kinesiology to study new facets of human and streetscape dynamics that were previously out of reach. Moreover, developments in sensing technologies have meant that researchers can now measure (rather than simply monitoring) significant attributes of people’s behavior on streetscapes, yielding empirical data that can inform theory and feed computer models and analyses. These developments, in data and understanding, have led to the growing appreciation for many components of movement and interaction between people and things, especially at the micro-scale of streetscape phenomena, including locomotion, body language, and collision detection and avoidance. Moreover, much measurement work now being done in kinesiology yields data that can be readily consumed by computer models, as weights, timing, action sequences, and geometry. These developments are useful, in particular, in explaining (empirically) how and why people’s bodies generate the movement and timing that determine up-scale behaviors on streetscapes, such as steering and navigation.

In psychology, the proliferation of eye-tracking technologies that can measure attention, focus, gaze, and how the eyes saccade relative to things of interest, has opened-up new insight into how people perceive space, place, time, social relationships, collisions, attractions, and other parameters of interaction. Moreover, eye-tracking technologies can now produce fine-grain measurements of how people acquire visual and positional information, how they weight affordances between people, what holds their interest and for how long, and so on. This work is yielding promising insight into spatial cognition on streetscapes, in particular. Other work in psychology and in neuroimaging with functional magnetic resonance imaging (fMRI) is beginning to provide some indication of the connection between action, cognition, perception, and neurophysiology, indicating which parts of the brain activate as people think through the activities they might pursue while in a streetscape. In parallel, theoretical and observational work in neuroscience is providing supporting evidence that people may have developed dedicated brain structures to enable things like wayfinding and navigation and the physics of collision [5,27].

In the examples from kinesiology and psychology, we see the domain science for streetscape dynamics being extended to a finer and finer scale, down to the biological interplay of information, thoughts, and actions. Emerging intersections between the equipment used to port data from real pedestrian locomotion into computer graphics has given way to some advanced insight into fine-scale dynamics of streetscape phenomena, including how people react to and move through collisions that they encounter and how they respond to risks and hazards when crossing roads. As in psychology, work in eye-tracking has also started to build biological understanding of how people train their attention and gaze on streetscapes, and in many cases eye tracking using virtual streetscapes is supplying useful data that can feed into biological models and analyses of synthetic organisms. There is broad potential that this work might soon extend to even finer scales, as developments in neuroscience are building a science of place-making, navigation, and movement at the scale of brain cells. Potential links between streetscape science and these components of biology research may fuse with even more associations as virtual worlds become portable and form as part of augmented reality systems. One could imagine, for example, testing what-if scenarios of movement along synthetic streetscapes using augmented reality systems, while also imaging brain activity using portable fMRI equipment.

At the other ranged scale of streetscape phenomena, work in geography and urban informatics is helping us to better understand the drivers of coarser dynamics of streetscapes, particularly how and why the rhythms and flows of people along streets relate to realities and perceptions of space and place. Traditionally, modelers in particular ascribed much of the coarse scale movement of crowds along streetscapes to dynamics borrowed from physics of material flow. However, the advent of location-enabled sensing technologies has encouraged a broader preoccupation with collecting large data-sets of human activity patterns and location traces. Initially, this work was developed for studies in public health, particularly around issues of accessibility in food landscapes, activity and physical well-being [416], and the relationship between activity, place, and social networks. Work in geography, particularly in time geography, has also developed atop the same technologies, to examine issues of behavioral geography and social geography. As with kinesiology and psychology, much of this work is measurement-based: public health studies now routinely yield activity diary data, energy expenditure data, social interactions, and location data; while work in geography has produced detailed trajectory data for trips and spatial interactions. In some instances, these data have been organized around large survey instruments that are carried out over large areas, long periods of time, and broad populations of participants, with the result that silos of real measurement data for the rhythms and motifs of movement and participation in streetscape phenomena are now being developed. Recently, many in the community have begun to collate their work under the banner of urban informatics in a deliberate effort to fashion common data instruments and related data science for harnessing the information cast by an array of urban sensors and infrastructure, wearable and mobile devices, and environmental observation platforms. Many transactional data-sets (taxi pick-up and drop-off locations, GPS traces, locations of mobile calls and cell tower traffic, transit system tap-in and tap-out data, etc.) are now being docked to common data platforms and are being used for fused analyses of patterns of urban activity and motifs of behavior that are incredibly interesting in framing streetscape dynamics. Again, these data are being used to fit models of streetscape dynamics. More broadly, these sorts of investigations are being hosted among communities of domain scientists that are working from the bottom-up and the top-down to build authentic and meaningful explanatory pathways for the dynamics that take shape within and on streetscapes (and the phenomena they foster), and much of this work is informing, and is informed by, computation.

In tandem, sensing technologies—whether based on computer vision or location-awareness—continue to shrink in size and expand in functionality at the same time that citizens become more adept at using them to explore the streetscapes and environments that they participate in, move through, and interact with. Some developments in citizen sensing are of particular relevance for future exploration of streetscape phenomena via computing because they provide unprecedented baseline and ground truth data that can be used to anchor computing (and some of the vagaries of the information that machines can cast) to reality [417]. These developments include citizen mapping projects, such as the OpenStreetMap initiative started by Steve Coast, which we might term active citizen sensing because people opt to contribute to them. A range of passive citizen sensing efforts have also emerged and the activities that they support (and that often operate “on” streetscapes) often yield large silos of data that are produced through citizen participation in various services such as check-in systems, location-based games (Pokémon Go [2] being the most recent and popular example), tagging services, and socialization platforms. Other systems produce telemetry data from personal devices such as fitness trackers, watches, and phones that are valuable in building sensed silos of movement and interaction data for human-urban interactions in streetscape dynamics with relevance at scales from the body to the city.

The built context for streetscapes is also being transformed by computing, and in many cases built environments and objects now function as significant components in smart cities [418,419], i.e., computing-mediated systems of urban processes, services, functions, and phenomena. Examples of smart streetscapes abound. Closed circuit television (CCTV) cameras are perhaps particularly relevant as they are now routinely deployed to streets and they tirelessly image and tag the activity that unfolds on streetscapes using machine vision. In many cases, CCTV cameras are designed to work in unison with software such that they can be used to proactively scan for particular patterns of movement within crowds and urban scenes, or to track entities and events. Some interfaces between streets and buildings are also used to count and classify people as they move through streets and move past sensors. For example, many urban transit systems are “smart” in this way, making use of near-field communications (NFC) technologies that allow for riders to be authenticated as paying customers on the system, automatically at the turnstile. Similarly, many entryways to buildings rely on NFC or radio frequency identity (RFID) tags to authenticate people as eligible to pass from streets to premises or particular spaces. In both cases—NFC and RFID—data are cast at the transaction, and over even small swaths of space and time the technologies can generate a rich and dynamic tapestry of movement, activity, identity, and purpose as people and things course by through streetscape phenomena. Cars are also being included in smart city systems and in some cases their sensing and computing abilities are being trained on streetscapes, particularly on pedestrians. Advanced Driving Assistance Systems (ADASs), for example, are now routinely designed to sense for the presence of pedestrians in crosswalks.

The evolving mass of big data that many of our research (and everyday life) activities now orbit around is also of broad utility in explaining streetscapes, while advances in big data analytics are making it easier to bring those data to bear in exploring streetscape phenomena. In particular, basemaps for street geometry and the points of attraction and transit on them are now widely available, in high spatial resolution. These maps of streetscapes are usually quite accurate and are updated quite frequently thanks, in large part, to crowdsourcing their cartography to citizen volunteers and automated map-making procedures that are capable of reconciling location-based data from diverse sources as the ground truth for maps. Detailed street and area of interest data are now available, broadly speaking, across the world and for the first time in places such as informal settlements. This provides the basis for organizing other forms of big data and for running models of streetscape phenomena atop an empirical substrate. At the same time, a set of tools are emerging for crawling big data, visualizing them, and adding value to them are coalescing into communal databases, which may be used to parameterize, calibrate, validate, and verify models. In some instances, big data have been used in conjunction with machine-learning to fill-in gaps that theory and models have left open in our understanding of streetscape phenomena. In other examples, machine learning on big data has proven useful in inferring location, activity, demography, and even socioeconomics within the signatures that streetscape phenomena yield. Now, data science is being increasingly trained on fusion across big data. For example, network science and GIS are leading the way in providing structure across unstructured big data that streetscapes often cast in the course of their everyday dynamics [420,421,422]. Indeed, these types of approaches may be what the community needs moving forward in an era of computational social science that is beginning to fuse a wealth of qualitative and observational work [21] with near-ubiquitous sensing [423] and big data capabilities [424,425].

Looking to the future, and beyond technical details or domain-specific demands, one grand challenge presents in computing streetscapes: how to unify data, knowledge, and modeling capabilities across scales. As is usually the case, different interest-sets in studying streetscape phenomena have different needs for and uses of computing. Traditionally, the majority of modeling schemes for animating streetscape dynamics, for example (for animating them as graphics, or for animating through their behavioral and process dynamics as model systems), have focused on high-level control (such as path-planning and scheduling) or on low-level control (such as kinematics and physics). The challenge of populating the dynamics in-between the high- and low-level phenomena that govern streetscape phenomena has been long-standing. As Edward Catmull (now president of Walt Disney Animation Studios and Pixar Animation Studios) described in his 1978 paper on computer-enabled animation of characters, “At the low end, the action is very simple. Movement is more indicated than actually performed.” [141] (p. 348). What might best work in between the phenomena is something of a terra nebulosus. This in-between layer is an area, perhaps, in which the socio-behavioral sciences and arts could make valuable contributions, by providing theoretical and computational synergy between the upper- and lower-realms of movement dynamics in animation and research simulation, and behavioral explanation and authenticity in the data models, parameters, heuristics, and algorithms that form the computational substrate for streetscape simulations.