Detecting Change between Urban Road Environments along a Route Based on Static Road Object Occurrences

Featured Application

A road environment-type (RET) detection function could improve the road awareness of inexperienced car drivers, especially in urban areas, and by doing so, it could slightly raise the urban traffic safety. A pragmatic implementation could make use of static road object data, e.g., traffic sign (TS) data, that is already collected and available on-board. It could rely on the TS recognition function offered by advanced driver assistance systems (ADAS). Furthermore, apart from its primary function, the RET detection system could provide reciprocal information—with respect to the current RET—for various ADAS and autonomous driving (AD) computations and subsystems. Making use of such reciprocal information could speed up the ADAS/AD computations, and render their results more accurate and more reliable, e.g., via introducing parameter constraints and marking regions-of-interest.

Abstract

For over a decade, urban road environment detection has been a target of intensive research. The topic is relevant for the design and implementation of advanced driver assistance systems. Typically, embedded systems are deployed in these for the operation. The environments can be categorized into road environment-types. Abrupt transitions between these pose a traffic safety risk. Road environment-type transitions along a route manifest themselves also in changes in the distribution of traffic signs and other road objects. Can the placement and the detection of traffic signs be modelled jointly with an easy-to-handle stochastic point process, e.g., an inhomogeneous marked Poisson process? Does this model lend itself for real-time application, e.g., via analysis of a log generated by a traffic sign detection and recognition system? How can the chosen change detector help in mitigating the traffic safety risk? A change detection method frequently used for Poisson processes is the cumulative sum (CUSUM) method. Herein, this method is tailored to the specific stochastic model and tested on realistic logs. The use of several change detectors is also considered. Results indicate that a traffic sign-based road environment-type change detection is feasible, though it is not suitable for an immediate intervention.

1. Introduction

Despite of the on-going research on self-explaining road layouts and designs [1,2], and on the computerized recognition methods of such designs and layouts, e.g., on methods that apply artificial intelligence methodology [3], setting up traffic signs (TSs) along the roads and traffic lights in road junctions and near pedestrian crossings by the transport authorities still remains a customary measure for reducing traffic safety risks in urban areas [4]. Clearly, there are other viable alternative measures, as well as supplementary ones for the purpose. These include—among many others—the installation of speed reduction markings onto the road-surface [5] and the installation of vehicle- to-infrastructure (V2I) communication facilities, e.g., to succor the TS recognition (TSR) function offered by advanced driver assistance systems (ADAS) [6] and self-driving cars [7]. In a wider sense, V2I communication succors the road, traffic and vehicle data gathering, fusion, and dissemination, and through these data processes, it is expected to have a significant beneficial impact on traffic safety [8]. More specifically, V2I communication can be used for raising the road-awareness of car-drivers, as well as that of the intelligent and the self-driving road vehicles. Furthermore, it can be used for providing the human drivers and the smart vehicular systems with current traffic information with respect to the region, town, and area, on the one hand, and with some very specific dynamic information on individual vehicles in the vicinity, on the other [9].

When speaking about raising road awareness of drivers, one is obliged to speak about the Global Navigation Satellite System (GNSS), a system that is used by masses of people around the world. According to [10], the GNSS devices per capita averaged out at 0.8 across the countries of world in 2019. The GNSS is used with wide variety of devices running map-based applications, a significant percentage of these devices are installed on-board cars. The brief history of the navigational systems and their respective precisions are presented in [11]. The paper provides a fresh outlook on the navigational needs of and the available navigational solutions for autonomous vehicles and systems. As it often happens to popular services, devices, and applications, threats against these surface from time to time. Such threats have surfaced also against the GNSS service [12]. Although the number of successful navigational spoofing attacks is still negligible, the navigational signal deteriorations due to other—i.e., non-hostile—factors are clearly not. For instance, the signal reception is often brought down, or even blocked by the high-rise buildings in densely built urban areas. Some examples in this context are presented in [13].

The speed reduction measures implemented in urban areas are motivated by the traffic safety concerns associated with the intense road traffic and the limited space available there for the driving maneuvers [14]. While driving, and particularly while driving in urban areas, drivers need to perform numerous mental and control tasks—ranging from those associated with limb-movement to those required for complex driving maneuver planning and execution—within stringent time and spatial constraints and with high reliability [15]. Furthermore, these tasks must be performed in presence of disturbances, such as unfavorable lighting, adverse weather, and traffic conditions [16]. In addition, the older age of the driver may contribute to the perceived difficulty of these tasks [17].

A system, which pays attention to the driver’s activity within the car and also to aspects of the urban road environment, was developed as part of the Urban Intelligent Assist Research Initiative some years ago [18], and since then, other systems with similar, or enhanced capabilities followed suit [19,20]. The effect of driving experience on drivers’ adaptation to road environment complexity—a notion closely related to that of the road environment type (RET) used herein—in urban areas was investigated in a simulation study [21]. The findings of the study underline the need for an automatic RET detection function, and indicate that such a function is particularly useful for car-drivers lacking prolonged driving experience, and also for older drivers.

Several algorithmic approaches and sensor arrangements were devised, applied, and tested for detecting, characterizing, and categorizing urban road environments based on image and/or point cloud data [22,23,24]. In the application considered herein, the urban road environment appears around and sweeps past an ego-car while it is driven in an urban area. The data streams used for the purpose of road environment detection and analysis originate—among others—from one or more camera and one or more light detection and ranging (LiDAR) sensor. In a viable implementation of a road environment detection and classification system that is capable of assisting a car driver while driving, either a comprehensive real-time on-board processing of the respective raw data streams is required (direct processing) or a timely access to and further processing of the data—rendered by some other real-time application/subsystem on-board—on certain distinguishing road objects (ROs) are necessary (indirect processing).

In the above cited papers, the real-time requirements were limited to data synchronization and data collection issues, while the bulk of the processing, e.g., simultaneous localization and mapping (SLAM), and object segmentation, were carried out in a post-processing manner. Nevertheless, a large portion of the processing presented in these papers is real time capable and could be used in direct implementations.

The approach presented in [22] builds and then segments the point cloud originating from a ground-level LiDAR device moving along a given trajectory. The aim of the authors was to produce editable—simplified, but visually still pleasing—object-models that lend themselves for fast visualization. The target areas were the residential urban areas in the United States. These areas are characterized by their low-rise buildings without strong and extensive repetitive patterns. The semantical labeling and various analysis steps follow the mentioned preprocessing steps. Simple models of the individual houses in the area are then created. The basic building blocks of the models are simple, symmetric, and convex geometric blocks. These blocks—together with their spatial arrangement and their connection graph—form an easy-to-handle geometric model of the individual buildings. By aggregating the certain features of the individual buildings for an area (e.g., by computing the average building dimensions and the average distance between nearest buildings), the residential urban road environment can be adequately characterized.

The system presented in [23] extracts the characteristics of individual buildings rather than those of more extensive road environments. Nonetheless, the building characteristics, such as building height and building complexity—again aggregated for a given area, or along a route—together with the spatial densities of the buildings there, are definitive in the respect of the RET.

A multi-sensor and multi-precision data collection campaign is described in [24]. It was a car-based campaign that made use of an array of different environment perception, navigational, and motion sensors. These included four LiDARs, a pair of stereo cameras, a fiber optics gyroscope and encoder sensors for the tires. The data collection trips covered diverse complex urban environments in Korea, with a clear emphasis on those environments, where GPS reception is highly unreliable. The collected data were organized into a publicly accessible dataset that includes the measured ego-car trajectories, the raw and processed point cloud data from the LiDAR sensors, as well as the ego-car trajectories with improved precision computed via SLAM.

Other approaches, e.g., the ones presented in [25,26], rely object-level data as inputs to the urban RET detection function, i.e., they follow an indirect processing approach. In a feasible realization, the raw data streams originate from the very same sensors as in the direct case, but the respective data streams reach the RET detection subsystem only after having been processed and considerably compressed by one or more ADAS subsystem. The resulting data are an object-level description of the road environment, i.e., an RO log. This log serves as an input to the indirect RET detection function.

The on-board data processing described above, as well as other road, traffic, and vehicular data processing carried out in various ADAS subsystems (e.g., lane detection, TSR, detection of nearby vehicles) can also be looked at from, analyzed with respect to, and formulated using a static reference point. Setting up and using a local dynamic map (LDM) [27] could serve these purposes, and provide additional conceptual support for the developers of ADAS functions. LDM is a widely used model for representing, and a standardized technology for integrating static, temporary, and dynamic road, traffic, and vehicular information into a static geographical context by means of a common coordinate reference. Customarily, it has four object layers describing and managing ROs that are subject to change and exhibit dynamics at different time scales. More concretely, these layers store and handle data on permanent static, transient static, transient dynamic, and highly dynamic ROs, respectively. For instance, when framing the static RO-based urban RET detection task in LDM, the ego-car is seen as a highly dynamic RO. A crossroads (CRs) intersection of streets is a permanent static RO in this model. The intersection can be associated with other ROs (e.g., with fixed traffic lights located there, which are transient static ROs). The lanes, lane markings, pedestrian crossings, and the conventional TSs are transient static ROs, while the TSs displayed by variable message sign boards can be classified as transient dynamic ROs. The RETs can be treated as permanent static features of an area or of a sub-network of roads.

One could look at the ROs in an urban settlement and collect and compile their location and categorical data into a map layer, e.g., in the way data contributors of OpenStreetMap maps do with roads, railways, rivers, and various locations of importance [28]. By selecting appropriate subsets of TSs—i.e., TS subsets that are characteristic to certain RETs—various sublayers of the RO layer can be created, displayed, and analyzed. The analysis could include a Delaunay triangulation of the TS locations within a sublayer, and then one could look for dense clusters of triangles in the generated structure. By carrying out similar processing for a number of sublayers, a TS-based RET categorization of the urban area can be created.

By further processing the map-based representation of TSs and other ROs, one could derive other interesting sublayers that relate to seasonal, weekly, or daily validity of the TSs and could derive a sublayer representing weather-related TSs (e.g., TSs applicable for wet, snowy and icy road conditions). For instance, the sublayer representing the within-the-day validity of TSs—indicated by auxiliary signs or time intervals attached to the TSs—should reflect the daily dynamics of traffic source and sink structure of the area [29]. Clearly, the mentioned dynamics are closely related to the RET categorization used herein.

In our view, such sublayers—compiled, e.g., from data gathered in car-based data collection campaigns—could give useful hints to road authorities and administration as to where to place additional TSs and auxiliary signs or remove unnecessary existing ones. Herein, however, we stick to the route-based sampling of the TSs of the urban area, the map-based processing touched upon above will be addressed in further research.

In [25,26], the urban road environments were categorized into three RETs, namely, into downtown (Dt), residential (Res), and industrial/commercial (Ind) areas. The ROs represented in the object-log were the TSs and CRs encountered along the route. In an advantageous implementation foreseen, both the TS and the CR data originate from their respective dedicated ADAS subsystems. While in case of the TS data, the corresponding subsystem, i.e., the TSR ADAS subsystem, is quite common in recent production cars, the CR detection ADAS function is fairly uncommon at this point of time. It is expected though that in the coming years, the LiDAR sensors developed for automotive applications will pave the way for the spread of such an ADAS subsystem.

A good insight in ADAS system architectures, various ADAS subsystems and functions, as well as the respective methods and computations involved is given in [30]. A survey on TSR methods and systems is given in [31], while in [32], a mapping and navigation system developed for large-scale global positioning-denied sites is introduced. The system is capable of detecting CRs, intersections, and other road infrastructure.

The static RO-based urban RET detection approach proposed in [25], and some further approaches make use of a variety of classification and change detection (CD) methods known from the statistical inference literature. In the cited paper, it is presumed that the static ROs in general, and the considered TSs and the CRs, in particular, occur along the route according to an inhomogeneous discrete-variable binomial process. The minimum description length (MDL) methodology is then applied to detect and locate change in the character of the road environment sweeping past the ego-car.

The lane-keep assist ADAS and the lane following autonomous driving (AD) subsystems, which perforce continually identify the current and neighboring lanes, and estimate their widths, as well as the TSR ADAS and AD subsystems, which locate, identify, and track the TSs encountered by the ego-vehicle, are of particular interest in the context of RET detection. First, such ADAS subsystems are already available on-board many production cars, second, the categorical and spatial distribution of TSs, as well as, the lane-widths and the number of lanes—in the current cross-section of the road or in an aggregated form (e.g., average lane-width, average number of lanes)—carry information that can be useful in determining the RET of the given urban area.

It should be emphasized that a timely feedback of the RET information to the above ADAS and AD subsystems could increase their effective processing speed and lower the rate of misclassifications via setting practical parameter constraints for the computations involved. Such constraints could be of geometrical nature and could take the forms of Boolean, probabilistic, and fuzzy regions-of-interest (ROIs), respectively, e.g., within image frames of video sequences [33]. While in case of point clouds, volumes-of-interest, again meant in a Boolean, in a probabilistic, and in a fuzzy way, respectively, could be marked and used [34]. As a further application of such reciprocal information, the characteristic size range of TSs—for a given RET—could be used for validating the detected TS candidates [35].

Similar processing benefit could be gained from the above outlined information feedback in case of other presently not so wide-spread driver assistance functions, such as the CR detection. Furthermore, information on the current RET is also important for suggesting/choosing appropriate vehicle speed and acceleration/deceleration for the ego-car. An embedded testbed architecture for testing functions of self-driving cars was proposed in [36]. It could also facilitate the seamless integration of the static RO-based RET detection function into the intelligent vehicle control systems.

In the following, it will be assumed that TS occurrences are reliably detected and logged by the on-board TSR ADAS subsystem, moreover, this log is passed on to the RO-based—in the following practically TS-based—RET detection system in real-time.

It was our aim to choose, adapt, and validate a mathematically sound CD method that makes provision for and relies on a simple, but realistic stochastic model of the static RO placement and occurrences, in general, and of the TS placement and occurrences, in particular, for the purpose and in the context of detecting transitions between road environments of different character—or more concretely, between road environments of different RETs—in order to assist car drivers, human, and robotic drivers alike, in their driving tasks and activities. The continuous-time inhomogeneous marked Poisson process (IMPP) was identified as a possible stochastic model to work with.

It should be noted, however, that in real life, the static RO placements—including those of TSs and traffic lights—are governed by technical and administrative guidelines [37], from time to time they are subjects of potentially lengthy conciliatory procedures between locals and road administration. The final decisions are therefore taken at different administrative levels. Some aspects of this occasionally complicated process are outlined in [38]. As in [25,26] also herein, the occurrences are considered along routes. These routes are assumed to be random, but they are, in fact, based on intelligent choices made by the drivers.

Results gained via simulation implementing the IMPP model and making use of realistic data indicate that a TS-based RET CD is feasible and can be used for driver assistance, though it is not suitable for initiating an immediate intervention in critical situations. A more varied selection of static ROs—including, e.g., CRs, traffic lights, and pedestrian crossings—would further improve the feasibility of the RET CD. Similar utility and feasibility are expected for the RET detection and identification function computed with several RET change detectors and an artificial neural network (ANN) that merges and mushes together the detected RET transitions.

2. Materials and Methods

2.1. Car-Based Collection of Static Road Object Data from Various Urban Road Environments

A series of car-based static RO data collection trips was carried out in Hungary in 2017. The data were collected from a number of urban areas. Data concerning a richer set—than presented here—of TSs and of some more characteristic ROs was gathered. The TSs and other ROs were recorded manually along the routes—together with the RETs of the given areas—with the help of a dedicated tablet-based Android application, while the trajectory data of the trip was collected automatically by the app in every few seconds, and at the times of the TS and other RO entries [25].

The data collection personnel consisted of two persons: a driver and a data entry assistant. The manual data entry was made easy by the array-like screen design with TSs and RO symbols. In case of parametrized TSs, e.g., speed limits, the standard options (i.e., 10 km/h, 20 km/h, …, 70 km/h) were offered—also in pictorial form—after the general TS type. Specific symbols, i.e., touch-screen keys, were offered for entering the considered RETs, the repeated entries, for the cancelations of the last entry, and for entering verbal comments. The location, time, TS/RO, and RET categorical data were stored in a text-file in a comma separated values (cvs) format. After the trips, the csv files were stored as spreadsheets and were converted to various formats (e.g., kml) for post-processing and visualization.

From the above data collection, the relevant TS rates—along routes in the considered RET areas—were known. The empirical rates for the (“No stopping” (NS)), , (“Parking lot” (PL)), (“Give way” (GW)), and (“Max speed 30 km/h” (SL)) TSs were used herein as a priori estimates of the reference rates for the marked Poisson processes. These rates are given in

Table 1. The empirical traffic sign (TS) rates along a random route for the homogeneous marked Poisson processes describing (downtown) Dt and residential (Res) urban road environments.

TS Type	Abbeviation	Index	Expected Number of Occurrences per km in Dt	Expected Number of Occurrences per km in Res Areas	Natural Logarithm of the Rate-Ratio	Characteristic to
	NS	1	2.00	0.35	1.74	Dt
	PL	2	1.70	0.25	1.92	Dt
	GW	3	0.70	0.80	−0.13	Res
	SL	4	0.20	0.40	−0.69	Res
Any			4.60	1.80		Dt

for routes within Dt and within Res areas.

2.2. Mathematical Models and Methods

As it was mentioned in the Introduction, the continuous-time IMPP stochastic model had been chosen for characterizing the along-the-route placement and occurrence of TSs jointly for the purpose of RET CD in the present study. For a profound treatment of the mathematical theory of Poisson processes, see [39].

In the chosen stochastic approach, the TS data logs are seen as realizations of an IMPP. The CD method used commonly in conjunction with Poisson processes is the cumulative sum (CUSUM) method. Detailed expositions of such methods can be found in [40,41]. By assuming the validity of the IMPP model—at least with respect to the considered RETs and considered TSs—for describing and characterizing TS placements and occurrences along routes within and between urban environments, our task narrowed down to adapt a suitable CUSUM method for the purpose, and validate it with realistic TS data.

It was our intention to adopt and validate a continuous-time variant of the CUSUM method for CD. Furthermore, trade-off is sought between the false alarm rate and the expected detection lag associated with the TS sequences and to provide hints for choosing appropriate thresholds for the change detectors.

The continuous-variable variant of the CUSUM method for CD in IMPP realizations is derived in this subsection. The working of the RET change detectors implementing this method is demonstrated in Section 3.1 and Section 3.2. The change detectors presented therein have been tuned to detect two different RET transitions, namely, Dt to Res and Res to Dt transitions, and are tested with synthetic input sequences.

2.2.1. Modelling TS Occurrences within a Given Urban Road Environment

As a first step, we are going to model the TS placements, occurrences, and detections—along a random route and within a certain urban road environment—jointly as events of a continuous-variable homogeneous marked Poisson process (HMPP).

In the literature dealing with Poisson processes usually the mentioned continuous variable is the time. Although, with the notations used herein, as well as with the verbal expressions describing relations between values, we are going to comply with this “temporal” convention, and it should be emphasized that the path-length—that has been covered by the ego-car—is chosen to be the continuous spatial variable.

Let $\left\{T_n,k_n\right\}$, where $k_n\in \left\{1,2,\dots ,K\right\}$ and $K\in \mathbb{N}$, be a marked Poisson process with counting measures $N^k(\cdot )$. These are defined as $N^k(A)=\#\left\{n:T_n\in A\right\}$, where $A$ is typically an interval. Let the rates associated with the marked Poisson process be ${\lambda }_k$, assuming spatial—and particularly along-the-route—homogeneity, and let the corresponding reference rates of the Poisson process be ${\lambda }_k^0$. Then, the negative logarithm of the likelihood-ratio of an observation sequence $\left(\left\{T_n,k_n\right\}\right),$ $T_n<T$ is given by

$$D_T^N\triangleq T\cdot {\displaystyle \sum _k\left({\lambda }_k-{\lambda }_k^0\right)}-{\displaystyle \sum _k{N}^k}\left(T\right)\cdot \log \left(\frac{{\lambda }_k}{{\lambda }_k^0}\right)$$

(1)

where $N^k\left(T\right)$ is the number of events of type $k$ prior to $T$.

Assuming now that ${\lambda }^0=\left\{{\lambda }_k^0\right\}$ is the true set of process parameters, furthermore assuming that $\lambda =\left\{{\lambda }_k\right\}$ is a set of tentative parameter values, and writing $D_T^N=D_T^N\left(\lambda \right)$, we have the following inequality:

$$E\left\{D_T^N\left(\lambda \right)\right\}\ge 0.$$

(2)

The left-hand side is simply the Kullback-Leibler (KL) divergence of the true distribution from the estimated one. Using the common notation of the KL divergence, the left-hand-side of the above inequality can be rewritten as

$$E\left\{D_T^N\left(\lambda \right)\right\}=D_{KL}\left(\mathrm{mPois}\left({\lambda }^{0}T\right)\parallel \mathrm{mPois}\left(\lambda T\right)\right),$$

(3)

where $\mathrm{mPois}(\cdot )$ represents the distribution corresponding to the marked Poisson process, while $D_{KL}\left(\mathrm{mPois}\left({\lambda }^{0}T\right)\parallel \mathrm{mPois}\left(\lambda T\right)\right)$ is the expected number of extra nats—NB: not bits, but nats, as the natural logarithm is used in Equation (1), not ${\log }_2$—required to encode the observation sequence from the distribution $\mathrm{mPois}\left({\lambda }^{0}T\right)$ using a code optimized for the distribution $\mathrm{mPois}\left(\lambda T\right)$ rather than using the code optimized for $\mathrm{mPois}\left({\lambda }^{0}T\right)$.

Associated with $D_T^N\left(\lambda \right)$ is the computable quantity

$$L_T\left(\lambda \right)\triangleq T\cdot {\displaystyle \sum _k{\lambda }_k}-{\displaystyle \sum _k{N}^k}\left(T\right)\cdot \log \left({\lambda }_{k}T\right)$$

(4)

$L_T\left(\lambda \right)$ can be interpreted as the approximate length of an optimal code encoding the observation sequence, were $\lambda$ the true set of the process parameters. Considering, however, that $L_T\left(\lambda \right)$ is dependent on ${\lambda }^0$—i.e., on the “real” true set of process parameters—via $N^k$, we can write $L_T\left(\lambda \right)=L_T\left({\lambda }^0,\lambda \right)$, and similarly, we can indicate the same kind of dependency for $D_T^N$, i.e., $D_T^N=D_T^N\left(\lambda \right)=D_T^N\left({\lambda }^0,\lambda \right)$.

2.2.2. Modelling TS Occurrences in a Neighboring Urban Road Environment

In conjunction with a second road environment that borders the one looked at in the previous subsection, let us now consider another marked Poisson process with parameters ${\mu }^0=\left\{{\mu }_k^0\right\}$. The probability distribution corresponding to this process is $\mathrm{mPois}\left({\mu }^{0}T\right)$. The counting measures $M^k(\cdot )$ are used for counting the events of different types, i.e., for counting the occurrences of the various TSs, separately.

Let the a priori estimate of ${\mu }^0$ be $\mu$. Then, similarly to our comments with respect to $L_T\left(\lambda \right)$ defined in Equation (4), the approximate length of an optimal code encoding the observation sequence observed within this second road environment—were $\mu$ the true set of the process parameters—is given in Equation (5)

$$J_T\left(\mu \right)\triangleq T\cdot {\displaystyle \sum _k{\mu }_k}-{\displaystyle \sum _k{M}^k}\left(T\right)\cdot \log \left({\mu }_{k}T\right).$$

(5)

Considering that $J_T\left(\mu \right)$ is dependent on ${\mu }^0$—i.e., on the “real” true set of process parameters—through event counts $M^k$, we can write $J_T\left(\mu \right)=J_T\left({\mu }^0,\mu \right)$. Furthermore, the negative logarithm of the likelihood-ratio of an observation sequence—observed within this second road environment—can be written as $D_T^M=D_T^M\left(\mu \right)=D_T^M\left({\mu }^0,\mu \right)$.

2.2.3. Modelling TS Occurrences over Two Neighboring Urban Road Environments

If $\lambda$ and $\mu$ are reasonable estimates of ${\lambda }^0$ and ${\mu }^0$, respectively, then $D_T^N\left({\lambda }^0,\lambda \right)$ and $D_T^M\left({\mu }^0,\mu \right)$ are going to remain relatively small. We shall consider the case, when the parameter-sets of the two marked Poisson process differ considerably, i.e., ${\lambda }^0$ and ${\mu }^0$ differ considerably with $\lambda$ and ${\lambda }^0$ still being close to each other, and with $\mu$ and ${\mu }^0$ still being close to each other. Then $D_T^N\left({\lambda }^0,\mu \right)$ and $D_T^M\left({\mu }^0,\lambda \right)$ are going to be large.

Furthermore, using the encoding argument outlined above, $L_T\left(\lambda \right)-L_T\left(\mu \right)$ will have a tendency to decrease in time (i.e., with $T$), and similarly, $J_T\left(\mu \right)-J_T\left(\lambda \right)$ will also have such a tendency. The negated version of the latter, i.e., $J_T\left(\lambda \right)-J_T\left(\mu \right)$, on the other hand, will have a tendency to increase in time. More clearly, using the defining formulae in Equations (4) and (5), respectively,

$$L_T\left(\lambda \right)-L_T\left(\mu \right)=T\cdot {\displaystyle \sum _k\left({\lambda }_k-{\mu }_k\right)}-{\displaystyle \sum _k{N}^k}\left(T\right)\cdot \log \left(\frac{{\lambda }_k}{{\mu }_k}\right)$$

(6)

tends to decrease with $T$, while

$$J_T\left(\lambda \right)-J_T\left(\mu \right)=T\cdot {\displaystyle \sum _k\left({\lambda }_k-{\mu }_k\right)}-{\displaystyle \sum _k{M}^k}\left(T\right)\cdot \log \left(\frac{{\lambda }_k}{{\mu }_k}\right)$$

(7)

tends to increase with $T$.

2.2.4. Detecting Change between Urban Road Environments and Locating the Change Point

Assume now a switch from the first marked Poisson process to the second, i.e., from $\mathrm{mPois}\left({\lambda }^{0}T\right)$ to $\mathrm{mPois}\left({\mu }^{0}T\right)$, and accordingly a switch from respective counting measures $N^k(\cdot )$ to $M^k(\cdot )$ at time $\tau$. Then for $T\le \tau$

$$g_T\triangleq g_T\left(\lambda ,\mu \right)\triangleq L_T\left(\lambda \right)-L_T\left(\mu \right)$$

(8)

tends to decrease with $T$. Let us now introduce the notation $T^{*}=T-\tau$. For $T>\tau$, i.e., for $T^{*}>0$, $g_T$ is defined as follows

$$g_T\triangleq g_T\left(\lambda ,\mu \right)\triangleq g_{\tau }+J_{T*}^{\tau }\left(\lambda \right)-J_{T*}^{\tau }\left(\mu \right),$$

(9)

where $J_{T*}^{\tau }\left(\lambda \right)-J_{T*}^{\tau }\left(\mu \right)$ is to be computed according to a modified version of Equation (7), as shown below

$$J_{T*}^{\tau }\left(\lambda \right)-J_{T*}^{\tau }\left(\mu \right)=T^{*}\cdot {\displaystyle \sum _k\left({\lambda }_k-{\mu }_k\right)}-{\displaystyle \sum _k{M}_{\tau }^k}\left(T^{*}+\tau \right)\cdot \log \left(\frac{{\lambda }_k}{{\mu }_k}\right)$$

(10)

here the counting measures $M_{\tau }^k(\cdot )$ count events in the same way as $M^k(\cdot )$ with the only difference that they now count events only from $\tau$ onwards.

For $T>\tau$, $g_T$ tends to increase with $T$. To estimate the location $\tau$, one should monitor function $g_T$. Then, in order to determine $\tau$, we need to wait for an increasing trend in $g_T$ to appear.

In order to find out whether a change in the stochastic model has occurred, or not, and if it has, when/where, one should compute the minimum of $g_T$ on-the-fly with a Page-Hinkley change detector (PHCD), see [42,43] for the detailed derivation of the change detector.

Using $g_T$, the PHCD, $h_T$ is defined as

$$h_T\triangleq g_T-\underset{s\le T}{\inf }{g}_s.$$

(11)

A change is thought to have occurred if $h_T$ exceeds a threshold $\delta >0$. As it will be clear from the examples presented in Section 3.1 and Section 3.2, the choice of $\delta$ is crucial for the proper working of the detector.

It can be shown—following the line of thoughts presented in [44]—that under the hypothesis of no-change, $h_T$ is L-mixing, and the false alarm probability is exponentially decaying in $\delta$: $P\left(h_T\ge \delta \right)\le C{e}^{-a\delta }$ with some $a>0$. Hence the false alarm rate itself is exponentially decaying in $\delta$. As a consequence, the false alarm rate can be effectively reduced by choosing larger $\delta$. On the other hand, if $\delta$ is chosen to be too large, then the detection lag can be too long or even transitions can be missed.

2.2.5. Basic Properties of Functions $g_T$ and $h_T$

Before getting on to elaborate concrete TS-based RET CD examples in Section 3.1 and Section 3.2, it is worthwhile to look more closely at the functions involved in the CD computations, namely, to $g_T$ and $h_T$. The diagrams of these functions are composed of linear segments.

In case of $g_T$, each of these linear segments has the same slope, and at either end of a segment, there can be a “jump”. The jump can be either an upward jump, or a downward jump depending on the particular event and on the process parameters.

In case of $h_T$, the situation is slightly more complicated. Apart from the linear segments with the same slope, if any such segment remains in the diagram, there can be broken lines reaching the horizontal axis, and a number of linear segments along this axis. Furthermore, all the constituting linear segments and broken lines of $h_T$ are located in the upper half-plane that includes also the horizontal $T$ axis.

3. Results

3.1. Examples

Let us now see two CD examples from the given application field. In these examples, we intend to detect change in the RET based on TS occurrences along a route. The full length of the trip represented in the table is 4.6 km. The TSs are assumed to be detected and located by an on-board TSR system.

The TS locations are marked with the corresponding TSs in the top band of

Table 2. A synthetic sequence of TSs used in the examples (top band), and the unified counts for the No stopping (NS), Parking lot (PL), the Give way (GW), and Max speed 30 km/h (SL) TSs in case of trips starting from the left and from the right, respectively (middle and bottom bands).

——		——		—		——		—		—		——		————		———		——		———		——
→			1								2
→					1				2
→	1						2								3						4
→													1				2		3
			2								1											←
					2				1													←
	4						3								2						1	←
													3				2		1			←

. The sequence given there is synthetic, and has been compiled for the purpose of demonstrating

• a RET transition from a Dt to a Res area (denoted by Dt → Res), if the virtual journey is taken from the left, and
• a RET transition from a Res to a Dt area (denoted by Res → Dt), if the journey is taken from the right.

In the middle and the bottom bands of Table 2, the actual counts of the NS, PL, GW, and SL TSs for the virtual trips starting from the left, and from the right, respectively, are given. These counts have been produced in a unified and generic manner, i.e., without demarking the validity intervals of the respective counting measures.

As these counts are denoted simply by $N^{NS}$, $N^{PL}$, $N^{GW}$, and $N^{SL}$ in the present and in the subsequent subsections, care should be taken to use them properly, i.e., according to the direction of the virtual trip.

In the sequence given in the top band, each “—” signifies a 0.2 km path-length along the route without any relevant TSs—facing the ego-car—installed/detected along the corresponding patch of road. It should be underlined that these fixed path-length route-segments are used in the examples simply for convenience, i.e., to make diagrammatic representation of the TS sequence, as well as the virtual trip easier to follow, and to make the calculations and diagrams easier to verify.

In each of the two examples that are presented in this subsection, the specific PHCD is applied to a RET transition that is homologous to the one, the PHCD has been tuned to.

First, let the ego-car start its virtual trip from the left. This case is described in detail in Example No. 1.

3.1.1. Example No. 1: A Dt → Res Change Detector Applied to a Homologous RET Transition

In the TS sequence given in the top band of Table 2, the intended RET transition (i.e., the Dt → Res) occurs at about the path-length of 2.2 km (i.e., having covered eleven dashes from the left in the band). The presence of such a RET transition in the synthetic sequence was secured by having inserted into it five TSs that are more characteristic of the Res areas than of the Dt areas—see the rightmost column of Table 1—in consecution and reasonably close to each other starting from this path-length along the virtual route.

According to the CUSUM-based RET CD method derived in Section 2.2, functions $g_T$ and $h_T$ are to be used for the purpose of CD. These functions are now denoted by $g_l^{Dt\to Res}$ and $h_l^{Dt\to Res}$, respectively. The lower and upper indices indicate the function’s dependence on path-length $l$ and the RET transition actually monitored by the change detector, respectively. These functions are given in Equations (12) and (13), respectively. In the former, the coefficients are given with the same numerical precision as was used in Table 1.

$g_l^{Dt\to Res}\approx 2.80 {\mathrm{km}}^{-1}\cdot l-1.74\cdot N^{NS}\left(l\right)-$

$$-1.92\cdot N^{PL}\left(l\right)+0.13\cdot N^{GW}\left(l\right)+0.69\cdot N^{SL}\left(l\right)$$

(12)

$$h_l^{Dt\to Res}=g_l^{Dt\to Res}-\underset{s\le l}{\inf }{g}_s^{Dt\to Res}.$$

(13)

For the generation of these functions, the actual counts of the four considered TS types—at a particular path-length—are required. These counts are given in the middle band of Table 2 for virtual trips starting from the left.

In Figure 1, functions $g_{\to l}^{Dt\to Res}$ and $h_{\to l}^{Dt\to Res}$ have been plotted, respectively, for the TS sequence given in Table 2. The virtual trip in this case had started from the left, as indicated by “$\to$” in the lower indices.

According to the diagram of $h_{\to l}^{Dt\to Res}$ in Figure 1, a threshold, say, $\delta =3.0$ could be selected to detect the RET change. With this threshold, the change is detected at about the path-length 2.7 km, i.e., with a detection lag of 0.5 km. If smaller thresholds, e.g., $\delta =1.0$ or $\delta =2.0$ are used instead, then 3 and 1 false alarms will occur, respectively, along the 2.2 km of the Dt route. If, on the other hand, larger thresholds are used, such as $\delta =4.0$ or $\delta =5.0$, then unnecessary extra detection lags of 0.3 and 0.6 km will occur.

3.1.2. Example No. 2: A Res → Dt Change Detector Applied to a Homologous RET Transition

To make full use of the synthetic TS sequence given in the top band of Table 2 and to provide a deeper insight into the proposed CD method, let now the ego-car start its virtual journey from the right.

As a side note, we can reverse such a virtual journey fairly easily in a table. In real life and in real traffic, however, it would be much more problematic as one might encounter very different TSs on the way back (if at all the reversed route is permitted by the TSs installed). Furthermore, the TSs that are facing us now are located on the opposite side of the road as in the first journey. However, even in case of this wieldy virtual journey, some modifications in the table, namely, with respect to the TS counts, are necessary.

These new TS counts, i.e., the counts for right-to-left virtual trips, are given in the bottom band of the table. In this reversed case, the ego-car is driven from an intended Res area to an intended Dt area, i.e., a Res → Dt transition is expected.

Again, as in the first example, we wish to form functions $g_T$ and $h_T$ that signal the RET transition that is actually expected. Accordingly, these functions are now denoted by $g_l^{Res\to Dt}$ and $h_l^{Res\to Dt}$, respectively.

In order to detect such a transition, the roles of the process parameters ${\lambda }_k$ and ${\mu }_k$—given in the fourth and the fifth column of Table 1, respectively—need to be swapped in Equations (8) and (9), as now, it is supposed that the first road environment is a Res area, rather than a Dt area. Again, the unified counts are to be used, namely, the TS counts given in bottom band.

Functions $g_l^{Res\to Dt}$ and $h_l^{Res\to Dt}$ are given in Equations (14) and (15), respectively.

$g_l^{Res\to Dt}\approx -2.80 {\mathrm{km}}^{-1}\cdot l+1.74\cdot N^{NS}\left(l\right)+$

$$+1.92\cdot N^{PL}\left(l\right)-0.13\cdot N^{GW}\left(l\right)-0.69\cdot N^{SL}\left(l\right)$$

(14)

$$h_l^{Res\to Dt}=g_l^{Res\to Dt}-\underset{s\le l}{\inf }{g}_s^{Res\to Dt}.$$

(15)

In Figure 2, the functions $g_{l\leftarrow }^{Res\to Dt}$ and $h_{l\leftarrow }^{Res\to Dt}$ have been plotted. The corresponding virtual journey had started from the right in the band, as indicated by the “$\leftarrow$” in the lower indices. The diagrams presented herein corresponding to virtual journeys from the right are presented in green.

Again, threshold $\delta =3.0$ would be an appropriate choice based on the diagram in Figure 2. With this threshold, the RET change is detected at about the path-length 3 km. Since the first signs of the Dt area begin to appear at the path-length 2.8 km, the detection lag is 0.2 km long.

Comparing the diagrams of $g_{\to l}^{Dt\to Res}$ and $g_{l\leftarrow }^{Res\to Dt}$ shown in Figure 1a and Figure 2a, respectively, one notices the symmetry between these. This symmetry can be traced back to two facts: first, from Equations (8) and (9)—as used in Example 1, and by swapping the roles of parameters ${\lambda }_k$ and ${\mu }_k$ for equations corresponding to Example 2—furthermore, from Equations (12) and (14), it follows that

$$g_l^{Res\to Dt}\equiv -1.0\cdot g_l^{Dt\to Res}.$$

and second, to the fact that the same route—with its TSs—was covered from opposite directions.

There will be further symmetries perceptible between the respective $g_{\dots l\dots }^{\dots \to \dots }$ diagrams. However, similar symmetries do not show up amongst the respective $h_{\dots l\dots }^{\dots \to \dots }$ functions.

3.2. Further Examples

In each of the two examples below, an “off-the-tune” RET change detector is considered. These change detectors are applied to the TS sequence used above, furthermore, the same marked Poisson process reference parameters are used. The examples below demonstrate that “off-the-tune” change detectors may behave fairly haphazardly. In [17], approaches are proposed to deal with such behavior of the change detectors. These approaches are essential if several differently tuned change detectors are to be used within a compound system, e.g., for the purpose of RET detection and identification, rather than CD.

3.2.1. Example No. 3: A Res → Dt Change Detector Applied to a Dt → Res Transition

Functions $g_l^{Res\to Dt}$ and $h_l^{Res\to Dt}$ to be used for signaling a RET transition Res → Dt have already been presented—in conjunction with the Example No. 2—in Equations (14) and (15), respectively. The only difference is that now we need to use these functions with the TS counts given in the middle band of Table 2, rather than with the TS counts given in the bottom band, as the car is now driven from the left to the right (i.e., a Dt → Res transition is expected).

In Figure 3, the diagrams of $g_{\to l}^{Res\to Dt}$ and $h_{\to l}^{Res\to Dt}$ are shown for an intended RET transition Dt → Res, respectively. The detector takes the first 0.8 km starting from the left for a Res area and then detects a change Res → Dt at that point (e.g., with a threshold $\delta =1.0$). If threshold $\delta =3.0$ is used instead, then the RET transition will be detected at 1.0 km. The corresponding detection lags for these two thresholds are 0.8 and 1.0 km, respectively. If we use larger thresholds, say, $\delta =4.0$, or $\delta =5.0$, then for the former, an extra detection lag of 0.9 km will be introduced, while the latter will completely miss the Dt segment of the route.

The intended RET change of transition Dt → Res remains undetected by this detector no matter what $\delta >0$ is used.

3.2.2. Example No. 4: A Dt → Res Change Detector Applied to a Res → Dt Transition

In Figure 4, the diagram of function $g_{l\leftarrow }^{Dt\to Res}$ is shown for the intended RET transition Res → Dt. This transition is present when driving from the right along the considered TS sequence. Note that for the given TS sequence and for the given process parameters, functions $g_{l\leftarrow }^{Dt\to Res}$ and $h_{l\leftarrow }^{Dt\to Res}$ happen to be identical. Using threshold $\delta =3.0$, the detector signals a change Dt → Res at about the path-length of 1 km.

4. Discussion

According to the approach derived in Section 2.2, the RET changes can be detected with CUSUM change detectors, which rely on the on-the-fly minimization effected by PHCDs.

In order to detect all kinds of the RET transitions between the three RETs considered herein, the simultaneous use of six differently tuned PHCDs is necessary. In Section 3.2, we have demonstrated what happens to the functions $g_{\dots l\dots }^{\dots \to \dots }$ and $h_{\dots l\dots }^{\dots \to \dots }$ when the actual RET change is not what the detector is tuned to detect. In fact, in the examples given there, we have applied change detectors that were tuned to the inverse transitions.

If one wanted to use the aforementioned PHCDs for the purpose of detecting not only the changes between different RETs, but also the actual RETs themselves, furthermore, wished to overcome the haphazard behavior of the “off-the-tune” PHCDs, there are promising possibilities; for instance, the respective functions $h_l^{\dots \to \dots }$ can be generated and considered within a sliding window, furthermore, several overlapping sliding windows can be used at the same time. In addition, these could be multi-scale windows.

An artificial neural network (ANN) proposed for TS-based RET detection was presented in [26]. The ANN-based method made use of sliding multi-scale windows, and for these windows, TS histograms were calculated. The network proposed there could well be extended to input and make good use of the “summaries” of functions $h_l^{\dots \to \dots }$, rather than the TS histograms. These summaries could be of syntactic nature. A tool capable of exploring time series data for pattern and query search tasks, as well as for generating syntactic descriptions of the time series was proposed and demonstrated in [45]. The syntactic descriptions of the functions $h_l^{\dots \to \dots }$ should preferably be computed for sliding multi-scale windows.

The TS-based RET change, the inferred actual RET within, and the complete inferred RET structure—i.e., a map layer, or sublayer—of an urban area could be utilized in various manners in automotive applications. First, the TS-based RET change, or the TS-based actual RET could initiate warnings to novice drivers, e.g., “You are now driving in a downtown area.” What actually is meant by this warning is as follows: “The area might be uncrowded now, but in half an hour, or so it could turn very busy and could be loaded with intense car traffic. Therefore, find a parking place now, if want to stay in this area.”

It also hints at reducing speed to, say, 40 km/h. In a Res area, the respective warning could, for instance, instruct the novice driver to watch out for groups of children playing on the streets.

In relation to the control of smart cars, the preferred speed could be set to some lower than 50 km/h speed in the Dt area, especially during and close to the usual peak hours. The maximum acceleration and deceleration values could be set to safer values.

In relation to the ADAS/AD computations carried out on-board smart cars, particularly to the computations related to TS detection and recognition, a specific geometrical size range for TSs can be used. In narrow streets of historical districts, often smaller TSs are installed by the road authorities, and that size should be allowed in the TS verification phase of the computing.

The detection of traffic lanes and the estimation of the distances to the TSs from the ego-car are examples for computations that implicitly make use of some spatial models of the road and its environment. In Res areas—at least in our country—multi-lane roads are infrequent, therefore simpler road structures/models should be matched against the camera images of the road scenery.

Concerning the road administration and management, the TS-based RET map layer compiled from data gathered through car-based data collection trips could be used to improve the match between seasonal, weekly, and daily traffic patterns and the inferred RETs, thereby creating a more perceivable and more self-explaining urban environment that is hopefully also safer.

5. Conclusions

The road environment appears around and sweeps past an ego-car, while it is being driven. The character of the urban road environments can be categorized into urban RETs. Abrupt changes in the character of the road environment, i.e., transitions between areas of different RETs, pose traffic safety risk, especially, for drivers lacking prolonged driving experience and also for drivers of old age.

The urban RET transitions per se manifest themselves in changes in traffic density and in the composition of the traffic. These are transient dynamic features describing an urban area, i.e., a subnetwork of an urban road network.

Nonetheless, urban RET transitions manifest themselves also in changes that concern static ROs, e.g., CRs (permanent static) and conventional TSs (transient static). So, e.g., the density and the “mixture” of TSs are expected to change between areas of different RETs. As a consequence, the RET change could also be detected via monitoring static RO occurrences along the route.

Herein, TS occurrences were considered only. These are noted in TS data logs. These logs can be interpreted as realizations of a continuous-variable IMPP, and the RET change can be detected—relying on this assumption—from them.

CD methods, e.g., the CUSUM method, are known and widely used for “simpler” inhomogeneous Poisson processes. The mentioned method was adopted and modified for detecting change between RETs based on a TS log. The behavior of the change detector was tested on a synthetic TS sequence. Nonetheless, the sequence was used in four different ways in Examples Nos. 1–4, and some observations and conclusions were drawn from these.

The presented simulation results indicate that a TS-based RET CD is feasible, and can be adopted for driver assistance, though it is not suitable for initiating an immediate intervention in critical situations.

The continuous-time approach presented herein serves as a clarification of the discrete-time model and method proposed in [25], and it was not meant and it was not expected to improve for the processing and detection characteristics achieved therein. This is due to the underlying similarity between the two stochastic models, i.e., between the marked binomial and the marked Poisson models. For this reason, the precision and the delay of the RET change detection are expected to be in the same range, respectively, for both approaches for any realistic parameter-choices in the given context.

Further research and development have been suggested in Section 4 and have been motivated with regard to the integration of the RET change detector into an ANN-based detector solution proposed earlier.