Despite of the on-going research on self-explaining road layouts and designs [1
], and on the computerized recognition methods of such designs and layouts, e.g., on methods that apply artificial intelligence methodology [3
], setting up traffic signs (TSs) along the roads and traffic lights in road junctions and near pedestrian crossings by the transport authorities still remains a customary measure for reducing traffic safety risks in urban areas [4
]. Clearly, there are other viable alternative measures, as well as supplementary ones for the purpose. These include—among many others—the installation of speed reduction markings onto the road-surface [5
] and the installation of vehicle- to-infrastructure (V2I) communication facilities, e.g., to succor the TS recognition (TSR) function offered by advanced driver assistance systems (ADAS) [6
] and self-driving cars [7
]. In a wider sense, V2I communication succors the road, traffic and vehicle data gathering, fusion, and dissemination, and through these data processes, it is expected to have a significant beneficial impact on traffic safety [8
]. More specifically, V2I communication can be used for raising the road-awareness of car-drivers, as well as that of the intelligent and the self-driving road vehicles. Furthermore, it can be used for providing the human drivers and the smart vehicular systems with current traffic information with respect to the region, town, and area, on the one hand, and with some very specific dynamic information on individual vehicles in the vicinity, on the other [9
When speaking about raising road awareness of drivers, one is obliged to speak about the Global Navigation Satellite System (GNSS), a system that is used by masses of people around the world. According to [10
], the GNSS devices per capita averaged out at 0.8 across the countries of world in 2019. The GNSS is used with wide variety of devices running map-based applications, a significant percentage of these devices are installed on-board cars. The brief history of the navigational systems and their respective precisions are presented in [11
]. The paper provides a fresh outlook on the navigational needs of and the available navigational solutions for autonomous vehicles and systems. As it often happens to popular services, devices, and applications, threats against these surface from time to time. Such threats have surfaced also against the GNSS service [12
]. Although the number of successful navigational spoofing attacks is still negligible, the navigational signal deteriorations due to other—i.e., non-hostile—factors are clearly not. For instance, the signal reception is often brought down, or even blocked by the high-rise buildings in densely built urban areas. Some examples in this context are presented in [13
The speed reduction measures implemented in urban areas are motivated by the traffic safety concerns associated with the intense road traffic and the limited space available there for the driving maneuvers [14
]. While driving, and particularly while driving in urban areas, drivers need to perform numerous mental and control tasks—ranging from those associated with limb-movement to those required for complex driving maneuver planning and execution—within stringent time and spatial constraints and with high reliability [15
]. Furthermore, these tasks must be performed in presence of disturbances, such as unfavorable lighting, adverse weather, and traffic conditions [16
]. In addition, the older age of the driver may contribute to the perceived difficulty of these tasks [17
A system, which pays attention to the driver’s activity within the car and also to aspects of the urban road environment, was developed as part of the Urban Intelligent Assist Research Initiative some years ago [18
], and since then, other systems with similar, or enhanced capabilities followed suit [19
]. The effect of driving experience on drivers’ adaptation to road environment complexity—a notion closely related to that of the road environment type (RET) used herein—in urban areas was investigated in a simulation study [21
]. The findings of the study underline the need for an automatic RET detection function, and indicate that such a function is particularly useful for car-drivers lacking prolonged driving experience, and also for older drivers.
Several algorithmic approaches and sensor arrangements were devised, applied, and tested for detecting, characterizing, and categorizing urban road environments based on image and/or point cloud data [22
]. In the application considered herein, the urban road environment appears around and sweeps past an ego-car while it is driven in an urban area. The data streams used for the purpose of road environment detection and analysis originate—among others—from one or more camera and one or more light detection and ranging (LiDAR) sensor. In a viable implementation of a road environment detection and classification system that is capable of assisting a car driver while driving, either a comprehensive real-time on-board processing of the respective raw data streams is required (direct processing) or a timely access to and further processing of the data—rendered by some other real-time application/subsystem on-board—on certain distinguishing road objects (ROs) are necessary (indirect processing).
In the above cited papers, the real-time requirements were limited to data synchronization and data collection issues, while the bulk of the processing, e.g., simultaneous localization and mapping (SLAM), and object segmentation, were carried out in a post-processing manner. Nevertheless, a large portion of the processing presented in these papers is real time capable and could be used in direct implementations.
The approach presented in [22
] builds and then segments the point cloud originating from a ground-level LiDAR device moving along a given trajectory. The aim of the authors was to produce editable—simplified, but visually still pleasing—object-models that lend themselves for fast visualization. The target areas were the residential urban areas in the United States. These areas are characterized by their low-rise buildings without strong and extensive repetitive patterns. The semantical labeling and various analysis steps follow the mentioned preprocessing steps. Simple models of the individual houses in the area are then created. The basic building blocks of the models are simple, symmetric, and convex geometric blocks. These blocks—together with their spatial arrangement and their connection graph—form an easy-to-handle geometric model of the individual buildings. By aggregating the certain features of the individual buildings for an area (e.g., by computing the average building dimensions and the average distance between nearest buildings), the residential urban road environment can be adequately characterized.
The system presented in [23
] extracts the characteristics of individual buildings rather than those of more extensive road environments. Nonetheless, the building characteristics, such as building height and building complexity—again aggregated for a given area, or along a route—together with the spatial densities of the buildings there, are definitive in the respect of the RET.
A multi-sensor and multi-precision data collection campaign is described in [24
]. It was a car-based campaign that made use of an array of different environment perception, navigational, and motion sensors. These included four LiDARs, a pair of stereo cameras, a fiber optics gyroscope and encoder sensors for the tires. The data collection trips covered diverse complex urban environments in Korea, with a clear emphasis on those environments, where GPS reception is highly unreliable. The collected data were organized into a publicly accessible dataset that includes the measured ego-car trajectories, the raw and processed point cloud data from the LiDAR sensors, as well as the ego-car trajectories with improved precision computed via SLAM.
Other approaches, e.g., the ones presented in [25
], rely object-level data as inputs to the urban RET detection function, i.e., they follow an indirect processing approach. In a feasible realization, the raw data streams originate from the very same sensors as in the direct case, but the respective data streams reach the RET detection subsystem only after having been processed and considerably compressed by one or more ADAS subsystem. The resulting data are an object-level description of the road environment, i.e., an RO log. This log serves as an input to the indirect RET detection function.
The on-board data processing described above, as well as other road, traffic, and vehicular data processing carried out in various ADAS subsystems (e.g., lane detection, TSR, detection of nearby vehicles) can also be looked at from, analyzed with respect to, and formulated using a static reference point. Setting up and using a local dynamic map (LDM) [27
] could serve these purposes, and provide additional conceptual support for the developers of ADAS functions. LDM is a widely used model for representing, and a standardized technology for integrating static, temporary, and dynamic road, traffic, and vehicular information into a static geographical context by means of a common coordinate reference. Customarily, it has four object layers describing and managing ROs that are subject to change and exhibit dynamics at different time scales. More concretely, these layers store and handle data on permanent static, transient static, transient dynamic, and highly dynamic ROs, respectively. For instance, when framing the static RO-based urban RET detection task in LDM, the ego-car is seen as a highly dynamic RO. A crossroads (CRs) intersection of streets is a permanent static RO in this model. The intersection can be associated with other ROs (e.g., with fixed traffic lights located there, which are transient static ROs). The lanes, lane markings, pedestrian crossings, and the conventional TSs are transient static ROs, while the TSs displayed by variable message sign boards can be classified as transient dynamic ROs. The RETs can be treated as permanent static features of an area or of a sub-network of roads.
One could look at the ROs in an urban settlement and collect and compile their location and categorical data into a map layer, e.g., in the way data contributors of OpenStreetMap maps do with roads, railways, rivers, and various locations of importance [28
]. By selecting appropriate subsets of TSs—i.e., TS subsets that are characteristic to certain RETs—various sublayers of the RO layer can be created, displayed, and analyzed. The analysis could include a Delaunay triangulation of the TS locations within a sublayer, and then one could look for dense clusters of triangles in the generated structure. By carrying out similar processing for a number of sublayers, a TS-based RET categorization of the urban area can be created.
By further processing the map-based representation of TSs and other ROs, one could derive other interesting sublayers that relate to seasonal, weekly, or daily validity of the TSs and could derive a sublayer representing weather-related TSs (e.g., TSs applicable for wet, snowy and icy road conditions). For instance, the sublayer representing the within-the-day validity of TSs—indicated by auxiliary signs or time intervals attached to the TSs—should reflect the daily dynamics of traffic source and sink structure of the area [29
]. Clearly, the mentioned dynamics are closely related to the RET categorization used herein.
In our view, such sublayers—compiled, e.g., from data gathered in car-based data collection campaigns—could give useful hints to road authorities and administration as to where to place additional TSs and auxiliary signs or remove unnecessary existing ones. Herein, however, we stick to the route-based sampling of the TSs of the urban area, the map-based processing touched upon above will be addressed in further research.
], the urban road environments were categorized into three RETs, namely, into downtown (Dt), residential (Res), and industrial/commercial (Ind) areas. The ROs represented in the object-log were the TSs and CRs encountered along the route. In an advantageous implementation foreseen, both the TS and the CR data originate from their respective dedicated ADAS subsystems. While in case of the TS data, the corresponding subsystem, i.e., the TSR ADAS subsystem, is quite common in recent production cars, the CR detection ADAS function is fairly uncommon at this point of time. It is expected though that in the coming years, the LiDAR sensors developed for automotive applications will pave the way for the spread of such an ADAS subsystem.
A good insight in ADAS system architectures, various ADAS subsystems and functions, as well as the respective methods and computations involved is given in [30
]. A survey on TSR methods and systems is given in [31
], while in [32
], a mapping and navigation system developed for large-scale global positioning-denied sites is introduced. The system is capable of detecting CRs, intersections, and other road infrastructure.
The static RO-based urban RET detection approach proposed in [25
], and some further approaches make use of a variety of classification and change detection (CD) methods known from the statistical inference literature. In the cited paper, it is presumed that the static ROs in general, and the considered TSs and the CRs, in particular, occur along the route according to an inhomogeneous discrete-variable binomial process. The minimum description length (MDL) methodology is then applied to detect and locate change in the character of the road environment sweeping past the ego-car.
The lane-keep assist ADAS and the lane following autonomous driving (AD) subsystems, which perforce continually identify the current and neighboring lanes, and estimate their widths, as well as the TSR ADAS and AD subsystems, which locate, identify, and track the TSs encountered by the ego-vehicle, are of particular interest in the context of RET detection. First, such ADAS subsystems are already available on-board many production cars, second, the categorical and spatial distribution of TSs, as well as, the lane-widths and the number of lanes—in the current cross-section of the road or in an aggregated form (e.g., average lane-width, average number of lanes)—carry information that can be useful in determining the RET of the given urban area.
It should be emphasized that a timely feedback of the RET information to the above ADAS and AD subsystems could increase their effective processing speed and lower the rate of misclassifications via setting practical parameter constraints for the computations involved. Such constraints could be of geometrical nature and could take the forms of Boolean, probabilistic, and fuzzy regions-of-interest (ROIs), respectively, e.g., within image frames of video sequences [33
]. While in case of point clouds, volumes-of-interest, again meant in a Boolean, in a probabilistic, and in a fuzzy way, respectively, could be marked and used [34
]. As a further application of such reciprocal information, the characteristic size range of TSs—for a given RET—could be used for validating the detected TS candidates [35
Similar processing benefit could be gained from the above outlined information feedback in case of other presently not so wide-spread driver assistance functions, such as the CR detection. Furthermore, information on the current RET is also important for suggesting/choosing appropriate vehicle speed and acceleration/deceleration for the ego-car. An embedded testbed architecture for testing functions of self-driving cars was proposed in [36
]. It could also facilitate the seamless integration of the static RO-based RET detection function into the intelligent vehicle control systems.
In the following, it will be assumed that TS occurrences are reliably detected and logged by the on-board TSR ADAS subsystem, moreover, this log is passed on to the RO-based—in the following practically TS-based—RET detection system in real-time.
It was our aim to choose, adapt, and validate a mathematically sound CD method that makes provision for and relies on a simple, but realistic stochastic model of the static RO placement and occurrences, in general, and of the TS placement and occurrences, in particular, for the purpose and in the context of detecting transitions between road environments of different character—or more concretely, between road environments of different RETs—in order to assist car drivers, human, and robotic drivers alike, in their driving tasks and activities. The continuous-time inhomogeneous marked Poisson process (IMPP) was identified as a possible stochastic model to work with.
It should be noted, however, that in real life, the static RO placements—including those of TSs and traffic lights—are governed by technical and administrative guidelines [37
], from time to time they are subjects of potentially lengthy conciliatory procedures between locals and road administration. The final decisions are therefore taken at different administrative levels. Some aspects of this occasionally complicated process are outlined in [38
]. As in [25
] also herein, the occurrences are considered along routes. These routes are assumed to be random, but they are, in fact, based on intelligent choices made by the drivers.
Results gained via simulation implementing the IMPP model and making use of realistic data indicate that a TS-based RET CD is feasible and can be used for driver assistance, though it is not suitable for initiating an immediate intervention in critical situations. A more varied selection of static ROs—including, e.g., CRs, traffic lights, and pedestrian crossings—would further improve the feasibility of the RET CD. Similar utility and feasibility are expected for the RET detection and identification function computed with several RET change detectors and an artificial neural network (ANN) that merges and mushes together the detected RET transitions.
According to the approach derived in Section 2.2
, the RET changes can be detected with CUSUM change detectors, which rely on the on-the-fly minimization effected by PHCDs.
In order to detect all kinds of the RET transitions between the three RETs considered herein, the simultaneous use of six differently tuned PHCDs is necessary. In Section 3.2
, we have demonstrated what happens to the functions
when the actual RET change is not what the detector is tuned to detect. In fact, in the examples given there, we have applied change detectors that were tuned to the inverse transitions.
If one wanted to use the aforementioned PHCDs for the purpose of detecting not only the changes between different RETs, but also the actual RETs themselves, furthermore, wished to overcome the haphazard behavior of the “off-the-tune” PHCDs, there are promising possibilities; for instance, the respective functions can be generated and considered within a sliding window, furthermore, several overlapping sliding windows can be used at the same time. In addition, these could be multi-scale windows.
An artificial neural network (ANN) proposed for TS-based RET detection was presented in [26
]. The ANN-based method made use of sliding multi-scale windows, and for these windows, TS histograms were calculated. The network proposed there could well be extended to input and make good use of the “summaries” of functions
, rather than the TS histograms. These summaries could be of syntactic nature. A tool capable of exploring time series data for pattern and query search tasks, as well as for generating syntactic descriptions of the time series was proposed and demonstrated in [45
]. The syntactic descriptions of the functions
should preferably be computed for sliding multi-scale windows.
The TS-based RET change, the inferred actual RET within, and the complete inferred RET structure—i.e., a map layer, or sublayer—of an urban area could be utilized in various manners in automotive applications. First, the TS-based RET change, or the TS-based actual RET could initiate warnings to novice drivers, e.g., “You are now driving in a downtown area.” What actually is meant by this warning is as follows: “The area might be uncrowded now, but in half an hour, or so it could turn very busy and could be loaded with intense car traffic. Therefore, find a parking place now, if want to stay in this area.”
It also hints at reducing speed to, say, 40 km/h. In a Res area, the respective warning could, for instance, instruct the novice driver to watch out for groups of children playing on the streets.
In relation to the control of smart cars, the preferred speed could be set to some lower than 50 km/h speed in the Dt area, especially during and close to the usual peak hours. The maximum acceleration and deceleration values could be set to safer values.
In relation to the ADAS/AD computations carried out on-board smart cars, particularly to the computations related to TS detection and recognition, a specific geometrical size range for TSs can be used. In narrow streets of historical districts, often smaller TSs are installed by the road authorities, and that size should be allowed in the TS verification phase of the computing.
The detection of traffic lanes and the estimation of the distances to the TSs from the ego-car are examples for computations that implicitly make use of some spatial models of the road and its environment. In Res areas—at least in our country—multi-lane roads are infrequent, therefore simpler road structures/models should be matched against the camera images of the road scenery.
Concerning the road administration and management, the TS-based RET map layer compiled from data gathered through car-based data collection trips could be used to improve the match between seasonal, weekly, and daily traffic patterns and the inferred RETs, thereby creating a more perceivable and more self-explaining urban environment that is hopefully also safer.
The road environment appears around and sweeps past an ego-car, while it is being driven. The character of the urban road environments can be categorized into urban RETs. Abrupt changes in the character of the road environment, i.e., transitions between areas of different RETs, pose traffic safety risk, especially, for drivers lacking prolonged driving experience and also for drivers of old age.
The urban RET transitions per se manifest themselves in changes in traffic density and in the composition of the traffic. These are transient dynamic features describing an urban area, i.e., a subnetwork of an urban road network.
Nonetheless, urban RET transitions manifest themselves also in changes that concern static ROs, e.g., CRs (permanent static) and conventional TSs (transient static). So, e.g., the density and the “mixture” of TSs are expected to change between areas of different RETs. As a consequence, the RET change could also be detected via monitoring static RO occurrences along the route.
Herein, TS occurrences were considered only. These are noted in TS data logs. These logs can be interpreted as realizations of a continuous-variable IMPP, and the RET change can be detected—relying on this assumption—from them.
CD methods, e.g., the CUSUM method, are known and widely used for “simpler” inhomogeneous Poisson processes. The mentioned method was adopted and modified for detecting change between RETs based on a TS log. The behavior of the change detector was tested on a synthetic TS sequence. Nonetheless, the sequence was used in four different ways in Examples Nos. 1–4, and some observations and conclusions were drawn from these.
The presented simulation results indicate that a TS-based RET CD is feasible, and can be adopted for driver assistance, though it is not suitable for initiating an immediate intervention in critical situations.
The continuous-time approach presented herein serves as a clarification of the discrete-time model and method proposed in [25
], and it was not meant and it was not expected to improve for the processing and detection characteristics achieved therein. This is due to the underlying similarity between the two stochastic models, i.e., between the marked binomial and the marked Poisson models. For this reason, the precision and the delay of the RET change detection are expected to be in the same range, respectively, for both approaches for any realistic parameter-choices in the given context.
Further research and development have been suggested in Section 4
and have been motivated with regard to the integration of the RET change detector into an ANN-based detector solution proposed earlier.