Traffic routing is a central challenge in the context of urban areas, with a direct impact on personal mobility, traffic congestion, and air pollution. In the last decade, the possibilities for traffic flow control have improved together with the corresponding management systems. However, the lack of real-time traffic flow information with a city-wide coverage is a major limiting factor for an optimum operation. Smart City concepts seek to tackle these challenges in the future by combining sensing, communications, distributed information, and actuation. This paper presents an integrated approach that combines smart street lamps with traffic sensing technology. More specifically, infrastructure-based ultrasonic sensors, which are deployed together with a street light system, are used for multilane traffic participant detection and classification. Application of these sensors in time-varying reflective environments posed an unresolved problem for many ultrasonic sensing solutions in the past and therefore widely limited the dissemination of this technology. We present a solution using an algorithmic approach that combines statistical standardization with clustering techniques from the field of unsupervised learning. By using a multilevel communication concept, centralized and decentralized traffic information fusion is possible. The evaluation is based on results from automotive test track measurements and several European real-world installations.
In the context of the Internet of Things (IoT), the integral transformation process of urban infrastructure into intelligent and connected devices plays an important role for future mobility. Cities experience an increasingly high level of road congestion due to the focused traffic coming along with progressing urbanization. This congestion induces a high social, economical, and environmental cost. From a European Union (EU) perspective, it is amounted to about one percent of the GDP [
In general, two main types of techniques achieving more comprehensive traffic information have been established in the last years. A first category is the fixed or mobile infrastructural distribution of sensors, like inductive loops and traffic cameras. The second type is represented by systems relying on end-user based distributed information, for example, movement data from smartphone users or information gathered using V2X communication approaches. Details and characteristics of several techniques are discussed in this paper, considering not only metrics like traffic information quality and performance in different scenarios but also possible privacy issues which may arise with certain techniques.
The novel ultrasound-based traffic sensing technique, which is the focus of this paper, belongs to the category of infrastructural sensing. A significant difference to existing ultrasonic sensor systems is the placement of the sensors in a so-called sidefire setup. This means that the sensors are mounted in a height of at least three meters and face the street sideways with downtilted sensors, while being positioned on the side of the street. Previously existing systems were mainly single-lane ultrasonic sensors measuring the height profile top-down, for example, from a special sensor gantry above the street, with a simple first-reflection processing. The monitoring of multiple lanes is possible with our setup, while, at the same time, the requirements in terms of algorithmic evaluation and processing are increased in comparison to these previously existing systems. However, with these improvements, our approach allows operation in a highly reflective acoustic environment, which is present in urban areas and permits simply mounting the sensor on the side of the street. This is enabled by a new approach with a combination of statistical signal processing, clustering, and inference algorithms for traffic participant object detection.
This sidefire ultrasonic traffic sensing technique is part of a development which focuses on intelligent infrastructure solutions, more specifically on intelligent street lights. An exemplary integrated setup is shown in Figure
Integrated street lamp head with a single ultrasonic sensor head.
The rest of this article is organized as follows. Section
Ultrasonic traffic participant detection falls into the field of traffic sensing with a large number of techniques available in research and in practical application. In this section, we therefore provide a state-of-the-art overview from two perspectives. First, we give a comprehensive overview of traffic monitoring techniques in general with a discussion of their performance, strengths, and weaknesses in order to show the technological gaps and potential. In the second part, specific technical approaches in ultrasonic traffic sensing, which are rare both in concepts and in practical implementations, are given. This application-centric perspective shows the drawbacks and unresolved problems in current state of the art.
In the introductory section, two main categories of traffic information acquisition systems were introduced. These are on the one hand infrastructure-based sensing systems, working in a temporarily or permanently fixed location. On the other hand, the type of end-user assisted sensing systems incorporates distributed information acquired from the drivers, their technical devices, and the vehicle’s internal sensors itself. In the following, a state-of-the-art overview together with a technological comparison is given for nowadays most relevant traffic monitoring techniques of both categories. A condensed overview of the properties is also given in Table
Overview of state-of-the-art traffic monitoring techniques.
Sensing technology | Characteristics ( |
References |
---|---|---|
|
||
|
||
Infrared sensors |
|
[ |
|
||
|
||
|
||
|
||
Camera-based systems |
|
[ |
|
||
|
||
|
||
In-pavement sensors |
|
[ |
|
||
|
||
Directional high-end radar sensors |
|
[ |
|
||
|
||
|
||
Top-down ultrasonic and radar sensors |
|
[ |
|
||
|
||
|
||
Proposed sidefire ultrasonic sensing |
|
|
|
||
|
||
|
||
|
||
|
||
Crowdsourcing systems |
|
[ |
|
||
|
||
|
||
V2X-based cooperative systems |
|
[ |
|
The most widely used approach to traffic sensing is the use of infrastructure-based solutions, which represents a mainly centralized approach. In general, authority of the data is kept with the operator and, thus, several of the problems coming with user-centric techniques do not arise. However, the sensing systems have to be deployed over the whole city or at least at critical locations in terms of traffic. Achieving the required density for an extensive urban traffic monitoring is therefore paramount, which brings along high effort and cost as a centralized solution.
In the previous introductory Section
There also exist initial concepts that use ultrasonic sensors mounted on a lateral street position with a so-called sidefire orientation, facing the opposite street side either horizontally or in a downtilted position [
Sensor systems have specific requirements regarding the setup, scenario, and working environment they are being operated in. In this chapter, the system setup and application concept, together with the generic requirements for our proposed ultrasonic traffic sensing system, are given. The scenario stays the same as initially proposed in Section
An exemplary setup of the smart street lamp platform was already shown with Figure
The typical seamless integration of the ultrasonic sensing concept into an urban street area is shown in Figure
Downtilted sidefire sensor configuration in typical urban environment with two-lane street (with downtilt angle
In order to provide the capabilities for ultrasonic sensing, the core system is extended with a custom sensor platform module, which is shown in Figures
General structure of sensor platform including the FPGA and attached ultrasonic sensors with driving circuits.
Custom sensor platform which is integrated as an extension module into the intelligent street light.
Control of the attached components, signal generation, and preprocessing capabilities is realized with an integrated Xilinx Artix-7 series FPGA. It provides communication capabilities to the core processing system over an Ethernet link, which is also used for the power supply of the sensor system extension. Together with the core system, GPS including the high-precision PPS signal is used for a timing synchronization and exact coordination of multiple sensors in the vicinity of a street. Interference can therefore be coordinated by a global optimization of the sensor timing patterns if the sensor heads are transmitting in the same frequency band.
For simplicity, the signal structure in the single-sensor case is discussed. The transmit signal
Generalized signal structure with transmit/receive duplexing mode, corresponding envelopes, and modulated signals.
The base pulses
With the use of identical repeated transmit pulses in our case,
The ultrasonic traffic sensing technique presented in this paper is based on a combination of statistical analysis and clustering techniques for object candidate detection. Together with a special signal chain structure, objects representing traffic participants can be detected, together with an extraction of further characteristics and object properties. In this section, the model for the received signal is given first. Then, the characteristics of the reflective environment are modeled and empirically analyzed. Based on these findings, statistical hypothesis testing is performed during operation and the resulting statistical information is combined with a modified clustering algorithm for object boundary detection. Then, the system concept for the extraction of further characteristics is given and the reduced-complexity embedded implementation for real-time operation is discussed.
With the general signal structure described in Section
The received signal
The equivalent time-discrete representation is
As a first step of preprocessing of the sliced receive signal on the FPGA, each impulse response is convolved with a bandpass filter
This envelope signal
An example for the envelope signal
Two-dimensional envelope function mapping of received signal
Given the high complexity of the reflective patterns in the sensor data, the statistics of the signal need to be described properly in order to perform an outlier or anomaly detection, which would indicate the presence of objects in the sensor field. The goal is to incorporate all noise components (measurement noise and other acoustic emissions in the ultrasonic band) and also the typical static and time-varying reflections (e.g., moving leaves of a tree caused by wind and reflection patterns of the street) for every specific distance point of the signal into the statistics. This allows the classification of segments of newly acquired impulse responses in terms of the data following the a priori distribution, called base distribution, which was estimated in the past, or whether an outlier is present.
In contrast to typical binary decision problems, here, only this base distribution without any presence of an object can be estimated, representing the null hypothesis. The alternative hypothesis of object presence is however different for every object. This problem can be solved using parametric or nonparametric hypothesis testing techniques for properly representing the underlying base distributions of the signal points and testing the outlier probabilities. However, for the real-time implementation on the embedded system of the sensor setup, we present a simpler and less computationally complex approach that is based on the distance-wise standardization of the signal based on the calculated statistics in a predefined time window in the past. These standardization techniques are often used in the field of data science as a preprocessing step for feature scaling. The distance-wise standardized envelope signal
Under the simplified assumption that the underlying process of
For the object detection, only the right tail of the distribution is of interest for outlier detection, as shadowing effects are not considered. Therefore, the complete standardized distribution data
This yields a variable with a standard normal distribution left-censored at
To summarize,
As a next step, the standardized data given in the previous section is used for object detection. The two-dimensional data
The choice of a suitable clustering algorithm from the field of unsupervised learning techniques for our application was subject to several requirements. While many clustering algorithms can aggregate nearby points in a space with arbitrary dimensionality and sparsely distributed data points, it is required here to incorporate the weight of data points, coming from our the nonsparse standardized and clipped data field
Furthermore, the algorithm should be able to detect important data peaks based on a local density, while, for the whole cluster, no penalty for an infinite extension in the time dimension should be given. This is necessary because the dwell time of an object in the sensor range is arbitrary, especially due to different movement speeds of traffic participants or even in a traffic jam situation.
As a third requirement, an anisotropy of the cluster properties and shape is desired, as we have two dimensions that are time (index of different impulse responses) and distance (specific point in the impulse response itself), which have highly different characteristics. A typical scenario where anisotropy would not be required is 2D image processing, where both image dimensions have similar properties.
First, we want to provide a basic understanding of the original DBSCAN algorithm in general and the specific terms coming with the algorithmic definition. Then, we can introduce the modifications for our specific application. Readers interested in the complete formal description are referred to [
Any cluster
In our special case, the algorithm is modified to suit the needs for clustering our two-dimensional space; therefore
With this definition, all data points of our two-dimensional data set are now taken into account, not just by their count, but by the sum of their assigned weights in the specific
In the clustering process, every element of
In Section
Finally, neither for the left-censored standard normal distribution
Survival function
The neighborhood choice in the time direction,
In Figure
Signal chain with acquisition, analysis, and inference stages on both platforms.
In contrast to the sensor platform structure described previously in Section
On the right side of Figure
The main parts of the processing chain shown on the right side of Figure
For the sensor platform with the FPGA shown on the left side of Figure
Evaluation of the complete system concept for traffic participant detection requires analyses both in real-world scenarios and in isolated conditions with synthetic scenarios, such as on automotive test tracks. A variety of European-wide test measurements have been performed with the system, described in Section
In the following, the framework for parameter space exploration of the whole system with its corresponding algorithms is specified. The basic structure is shown in Figure
Evaluation framework and methodology for parameter space exploration.
Dimensions of Properties of the statistics memory, such as the statistics memory size (past information, see Section Optional use of statistical feedback (see Section Optional clipping threshold for calculated statistical norms in standardization to stabilize against outliers Optional use of matched filtering for the received envelope signal
In Table
Overview of test scenarios and their characteristics.
Scenario name | Number of lanes | Sensor distance range1 | Sensor mounting height | Typical speed range | Pulse interval |
Description |
---|---|---|---|---|---|---|
|
|
7.5 m | 3.5 m | 20–50 km/h | 100 ms | Urban alley street with trees on both sides and parked cars on the adjacent side, mixed use by cars, buses and bicyclists, sensor placement in a horizontal distance of 1 m to curb |
|
||||||
|
|
8 m | 3.5 m | 20–40 km/h | 100 ms | Sensor placement behind sidewalk, distance to curb 2 m, mixed use by cars, buses and bicyclists, reflective trash containers on adjacent side |
|
||||||
|
|
7 m | 5 m | 20–50 km/h | 100 ms | Typical “urban canyon” with large buildings on both side close to the street and narrow sidewalks, distance to curb 20 cm |
|
||||||
|
- | >15 m | 3.5 m | 5–100 km/h | 50–100 ms | Automotive test track without reflective objects, only asphalted road surface in sensor range, low rate of vehicles passing by with highly different speeds |
Results of the field tests and evaluations are given in this chapter. First, the according performance metrics are introduced and specified, followed by the results for the different scenarios and finally a discussion of the results.
In this paper, the main focus lies on the detection of traffic participants as objects and their correct assignment to the lanes of the street. For describing performance in our object detection scenario, no calculation of the true-negative count is possible, as we can have an arbitrary number of objects in our data timeline. Therefore, the widely used
The second important metric is the assignment of objects to the correct lanes for determining the direction of the traffic flow. With multiple lanes as a multiclass problem, the
The performance evaluation results for the different scenario with optimized parameters are shown in Table
Field test evaluation results for different scenarios.
Scenario name | Object count |
|
Weighted |
---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The multilane assignment performance is also very high and of special importance. As no directional or velocity information is available with these sensors, the known information of a fixed lane’s driving direction yields the final traffic flow information with direction. A possible problem is showing up in the scenario
Further future long-term analyses are required to analyze the long-term stability in scenarios with a highly fluctuating traffic flow and day-and-night cycle. However, with the system structure which relies on the standardized data together with the statistics memory, the sensitivity to long-term sensor drift and also production variations is limited. Further investigation is also required on the impact of the statistical feedback for stabilization of the statistical base information. As the parameter optimization is based on only a few parameters, the susceptibility to overfitting effects is limited in the shown results. Still, for future investigations, also the test performance with regard to future measurements in the same scenario is of high interest.
In this paper, it was shown that sidefire ultrasonic sensing is a viable option for multilane traffic participant sensing, giving very good performance results in real-world urban scenarios with a single sensor. The functionality is enabled by a novel combination of standardization techniques based on windowed statistics and a modified density-based clustering algorithm, together called DBStaC. Several evaluations were performed, both in real urban environments and for special cases on an automotive test track. The proposed system is integrated into an evaluation and parameter optimization methodology, whose reference system is flexible to be extended with further object characteristics in future.
The ultrasonic sensor system deployed together with the street lamp platform was presented to be a Smart City technology suitable for high-coverage deployment in cities, enabling traffic monitoring and further applications. With this platform, distributed processing and sensor fusion can expand the possibilities of using available traffic participant information even more, for example, for future applications like trajectory prediction. Further future work can be the use of ultrasonic sensor arrays for acquiring directional and velocity information. In the field of algorithmic improvements, the investigation of more advanced hypothesis testing techniques and channel statistics analyses could yield further performance improvements. Also, the comparison of the presented algorithmic concept with state-of-the art supervised machine learning algorithms such as recurrent neural networks is planned.
The authors declare that they have no conflicts of interest.