Driving Performance Assessment Based on Accurate Position Tracking

A driving performance assessment (DPA) system has been proposed in this paper to evaluate drivers’ skills in training. The system is based on centimeter-level localization of the vehicles, thanks to differential BeiDou Navigation Satellite System (BDS). Given a vehicle’s dimensions, its envelopment has been discretized both temporally and spatially as binary images, while the training area is modeled as a grayscale image where the intensity denotes the penalty of the certain area in unit time. The performance index can be obtained from the summation of images along with time. Experiments have been conducted to demonstrate the accuracy of vehicle tracking and the effectiveness of the proposed assessment system.


Introduction
Autonomous vehicles (AVs) have been witnessed vigorous and fruitful in the last decade, with many applications in public transportation, logistics services, environmental health, disaster response, and space exploration [1][2][3][4][5].Driving performance assessment (DPA) plays an essential role in building a robust AV system by providing feedback on the performance of an autonomous driver.Explicitly, learning-based approaches require a criterion such that di erent driving trajectories could be appropriately rewarded or penalized [6][7][8].Besides potential applications on AVs, DPA is also helpful during intelligent driver monitoring and training [9][10][11].
In related work, driving performance has been assessed through mainly two perspectives, namely, (1) driver-centered and (2) vehicle-centered approaches.As for drivercentered methods, driver's physical and mental status [12] and other subjective indicators [11] have been used for assessment.
ese evaluation indexes may re ect driving performance from the source but are sometimes hard to quantify, thus are trying for a fair evaluation in a variety of driving scenarios.On the other hand, vehicle-centered approaches focus on the vehicle's velocity, acceleration, trajectories, and many other designed metrics.Authors in [11] have used completion time and the number of collisions to compare driving skills in di erent groups.In [13], an approach measuring maneuvering time, lateral velocity, and acceleration has been proposed to evaluate a lane change algorithm.In [14,15], traveling distance and clearance to obstacle have been used as the cost function to optimize in AV trajectory planning problem.
is work proposes a DPA approach by evaluating vehicles' trajectories.Firstly, a BeiDou Navigation Satellite System-(BDS-) based position tracking module which accurately records the vehicle's trajectory has been built.BeiDou Navigation Satellite System (BDS) [16,17] has been in rapid expansion since its rst launch in 2000.As an alternative of GPS in the Asia-Paci c region, BDS has been widely used in transportation [18], agriculture [19], disaster prevention and mitigation [20], and many more.e tracking module includes a xed base station and a mobile base station, where the xed base station receives BeiDou satellite ephemeris data, and the mobile base station, which is mounted on the vehicle, receives di erential data from the xed base station.en, an assessment approach has been proposed to evaluate trajectories such that the performance index could be obtained for each recorded trajectory based on the prebuilt map.e map has to be manually defined as a grayscale image, where the intensities indicate local penalties in unit time.anks to the high-accuracy trajectory recorded from the tracking module, driving performance can be scored by integration concerning space and time.Software to collect and process data has been developed for rapid and easy implementation.Experiments have been done on passenger vehicles to validate the effectiveness of the proposed DPA system.e rest of this paper is organized as follows: Section 2 presents an accurate position tracking system.e proposed DPA algorithm is detailed in Section 3. Experimental studies and results are shown in Section 4. Finally, Section 5 concludes the paper and discusses possible future extensions.

BDS-Based Accurate Position Tracking
is section presents our proposed position tracking system structure, which includes both hardware configurations and front-end software.

Systematic Architecture.
e architectural diagram of the proposed position tracking system is shown in Figure 1 [21].Two BeiDou base stations need to be installed at different places.One is fixed as the reference point, for which the position is chosen depending on the practical situation.
e other station, a mobile station, is placed properly on the vehicle to be tracked.e fixed station consistently receives the ephemeris data from BeiDou satellites, and the data are used for differential data calculation.e resulting differential data are then broadcasted out to the mobile station through the wireless or UHF radio data link.When data are captured and processed with necessary calculations, the tracked vehicle can be located in real time [22,23].e position and other related information calculated on the terminal computer will be sent to the monitoring center where data are displayed and logged.

Fixed Base Station.
As shown in Figure 2(a), the fixed base station includes a Global Navigation Satellite System (GNSS) receiver, GNSS antennae, a wireless or UHF digital radio station, the power supply equipment, and the lightning protection equipment.
BeiDou fixed base station, which is the origin in the system reference frame, receives and processes satellite signals continuously.It is powered by automotive batteries or any stable voltage of 12-220 V. e difference correcting value (DCV) of the base station is broadcasted through the onboard wireless/UHF digital radio station in real time, such that high-precision carrier phase differential data and initial coordinates for the mobile station can be obtained.DCV is the basis to achieve centimeter-level vehicle positioning precision, since the continuity and reliability of the differential information broadcast are directly related to the GNSS accuracy.As for base station location selection, the following items should be considered: (1) e site is stable and solid without any visible sinking (2) e height cutoff angles of visual obstacles' heights should not exceed 15 degrees (3) e distance to any high-power radio source is not less than 200 m (4) e distance to any high voltage transmission line and microwave radio transmission channel is not less than 50 m (5) Lightning rods should be installed 1-3 m away from BeiDou fixed reference station and above its position

Mobile Station.
e mobile station consists of a mobile terminal GNSS receiver host, GNSS antennae, and the power supply, as shown in Figure 3(b).In this work, the mobile terminal station GNSS receiver host is placed inside the mobile terminal.e GNSS antennae are at the top of the mobile terminal, installed, and fixed by a sucker or welding at a distance of 70 cm.
Similar to the base station, the mobile station is also powered by a car battery or any stabilized voltage source of 12 V to 220 V. e data stream between the BeiDou fixed base station and the mobile station is shown in Figures 3(a) and 3(b), respectively.

Error Analysis.
e principle of the proposed position tracking system is shown in Figure 4. Since the BDS-based system outputs ENU coordinates only, transformations are necessary if coordinates in global frame are required.In particular, vectors AB � �→ and AC ��→ in ENU frame can be obtained from BDS.With a measured CD ��→ in the vehicle's body frame, CD ��→ is then transformed to the global frame as the tracking result.Without considering the roll angle, it is noted that baseline position error δ would lead to angle error δ/L rad, where L denotes the baseline length.By writing the position error of the BDS-based system as δ, we have errors of baseline, pitch, and yaw in the ENU frame as e errors of baseline, pitch, and yaw caused by the transformation from the ENU to the global frame are e 2 � 0, e (2) 2 Wireless Communications and Mobile Computing From ( 2)-( 4), it is observed that e � e 1 , e yaw � e yaw 1 , but e pitch ≠ e pitch 1 after transformation.e error of yaw would contribute to the total position error: In

Database and Application Design.
e mobile application has been developed in Android 4.4.4,a Linux-based open-source operating system which consists of an application layer, an application framework, a system runtime, and a Linux kernel.
e model-view-controller (MVC) simplified model-view-presenter (MVP) framework is used in development.e framework is divided into a view layer for graphical user interface (GUI) and interaction design, a presentation layer which handles the responsibility logic in interaction, and a model layer, which operates the database.In order to complete the training and evaluation task, four functional modules, namely, "sign in," "training," "inquiry," and "configuration," have been designed in the application, as shown in Figure 5.

Driving Performance Assessment
Since the exact position of the vehicle has been obtained, this section focuses on designing a straightforward approach to evaluate the driver's performance.Suppose vehicle's position has been converted to East-North-Up (ENU) frame.en, the two-dimensional approximation is applied by removing z-axis from the coordinates by assuming that the vehicle's trajectory is relatively short compared to the curvature of the earth's surface.By writing the state vector as x � x, y, θ   T that contains the vehicle's two-dimensional coordinates in translation plus one-dimensional coordinate in rotation, each trajectory can be represented as a function in state space x(t) � x(t), y(t), θ(t)   T .e DPA problem is then formulated as follows: Design a function g(x) such that for each given trajectory x(t), a performance score can be computed.

Trajectory Envelopment and Map Representation.
As vehicles have distinct dimensions which should be taken into consideration during DPA, the envelopment of the vehicle is defined as a static binary scalar field u e (x, y) which satisfies where P denotes the set containing all points within twodimensional (x − y − z to x − y) projection of the vehicle in its body frame.Note that θ is not included here as u e (x, y) is in the vehicle's body frame instead of the global frame.
Based on the concept above, we further define the trajectory envelopment u t (x, y), which satisfies u e (x, y) � u t (x(t) + x cos θ(t) − y sin θ(t), y(t) + x sin θ(t) + y cos θ(t)), (7) where x and y denote coordinates; x(t), y(t), and θ(t) are obtained from trajectory x(t).It is noted that u t is varying with time due to vehicle motion, but we omit the variable t for a concise notation.From ( 7), u t is actually u e represented in the global frame.
To evaluate if the vehicle is at the desired position quantitatively, a map in the global frame is introduced as an integrable static scalar field in two dimensions u m (x, y) satisfying u m (x, y) ≥ 0. e map reflects the area where the vehicle should stay on: e larger u m is, the more penalty will be received for the trajectory in unit time.e explicit form of u m (x, y) is case-by-case depending on the application.In this paper, we apply the following map: where G denotes the goal set, which is defined manually; d(x, y) represents the distance between point (x, y) and the nearest drivable route that could be extracted from the map; the positive scale s represents how we measure the influence of distance; and a is a constant offset.4 Wireless Communications and Mobile Computing

Graphical Trajectory Evaluation.
With a map u m (x, y), the following operation is proposed to evaluate the trajectory envelopment u t (x, y) derived from trajectory x: Firstly, we multiply u m (x, y) and u t (x, y) to create a new scalar eld which indicates the driving performance of the vehicle's current position.Both the dimension and position of the vehicle are considered during evaluation.en, the new scalar eld is integrated with respect to space and time such that the vehicle needs to reach the goal area as fast as possible while keeping close to drivable route in order to achieve higher evaluation score.

Approximated Version of DPA Algorithm.
In practical implementation, the approximated DPA is required to ensure e ciency on a variety of platforms.In this work, we approximate u t (x, y) and u m (x, y) as binary and 8-bit grayscale images U t (X, Y) and U m (X, Y), respectively, for convenient visualization.Each pixel in the image represents a grid with particular dimensions such that the coordinate transformation (X, Y) ⟼ (x, y) is linear.As the trajectory envelopment is varying with time, we nally obtain a sequence of images U t (X, Y, k) where k denotes the discrete time index.It is required that the time sampling interval remains the same in order for a fair evaluation at di erent times.
As matrices are used to represent images, the approximated evaluation function is written as in which ∘ denotes the Hadamard product (or the Schur product).

Experiments
In this section, the e ectiveness of the proposed DPA approach has been demonstrated by actual tests.Since GPS has been widely applied to various applications, a criterion system with a GPS-based tracking module has been used in experiments for performance comparison.

Evaluation on Tracking
Accuracy.We rst evaluate position tracking accuracy before implementing DPA on the vehicle such that its trajectory can be precisely obtained.
During all experiments, data recorded by the tracking system have been converted to 2D by assuming the vehicle moves around a relatively small region where the ground could be deemed as at.
As shown in Figure 6, the testing vehicle runs three loops along the same route over 900 m.As the output frequency for the BDS and GPS systems are di erent, the Hausdor distance [24] is used to measure the error of trajectories of the proposed system.
e Hausdor distance measures the maximum distance from points in one trajectory to the nearest point in the other trajectory.It is shown that the two trajectories are almost the same for most of the time, and it is computed that the average Hausdor distance for the three loops is 5.44 m.

Evaluation on DPA System.
Experiments have been conducted to verify the e ectiveness of the proposed DPA system.During map generation, di erent resolutions may be chosen.Figure 7 illustrates maps from coarse to ne.It is noted that a ner map leads to smoother and more accurate approximation but would also increase computational complexity.We approximate the map as 750 × 1100 8-bit grayscale image U m (X, Y), where each pixel represents 0.02 meters.e map intensities at the starting area and the target area are set to o set a and 0, respectively.As shown in Figure 8, two trajectories from the starting area to the target area have been collected in this work, and the physical dimension of the vehicle has been deemed as zero.In other words, U t is a single-pixel image that moves along with the trajectory.e results are shown in Figure 9, which illustrate the corresponding evaluation scores with respect to discrete time.It is noted that the evaluation score is monotonically increasing due to time integration.As for the groundtruth trajectories from GPS, the 1st trial obtains lower score owing to its slight deviation to drivable roads and faster completion.However, as shown in the 2nd trial, the BDS-based tracking system sometimes outputs intermittent tracking results, which explain why DPA gives di erent results.Figure 10 shows the in uence on evaluation score with di erent map resolutions ranging from 3000 × 4400 (0.005 m/pixel) to 15 × 22 (1 m/pixel).e scores are quite steady while the resolution is smaller than 0.1, and the slight disparity comes from approximation errors.Wireless Communications and Mobile Computing We further show the real-time performance of DPA as in Figure 11.It takes 0.08 s to compute scores for the two pairs of trajectories on average, with 0.02 m/pixel resolution.It is evident that the time consumption for score computation decreases with a coarser map.Based on the above discussion, it is a case-by-case trade-o between algorithm e ciency and approximation accuracy in practical implementation.

Conclusion
In this paper, a driving performance assessment system, which includes a di erential BeiDou Navigation Satellite System-based accurate position tracking and a back-end trajectory evaluation method, has been designed.Experimental validation has been made during eld tests to demonstrate the e ectiveness of assessment, given a prede ned map.e proposed system be applied to automated driver training, or as a quanti able criterion to be fed into learning-based autonomous driving approaches.e plot with error bar shows mean and standard deviation of time consumption in 100 evaluations.

yaw 2 � 0, and e pitch 2 �
δ/|AB � �→ |, respectively, since the global frame is simply rotated from the ENU frame with the baseline length |AB � �→ |. us, the resultant errors of baseline, pitch, and yaw are derived as squared summations: e � δ,

Figure 6 :
Figure 6: Tracking accuracy evaluation trajectories of the proposed (a) and the groundtruth (b) systems.e vehicle runs three loops along the same trajectory.

Figure 7 :Figure 8 :Figure 9 :Figure 10 :
Figure 7: Maps with di erent resolutions.Testing area is 15 × 22 m, with starting area on the left and target area on the right.e scale and o set are set to s 10 and a 64, respectively.Resolutions from left to right: 0.2, 0.1, and 0.05 m/pixel.Vertical axis is inverted in image display.

Figure 11 :
Figure 11: Real-time performance of DPA with respect to map resolutions.eplot with error bar shows mean and standard deviation of time consumption in 100 evaluations.