^{1}

^{2}

^{2}

^{3}

^{1}

^{1}

^{1}

^{2}

^{3}

This paper proposes a new enhanced method based on one-dimensional direct linear transformation for estimating vehicle movement states in video sequences. The proposed method utilizes a contoured structure of target vehicles, and the data collection procedure is found to be relatively stable and effective, providing a better applicability. The movements of vehicles in the video are captured by active calibration regions while the spatial consistency between the vehicle’s driving track and the calibration information are in sync. The vehicle movement states in the verification phase are estimated using the proposed method first, and then the estimated states are compared with the actual movement states recorded in the experimental test. The results show that, in the case of camera perspective of 90 degrees, in all driving states of low speed, high speed, or deceleration, the error between estimated speed and recorded speed is less than 1.5%, the error of accelerations is less than 7%, and the error of distances is less than 2%; similarly, in the case of camera perspective of 30 degrees, the errors of speeds, distances, and accelerations are less than 4%, 5%, and 10%, respectively. It is found that the proposed method is superior to other existing methods.

As the video image records the vehicles’ motions within the monitoring range and can provide objective raw data for further quantitative analysis, video surveillance equipment has been applied to provide important clues for detection and forensics of vehicle-related applications. For instance, Byon et al. [

Vehicles’ speed has a significant impact on severity of road traffic accidents [

The combination of photogrammetry and computer technology provides a better method for road geometry measurement and vehicle speed calculation [

Overall, the traditional and existing methods of estimating the vehicle motion cannot fully reflect the driving states of the vehicle, or the required conditions are greatly restricted or do not have enough accuracy. This paper develops a one-dimensional direct linear transformation method to solve the entire trajectory of a vehicle in the image sequence of recorded video, by capturing linear structure features of vehicle body and using the trajectory of the vehicle as calibration information. The calibration information of the method is derived from the profile of a target vehicle. The data collection process is found to be stable and effective, which eliminates the dependence, as in other existing methods, on other information such as fixed distances in the image, and the requirement for the quantity and distribution of the road markings. The calibrated area covers the whole motion of the vehicle in the video image, which keeps the consistency between traffic trajectory and calibration information space as much as possible. Based on video frame rates, the minimum step length of the entire vehicle trajectory is used to calculate the distance, which reduces the phenomenon of external interpolation during the solving process and improves the accuracy of results.

In order to discretize the vehicle’s continuous movement from a recorded video with certain image sequences, the video frame rate (i.e., recording frequency) and the vehicle’s position in each image frame are used to compute the change in displacement during each time interval between each consecutive pair of image frames.

Because of the continuity of a vehicle’s movement and the high video frame rates commonly used in the emerging camera devices, it can be assumed that the moving trajectory within the two adjacent image frames are approximately a linear motion. Therefore, the corresponding displacement between adjacent frames of the video can be assumed to be equal to its actual distance, as shown in Figure

Approximate linear motion of the vehicle between adjacent image sequences.

A direct linear transformation does not need interior and exterior orientation elements of the camera, and it is suitable for photographic measurements of images taken by nonmeasurement cameras. The basic formula for solving the direct linear transformation is shown in the following:_{1∼11} are the direct linear transformation factors.

It is assumed that two adjacent frames can be approximated to form a linear motion, so the linear feature of the vehicle body profile can be considered to be on the same line in two adjacent frames. To make the variable

Equation (_{N}, _{N}, _{N}, and _{N}. The four marked points make up three segments adjacent to a straight line. Then, any three of them are selected as control points. And based on the distance of the pixels and the actual distance between them, one-dimensional direct linear transformation coefficients _{N} = [_{1}, _{2}, _{3}] of _{N + 1} and _{N} and the direct linear transformation coefficients _{N}, the object space distance of _{N + 1} and _{N} are calculated.

Distance calibration between adjacent image sequences.

Then, we can find the _{N + 1} in accordance with the image distance and the actual distance of any three points of

The solution procedure described in the “

Extract the image and time-stamp of the target vehicle in the desired duration of video frame by frame.

In each frame, four intersection points are formed between the straight line at the outer end of the wheel axis on one side of the target vehicle and the rim edge as the reference points, respectively set to

With the

Based on the image point coordinate

On

Based on the image point coordinate of

Calculate the direct linear transformation coefficient

According to the driving distance of the target vehicle in the image sequence, combining with the frame rate of the video, calculate the vehicle’s speed and acceleration corresponding to the specified images, frame by frame.

Output file information, vehicle feature point spacing, extracted video image range, image sequence and time-stamp information, feature point image coordinates on each frame, one-dimensional direct linear transformation coefficient

This section describes the validation test set-up as the following:

Test material: a manual transmission car, Kistler S-350 photoelectric five-wheel instrument, LED monitors, two cameras, SIRIUS digital collector, a notebook computer (for channel setup and data collection), three-dimensional laser scanners, checkerboard, tape, and BMW label.

Test site: a 150-m-long straight lane.

Test scenario: Figure

Test vehicle, equipment, and scene.

Original images taken by the video camera.

Images corrected in distortion.

Analytical process of the

First of all, make a gray-scale change of the image of three-dimensional matrix information and translate it into a two-dimensional matrix. Calibrate the vehicle contour of three feature points, as shown in Figure

Second, the image plane coordinate system is established in the upper left corner of the image as the coordinate origin. Extract pixel coordinate values for three feature points

At the same time, through the one-dimensional direct linear transformation equation (

Put above three points’ pixel coordinate values and the object-square coordinate values in the formula and obtain the matrix of coefficient

For

Relative to _{a} = 33 in equation (_{a} = 31.25 cm. Therefore, it can be considered that the point

With the same method to all frame image processing, calculate the displacement between two adjacent frames of the test vehicle for d

Calculation results of the displacement between 28 image sequences (unit: cm).

1 | 2 | 3 | 4 | 5 | 6 | 7 | |

35.38 | 34.55 | 35.71 | 34.88 | 32.80 | 37.60 | 33.59 | |

8 | 9 | 10 | 11 | 12 | 13 | 14 | |

35.96 | 33.97 | 37.19 | 36.12 | 34.95 | 36.41 | 34.33 | |

15 | 16 | 17 | 18 | 19 | 20 | 21 | |

33.35 | 34.25 | 34.40 | 36.01 | 34.96 | 35.58 | 34.56 | |

22 | 23 | 24 | 25 | 26 | 27 | — | |

34.65 | 35.25 | 31.60 | 34.84 | 34.09 | 33.62 | — |

Three feature points calibrated on vehicle exterior contour.

Calculated average speed and its fitting curve.

Compared analysis between

From Figures

Then, we take the derivative equation (

At the same time, since the data collection instrument does not record the acceleration value over time. This paper takes the derivative of _{0}. Then, we fit a third-order polynomial from the scatter value _{0}, as shown in Figure

Compared analysis of _{0}.

There is a large deviation between the _{0} fitted curve and calculated _{0} is scattered. Because the change of test speed is small (<2 m/s^{2}), close to a uniform motion, while sampling period is as short as, the speed error reflected on the acceleration is amplified 100 times by the sampling frequency.

We notice the time

Linear regression of _{0}.

_{0}.

Compared analysis of _{0}.

_{0} reflect that the car is approximately under uniform motion and the linear regression equation is

The two slopes and intercepts are similar. The cumulative driving distance in 0.9 s results in _{0} = 9.47 m, between which the difference is 1.9%. It is found that the calculation of the driving distance is valid.

Due to the 30° of deflection angle between the lens orientation and the driving direction of vehicle, it is difficult to ensure that the feature points of the body of the vehicle are located on the same line. As shown in Figure

Difficult to ensure the selected feature points are on the same line.

First, 3 feature points for each frame are selected (126 feature points in all). Second, a linear regression of 126 feature points is conducted, as shown in Figure

Coordinates and linear regression of 126 feature points.

Partial ordinates of feature points corrected.

33 | 249 | 244 | 102 | 236 | 232 | 157 | 226 | 224 | |

62 | 236 | 238 | 126 | 226 | 228 | 175 | 220 | 221 | |

204 | 214 | 216 | 241 | 209 | 210 | 275 | 204 | 205 | |

61 | 243 | 238 | 120 | 232 | 229 | 171 | 226 | 221 | |

84 | 232 | 235 | 144 | 225 | 226 | 191 | 218 | 218 | |

216 | 212 | 213 | 253 | 207 | 209 | 284 | 202 | 204 | |

82 | 237 | 235 | 141 | 226 | 225 | 190 | 223 | 219 | |

106 | 232 | 231 | 158 | 220 | 223 | 204 | 217 | 216 | |

230 | 209 | 212 | 262 | 203 | 207 | 293 | 204 | 204 |

The curve of

Compared analysis between

As Figure

Then, the derivation of the equation results in the following for

Linear regressions of curve _{0} are shown in Figure

Linear regression of _{0}.

The difference between the mean values of two accelerations is 5.6%. Although the two fitting line slopes are opposite, the lines are below 0, indicating the acceleration is always negative.

_{0}. Two curves approximately coincide, in 1.4 s with _{0} = 14.4 m, resulting in the difference of 4.9%, which satisfies the error requirements.

Compared analysis of _{0}.

Compared analysis between

The average speed of

The coefficient of determination ^{2} = 1 indicates the regression is significant, and the vehicle is approximately under a uniform motion, while the speed is 21.37 m/s (76.93 km/h). The difference between the estimated _{0} = 6.40 m is 0.3%, meeting the error requirements. Figure

Compared analysis of _{0}.

The comparisons of its acceleration values are not discussed here, as the experiment is very close to uniform motion and similar to the situation described in the “

Coordinates and linear regression of 75 feature points.

Based on the modified feature point coordinate, we obtain

Compared analysis between

The average speed of

_{0} are shown in Figure

Compared analysis of _{0}.

After the regression, with coefficient of determination ^{2} = 0.98, the vehicle is approximately under a uniform motion. In 0.8 s, the difference between calculated _{0} = 17.05 m is 1.9%.

Compared analysis between

We took

The calculated result of

Compared analysis of _{0}.

_{0}

The resulting acceleration difference is 6.9%.

By the same method, driving speed and distance can be obtained as follows in Figures

Compared analysis between

Compared analysis of _{0}.

In this paper, a method based on a direct linear transformation is proposed to calculate the moving state of a vehicle in a video image, which can fully reflect the running state of the vehicle based on the collected data and is found to be stable and easy to obtain. The application condition is thoroughly described throughout the paper. Examples are given to show the estimation processes of the speed of the vehicle, acceleration, and driving distance under 2 angles for 3 different motion states.

In order to verify the validity of the method, vehicle motion states are estimated in experimental test cases; the results show that in the 90° angle of camera perspective, low-speed, under-medium-speed, and high-speed and deceleration running three states, the calculation of the speed and record value error is found to be less than 1.5%, the acceleration value error is less than 7%, and the driving distance value error is less than 2%. In 30° angle of camera perspective of view, low-speed, under-medium-speed, and high-speed and deceleration running three states, the calculation of the speed and record value error is found as less than 4%, the driving distance value error is less than 5%, and the acceleration value error is less than 10%. The performance of estimations made by the proposed method in terms of errors for vehicle speed and distance value is found to be higher than other existing methods, and the error of acceleration value complies with the error requirement, which shows that the method presented in this paper is effective in solving the driving state of the vehicles in the video.

This paper does not currently attempt to estimate the vehicle movement states of running trajectory under the view of overlooking camera. We will continue our research by adding vehicles’ cross-lane curve states in the near future, and the selection criteria of the feature points of the body profile under overlooking camera will be newly defined.

All of the data related to this paper are available for peer researchers to validate.

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

The authors declare no conflicts of interest.

This research was funded by the National Key Research Development Program of China: Research on Digital Simulation and Reappearance Technology of Road Traffic Accidents, Grant no. 2016YFC0800702-1, Shanghai Committee of Science and Technology Program for Science and Technology Development: Research on the Key Technology of Monitoring Video Forensic Identification, Grant no. 17DZ1205500, National Natural Science Foundation of China General Program: Study on traffic injury mechanism based on digital scene reconstruction, Grant no. 81571851, and ADEK Award for Research Excellence (AARE-2017), Grant no. 8-434000104, Central-Level Research Institutes Public Welfare Project, Grant no. GY2020G-7, and Central-level Research Institutes Public Welfare Project, Grant no. GY2018Z-3.