An Evaluation Method of Ship-Tracking Algorithms for High-Frequency Surface Wave Radar considering High Maneuvers Generated by the MMG Model

,


Introduction
Te HFSWR is an efective maritime remote sensor covering a long distance of 200 nmi and has various functions, including ship detection and tracking. As a primary radar, it can provide continuous and real-time monitoring. Benefcially, it can detect noncooperative ships, which turns of the automatic identifcation system (AIS) radio. Tus, it can fll in the shortcoming of AIS [1].
Beyond the advantages of HFSWR for monitoring ships, it has some challenges in the detection and tracking process. Te Bragg components produced by the interaction between dynamic sea waves and radar signals cause a high false alarm for ship detection [2]. Besides, the ionospheric clutter and radio frequency interference can mask the scattering signal of the ships [3]. Another challenge in developing ship detection on HFSWR is the low spatial resolution of radar detection, including the radial range and the azimuth resolution [4]. Te relatively narrow bandwidth on the HF band produces a radial range resolution that is much larger than the ship dimension. Moreover, the azimuth resolution in the radial coverage area causes a lower resolution at a farther distance from the radar location, which produces a higher deviation of the detected ship location.
Although the sailing ships are statistically moving in regular linear track and low maneuver, those with illegal mission sail with ghost mode or turn of the AIS transmitter and make a high maneuver to hide from the radar monitor.
In addition, the military vessel is also possible to do so. Terefore, the tracking of high-maneuvering ships is necessary to be investigated. Unfortunately, the low resolution of HF radar detection afects the performance of the ship tracking processes. Te maneuver or sudden change of ship motion can degrade the tracking accuracy [5]. Terefore, a more robust tracking algorithm is required to track high maneuvers, which is still a challenging problem in ship detection by HFSWR.
Generally, the tracking algorithm in the research is verifed by two kinds of data. Firstly, the empirical data gathered by AIS are utilized as the tracking reference [5]. Te sailing ship is mandatorily equipped with an AIS transmitter which regularly broadcasts its information, including position and velocity. Te AIS receiver collects the signal and the data, which can be used to assess the tracking results of HF radars. However, the linear tracks dominate the data for any assessment of the tracking algorithm [6,7]. Te AIS data are sometimes sufered from track fragmentation which can reduce the availability of ground truth data for this study [8]. Wang et al. proposed a grammar-based model based on AIS data mining representing the ship maneuver scenario to assess the tracking algorithm performance [9]. Of course, the considered ship maneuvers are limited by the existing scheme in the dataset and possibly missing some other possible plots. Furthermore, Wang's work is more concerned with evaluating the algorithm's ability to track multiple ships simultaneously. Secondly, the theoretical tracks produced by a specifc mathematical model are used to test the performance of the tracking algorithm [10]. An efort to include ship maneuvers has been proposed by Zhang et al. with the assumption of constant velocity and constant turn of ship motion [11]. Te theoretical data are more controllable and provide a more exact ship position but a less natural characteristic of ship maneuvers. Te efect of ship maneuvers is also discussed in terms of data integrity on AIS systems regarding the multitrack detection.
Meanwhile, ship maneuverability in naval architecture and ocean engineering is commonly expressed by mathematical models categorized into the whole ship and modular models [12]. Te whole-ship model constructs a maneuver from the total hydrodynamic forces and moments acting upon a complete entity of the vessel. In comparison, the modular model considers the hydrodynamic forces working on separate modules of ship elements, such as hull, rudder, and propeller, to produce ship maneuvers. A famous modular model has been proposed by a research group called the maneuvering modeling group (MMG) at the Japanese towing tank conference (JTTC) as named as MMG model [13]. Te MMG model calculates the ship maneuver based on the hydrodynamic force on the diferent modules, i.e., hull, rudder, and propeller. Te model's ability to simulate various possible ship motions can be adopted to assess the tracking algorithm performance in HFSWR. To the best of the authors' knowledge, this idea has not been presented in any literature before. Tus, it becomes the main contribution of this paper.
Our research proposed evaluating the tracking algorithm by considering the ship's high maneuvers generated with the MMG model HFSWR. Tis paper uses an MMG modelbased software, i.e., MANSIM [14], to generate the ship tracks. Te validity of MANSIM in the empirical measurement of full-scale or miniaturized ships has been reported [15]. Next, the maneuver model is combined with the HFSWR signal model in our previous work [16]. Te detection results of combined two radar stations are computed and used for input of tracking systems. Te performance of four tracking algorithms, i.e., Kalman flter (KF), extended Kalman flter (EKF), unscented Kalman flter (UKF), and particle flter (PF), are evaluated on three ship maneuvers: linear, turning, and zigzag. Since the radar coverage is radial basis, the radar resolution is gradually decreasing in the farther distance. Te Monte Carlo simulation is conducted to randomize the initial ship position and heading to handle this issue. Finally, the tracking performance is evaluated by calculating the root mean square error (RMSE) of absolute position, radial range, bearing angle, and speed [4].
Te rest of the paper is organized as follows. Section 2 describes the research method that consists of the tracking algorithm and evaluation method. Section 3 reports the results of model implementation on the defned scenario and the evaluation of tracking algorithms. Finally, Section 4 summarizes the research fndings and future works.

Research Methods
Tis paper aims to evaluate the performance of the tracking algorithm considering the efect of ship maneuvers. Subsequently, the MANSIM software was taken into account to generate the ship tracks based on the MMG model and fed as the input to the HFSWR simulator. Te sequential data of ship detection were used to evaluate the performance of the tracking algorithms. Figure 1 shows the block diagram of the research method for tracking performance evaluation.
Initially, the MMG model generated linear, turning, and zigzag maneuvers. Ten, the Monte Carlo simulation was brought into account to randomize the initial position and heading of the ships. Te tracks were inputted to the two independent HFSWRs, i.e., Radar A and Radar B. Four popular algorithms were assessed in the current works: KF, EKF, UKF, and PF. Te performance of those trackers was observed by calculating the RMSE of the detected tracks compared to the exact positions in rectangular and spherical coordinate systems. Figure 2 describes the developed ship tracking system and data generation with the MMG model. Te tracking algorithm functions to estimate the ship's trajectory (x k ) from the sequence of the ship's position (z n ), as shown in the end process in Figure 2. Te input to this system is in the radar received signal, fed to the receiving antenna elements, which form the receiving array. Te receiving array consists of n-element antennas where each n th antenna element receives the radar receiver signal scattered by the ship (S r,n ). Te signal received by all elements of the receiving array is processed by a series of processes to detect the target position sequence (z n ). Te details of signal processing and detection methods are not discussed here but can be found in our previous publication [16].
Te considered radar is the FMCW radar; so the radar reception signal is a copy of the broadcast signal with attenuation and time delay. Te attenuation and the time delay are calculated from the radar receiving antenna and the scattering target positions. In this case, the target is referred to as the ship. In the current model, the MMG model generates the ship position, as shown by the left block of Figure 2. In this data generation process, the MMG model generates ship track data which consists of successive points in the coordinate system from (x 0 , y 0 ) to (x k−1 , y k−1 ), where k is the number of data samples. Tis track data are then stored in the form of vector x s . Te ship's position sequence on the trajectory made with the MMG model is then used to calculate the radar-received signal sequence. So, the input for the ship tracking system is HF RADAR signal data from the refection of a moving ship with the trajectory obtained from the MMG model.
Te last process is for testing the ability of the radar tracking algorithm. Te radar target detection system produces a series of ship positions (z k ) used as input for the tracking algorithm being tested. Te accuracy of the tracking algorithm is measured by comparing the track results (x k ) with the actual path (x s ) generated by the MMG model.

Radar Specifcation.
In this paper, the HFSWR confguration is considered in this work's prior publication [16]. Te radar is monostatic with a frequency modulated continuous wave (FMCW) waveform and a linear array type antenna on the receiver. Te radar confguration is listed in Table 1, which is chosen for proper ship detection following the work of Dzvonkovskaya et al. [17].
A single radar covered an area of 120°azimuthal wide, and the maximum range was set to 80 km. Te 1.5 km radar range resolution was obtained from the 100 kHz bandwidth based on the relation of c/2B, where B is radar signal bandwidth (Hz) and c � 3·10 8 m/s is the radio wave speed in free space. Te radar deployed 16 elements of an array antenna to provide the azimuth resolution approximated to 7.5°by assuming the beamwidth of uniform linear array (ULA) [18].

Exact ship trajectory
Radar signal generator Z k = X k = Data generation Target Detection and Tracking Figure 2: Data fow from MANSIM to simulator in the current evaluation.
Te 2 Hz chirp repetition frequency with 28 sampling per chirp provided the CIT of 128 s. Te radar could detect the target with the maximum absolute speed of 15 m/s or 29.16 knots in this confguration. Following the reference work, the target detection had a 75% overlapped Doppler map. Tus, it could provide position updates every 32 s [13]. In addition, two identical radars were assumed in the simulation scene with a 40 km separation and an 18°slanted angle to obtain the maximum overlapped area. Te predefned position and direction radars are shown in Figure 3 to indicate the radar coverage area.
Tere were two coordinate systems used to represent the position in space and moving ship. Te space coordinate system of space was annotated by x 0 -y 0 -z 0 with o 0 as the origin point. Te x 0 -y 0 plane coincided with the water surface, and the z 0 axis was vertically downward. Te moving ship position was presented by x-y-z, with the origin point o was located in the midship. Te x-axis was directed to the bow of the ship.
Te other parameters are also drawn in Figure 4. Te total ship velocity is represented by U � ������ � u 2 + v 2 m , where u and v m are velocity components along x and y axes. Te ship's center of gravity is assumed in the midship position and located in (x G , 0, 0) in the x-y-z system. Te lateral velocity in the center of gravity is defned by the following equation: where r is yaw rate and x G is the ship's center of gravity. Te ship motion consists of three components: yaw, sway, and surge. Tose components relate to the hydrodynamic forces with the following relationships [13]: where I zG is the moment inertia of a ship around the center of gravity. Te dot notation in _ u, _ v, and _ r means the frsttime derivation. X, Y, and N m are the hydrodynamic forces and moments acting on the center of gravity, further derived from the modular components.
Te subscripts H, R, and P mean hull, rudder, and propeller, respectively. All parameters on the right-hand side of (3) are the components of hydrodynamic forces that act on the ship. Tose components are derived based on many parameters from the simulated ships. Te complete expression of each parameter can be found in [13].  In this paper, the ship track was generated by using the MMG model run on a software named MANSIM [14]. Tis software has been validated to the empirical measurements on the full scale or miniaturized ships [15]. Figure 5 shows the interface of MANSIM, which contains a sort of predefned parameters to simulate the ship maneuvers by the MMG model and the resulting track. Te current work used the KVLCC2 tanker in the reference published by Yasukawa and Yoshimura [13]. Te tanker's principal specifcation is listed in Table 2 to provide a glance at the ship. Besides the parameters listed in Table 2, some hydrodynamic coefcients were also needed during the simulation. Tis paper does not mention those parameters, but they are provided in [13].
Te MMG model is elaborated in this paper to simulate three basic maneuvers: linear, turning, and zigzag with the driving parameters listed in Table 3. Tose motions are simulated by assuming the constant propeller revolution, which produces the initial speed of 10 m/s. Te motion is generated by the rudder angle set to 35°. Te zigzag maneuver is simulated with the steering pattern 10/10 (10 s of +35°and 10 s of −35°, alternately).

Tracking
Algorithms. Various tracking algorithms have been known in the radar felds. However, the HFSWR has a unique challenge due to its low spatial and temporal resolutions, high-density multitarget, and being interfered with by sea clutter. Various tracking algorithms have been proposed to work on the HFSWR that can be categorized based on the linearity of the target motion model. Based on the presumption that most of the vessels move on linear trajectories, some linear algorithms have been proposed for HF radar, such as the alpha-beta (AB) tracker [19] and Kalman flter (KF) tracker [5,20,21]. Te KF tracker was the improved version of the AB tracker to be more adaptive to the target maneuvers.
However, the ship sometimes moves in nonlinear motion during the maneuver, which reduces the accuracy of linear trackers. Terefore, there are some modifcations of the KF tracker to work on nonlinear motion, such as extended Kalman flter (EKF) [22,23], unscented Kalman flter (UKF) [23], and particle flter (PF) [24]. In this paper, the performance of those nonlinear trackers was evaluated on the turning and zigzag tracks. In addition, the KF flter was also included as a comparison.
Te explanation of those trackers is started from the target dynamic model from the state of two-dimensional moving targets with nearly constant velocity expressed in general matrix form as follows [25]: where f(·) is the dynamic model function, x k is the target state vector in the time k containing position, w k is the Gaussian noise of the process, and u k is the control vector. Next, the radar measurement as the input of tracking can be expressed by the following equation: where z k is radar measurement data in the time k, H(·) is the measurement model function, and v k is the measurement noise. Te tracking process consisted of four steps: prediction, innovation, correction, and updating, which can be briefy described as follows.

Prediction.
Te position is predicted at time k based on target state on time k-1 where the tilde means the predicted value. Te Kalman flter uses the linear motion model, so equation (6) can be derived as follows: where the transition matrix F k is defned as where T k is the radar time resolution. Te other algorithm uses nonlinear motion models. For example, the Taylor series expansion for the EKF is where J() is the Jacobian function. Te prediction step also estimates the following predicted covariance: where F k is the Jacobian of the state and Q k is the process noise covariance matrix.

Innovation.
New information is gathered from the measurement on time k for calculating the measurement residual and formulating the innovation matrix S k . Te measurement of residual and innovation matrix for the Kalman flter is calculated by the following equation: where H is the transition matrix and K k is the weight equation or commonly called the Kalman gain. For the EKF, this process was calculated by the following equation: where R k is the covariance matrix of measurement noise and H k is the jacobian of the state.

Correction.
Te target state is estimated with the correction from the innovation. Tis process generally can be expressed as Te hat was used to sign the predicted value. In addition, the covariance matrix was also corrected by using the innovation data:

Updation. Information is updated for the next iteration.
Te diference between the methods is on how to calculate the Kalman gain and assumed dynamic model function. Te detailed explanation of the tracking algorithm is not explained in this paper and can be found in [26].

Method of Performance Evaluation.
Tis paper aims to assess the tracking performance in three maneuvers with the following scenarios: Note that the radar detection gave the update position every 32 s. Terefore, radar detection provided 112 updates along the 60 minutes of considering track durations.
Te radar coverage area in Figure 3 is a radial shape. Consequently, the patch in the further range is more extensive than the closer one. It means that the radar resolution varies with the radial distance. Each maneuver was randomly spread in the radar coverage to handle this variation by taking into account the Monte Carlo (MC) simulation as done in the work of Zhang et al. [11].   Te MC simulation generated random initial positions and the heading of the ship. Te initial position was set −40 < x 0 < 40 km and 10 < y 0 < 70 km, while the heading was from 0 to 360°to keep the track inside the radar coverage.
Next, the tracking performance was evaluated by comparing the tracking results to the exact ship position and velocity. Te error measurements were quantized into the following four parameters: (i) Te RMSE of absolute position in Cartesian coordinate [27] was calculated by the Euclidean distance between the tracking result to the exact track.
where x and y are the coordinates in the Cartesian coordinate with the prime symbol referring to the tracking results. While N k is the number of the time step. (ii) Te RMSE of the radial range is the distance relative to the origin point between two radar stations [8].
where ρ is the radial distance relative to the origin point.
(iii) Te RMSE of the bearing angle or the bearing of the target position is relative to the radar look direction.
where θ � atan y x . (19) (iv) Te RMSE of speed or the nondirectional velocity was defned by the following equation: where v x,k and v y,k are the target velocity at k time step and in the x and y axis direction, respectively.

Results and Discussion
Tis paper presents the utilization of the MMG model to test the ship tracking algorithm in HFSWR. Te model can provide many realistic and modifable maneuvers to input on the radar simulator and further tracking assessments. Basically, the MMG model can create various ship movements; however, the turning and zigzag motions are chosen to be presented in this paper.

Te Spatial Resolution of Radar Detection.
As previously discussed, the limited bandwidth available for HFSWR causes low spatial and temporal resolutions. A radar detection test was initially tested to show the efect of those limitations. A ship with linear motion was randomly placed on the radar scene, as shown in Figure 6(a). Two radar stations were taken into account to make the comparison. As shown in Figure 6(a), both radars observe the linear target motion as a nonlinear track due to the low spatial and temporal resolutions. To show the detection error, the diferences in radial range, bearing angle, and radial speed were calculated and shown in Figures 6(b)-6(d), respectively. Te quantitative errors are listed in Table 4. Te average radial range error is 0.4625 km. Tis error makes sense because the radar range resolution is 1.5 km, as shown in Table 1. Likewise, the bearing angle error of 2.217°is acceptable for the angle resolution of 7.5°. Te average error of radial speed is 0.121 m/s, which looked slightly higher than the speed resolution of 0.1172 m/s, but it still confrms the absolute target speed of 10 m/s. It should be noted that radar signal detection uses different mechanisms. Te target range is measured from the receiving time delay, infuenced by the sampling frequency. In contrast, the speed is acquired from the Doppler frequency shift, controlled by the coherence integration time instead of the radar spatial resolution. Terefore, the result of speed detection in Figure 6(d) has a smoother graph compared to range measurement in Figures 6(a)-6(c).
Te comparison of the detection error of radar A and B, as listed in Table 4, shows that radar A has a higher error. It can be observed that the distance between the target location and radar A is farther than radar B. Te detection error is gradually increasing for the farther distance since the detection is conducted radially. In this research, the efect of the error gap due to distance is compensated by generating multiple measurements with the target location spread in the radar scene. Figure 7(a) shows the result of the turning maneuver based on the MMG model and generated by MANSIM. All turning tests were conducted with 20 s duration and 1 s time resolution. Te confguration set the ship to complete a full turning with the duration was kept constant. Te rudder was varied from 10°to 35°, while the propeller rotation speed was kept constant to give constant absolute velocity. Te fgure shows that the higher rudder angle gives a smaller turning radius. Since 35°is the maximum rudder angle in regular ship operation, this maneuver basically gives the smallest turning radius of the ship under test. It is worth noticing that the turning radius in any turning maneuver is not constant. It difers from the result of those modeled by a simple circle equation.    Figure 7(b) depicts the results of the zigzag maneuver with various maximum rudder angles. Te higher steering angle gives a narrower transverse variation. Since the MMG model considers many hydrodynamics aspects during its movement, it provides a more realistic ship maneuver than the result of general shape equations.

Te Maneuvering Ship Tracks Generated by MANSIM.
Next, the position and heading of each ship track were randomized by the Monte Carlo simulation to observe the efect of spatial resolution variation statistically. Figure 8 visualizes the randomized linear tracks generated by the MC simulation.

Test on Linear Ship Motions.
Initially, all-considered tracking algorithms were tested on 100 randomly positioned linear tracks on the radar scene. Te RMSEs of tracking results are shown in Figure 9, which quantizes that all trackers can obtain the RMSE about the radar resolution. Note that the radar range resolution is 1.5 km, as listed in Table 1. However, the range error is increased in the patch far from the radar station due to the higher azimuth error. After comparing the result of the tracking error with and without the tracking algorithm, it can be concluded that the tracker can improve radar detection. Table 4 shows both trackers' RMSE of distance, confrming that the EKF tracker performs better on the linear track while the KF tracker has an error close to the measurement error. Te efect of low radar resolution causes the random detection error, as shown in Figure 6(a). Since the error is sometimes higher than the ship movement in the sampling period, the error can signifcantly decrease the tracking performance of all algorithms. However, the EKF sufers the lower efect due to its fexibility in the error model.

Test on Turning
Maneuvers. Secondly, the trackers were employed on the mixed linear and turning tracks. Te test results are shown in Figure 10 for various turning angles. Te results show that adding 20 minutes of turning is drastically decreased all tracker's performance.
Te UKF has the lowest error in the turning track, while the EKF is slightly higher than UKF. Te linearization method of UKF is naturally more stable to track in a higher nonlinear system. Te PF obtains the most error. Te tracks are spread out to occupy all radar coverage areas by the Monte Carlo simulation in the track generation. On the other hand, the PF algorithm generates the particle and gives the weight factor to the most repeated point.

Test on Zigzag Maneuvers.
Te last assessment was tracking the zigzag maneuvers. Te results are depicted in Figure 11. In this scenario, the zigzag plot is in the entire duration. Similar to the test on turning angle, the UKF is superior to other methods. It underlines the ability of the UKF tracker to recognize any changes in target motions.

Comparison of Tracking Algorithms.
It has been presented in the prior discussion that the ship detection and tracking process on the HFSWR is challenging due to its low spatial and temporal resolutions. Te tracking process becomes highly sophisticated when the target moves with high maneuvers. Terefore, it is crucial to test the tracking algorithm's performance on the maneuvering target. Table 5 and Figure 12 summarize the tracking errors of considered trackers to work on the radar with an 80 km maximum range, 1.5 km range resolution, and 32 s time resolution. Te results in Table 4 show that all trackers can work well on linear ship tracks with RMSE of about 1.3 to 1.8 km. Te testing tracks consist of linear and maneuvering ship motions. Te tracking performance on the linear track shows that the EKF performs the best among other algorithms. Te indicator parameters calculation is 1.368 km, 0.526 km, 1.550°, and 0.005 m/s for the absolute position,   radial range, bearing angle, and speed. Te results of tracking the maneuvering target ship showed that UKF achieved the lowest error indicators, but it just slightly difers from the EKF. Te RMSE of absolute position, radial range, bearing angle, and speed is 1.649 km, 0.639 km, 1.919°, and 0.165 m/s, respectively.
Finally, it is worth noticing that the MMG model can generate free-running maneuvers with many diferent types of vessels as long as the required parameters are known. Accordingly, it is possible to evaluate the tracking performance with any plot by following the proposed procedures in this work.

Conclusions
Te paper presents the procedures to evaluate the tracking algorithm performance in HFSWR in any possible ship maneuver by utilizing the MMG model. Te target motion generated by the proposed model can be inputted into the radar simulator to get successive target detection on which the tracking algorithms are applied. Tis proposed method allows the tracker to test realistic ship maneuvers which are not provided by existing mathematical models or AIS empirical records. Te proposed evaluation mechanism is implemented to assess the performance of KF, EKF, UKF, and PF flters to trail three maneuvers: linear, turning, and zigzag. Te ship detections are conducted on the radar simulator with an 80 km maximum range and 1.5 km range resolution. Te tracking performance is analyzed by calculating the RMSE of four parameters, i.e., absolute position, radial range, bearing angle, and speed. Te evaluation results show that the EKF tracker has the slightest error for the linear track with RMSE of absolute position, radial range, bearing angle, and speed are 1.368 km, 0.526 km, 1.550°, and 0.005 m/s, respectively. Otherwise, the UKF performs slightly better than EKF for the high maneuver targets. Te RMSEs of absolute position, radial range, bearing angle, and speed are 1.649 km, 0.639 km, 1.919°, and 0.165 m/s, respectively. Te assessment results agree with the fndings of other works.
Te current work evaluates four basic tracking algorithms for a method trial. Te work has been accomplished to develop the HF radar simulator, which is interconnected to the MMG model to include the ship maneuver in the radar simulation. Te simulator has also been incorporated to assess the ship tracking performance in the HF radar. Based on the current achievement, the simulator can be

Data Availability
Te data used to support the fndings of this study are available from the corresponding author upon request.

Conflicts of Interest
Te authors declare that there are no conficts of interest regarding the publication of this paper.