Experimental Autonomous Road Vehicle with Logical Artificial Intelligence

This article describes some technical issues regarding the adaptation of a production car to a platform for the development and testing of autonomous driving technologies. A universal approach to performing the reverse engineering of electric power steering (EPS) for the purpose of external control is also presented. The primary objective of the related study was to solve the problem associated with the precise prediction of the dynamic trajectory of an autonomous vehicle. This was accomplished by deriving a new equation for determining the lateral tire forces and adjusting some of the vehicle parameters under road test conductions. A Mivar expert system was also integrated into the control system of the experimental autonomous vehicle. The expert system was made more flexible and effective for the present application by the introduction of hybrid artificial intelligence with logical reasoning. The innovation offers a solution to the major problem of liability in the event of an autonomous transport vehicle being involved in a collision.


Introduction
The desire for the development of more efficient transport services through the use of innovative organizational and technical solutions has fueled the development and implementation of intelligent transport systems (ITS) and intelligent (autonomous) vehicles [1].A decrease in the effect of the human factor on the process of driving is expected to improve road safety and facilitate the achievement of more efficient transport services.Reduction of the severity of road traffic accidents, preservation of human capacity, and increased labor productivity are some of the advantages of autonomous technologies in everyday life.
The automation of road vehicles has been driven by approximately a century of research in different countries [2].Several scientific studies [2][3][4][5][6][7][8] have been dedicated to address the issues of automatic vehicle control, such as the object component base, the mathematical apparatus and its implementation, and technical vision systems.However, only a few Advanced Driver Assistance Systems (ADASs) have reached the market, such as serial production vehicles.
The significant stimulus that the field of autonomous road vehicles has received in the last few years has been related to the results of the DARPA Grand Challenge competitions [3].Developmental projects on autonomous vehicle technologies are currently undertaken by all global motor manufacturers, especially in the US, Germany, Japan, and China.Automobile groups such as Ford, Daimler, Volkswagen, Toyota, Honda, GM, Geely, Tata, and Tesla, as well as other major technological organizations such as Google, Continental, Delphi, Siemens, and Bosch, are involved in these projects as well.Other players in the field include defense departments and agencies of different countries (e.g., the United States' Defense Advanced Research Projects Agency (DARPA)), universities (e.g., Stanford University, Carnegie Mellon University, Technical University of Munich, University of Karlsruhe, Fraunhofer Institute, University of Minnesota, Universidad Politécnica de Madrid), and many other institutions.
In this section, we will consider the main levels of road vehicle control, with the purpose of highlighting some aspects of the present study.In accordance with SAE J3016 approaches [9,10] the overall act of driving can be divided into three levels of driver efforts: Strategic, Tactical, and Operational levels.The Strategic level involves trip planning; the Tactical level involves maneuvering the vehicle through traffic (lane change, overtaking, speed limit, etc.); and the Operational level involves innate actions such as steering, braking, and other vehicle system control for trajectory tracking.The present study focused on the Tactical and the Operational levels of autonomous vehicle control.
Through observation of the successes of foreign developmental efforts on autonomous technologies and domestic test driving of an experimental car in the autonomous mode, the following main challenges have been determined: (1) Operational-level vehicle movement control models are either simplified or parameterized, with the parameters of the mathematical models manually calibrated for each specific vehicle, weight condition, tire type, and so on.In addition, the modeled vehicles move along curved trajectories at only low velocities [3,11,12].
(2) The Tactical level of control is strongly interconnected with the scene recognition of the machine vision systems, which generally utilize deep learning neural networks [13], which in turn, despite their advantages, increase the hardware requirements.It is thus challenging to carry out analyses to justify specific management decisions.
The rest of this paper is organized as follows.In Section 2, we provide a detailed technical description of production car procedures that potentially enable the development of autonomous vehicles.In Section 3, we introduce the halfempirical mathematical model of vehicle dynamics, adaptable to varying external conditions (individual tire traction on different road types, road inclination, etc.), and the properties of vehicles (overall and mass parameters, steering and stability properties, etc.) that enable precise real-time vehicle trajectory prediction.The adjustment of the parameters of the mathematical model using the proposed method constitutes a novel contribution to the literature.In Section 4, we describe our application of Logical Artificial Intelligence, which is based on a Mivar expert system, to Tactical-level control strategies.The results of the application are presented and discussed in Section 5, as well as the conclusions of the study and scope of further work.

Open Platform for Development and Testing of Autonomous Road Vehicle Technologies
The experimental autonomous road vehicle used for this study, which is based on Chevrolet Orlando production car (2012MY), was built at the Automobiles department of Moscow Automobile and Road Construction State Technical University (MADI) as a platform for the development and testing of relevant autonomous vehicle technologies.
The experimental car was equipped with the following hardware: (i) A vehicle electronic systems control module.The small, specially designed dual-processor board is hidden above the pedal assembly and used to control the power steering system, inner CAN-bus, and external Ethernet communications, as well as the external servo drives.
(iv) Ethernet network with two switches.
(v) Two external side video cameras used by the lane keeping assistance system.
(vi) One back video camera.
(vii) Three front video cameras (one for lane keeping and two high-speed 2 K cams for stereovision).
(viii) Forward 3D scanning device.The bionic vision device is a specially designed Light Detection and Ranging (LIDAR) device with one fast changeable beam orientation based on the UDP protocol.
(ix) Five video processing modules.
(x) An autonomous driving module.There are two versions of this module.The first is a simple highperformance computer located in the baggage compartment, while the second is a modern Android smartphone with Wi-Fi connection to the vehicle Ethernet network.
(xii) Access point to LTE/4G networks.This is used for the remote control function of the vehicle.
The cost of all the additional hardware installed on the experimental vehicle was about $6000.The open protocol of the communication with the vehicle and all the technical vision devices were specially developed.External control of the electric power steering (EPS) was achieved by a special "electronic gate" installed between the steering column torque sensor and the EPS electronic control unit (ECU).This system has several advantages: (1) When the electronic gate modulates a signal, the EPS ECU "thinks" that the driver has applied a real torque to the steering wheel and begins to turn the wheels.(2) The instances at which the driver wants to take back control of the vehicle can be determined by monitoring the difference between the real torque sensor data and the data modulated by the electronic gate.(3) The EPS functionality is preserved.(4) It affords the cheapest means of taking control of the steering system of the car.
The steering column torque sensor digital signal of the experimental car appears like a voltage ripple on the sensor power supply wire (8 V) within 7.5-8 V.The duration of one data package is about 207 s, while the time interval between packages is about 280 s.With the assumption of Manchester encoding [14], we chose a time gap of 3.916 s and interpreted a signal fall as a logical zero.We then transformed the data package into a sequence of four 12-bit parts, implying a total of 48 bits on one data line.Following is a sample sequence of three packages: 1EA 231 3E2 Further analysis revealed that the first four bits of each segment did not change because there were four segments in the package.The first 12-bit segment was a circular counter.The result of the final transformation of the given sequence of packages was as follows: F20 FAA ADF A8E E31 AE2 It was thus determined that, in each package received from the steering column torque sensor, there were two important digits, which were the values of the two channels.Our interpretation of the steering column torque sensor protocol is graphically illustrated in Figure 1.
In Figure 1, the red cross indicates the point with coordinates corresponding to the channel values, which are transmitted by the digital protocol from the steering column torque sensor to the EPS ECU.This implies that external torque is not applied to the steering wheel shaft.
When an external torque is applied to the steering wheel, the torque is directly proportional to the angle between the radius (of the approximate circle generated by the torque) that passes through the point of zero moment (red cross) and the radius that passes through the variable point with coordinates (channel 2; channel 1).This angle is hereafter referred to as the equivalent torque of the steering wheel shaft.
The torque applied by the driver through the steering wheel to the steering wheel shaft is given by where  SW is the torsion torque on the steering wheel shaft [Nm],  SW is a proportionality factor [Nm/rad], and  SWE is the equivalent torque of the steering wheel shaft [rad].
The following empirical equation of the equivalent torque of the steering wheel shaft was obtained: where  1 and  2 are, respectively, channel 1 and channel 2 values.
To enable control of the EPS motor, the electronic gate sends packages with the appropriate channel values to the PS ECU.The channel values are calculated as As can be observed from Figure 1, the channel values are dependent on the engine temperature.Our experiments revealed that two data sets were sufficient for an approximate circle of radius 888 for a cold engine (see ( 2) and ( 3)) and an approximate circle of radius 830 for a heated engine.
The separate processor (STM32F105RCT6 LQFP64) of the external vehicle electronic systems control module satisfied the requirements of the electronic gate of the EPS system.The main processor (STM32F407VGT6 LQFP100) also satisfied the requirements of all the other tasks.Communication between the processors was achieved through the Serial Peripheral Interface (SPI) at a bit rate of 1 Mbit/s.The vehicle electronic systems control module was equipped with a TJA1042T.118CAN-bus driver and HLK-RM04 Wi-Fi module with a universal asynchronous receiver/transmitter (UART).
To avoid the need for the installation of additional sensors on the experimental car [1], the messages from the high-speed CAN-bus were decoded to determine the pedal positions, steering wheel angle, wheel velocities, lateral acceleration, yaw rate, and other parameters.The results in DBC format are as follows: Braking and automatic-transmission gear shifting were accomplished through electric motors (LA12 GoMotor-World).The electronic gate between the electric accelerator pedal and the engine ECU was used (physically in the main STM407 processor) to control the vehicle acceleration.
We thus achieved a platform for the development and testing of autonomous road vehicles technologies.The experimental vehicle currently has software for the test implementation of the following functions:

High-Precision Vehicle Dynamic Trajectory Prediction
Figure 2 shows the degrees of freedom of a vehicle, the positive directions of the coordinate axes, and the conventional notations.With the exception of the fixed inertial system, all the coordinates move with the vehicle.A Burckhardt model [15] was used to describe the tire interaction with the road surface.In this approach, the friction coefficient between the tire and the road was calculated using where the coefficients  1 ,  2 , and  3 are determined by the type of road surface (asphalt, concrete, cobblestones, snow, etc.), with the relevant values available in literature [15][16][17];  4 is used to correct the equation for high driving velocities;  5 is used to consider the dynamic vertical load change;  CoG is the center of gravity (CoG) velocity;  Res is the resultant wheel sleep; and   is the vertical load (vertical wheel contact force).
The following system of differential equations was used to analyze the body motion of the vehicle: where  CoG is the vehicle mass; ẍ In , ÿ In , and z In are the projections of the CoG acceleration vector in the inertial (fixed) coordinate system; ẍ CoG , ÿ CoG , and z CoG are the projections of the CoG acceleration vector in the CoG (chassis) coordinate system;  In is the transformation matrix for rotation of the vector from the undercarriage to the inertial coordinate system; the indices , , and  determine the orientation of the vectors in the undercarriage coordinate system;  wind ,  wind , and  wind are the projections of the aerodynamic resistance force (wind force);  GX ,  GY , and  GZ are the projections of the vehicle gravitational force;   ,   , and   are the main moments of inertia; and ℎ CoG is the height of the vehicle CoG.
There is the known dynamics problem of precisely determining the lateral wheel force in an unsteady state, due to the pneumatic tire elasticity.The same also applies in a quasisteady state such as during driving with a constant velocity along a constant curvature with a lateral acceleration above 4 m/s 2 [16].There are many different approaches to solving this problem.The approach employed in this study was a modification of that in [16,18] using [19,20].The following proposed equation for calculating the lateral wheel force was thus obtained: where   is the wheel velocity;  is the tire side slip angle;  0 is the pneumatic tire relaxation length;  red is the reduction factor, which prevents an increase of the lateral force above the adhesion limit during simulation; and  1 and  2 are the dynamic slip resistance coefficients.The proposed equation ( 6) describes the decrease in the amplitude of the lateral wheel force under rapid changes in the external impacts (actions), as well as the delay in the growth of the lateral wheel force due to the properties of the tire elasticity.
The complex mathematical model was implemented in MATLAB Simulink.We developed a methodology for searching the unknown model parameters by nonlinear optimization through comparison of the simulation results with those of road tests using real cars.The adjusted parameters of the mathematical model for different cars are presented in Table 1.These parameters obtained by the present methodology constitute a novel contribution to the literature.The model adequacy was examined by comparing the results of road tests with those of simulations and further statistical analysis of the error functions.The relative errors of the estimated motion parameters were within 12%.
As an example, Figures 3 and 4 show comparisons of experimental and simulation data.The road test was conducted on a proving ground with spontaneous maneuvering comprising a combination of consecutive actions, namely, rapid acceleration, right turn, slalom, and emergency braking.
The mathematical model of the present experimental autonomous vehicle (Chevrolet Orlando) with the adjusted parameters was used to obtain a huge dataset of possible future trajectories of the vehicle body over the next 3 s of motion with respect to the incoming parameters (current CoG velocity, CoG longitudinal and lateral acceleration, yaw  rate and its derivative, steering wheel angle and its derivative, and type of road under each wheel and corresponding sets of control actions over the next 3 s).The dataset was transformed into a multidimensional array for use in the autonomous vehicle motion planning system (Operational level) during driving along a fixed trajectory.This vehicle trajectory prediction method can be used in real-time systems for collision avoidance.
In Figure 5, the blue dotted line represents the actual trajectory of the experimental vehicle's CoG in an inner coordinate system of the hybrid navigation during the implementation of a U-turn in real traffic in the city of Moscow.For safety reasons, the vehicle was controlled by a human and the calculations were performed after the experiment.The red lines represent the predicted trajectories for the case in which the control actions that are relevant to a certain point of prediction are unchanged over the next 3 s.The black lines represent the predicted trajectories with the input of a multidimensional array of motion planning of the real control actions, which were recorded during the road experiment.
The match of the predicted (black) and real (blue) trajectories in Figure 5 within acceptable limits confirms the efficiency of the proposed approach.We thus successfully built an autonomous vehicle motion planning system with precise trajectory prediction (Operational level).The development of the system was based on a complex mathematical model of vehicle dynamics, the parameters of which can be adjusted for different types of road vehicles.The model also takes into consideration the changeable environmental conditions such as the road type under each tire, the wind velocity, and the road inclinations, as well as changes in the vehicle itself, such as load and tire changes.

Logical Artificial Intelligence in Control Strategy of Autonomous Road Vehicle
It has become clear that, to achieve the fifth level of SAE automation [9], road vehicles should be equipped with hybrid artificial intelligence systems, which would enable the solution of driving tasks under undetermined conditions.
According to [21], there have been three main waves in the development of artificial intelligence (AI): (1) Wave of "handcrafted knowledge": this was the stage in the development of expert systems in which engineers created the sets of rules that represented the available knowledge in well-defined domains.
The developed systems were characterized by poor perception, no learning, and no abstracting abilities, but good reasoning.
(2) Wave of "statistical learning": this was the era of the development of neural networks such as CNN and deep learning algorithms, when engineers created statistical models for specific problem domains and then trained them using big data.The creations were characterized by good perception and good learning, but poor abstracting and reasoning abilities.
(3) Wave of "contextual adaptation": this is the current era of AI hybrid systems, which combine the advantages of the above two innovations.The outlook of the current innovations seems to be positive on all counts.We suppose that, in the driving of a road vehicle in the fully autonomous mode, an expert system with a knowledge database containing formalized traffic rules would initially be sufficient.Attempts to formalize traffic rules in machine language began in 1970.The main challenge is that traffic rules contained in legal texts are mostly written in natural language and are thus often abstract and imprecise.A history of the attempts at formalizing traffic rules and the development of a new approach to the task using Isabelle/HOL are available in [22].
In the present study, we used our own expert system technological platform to formalize the traffic rules.The expert system, which is referred to as the Mivar expert system, consists of multidimensional databases, data processing algorithms, and a cognitive model for describing objects and their relationships.As a software solution, it also consists of tools for designing knowledge models and for model usage (server kernel).The Mivar models are structured as networks of objects (conditions, etc.) and the links (rules) among them.Similar to rational human reasoning, a Mivar kernel initially identifies a means in the uploaded model for solving a request and only thereafter performs a computation.This approach to data processing produces a higher performance compared to the use of classical programming languages and structures like "if-else" ("switch-case") with billions of rules (combinations).More information about the Mivar expert system can be found in [23,24].
In the formalization of traffic rules, we divided all the data into two main groups, namely, incoming and resulting parameters.The incoming data was obtained by the scene recognition module and consisted of actual road signs (319 pieces with triggers like "yes"/"no"), traffic lights (up to three objects with 10 different types and four substates for each one), road markings (up to 10 objects with seven different types of road markings for each one), weather conditions (five states), road type (five states), and all the surrounding objects in the relative grid sections around the vehicle, as well as collision prediction flags for the next 3 s of driving (48 sections with seven classes and three states of collision flags).The incoming data formed a structure with 382 cells with a maximum of 1846 independent variables.The exhaustion of all the possible combinations of the variables involved 13e10 iterations.There were a total of approximately 800 traffic rules, and parallel computing could not be applied to them owing to the requirement for consecutive reasoning.
Indeed, the application of classical programming languages and hardware to such would require approximately 8e2000 computational iterations per request.However, the Mivar expert system enables the performance of the task about 20 times per second (20 Hz) using common hardware.
The Mivar expert system outputs 10 parameters with permissions for performing the nearest maneuvers.This is graphically portrayed in Figure 6 for a subsequent road situation, where only forward driving and right turn are allowed, the speed limit is 60 km/h, the give way indicator is active, and stopping (slowing down) is required to avoid a forecasted collision.If a collision occurs under this condition, our vehicle would be guilty because it had no right of way based on the traffic rules.Figure 7 shows an example of a Mivar graph (representation of the objects, rules, and links) for crossing a road intersection in the forward direction according to the traffic rules.
One of the challenges of autonomous road vehicles is the making of fast and correct decisions during operation, with further logging of each reasoned step.Our principal scheme for integrating Logical Artificial Intelligence into an autonomous driving control system is illustrated in Figure 8.
However, an expert system is by itself a very rigid system.In the event of a scenario that is not recognized in the knowledge database, there will be no solution.For example, if the autonomous car stops in front of a broken-down traffic light with the red light on, the car would remain there for eternity.To achieve greater flexibility of the expert system and enhance its applicability to autonomous driving, we added two additional units, namely, "risk management" and "experience analysis."The added scheme, which was named Logical Artificial Intelligence (LAI), is color-shaded in Figure 8.The knowledge database of LAI consists of three sectors: (1) basic traffic rules; (2) based-on-experience rules; and (3) temporary rules.For example, in the situation of the broken-down traffic light, the uncertainty trigger will lead the program to the "risk management" unit, where all the potential future maneuvers are predicted with respect to the proposed criteria such as the risk of accident in civil driving, time loss in military tasks, and economic factors in business.The optimal maneuver will then be chosen.This will in turn cause the knowledge database sector 3 to change with or without trajectory or route change.After the execution of the maneuver, the "experience analysis" unit will decide on whether to  transfer the temporary rule to sector 2 or not.The experience analysis unit may change sector 2 of the knowledge database without the involvement of the "risk management," and this would amount to a social adaptation.The high performance of the LAI expert system enables driving at high velocities.Overall, the system offers the advantages of logically proven reasoning, high performance, and self-learning.One significant obstacle to the broad implementation of autonomous driving technologies is the question of liability in the event of a collision.To address this issue, we propose the use of a "black box" as a mandatory device of autonomous vehicles, for use in conjunction with the expert system unit containing the formalized traffic rules (Figure 9).The main challenge of this approach is the standardization of incoming data from the scene recognition unit.
The first road experiments performed using our autonomous vehicle with the integrated LAI system encountered problems with the detection of dynamic scene changes (traffic signs active areas detection/not those from digital map/poor prediction of other traffic participants' intentions).There were also risk management problems in poorly determined road scenes, as well as some technical issues.Nevertheless,  overall, the proposed approach was determined to be efficient.

Results.
(1) An experimental autonomous road vehicle was built based on Chevrolet Orlando production car as a platform for the development and testing of relevant technologies.A universal approach to performing the reverse engineering of electric power steering (EPS) for purposes of external control was also developed.The cost of all the additional hardware installed on the experimental vehicle was about $6000.(2) The vehicle motion planning system is capable of highly precise trajectory prediction, which is achieved by a complex mathematical model of the vehicle dynamics.The model is capable of adjusting its parameters for different types of road vehicles and takes into consideration changes in the environmental conditions such as the road type under each tire, the wind velocity, and the road inclinations, as well as changes in the vehicle itself, such as load and tires changes.The adjustment of the parameters of the mathematical model and the proposed equation for determining the lateral tire forces through trajectory prediction constitute novel contributions to the literature.Multiple road tests have shown that the relative errors of the estimated motion parameters were within 12%.(3) A new Logical Artificial Intelligence (LAI) scheme was developed and integrated into the control system of the autonomous road vehicle.The Mivar expert system enables the performance of the requests to knowledge database of traffic rules with 800 rules and 1846 variables about 20 times per second (20 Hz) using one core of common i7 processor without use of any GPU hardware.The resultant Mivar expert system with LAI was found to be efficient.(4) A method for addressing the issue of accountability in the event of a collision involving an autonomous vehicle was proposed.

Discussion. (1)
The technical processes of the adaptation of a production car to an autonomous vehicle can be applied only to research and testing purposes.In addition, for legal and practical reasons, such a vehicle can only be driven in closed areas and on proving grounds.(2) Sometimes abstract and imprecise expression of traffic rules constitute a challenge to formalizing them, but the application of a global collision avoidance system can be used to overcome the challenge.(3) The standardization of incoming data from the scene recognition unit poses a significant challenge to the use of black boxes in conjunction with expert systems to address the issue of liability when an autonomous vehicle is involved in a collision.

Conclusion.
The results of the present study indicate that any modern vehicle with electric power steering (EPS), automatic transmission (AT), and electronic braking assistance (EBA) can be modified for autonomous driving through appropriate wiring.The real-time dynamic trajectory prediction method adopted in this study can be applied to different autonomous motion planning systems, as well as active collision avoidance safety systems.In addition, the proposed modification of the expert system through the integration of a hybrid artificial reasoning system, namely, the LAI system, enhances the efficiency of autonomous driving control.

Future Work.
We plan to integrate some V2X technologies in the developed autonomous driving platform and conduct further experiments using more experimental vehicles in the form of an autonomous connected fleet.We would also like to address the highlighted challenges of the LAI scheme.

Figure 1 :
Figure 1: Protocol interpretation of the steering column torque sensor.

Figure 2 :
Figure 2: Degrees of freedom of a vehicle.

Figure 3 :Figure 4 :
Figure 3: Model validation and comparison of lateral acceleration.

Figure 5 :
Figure 5: Trajectories of the experimental autonomous vehicle.

Table 1 :
Adjusted parameters of mathematical model for different cars.