An Overview of Agent-Based Models for Transport Simulation and Analysis

This article presents an overview of the agent-based modeling and simulation approach and its recent developments in transport ﬁelds, with the purpose of discovering the advantages and gaps and encouraging more valuable investigations and applications of agent-based models. We clarify the agent-based model from agents, the background of development, and the basic structure applied in transport systems. Then, the agent-based transport modeling toolkits are discussed. The applications of agent-based models in transport systems are reviewed in three time scale models followed by an additional discussion of hybrid modeling approaches. The extensive modeling of the beliefs, desires, learning, and adaptability of individuals and the optimization problems using agent-based models are explored. Besides, we point out some limitations in terms of calibration and validation procedure, agents’ behavior modeling, and computing eﬃciency. In conclusion, some recommendations are given and suggest potential and insightful directions such as Big Data and Digital Twin for future research.


Introduction
An appropriate transport model features prominently in characterizing travel demand, related traffic/passenger flows, and their dynamic performance. It serves as a useful tool for transportation planning and management in terms of alleviating traffic congestions, fulfilling travel needs, and creating a better transport environment. However, it is very challenging to describe such a highly complex, dynamic, and stochastic transport system with analytical mathematical models. Simulation models/methods, by contrast, are capable of modeling the complex traffic dynamics in a realtime manner and therefore have been attracting more and more research efforts [1].
Agent-based models are not limited to modeling and simulating the mechanical movement but also introduce the autonomous decision of the agent. Usually, the agent in the systems does not "play by the script" but performs in realtime. It starts from the local rules of agents to the more complex adaptive and emerging behaviors formed by the interaction of the neighborhood agents, thus yielding the system dynamics in the environment. In transport systems, such as transportation management systems and transportation control systems, there exist a large number of independent entities that behave in heterogeneous and inherently complex manners.
is usually requires the structural framework of the combination of the distributed subsystems representing the agents with local plans or strategies based on knowledge and rule. e overall system performance is achieved by the cooperative and even learning the logic of agents, which are the vital components to perform a global improvement [2][3][4][5][6][7].
In general, the autonomy, collaboration, and reactivity of agent-based models are the significant advantages to use for modeling different behaviors and interactions such as perception, reasoning, and decision-making, therefore performing different policies from the traffic planning, control, and management perspective. For instance, in transportation study areas, agent-based approaches have been widely used in traffic management frameworks [8][9][10][11], congestion management [12], traffic policy [13], traffic control, and especially traffic signal control [3,[14][15][16][17][18][19] in order to explore the complexity, interdependencies, and systematic structural evolution pattern under complex urban traffic environment [20].
In summary, this article is motivated by the increasingly promising potential of agent-based models applied in transport system studies. It reviews the recently emerged development of agent-based transport modeling and analysis, aiming to explore the inherent features of agent-based models and identify their advantages and the current gaps of their applications in the transport domain. e contribution of this review lies in the following three aspects: (1) It provides a summary of the basic structure for agent-based models and popular modeling and simulation tools in transport system modeling and analysis. e general agent-based model structure for the transport system is proposed by summarizing the related studies. Preanalysis, agent definition, rule modeling, and model tuning are four fundamental components in an agent-based model. Agent-based models are proved unique in their ability to integrate heterogeneous entities and investigate the emergent dynamics of the transport systems by modeling all the interactive behaviors involved. e agent-based modeling and simulation tools provide a shortcut for developing such an agent-based model, which aims to avoid the cumbersome and time-consuming programming work from the ground up, but some limits still need to be breached.
(2) It reviews the ideas of developing and improving agent-based model applications in the transport domain. We categorize the previous research contributions into the short-term, medium-term, and long-term agent-based models based on the time horizon in which the agent behaviors are considered, and we also identify the gaps in the existing studies about modeling and analysis. In general, the design and implementation of agent-based models used in traffic studies are still immature and require further investigation. On the one hand, incorporated with heterogeneous agents in terms of types, decisions, purposes, etc., agent-based models often model the communication strategies or cooperative rules in more detail to describe the global dynamics. On the other hand, the hybrid modeling approach, which can combine the short-, medium-, and long-term models, integrating microscopic, mesoscopic, and macroscopic simulations, is proved to effectively achieve a joint and multiscale system to meet multidimensional simulation requirements. Instead of making a unified choice of one single microscopic modeling approach, components in different levels of resolution should be incorporated according to the characteristics of the targeted transport system and the modeling and simulation purpose. is modeling approach can save a large number of computational resources and allow the simulation to be more effective. (3) It provides promising research directions for agentbased model studies. As for the insights for future research, we point out that one of the less-developed aspects of the agent-based model approach is modeling the adaptability of the individuals by embedding the knowledge learning abilities in agents' attributes. Additionally, combining optimization algorithms to solve problems like routing, mode choice, or fleet allocation with simulation is a promising research direction. Moreover, the agentbased simulation can be incorporated into the simulation-based optimization framework as an iterative component to provide intermediate input to objective function in every iteration [21]; Zhang and Xu [22]. e use of the agent-based approach not only opens a channel for searching the optimal solution in an optimization program but also facilitates more realistic and reliable policy-making results.
e article is organized as follows. Section 2 demonstrates a brief review of the agent description and the agentbased model introduction, including the background, key properties, and the model structure applied for the transport system. Section 3 is devoted to the comparison in terms of their functions, strengths, and limitations of some popular agent-based model tools.
e applications of agent-based models in transport systems are thoroughly researched through previous studies and limitations of these applications are also discussed in Section 4. Finally, Section 5 provides some recommendations followed by insights and challenges for future research.

Definition and Features of Agents.
ere is no consensus on the definition of the term "agent" since the diversity in the application or environment makes it difficult to extract consistent and concise features. Nevertheless, the essence is that an agent, the basic unit in the agent-based model, can be a representation of any type of autonomous entity, autonomously working within an environment in pursuit of its agenda or goals and actively interacting with other agents if necessary [23]. A typical agent in the agentbased model can hopefully possess the following features: (a) autonomy: operating without direct external intervention; (b) sociability: cooperating with the environment and/or other agents to achieve its task(s); (c) reactivity/perceptibility: perceiving external influence and responding to changes; (d) proactivity: exhibiting goal-directed behavior on its initiative; (e) adaption/learning: learning knowledge from experience and environment and adjust yourself to the environment and simulation objectives [2,[24][25][26]. It should be noted that the modeling of agents is flexible, and the features listed are not exhaustive or exclusive. Within a simulation experiment, there might exist various types of agents that possess different characteristics with different perceived importance [27]. For example, an agent-based simulation of a demand-responsive transport system encompasses different types of agents. e clients can be taken as autonomous agents who constantly generate trip requests. ey are also be designed to be proactive and adaptive, learning to alert and manage the choices/plans for the reproduction of trip requests. Vehicle agents monitor the status of the real vehicles and react to the clients' requests with prior knowledge supplied. Besides, the planner agents can interface between clients and vehicles through processing trip requests and handling bids from available vehicle plans.

Basic Functionalities of
Agents. An agent can be regarded as an intelligent synthesis in possession of explicit and identifiable separate function parts. e basic functionalities of an agent involve six parts: effector, sensor, communication, intention, motivation, and cognition [28]. ese functionalities can be divided into four different modules: (a) effector, sensor, and communication module: making contact and transmitting and receiving information with the external environment and other agents; (b) intention module: comprising the short-/ long-term goal(s) of the agent; (c) motivation module: implementing the strategic intentions of the agent; (d) cognition module: consisting of the agent's knowledge base, reasoning, and deciding component and protocol execution component. e last module has the functions of evaluating the current situation and selecting, executing, and monitoring the final actions of the agent. e functionalities of agents and interactions between multiple agents and with the environment are depicted in Figure 1.

Background and Concept of the Agent-Based Model.
Agent-based model, which is different from the term activity-based model, is used to simulate the behaviors and interactions of autonomous agents, thus assessing their overall impact on the system [29,30]. It is unique in the ability to integrate heterogeneous and dynamically changing processes of complex systems composed of autonomous entities, intending to investigate the emergent and collective effects on the system [31].
In recent years, there has been an increasing prevalence of agent-based models in transport studies. e reasons are threefold: first, the transport system is a very complex (and adaptive) system.
e key characteristics such as multimodality, heterogeneity, complexity, and adaptivity in drivers' behavior are too challenging to be fully and accurately modeled.
rough agent-based modeling, one can formulate a more realistic model considering specific characteristics of heterogeneities and interdependencies among road users and the transport system. Second, new technologies (such as AV, vehicle-to-vehicle (V2V), and vehicle-to-everything (V2X)) expand the scope of research, add complexity to the transport system, and bring more challenges to the modeling process. e technology development also brings a large amount of detailed data, which are very important for calibrating agent-based models. Finally, the computational power has been booming following Moore's law, which enables exploring transport system processes through constituent elements no matter how microscopic they are.

Agent-Based Model Structure for Transport Systems.
e agent-based model can be tailored for various problems, objectives, and scenarios. On the one hand, the applications in the transport domain cover specific systems or phenomena, such as ridesharing, bike-sharing, autonomous taxi, and high-speed railway models to capture only the most salient features connected with the goals of the simulation. On the other hand, large-scale simulation of a certain region or city can also be realized through an agent-based approach to solving real-world policy problems. Nevertheless, agentbased models for different traffic and transport problems share a similar structure with the following four parts, as shown in Figure 2. Step 1. Preanalysis: it determines the application scenario, the purpose(s) of the model, the scale, and the resolution of the simulation system/environment. e application and purpose of the agent-based model are the grand plans for future work, regarding what problem to be solved by applying the agent-based model; what results to achieve; whether to describe, observe, and analyze a specific transport system or predict its unknown emergent phenomena. en, the scale of the system restricts the range of the moving and interactive environment of agents and the quantities of agents involved. As for the simulation resolution, it should be determined as per the characteristics of the system in terms of agents and the environment. e model can feature agents' motion level modeling and simulation like gap acceptance and lane change to study the traffic dynamics, detailed flow changes, and resulting impacts. Alternatively, it can be designed to focus on the modeling of immobile behaviors, decisions, and feedback mechanisms. In this case, the travel motion does not require a precise description and could even be presented at a link level. Moreover, more macroscopic and long-term perspective models can be chosen, which involve activities or behavioral patterns like the choice of housing location and job location [32].
Step 2. Agent definition: it identifies and classifies the heterogeneous entities (agents) in the transport system environment in terms of types. DeLaurentis [33] classified agents into eight basic types based on their behavioral rules, mobility, and information accessibility (see Figure 3). Agents can vary in property and mutate into different entities with different functions when applied in different agent-based models to achieve different objectives. For example, in a traffic control system, reactive agents represent the role of policemen who enforce a set of laws on the road. Various disrupters like weather and accidents can be modeled as autonomous agents. Drivers are adaptive, mobile, and utility agents who may avoid roads that are known to be congested and would like to minimize their drive times or distances. Determining the types of agents and their general characteristics appropriately will pave the way for the following behavioral rules-modeling process, making it clear to define the specific characteristics further and set up detailed behavioral rules.
Step 3. Rule modeling: it models the behavior and interaction of agents. Typically, published literature, expert knowledge, and data analysis can all be the major sources of rules. On the one hand, they are generally based on "if-else" statements that the actions of agents are triggered by some external stimulus. On the other hand, rules can also be put into action directed by goals like the signal controller agent [34]. e rules are reflected by the behavior and interaction of agents, so they are usually influenced by each other and can provide positive and negative feedback to the environment. Many agent-based models focus on the precise motion-related behaviors that deal with agents' movement description and performance in the physical perspective [35,36], and they adopt microscopic models like car-following (CF) and lane changing (LC) models as their mobility rules to facilitate the simulation presenting reality. However, this sacrifices computational proficiency, which is particularly for large-scale scenarios. erefore, one of the challenges of agent-based models is the tradeoff between the complete complexity of the inner logic of the models and computational efficiency. Liu and Meng [37] showed that the allocation of the processor in the distributed computing system performed well in the computational time reduction. Each simulation trial is conducted in parallel and in turns.
is study enlightens researchers to utilize parallel computing approaches to overcome the high nonlinearity and huge computational cost in agent-based simulation.
Sometimes, the event-based mechanisms can be combined seamlessly with the agent-based simulation method. Traditionally, the entities in the discrete event-based simulation are defined to follow a sequence of processes, which are similar to the definition of the local behaviors in agentbased modeling and simulation. Some agent-based modeling tools allow the environment for multiparadigm modeling with agent behavior, agent environment, and event-based description of the occasional or time-certain occurrences included. e hierarchical, sequential, or parallel simulation structure can also be realized with different modules designed based on agent-based models and other simulation methods.
Step 4. Model tuning: model verification, calibration, and validation. Overall, the calibration and validation deal with building the right model, while the verification deals with building the model right. e calibration process aims to examine whether the agent-based model fits well with the empirically supported data. e model evaluation with respect to calibration and validation varies due to different modeling purposes [24]. In general, the development of an agent-based model should include a calibration process and/ or a validation process to demonstrate that the agent-based model presents the system in validity and accuracy.

Pre-analysis
Agent definition Rule modeling Model tuning

Agent-Based Transport Modeling Tools
An agent-based model can be constructed using an object-oriented language, which is largely facilitated by the development of modeling tools. e development of agent-based modeling toolkits aims to avoid cumbersome and time-consuming programming work from the ground up. From the reviewed articles, the agent-based software toolkits can be categorized and distinguished based on the scalability, execution speed, programming language, primary application domain and key features [25,38] (see Table 1 in Appendix A.1).
ere are several agent-based tools specially designed for transport systems, which assist the development of agentbased models in a simplified way with desired attributes, features, and rules. Table 2 in Appendix A.1 summarizes and compares some most commonly used agent-based simulation tools. e agents in these platforms refer to various transport participants, drivers, travelers, and vehicles. In general, when working with relatively large-scale simulation, most of the existing agent-based transport modeling platforms share the similarity of creating an integrated system by combining the travel activity and network loading components.
e frameworks are often based on individual models and rooted in activity-based models. Figure 4 shows one typical type of modeling structure. e system generates demand and activity plans of the population based on demographic and infrastructure characteristics. Each agent then generates individual travel mode and route choice. e plans of agents are fed into a microsimulation, which will produce certain traffic performance. e overall performance feeds back to the decision-making process of activity plan and mode and route choice decisions so that the plans can be revised and improved to be ready for the next iteration of the simulation.
Note that many of them exhibit a similar theory of simulation-based dynamic traffic assignment (DTA) in agent-based transport modeling. It should be noted that simulation-based DTA and agent-based transport modeling share some similarities and dissimilarities. On the one hand, the route choices can be optimized by using the historical travel time information in both methods and this processes iteratively to accomplish the optimal performance. On the other hand, simulation-based DTA focuses more on the trip of individuals in the simulation, so that the travel time information is usually only provided for route choice decision of travelers, while the feedback information can be fed into multiple dimensions of decisions like activity location and schedule in addition to route choice domain. In addition, the heuristic approaches used in agent-based models can sometimes converge to the equilibrium faster. In terms of the internal models and algorithms, the existing platforms exhibit a certain difference in the design process, data structures, resolution levels, and emphasis on activities of agents to some extent.
Several agent-based tools aim for specific realms of transport research like traffic control and management or route guidance. For example, Cooperative Traffic Management and Route Guidance System (CTMRGS) can be applied in traffic management [10]. TRACK-R (TRaffic Agent Does it care about the utility value?
Does it move? Journal of Advanced Transportation   Journal of Advanced Transportation City for Knowledge-based Recommendation) can be used to deal with the dynamic route guidance problem [45]. Agent-Based Dynamic Activity Planning and Travel Scheduling (aDAPTS) can be used to study traffic signal control [46]. More and more studies have applied agent-based simulation tools to traffic and transport systems, and different platforms and tools can be chosen to serve the research purpose. For mobility simulation, some platforms are aimed at presenting microscopic traffic flow dynamic characteristics, which simultaneously lead to longer computation times, particularly for large-scale scenarios. In contrast, others adopt the faster microscopic or mesoscopic models like the spatial queue model to measure traffic dynamics, pushing the limits of computational feasibility. Nevertheless, many platforms are becoming more and more flexible to provide many connectors or fully open-source to allow the researcher to write extensions to solve such problems and explore the potential functions.

Applications in Transport Systems
Compared to other microscopic simulations, the agentbased model has inherent advantages in modeling multimodal traffic by incorporating different transport modes and multiple entities of operators and users into traffic flows. Table 3 in Appendix A.2 summarizes the existing studies of the agent-based model methods in the transport domain and compares/highlights their main objectives, agent types, method features, strengths, and potential improvements. How the agent-based model method is applied and constructed is determined by the objectives of the studies. Most paradigms focus on traffic control and management, route choice or guidance, and carsharing or ridesharing. Accordingly, the agents in these models usually represent components like drivers, passengers, and vehicles. e most common goals are developing effective strategies, novel systems, and algorithms. In addition, some psychological characteristics can be embedded to enrich the agent models and make their behavior close to real human beings, which might be favorable in evacuation simulation [63,64].

Simulation of Transport Systems.
In this section, previous research contributions of agent-based models differentiated by rule-based behaviors of agents at the different temporal scales are reviewed. Some studies may select the traditional categorization of the "micro-, meso-, macroscopic" models according to the distinction of spatial information. However, this classification has the limitation in providing insights when the agent-based simulation system to be built have different requirement of the timespan [65]. Moreover, we cannot have a clear mind about the system-related agentbased modeling difference for developing a specific agentbased transport system only based on another categorization of "strategic, tactic, operational" types of modeling. erefore, this article intends to provide a review of agent-based modeling methods from the perspective of the time horizon of simulation. We adopt a categorization of the models into short-, medium-, and long-term types, comparing the different agent-based modeling techniques and algorithms used in different frameworks. It not only reveals another aspect of information of the traditional one in terms of "strategic, tactic, operational" to some extent but also corresponds to the ideas of microscopic, mesoscopic, and macroscopic simulations in some ways. is categorization is beneficial to provide a reference for exploring agent-based systems at different levels.
Short-term movement control models handle the within-day movement of agents at a microscopic granularity. Detailed driving and travel behaviors under the impact of other moving agents or traffic control/management are described. Mid-term models function at the design/operational scheduling: simulating the day-to-day activity and travel patterns of agents, which are often generated by choice model, behavior-based demand model, etc. Long-term planning models, with the view of capturing land use and economic activities, are commonly applied to year-to-year transportation planning studies. e discussions of these three levels of agent-based models are followed by an investigation of the hybrid models, which merge two or all the three different scale time models into a single simulation process. Figure 5 presents the important key points of three different models, respectively, to show their difference. Note that for some agent-based models embedding the event-based mechanisms, the time scale might also depend on the event mechanism in the model, and the time between two events may range from seconds to days in the same model.

Short-Term Models.
Short-term models are usually developed at a microscopic level and on the temporal scale at a very fine resolution of the second level and some even up to 100 ms (e.g., in pedestrian flow or traffic flow models). As the agents involved in transport systems are basically mobile agents (not necessarily in other domains), the behavior and the relationships with other agents and the external environment for their interaction in the short-term models are usually related to mobility and are commonly used to  ey can be modeled to function at sensing and perceiving the conditions and states of their environment around. e timely and prompt reaction to the infrastructure equipment such as signboards, traffic signals, and traffic laws can be incorporated. Moreover, the pedestrian agents' walking behavior modeling also caters to the short-term agent-based models, which involve the modeling of walking direction and relative speed based on the condition of both the pedestrian and road traffic around the current position. Similar researches include the microscopic movement of pedestrians during an evacuation or the movement of cars on the street.
Various agent models facilitate the application of short-term agent-based models in mixed traffic flows that refer to the mixture traffic of two or more pedestrians, motorcycles, cars, buses, and trams. e study of such complex vehicular and pedestrian flow patterns should be started from the individual's behavior pattern. e type of agent and behavior modeling process in these short-term agent-based models vary with different objectives. Yang et al. [67] established the model of pedestrians' road-crossing behavior to study the pedestrian flow at different traffic volumes. Explicit descriptions of the pedestrians' road-crossing and decision-making behavior like whether to run through the red light and the followers' violative behavior are required. Some studies involve other vehicle agents such as motorcycles and public transport. In Fujii et al. [57], a framework for modeling and simulating the mixed traffic flows composed of cars, trams, and pedestrians was constructed to test the control and management strategies. e pedestrian agents have the behavior of free walk, collision avoidance, momentary stop, overtaking, and the following mobility with car and tram agents. e car agents can plan the route and determine acceleration, and the tram agents are extended from the car agents, excluding the LC and route-searching functions. e mobility of agents was expanded with more detail and flexibility in that the existing 1D CF model for cars can be extended to a 2D discrete choice model for pedestrians.

Medium-Term Models.
Medium-term models refer to the (mesoscopic) simulations that involve mode and route choice, activity schedules, or en route diversion at the temporal scale of minutes or even longer. Such multidimensional travel behavior simulation is effective in analyzing or predicting behavioral changes and the dynamics of the traffic condition [68,69].

Journal of Advanced Transportation
An agent-based simulation is frequently used in sharing mobility research to capture the feedback and informationtransferring processes. e study of sharing mobility is closely related to the problems of the optimal fleet size dispatch and the passenger-passenger (and driver-passenger) matching. Agent-based models specifically pertaining to sharing mobility phenomenon are grouped into mediumterm models. For mobility simulation in these models, the agents can still be modeled at the individual level in terms of mode or route choice process. e attributes and behaviors of agents are related to making decisions about travel plans based on their cognitive information (knowledge) and returning effective feedback of the matching information.
Medium-term agent-based models perform well in dynamic ridesharing (also known as on-demand ridesharing and dynamic carpooling), and the model usually involves a service operator dynamically matching potential driver and passenger agents with similar trips and allowing them to travel together and sharing the costs [70]. A binary integer programming was combined with the dynamic auctionbased multiagent optimization algorithms to solve a dynamic ridesharing problem [47]. e decentralized agentbased model allows the matching condition of drivers to passengers, which is proved to be flexible in solving ridesharing schemes. In some cases, the medium-term agentbased models contribute to studying multimodal transport systems in that the explicit mode choice and split condition can be presented from an individual perspective. Shen et al. [71] developed an agent-based system combining on-demand and shared AV service and public transportation (PT) to identify the synergy effect. e passengers' searching, choosing, quitting, waiting, and recalling processes are considered. In this case, all the occupied AVs will be searched, and then the passengers will agree to share the ride according to their sharing preference. As for sharing AVs, shared rides are matched subject to certain constraints like enough available seats and detour constraints.
Further study of psychological characteristics affecting the medium-term behaviors of drivers can be conducted to the incorporated preference of agents like passengers' sharing preference of riding alone or with other riders under certain conditions. Notably, the agent environment in medium-term models is not as specific as that in short-term models with continuous or grid cell tangible interaction space for micromovement particularly related to geospatial and spatial modeling and simulation. Instead, the interactive environments in medium-term models are relatively virtual and fuzzy as they are designed not to deal with physical and social behavior like moving in different directions and at different velocities within their environment.

Long-Term Models.
Long-term models, which are usually at the temporal scale of months to years, do not deal with the simulation of the behavior or state of agents at the microscopic or mesoscopic level. Agents in such models also can be modeled microscopically at the individual level to represent the evolution in the long run, which is widely used in the demography, household relocation, housing market, travel demand, and assignment research field. For instance, explicit modeling of the individual decision process is used to predict the land use evolution and property development closely connected with life-cycle decisions of agents in the demand-side [72]. Research efforts are made to study the fundamental connection between land use patterns and travel needs or activity agendas. e ILUTE (Integrated Land Use, Transportation, Environment) system targets the simulation of agent activities evolving in the long term [73]. e activities/behaviors of person agents can be simulated in terms of birth and death, in-migration and out-migration, the property (e.g., car ownership) change, housing relocation, job location choice, etc. [39,74]. e long-term agent-based models are usually integrated with the models in other levels mentioned above to provide the synthetic population information along with land use, agent's socioeconomic characteristics, vehicle ownership, etc. [75]. is can be used to analyze the activity schedule, daily plan, and other lower-level choices made at the midterm and even short-term time scale. e point of the integration approach will be discussed further and extended with more details in the next section.

Hybrid Models.
e entities in transport systems like a person, traveler, driver, and vehicle or driver-vehicle unit exhibit diverse behavior patterns at different levels. Models at a simplex temporal scale cannot capture the behavior patterns ranging from mesoscopic mode choice, route choice, and rescheduling to the microscopic simulation of lane changing, gap acceptance, acceleration, deceleration, and braking.
In some studies, the short-term models are integrated with the medium-term ones to achieve hybrid agent-based systems, though the difference in emphasis may lead to different internal modeling structures [76]. Galland et al. [77] proposed a computational agent-based model for the carpooling problem, which focused on simulating the interaction of autonomous agents. As Figure 6 shows, the daily travel agenda (schedule) of each agent is predicted by the FEATHERS module, which is a developed activity-based model based on traffic demand [78]. On the one hand, modeling the medium-term individual behaviors is dominant during the carpooling (decision) process, such as activity pattern, mode selection, partner-matching, plan negotiation, and feedback to all concerned agents. On the other hand, the trip/plan execution activity in the carpooling model involves simulating driving agents on the road network. e behavior of a driver activity primarily involves path planning and collision avoidance in the network.
Due to the different objectives or focuses of the studies, the structure and the combination of functionalities and/or policy decisions of the hybrid agent-based model may vary substantially. For example, suppose that researchers focus on capturing interaction and policy effects between land development and transportation. en the model of long-term system evolution in terms of the components like migration, location choice, and auto ownership should be embedded to explore their influences on agents' decisions in lower time scale models.
A hybrid model (simulator or platform) can be composed of long-term and medium-term models. e longterm one is designed to provide land use information by synthesizing a set of households, persons, jobs, etc., while the medium-term one is responsible for the daily activity-travel plan of individuals. In the SimAGENT (Simulator of Activities Greenhouse Emissions, Networks, and Travel) system, the long-term agent-based model modules, PopGen and CEMSELTS (Comprehensive Econometric Microsimulator for Socioeconomics, Land Use, and Transportation System), deal with the generation of a synthetic population and demographic attributes for persons and households, respectively [79]. e CEMDAP (Comprehensive Econometric Microsimulator for Daily Activity travel Patterns) serves as a medium-term simulation component, and an additional traffic assignment component is used to achieve a traveler's route choice process. Such hybrid modeling approaches can realize the simulation of lowerlevel decisions like activity participation agenda or daily travel behavior influenced by those at a higher level like residential mobility. Similarly, the daily travel behavior is qualified as a measurement to feed back the decisions in terms of residential mobility, location choice, and economic decisions.

Simulating Adapting and Learning Behaviors.
In some agent-based models, various models such as logit models might be applied to better capture individual characteristics of selecting an alternative based on preferences, habits, and prejudices. Some cognition-oriented behavior modeling approaches are also developed to capture the uniqueness, such as motivation, emotion, and risk avoidance that make us human (driver). e information source for adaptation behaviors comes from the agent environment, where positive and negative feedback are introduced. Xiong et al. [17] conducted a study in which the drivers' behavior was modeled and represented under the effect of the psychological factors. Apart from the threshold-based rules (a behavior or action is triggered when a related parameter exceeds a predetermined threshold, for example, decelerating to keep a safe distance and avoid collision with the car forward and stopping at the intersection with the red light on), more abstract concepts are embedded in individual behavior modeling to represent the agents' learning and adapting hallmarks.
Instead of using mathematical approaches, complex agents can be designed based on the Belief-Desire-Intention (BDI) paradigm. It is developed on cognitive frameworks and embeds beliefs, desires, and intentions into individual agents [80]. e BDI architecture regarding the decisionmaking mechanism indicated that the decision-making behaviors of agents, such as the perception of information towards the current system state, desirable and motivational state, and final act of agents, greatly influenced the model output [81]. Similarly, PECS (Physis, Emotion, Cognition, Social Status), a multipurpose reference model, was developed to simulate human behavior with the four main factors [82]. e knowledge learning ability allows an agent to save information on previous experiences, and the learned knowledge is used to generate a better scheme through this feedback mechanism. Bayesian models are important learning methods to be used in agent-based models. e Bayesian approach provides an excellent conduit to rule the definition of agents in the context of a large number of alternatives and complex internal relationships where agents conduct joint selection behavior. In a Bayesian model, each agent's spatial and temporal knowledge used for travel-related choices is updated according to the previously learned experience. Probabilities can be modeled into the agentbased model by using Bayesian methods-specifically, a unique probability can be created by an agent or agent population at each point of time or space to represent the decision-making process based on learning and perceived knowledge from the environment. Zou et al. [83] proposed an agent-based model with an iterative search shown in Figure 7 to predict travelers' choices of mode and departure time. Agents of travelers can make use of information generated in the traffic environment, such as traffic management and control policies, to learn and accumulate travel experience. For mode choice in the Bayesian learning process, a quantitative relationship between the quantified knowledge vector and the individual's subjective beliefs vector is established based on individuals' prior beliefs [84]. e learned beliefs are usually used to search for new alternatives of travel decisions, such as mode, departure time, and route, and new plans or strategies are fed into the agentbased model for the next iteration of the simulation.
In mimicking the actual human (learning) behavior, reinforcement learning (RL) appears to be a good fit in agent-based models. Such a machine learning paradigm can be fed into modeling agents who seek to maximize cumulative reward by developing a state-action policy through repeated interaction with both agents and the environment [30,85]. RL often deals with finding the best solution(s) to achieve optimal control or optimal travel behavior. erefore, RL agents can be equipped with the ability to learn how to survive and optimize their behavior by updating the knowledge base and inference rules in an unknown, complex environment, especially when prior knowledge is quite limited.

Integrating Agent-Based Approaches in Optimization
Problems.
e agent-based model can be combined with optimization techniques in two ways: (1) embedding an optimizing method into the physical behavior of agents (e.g., by translating optimization algorithms like route-search into agents' behavior) [86]; (2) utilizing the agent-based model in a simulation-based optimization framework to obtain the simulation results, which are designed as an intermediate component to provide input data iteratively [87].
One primary application of the agent-based model in optimization problems is transport management and control. To achieve the objectives such as the optimal allocation of vehicles or the optimal schedule of flights, the optimization strategies can be developed by embedding the agentbased models to the model the process from pursuing the local goals to the global solution. Böcker et al. [88] illustrated a multiagent train coupling and sharing (TCS) scheduling system, which included the planning of train units and the optimization of solutions to reduce the track allocation cost and increase the capacity of tracks. e optimization technique was achieved by a negotiation protocol, which was a process of peer matching: finding the agents that share the same routes and making them share the respective routes by coupling together and splitting up afterward. In the traffic signal control optimization research field, agent-based technology can effectively handle multiple traffic signal systems. It allows the massive information distributed among agents responsible for controlling each intersection, which is influenced by the traffic demand at intersections [89]. In the study of [18], the traffic signal timing optimization problem was solved by integrating a multiagent traffic control component into a mathematical program. Additionally, the multiple signal coordination was developed using the initial typical coordination scheme. It further dealt with the coordination scheme through information exchange among agents controlling different intersections.

Limitations.
ere are some limitations of the existing agent-based model studies in the following two aspects.

Incomplete and Limited Calibration and Validation
Procedures. We can easily notice that the techniques of developing agent-based models for all the different applications in the transport systems are intensively explored. However, the ideological framework cannot be successfully and securely implemented for the lack of explicit calibration and validation varying from different problem scales. Although some attempts have been made to evaluate the proposed agent-based models, there are no acknowledged, standard guidelines formed to conduct the model calibration or validation. For instance, some studies have weakened the calibration of the individual parameters in the large-scale transport system [60,90]. Some articles did explicitly mention the role of calibration, verification, and validation, but their effects on the final results were not clearly discussed, while other studies just tagged the validation to be the future work [24]. Due to the various applications of the agent-based model approach in transport, it is understandable that there are no specific unified methods/steps of the calibration, verification, or validation process. Nevertheless, different methods like comparing with analytical models should be explored to increase the credibility of the results of agent-based models.

Limitations in Agents' Behavior Modeling.
Although the previous studies have investigated the new ideas of agent modeling and the use of ABMs, agent modeling, especially the complex and adaptive behavior, is still one of the notorious and common difficulties to solve. Human behavior usually needs to be understood concerning intelligent perspectives like preference and memory. Frameworks like PECS and BDI are applied in some studies, but in most cases, they are not completely embedded in the context of agent-based transport systems. Many of those systems are still developed based on simple threshold-based rules, which are not realistic enough. erefore, one potential improvement to the agent-based model approach is to consider how to merge the behavioral realism in individual-based models to improve the maturity level of the model. e development of AI technology can also provide more and more insights to improve the previous modeling approaches by optimization methods like model training of machine learning.

Limitations in Innovative Approaches to Improve
Computing Efficiency. Within many conventional agentbased models, the movement of individual agents is modeled in great detail. e explicit simulation of agent interactions and moving dynamics increase the model complexity, which causes large usage of computational memory and subsequent reduction of processing speeds, in particular for largescale transport networks consisting of a large number of entities. erefore, the motivation falls in finding the solutions of some modeling approaches, which offer an answer to protect a certain level of modeling resolution and remain acceptable computational efficiency, seeking the balance of behavioral realism and computational efficiency. For example, Manley et al. [60] studied a hybrid agent-based model framework, which integrates a detailed description of driving behavior with a macroscopic traffic flow model. e agents are granted cognitive abilities in the short-term agentbased model, and a macroscopic-level traffic dynamic model of traffic flow is employed to constrain the aggregated movement of agents. eir hybrid approach allows analyzing drivers' behavior and interactions without scarifying computational efficiency. However, such innovative approaches are relatively sparse.

Conclusion and Discussion
e agent-based model can be used to study the systems consisting of agents that simultaneously interact and affect each other. e transport systems, which are complex, dynamic, stochastic, and heterogeneous, conform to these characteristics and hence are ideal to be modeled and analyzed using agent-based models. is article explains the thriving applications of the agent-based model in the transport research domain, such as traffic management, dynamic route, traffic signal control, mode choice, environmental/economic analysis, and transportation infrastructure and policy development.
e advantages of the agent-based model approach in transport modeling and analysis can be summarized as follows: (1) consideration of the interactive features and rule settings in the study of heterogenicity and emergent dynamics in the form of different time scale models; (2) exploration of modeling the adaptability, beliefs, and desires of individual entities to establish a more robust system; (3) capability of being integrated with optimization to achieve realistic and reliable results.

Recommendations.
Agent-based modeling and simulation approaches are widely applied for transport systems with mixed traffic, mobility adjustments, dynamically personalized routing guidance, cooperative behavior. e differences in the predefined purpose lead to different agent-based models. is article made a comparison of the agent-based models developed in the literature by categorizing them into three different types as per time scale in behavior (rule) simulation. e hybrid modeling approach is discussed further with refinement in model establishment patterns. e short-term, medium-term, and hybrid agent-based models are the most commonly used in transport research. e short-term agent-based models can be used for systems that focus on detailed motion level simulation (vehicle model, pedestrian model, information-sharing rules, mobile coordination, etc.) for analyzing the traffic condition or assessing the local impact of some proposed extension plan for a real city. However, the high-resolution simulation demands considerable computer memory if the agent number and the network size reach the scale of a city, thus slowing down the model's execution speed. For general cases, we recommend covering a scale of hundreds of thousands of agents within a specific area. Otherwise, some simplifications should be made to the agent models.
In (on-demand) sharing systems, due to the complex and discontinuous characteristics or the discrete protocols for interaction related to decision-making behavior, it is impossible to deal with them by nonlinear analytical methods. Medium-term agent-based models are well suited to deal with the great complexity of the systems under study and assess the performance of some proposed services or plans in various scenarios. e feedback information from interaction among different agents directly influences the following decision-making process. In addition, positioned at the strategic level of simulation, medium-term agent-based models can capture the transformation pattern of agents' choice (e.g., due to an incident) that affects the system performance.
In practice, the hybrid agent-based models are gaining wide consideration within transport system simulation areas as the models allow explicit modeling of the individual decision process from microscopic to macroscopic levels and demand to supply sides. Incorporating the short-, medium-, and long-term models, the hybrid modeling approaches integrate microscopic, mesoscopic, and macroscopic simulation to achieve joint and multiscale systems that meet multidimensional simulation requirements. In some cases, if short-term models are omitted or simplified properly, like searching for very fast microscopic (or mesoscopic) mobility models to speed up the computation, these hybrid models can be applied across relatively large regions like a whole transport system in a city with millions of people. Moreover, developing the simulation system in a single type of model may lead to a waste of computational Journal of Advanced Transportation 13 power if the models are only designed in a short-term way or the failure of an accurate description of the phenomenon or systems otherwise. Using hybrid models, components in different levels of resolution can be grouped according to the characteristics of the targeted transport system and the modeling and simulation purposes. erefore, hybrid models have advantages in terms of computational power savings by focusing on different simulation parts or evolving stages of the whole transport system.
As an important application domain of agent-based models, the intelligence, adaptation, and learning behavior simulation can be further applied in AVs [91][92][93]. e key features of intelligent and autonomous vehicles (IAVs) compared to humans are that they are more likely to cooperate, be fast in processing communication information, and make more precise operations and decisions. In the domain of AVs, collaborative driving is a key developing direction of ITS (Intelligent Transportation System).

Insights for Future Research.
For future research of agent-based models in the transport domain, one important aspect is researching how to develop efficient and accurate individual behavioral models.
is is hard to achieve by simply introducing more probabilistic descriptions. It is necessary to figure out how to identify and characterize the key behavior of stakeholders such as drivers in real-life transport systems and feed them (directly) into the representations of behavior in agents. e large-scale simulation, like city-scale or even country-scale, is another research gap of the agent-based model, which requires increased support in computer processing power and adequate and accurate data sources. Besides, the credibility of the agent-based models and their results need to be increased, which involves extracting useful and valid data needed for model calibration and validation [94]. e agent-based modeling and simulation have formed a relatively perfect theory, method, and technology system, which will provide a solid foundation and favorable support for the research and application of Digital Twin (DT) technology in transport systems. Considering the disruptive forces that are transforming the mobility ecosystem, transportation industries have to manage the social and economic impacts of traffic growth and provide more "intelligence" components to manage future challenges. e key elements, the rules of the agents' movement, and the interactions of the "twin" system can be clearly defined close to the real-world based on agent-based modeling. erefore, in combination with agent-based models, DT can be used to develop an upgraded "Dynamic Information Physical System" closely integrated with the actual traffic system and eliminate the effects of some inevitable deviations of the accurate matching and real-time performance of data in simulation. e digital and structured data can be obtained through the process of model reconstruction and accurate perception on the computing platform, which has twin functions established with these agent-based rules in place. ese calculated real-time data, applied to all traffic services, and can be used to quickly realize the representation and detection of traffic behavior and the deduction and automatic analysis and prediction of traffic state [95,96]. e gaps in the context of the different transportation settings in developing economies vs. developed economies may also lead to more challenges and insights for future research. Industry digitalization has become the key leading force driving the development of the global digital economy. e transportation settings like the infrastructures are more complete under the developed economies and more promising to implement the agent-based transportation scheme. Applying the agent-based model framework in a real-world complex system requires a number of critical issues like collecting and integrating multisource data that is readily used to construct a complete business platform. e higher the level of economic development, the higher the digitalization of the industry. erefore, based on the advantages of artificial intelligence and Big Data technology, the developed economies better promote the successful implementation of agent-based technologies and achieve the objectives of alleviating urban traffic congestion. Specifically, no matter in the developing economies or the developed economies, there can be significant challenges to achieve the accurate perception of traffic events and traffic flow based on holographic data, traffic knowledge maps establishment to mine historical and real-time portraits of road network supply capacity, and travel characteristics of people and vehicles and the development of the terminal of the intelligent transportation operation architecture.

Conflicts of Interest
e authors declare that they have no conflicts of interest.