Artificial Intelligence Applied on Traffic Planning and Management for Rail Transport: A Review and Perspective

Artifcial intelligence (AI) has received much attention in the domain of railway trafc planning and management (TPM) from academia and industries. While many promising applications have been reported, there remains a lack of detailed review of the many AI models/algorithms and their uses and adaptations in rail TPM. To fll this gap, this systematic literature review conducts, reports, and synthesizes the state-of-the-art of AI applied in railway TPM from four perspectives, i.e., the intersection between AI research felds (e.g., expert systems, data mining, and adversarial search) and rail TPM, the intersection between AI techniques (e.g., evolutionary computing and machine learning) and rail TPM, the intersection between AI applications (e.g., operations research, scheduling, and planning) and rail TPM, and the intersection between AI related disciplines (e.g., big data analytics and digital twins) and rail TPM. Te study evaluates 95 research papers published during 1970–2022. Accordingly, a comprehensive synthesis of each intersection between AI and rail TPM is presented, and the practical roadmap for application of AI in rail TPM is proposed. Furthermore, the study identifes the research gaps and areas that need more investigation. Te contribution helps researchers and practitioners to get a better understanding of the status quo of research stream, research development trends, and challenges for further related study.


Introduction
Artifcial intelligence (AI) is becoming the most central player in Industrial 4.0, and the railway industry is included [1]. It is recognized that the potential of AI in the railway sector should never be underestimated [2]. Tough AI is still largely at its infancy stage in the railway sector currently, AI is assumed to become a common tool used throughout the rail industry in the near future [3,4]. As one of the main pillars of Industry 4.0, AI is the type of exponential technologies [5], which help to sharply increase productivity and efciency. So far, current researches about AI applied in railway contributed most to the feld of railway maintenance and inspection, safety and security, and automation with AI technique, e.g., image processing, natural language processing, computer vision, decision tree, machine learning, and deep learning [6].
Varying in their "intelligent" trait, artifcial intelligence (AI) does not refer to a single concept or technology. Te set of the mainstream AI domains can be divided into four main categories [2], i.e., AI research felds (including expert systems, data mining, pattern recognition, and adversarial search), AI techniques (including machine learning, evolutionary computing, and logic programming), AI applications (including operations research, scheduling and planning, computer vision, image processing, natural language processing, speech recognition, autonomous systems, and robotics), and AI related disciplines (such as big data analytics, digital twins, and augmented reality). According to the deliverable reports [3], machine learning techniques are the most exploited in the railway sector due to their versatility and potential to analyze diferent kinds of data.
Based on RAILS [4], trafc planning and management is the core feld of railway subdomains, surrounded by maintenance and inspection, autonomous driving and control, revenue management, transport policy, safety and security, and passenger mobility. Rail trafc planning and management (TPM) covers all the activities that deal with efective and efcient capacity management, train timetabling, scheduling of trains and crews, optimal use of rolling stock and energy, resource allocation and management, control of railway operations (energy-efcient driving, autonomous driving and control, and train trajectory), trafc state prediction, estimation of trafc demand and capacity, analysis of passenger and freight railway transport, routing, shunting, disruption management, trafc rescheduling (retiming, reordering, and rerouting), and equipment layout, so as to increase the competitiveness and efciency of passengers and freight transport from strategic to tactic perspective [7].
For AI applied on the railway domains, during the years between 2010 and 2020, only secondary to subdomain of maintenance and inspection (57%) and followed by subdomain of safety and security (8%), the number of articles in railway subdomain of trafc planning and management occupied 25% of the total 141 papers obtained [7], which indicated that this subdomain has attracted many studies over the years. However, in most literatures about AI applied on railway systems, the felds of trafc planning and management have been only reviewed as a subdomain [6,7]. Te number of datasets divided by railway application showed that trafc planning and management is one of the most prevalent railway domains in AI-oriented studies [1]. However, only a few of the exclusive literature reviews about AI applied on rail TPM have been discovered, from the AI aspect of data mining [8], evolutionary computing [9], operations research, and scheduling and planning [10][11][12][13][14]. Also, the focused area in railway subdomain of TPM included rescheduling, delay management (delay analysis/ prediction), timetabling, railway capacity, confict prediction, train trajectory, train routing, railway disruptions, train shunting, and stop planning. Among the detailed segmentation of the rail trafc planning and management that the existing literatures have explored, the train timetabling and rescheduling are the kinds of complex combinatory NP-hard problem, while delay analysis and prediction are the kinds of data-driven problem, all of which demonstrated the sound matching between AI techniques and rail subdomains. Tis review surveys the research carried out within the area of railway trafc planning and management crossing over AI. Even though this is a rather well-known problem domain, the number of reviews handling with this topic is limited. Our literature review would be conducted with a more comprehensive perspective in more details. In particular, we are focusing on all the connections between the subdomain of rail TPM and the concerned AI categories. Te classical algorithms for railway scheduling and planning cover exact approaches (integer programming, linear or nonlinear programming, mixed integer linear programming, etc.), ad hoc heuristics, simulation models, expert systems, constraint propagation, and alternative graphs [15], most of which are only suitable for the not very large-scale problem instances.
Te main objectives and contributions of this review paper are as follows: (1) Synthesis of the literature on AI applied in rail trafc planning and management (TPM). To our knowledge, this the frst exclusive review paper about AI applied in rail trafc planning and management (TPM). (2) Conducting the paper distribution analysis and academic research analysis and matching between AI subdomains and rail TPM applications. (3) Mapping of artifcial intelligence on rail trafc planning and management (TPM), including survey of the intersection between AI research felds and rail TPM, survey of the intersection between AI techniques and rail TPM, survey of the intersection between AI applications and rail TPM, and survey of the intersection between AI-related disciplines and rail TPM. Also, a synthesis was conducted for each subsurvey, which deepens the crossover between AI and rail TPM. (4) After reviewing the literature, the output of the review is a guideline to support the future AI application to smart rail TPM, which support rail industry stakeholders to promptly determine promising and useful AI solutions to solve certain problems of rail TPM. (5) Determining a taxonomy of AI to enable its application in rail TPM, and the state-of-the-art of AI techniques in rail subdomain of trafc planning and management. (6) Tis paper not only identifes application areas of AI in rail trafc planning and management but also matches railway TPM problems and applicable AI techniques and contributes to select AI models and algorithms to deal with specifc railway TPM problems. More importantly, this work could bridge the gap between AI application and intelligent decisionmaking of rail trafc planning and management.
Te rest of the paper is organized as follows. Section 2 demonstrates the search strategy and overview on papers. Section 3 reports the mapping of artifcial intelligence (AI) on rail trafc planning and management (TPM), including intersection between AI research felds and rail TPM, intersection between AI techniques and rail TPM, intersection between AI applications and rail TPM, and intersection between AI-related disciplines and rail TPM, respectively. Section 4 conducts the summary and discussions about the review. Finally, Section 5 concludes the paper.

Search Strategy and Overview on Papers
Te methodology of our review is based on [16]. For the systematic literature review, a wide variety of papers on the 2 Discrete Dynamics in Nature and Society subject of artifcial intelligence (AI) applied to rail trafc planning and management (TPM) were consulted. Te wellknown scientifc search engines used for the search include Elsevier Scopus, Web of Science, IEEE Xplore Digital Library, ResearchGate, ScienceDirect, and SpringerLink and consequently screened their references. Te search terms used to fnd the publications are a combination of the following key words in the titles, keywords, and abstracts of railway-related publications: "artifcial intelligence," "AI," "railway," "metro," "urban rail transit," "train," "planning and management," "delay," "timetabling," "scheduling," "rescheduling," "machine learning," "reinforcement learning," "evolutionary computing," "expert systems," "data mining," "adversarial search," "operations research and scheduling," etc. Some of the keyword pairs used in initial searching is showed in Table 1. For search items, we build more queries combining keywords related to AI research felds/techniques/applications and railway subdomain of TPM. Some of the reviewed publications were also manually selected from the reference lists of [6,7], most of which are highly correlated to this review topic. We limit the inclusion of contributions to journal publications and conference proceedings written in the English language and delete the duplicate publications. Finally, a fnal refnement was made based on full texts and sorting of all papers based on full texts. After all steps, we reviewed 95 papers. Among the reviewed publications, almost 76.6% are journal publications. Te number of yearly contributions has increased steadily in the recent few years, as shown in Figure 1. For the sake of visualization, the label "before 2015" collects 27 publications from the years 1970, 1998, 2000, 2001, 2003, 2005, 2007, 2008, 2009, 2011, 2014, and 2015. Table 2 reports the scientifc journals in which most contributions have appeared, also divided in publication periods.

Intersection Analysis between AI Research Fields and Rail TPM
3.1.1. Paper Reviews by Expert Systems. Expert systems can be seen as the combination of knowledge base and inference engine from a more structured point of view. While the knowledge base contains the coded domainspecifc knowledge of a problem, the inference engine consists of one or more algorithms to process it [17]. As dispatching links the long-term or midterm planning with real-time train operation, expert systems allow fexible design and application of expert knowledge by introducing computer-aided dispatching systems to handle train operation plan and disrupted situations [18]. For the limitations of the current research in automatic train operation, i.e., reducing the passenger comfort and impairing the train operation intelligence, Yin et al. [19] proposed an intelligent train operation algorithm ITOe for expert system consisting of expert rules and heuristic inference method.

Paper Reviews by Data
Mining. From the view point of knowledge discovery from data process, data mining represents an essential procedure where intelligent methods are applied to extract information (patterns/models) from data [20], by which the model-free method can work for the rail trafc planning and management, e.g., delay management. Te authors in [7] summarized the various historical data sources for rail trafc planning and management, e.g., realized trafc movements, infrastructure occupation data, topology of railway network, and existing train scheduled timetables.
Considering the multiattribute data of dynamic systems, i.e., static, time-series, and spatiotemporal format [21], developed a deep learning approach that combines 3dimensional convolutional neural networks, long-short-term memory recurrent neural network, and fullyconnected neural network (FCNN) architectures to address the train delay prediction for four railway lines with diferent operational features, from the perspectives of temporal data mining and spatiotemporal data mining. Te results showed that the proposed combinatory methods outperformed conventional machine learning models.
Te typical data mining methods for big data include neural networks [22], Bayesian networks [23], and supporting vector regression methods [24]. Te authors in [25] employed a big data mining technique, i.e., K-means clustering, to identify recurrent delay patterns on a high trafc  Discrete Dynamics in Nature and Society railway line in north of Copenhagen. By showing the cases where recurrent delay patterns take place, the K-means clustering results can discover the conditions where regulations are necessary, and they can provide managerial insights to improve timetables and processes for train operational analysis.
According to the data collection conditions, the train operational data can be classifed as those collected under recurrent conditions relating to minor disturbance events and those collected under nonrecurrent conditions relating to major events. Using automatic train supervision data, Liu et al. [26] presented a data analytics approach for train timetable performance evaluation, consisting of data processing and cleaning, waiting time assessment method (headway irregularity), process time estimation method (running and dwell time variability), and arrival punctuality examination method (arrival time reliability). Te case study of Shanghai Metro demonstrated that the proposed data analytics framework and fndings had operational and planning implications with regard to evaluating timetable parameters and improving the passenger satisfaction.

Paper Reviews by Adversarial Search.
Adversarial search comprises algorithms, techniques, and ideas from both game theory and agent-based modelling. For more and more complex transport behaviors and phenomena that cannot be analyzed successfully and explained using analytical models, agent-based modelling and simulation is a good choice. Also, some principles of natural swarm intelligence in the development of artifcial systems can be used to solve the complex multiagent system in transportation planning and management [27], including but not limited to the railway industry. In the background of competitive railway market in 1990s, the authors in [28] proposed a game theoretical model for a coalition formation problem. Also, the agents (i.e., transport operators) in the model could exchange information on their needs and be rewarded by a possible increasing of their utility. By using game theory simulation, the authors in [29] explored the theoretical framework and a set of experimental studies towards railway capacity allocation for freight paths in Britain. By merging two streams of research for railway trafc rescheduling, i.e., train scheduling and routing (operations-centric railway trafc models), and passenger routing and route choice (passenger-centric railway trafc models), the authors in [30] investigated microscopic railway trafc optimization models and algorithms with a game theoretical approach, tackling the microscopic delay management problem [31], and focusing on seeking Nash equilibria among multiple stakeholders, e.g., passengers, infrastructure managers, and railway operators. In order to address the tradeof between the interest of railway operator and the service quality of passenger needs by an optimal departure frequency, the authors in [32] developed a Stackelberg game model on the train operation scheduling, modelling the railway operator as the game leader and the passengers as the game follower.
Te authors in [33] proposed a multiagent based railway timetable scheduling algorithm, handling the in-between time delay of the newly introduced train. So far, diferent multiagent transport simulation toolkits for trafc prediction have been developed, e.g., MATSim [34], DynaMIT [34], TranSim [35], and AIMSUN [36]. Most of the agentbased demand models employing activity-based approach used data from national census. Alternatively, the authors in [37] created agent-based microsimulation model using public transport organization's passenger survey datasets. In summary, agent technique can be used as the distribution artifcial intelligence for the self-organizing system, including the railway transportation system, e.g., trafc planning and scheduling, delay management, and rescheduling optimization. Te application of agent theory and multiagent system as a solution to railway transportation has been observed in a number of studies [33][34][35][36][37][38][39]. Multiagent technique provides the promising and feasible solutions for decentralized railway TPM in a distributed manner, especially when the centralized options are far more complex for too many combinatory possibilities, e.g., spatial and temporal allocation of the tracks, and solving conficts of simultaneous resource requests. Te current standard reactive practice in trafc management consists of timetabling and scheduling/rescheduling. Timetabling is a process of of-line development work, and scheduling/ rescheduling is the real-time adjustment of the developed timetable in case of disturbance. For the desired proactive operational trafc management systems [40], the key participants, i.e., train drivers, dispatchers, and network controllers can be taken as self-organized autonomous agents interacting in a common environment within certain control structures, e.g., multilayered hierarchical or nonhierarchical structures.

Synthesis of Intersection between AI Research Fields and
Rail TPM. Referring with the AI taxonomy to the rail sector [2,6], AI research felds range from expert systems to adversarial research, including expert systems, data mining, pattern recognition, and adversarial research. As few publications about the method of pattern recognition applied in the subfeld of rail trafc planning and management can be found, this review has to omit the details about pattern recognition. Typically, both data mining and adversarial research are more popular in the area of rail TPM in recent years, comparing with expert systems and pattern recognition. Diferent data mining methods, e.g., K-means clustering, neural networks, Bayesian networks, supporting vector regression, and deep learning, have been investigated in train delays, while adversarial search mainly focused on timetabling, capacity allocation, trafc scheduling, delay management, trafc prediction, trafc simulation and optimization, and trafc control. Table 3 summarises the specifc methods of AI research felds applied in the rail trafc planning and management.

Paper Reviews by Genetic Algorithm (GA).
Genetic algorithm (GA) is a typical evolutionary algorithm. One of the popular application areas of GA is to solve the train timetabling problem. For obtaining the timetable for the new trains on a railway line that is occupied (or not) by other trains with fxed timetables, the authors in [41] proposed a GA with a guided process to build the initial population. For minimizing the travel time of each train and maximizing capacity of the network, the authors in [42] presented an optimization-based train scheduling approach, i.e., fxed path + genetic algorithm. Also, the GA was used for selecting the assumed fxed path for each train. For dealing with the time-consuming problem of the exact approaches on realsize instances of optimal train timetabling problem, the authors in [43] proposed an alternative mathematical model and GA implementing method to solve the near-optimal train timetables. For optimizing three indicators of the mixed-speed train trafc structure for a cyclic timetable, i.e., heterogeneity, cycle time, and bufer time, the authors in [44] proposed the random-key genetic algorithm (RKGA) to solve the mixed-speed train trafc planning (MSTTP) model. It has been found that GA could produce the same or very similar results as nonlinear programming models for timetabling, with performing much better computationally [45].

Paper Reviews by Swarm Intelligence (SI).
Swarm intelligence-based approaches are the type of important bioinspired computations which focus on the collective behavior of decentralized, self-organized systems [46]. From the perspective of the nature of the analyzed systems, swarm intelligence (SI) can be classifed as natural SI (focusing on biological systems) and artifcial SI (focusing on human artifacts) [47]. While from the perspective of goals pursued, SI can be categorized into scientifc SI (focusing on the understanding of natural swarm systems) and engineering SI (focusing on the design and implementation of artifcial swarm systems). Also, SI can be divided according to maturity of theory, the authors in [48] presented the existing swarm intelligence-based algorithms with their main applications, e.g., ant colony optimization (ACO), artifcial bee colony (ABC), and particle swarm optimization (PSO). Tese SI algorithms are common in the features as they are inspired from animals, iterative and population-based. Te diference of them lies in the exploration and exploitation of work place. Te authors in [49] proposed a particle swarm optimization-based method for railway trafc to reduce the waiting time of trains and established a simulation environment. Considering the requirements of all stakeholders simultaneously, the authors in [50] employed a particle swarm optimisation (PSO) approach for timetabling in an open market to address the combinatorial timetable generation problem and tested the suitability and performance of PSO on a multiagent-based railway negotiation (i.e., between infrastructure provider and train service provider) simulation platform.

Paper Reviews by Machine Learning (ML)
(1) ML for Delay Management. Te delay management problem involves the delay pattern recognition (methods include probability density, phase-type distribution, association analysis, clustering model [25], delay prediction (methods include regression, statistics, Bayesian network, Markov chain, Graph model, Petri Net, neural network, support vector machine, decision tree, deep learning, and Kalman flter), and the decisions about which transfer connections should be kept or canceled when running delays occur (mainly use mathematical optimization method). Delay pattern recognition, e.g., identifcation of behavioral patterns in large datasets (big data), showing the recurrent delay pattern cases and revealing the conditions where operational adjustments are necessary, can provide the chances to discover the factors afecting reliability, modify the timetable, and improve train operation processes. Te authors in [25] employed K-means clustering to identify recurrent delay patterns on a high trafc railway line of Copenhagen.
Based on the properties of the train, the attributes of the network, and the properties of potentially conficting train trafc on the freight rail network, the authors in [51] proposed a data-driven approach to predict estimated times of arrival of individual freight trains with support vector regression, i.e., regression via support vector machines.
Discrete Dynamics in Nature and Society  6 Discrete Dynamics in Nature and Society Te authors in [52] proposed a real-time Bayesian networks model to predict the primary delay, the number of afected trains, and the total delay trains, resulting from the spatial and temporal propagation of disruptions and disturbances during train operations. For train delay propagation pattern discovery, the authors in [53] designed a deep learning network model FCF-Net, consisting of fullyconnected neural networks (FCNNs) and convolutional neural networks (CNNs).
(2) Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) for Rail TPM. As far as the learning paradigms of machine learning are concerned, reinforcement learning (RL) is a type of unsupervised machine learning. Deep learning (DL) has the power in tackling large and complicated problems, while reinforcement learning (RL) can provide a generic and fexible framework for sequential decision-making, especially suitable for autonomous decision-making and operation control. Deep reinforcement learning (DRL), i.e., an integration of DL (e.g., convolutional neural network and recurrent neural network) and RL, is an emerging and promising methodology for tackling many complicated transportation real-time decision-making problems. Te authors in [54] conducted a comprehensive and synthesized review of DRL applications in transportation (e.g., train timetable rescheduling, automatic train operations, and train shunting operations) by investigating about 150 DRL studies that have appeared in the transportation literature, in terms of DRL's fundamentals, applications, strengths, and weaknesses. One thing has to be pointed out is that existing DRL transportation research is mainly performed in tailored, simplifed, and simulated environments (such as Flatland [55] with synthetic data, rather than in large-scale real-world applications. Te authors in [56] proposed a reinforcement learning method for train scheduling (either from scratch or from a given operating state). Te state space is a vector including the integer priority value and the local resource status, while the action space consisted of two binary elements, with 1 representing a decision to dwell in the current resource for a predefned time period, and 0 representing a decision to move the current train to the next resource on its journey. Also, the table-based Q-learning algorithm adopted a slightly modifed version of the Ɛ-greedy policy for action selection. Finally, they summarized four advantages about using reinforcement learning rather than heuristics.
Reinforcement learning is a sequential decision process, and the mathematical foundation of RL is Markov decision process (MDP). Timetabling, scheduling, and rescheduling can be classifed into the tactical or operational level of railway planning. In nature, timetabling is a type of scheduling from scratch. To facilitate the reinforcement learning process by Sarsa on-policy methods, the train timetabling can be modelled as a discrete-time Markov decision process [57]. For the nonperiodic macroscopic train timetabling problem of diferent railway systems (i.e., both a single-track railway system and a double-track system), the authors in [58] proposed a multiagent deep reinforcement learning approach, constructed a general train timetabling learning environment by modelling the problem as a Markov decision process, and built a multiagent actor-critic algorithm framework to decompose the large-scale combinatorial decision space into multiple independent ones with parameterized deep neural networks.
By integrating deep learning, reinforcement learning can be enhanced to deep reinforcement learning, so as to mitigate "the curse of dimensionality." It was reported that over 150 papers have appeared in the literature about deep reinforcement learning (DRL) in transportation research from 2016 to July 7, 2020 [59], because of the capability of DRL to solve large, complex transportation problems, e.g., timetabling, and rescheduling (retiming, rerouting, reordering, or canceling in case of uncertain disturbances). For deep reinforcement learning, the authors in [59] summarized that the main approaches for railway transportation problems were DQN (deep Q-network) and DDPG (actor-critic-based deep deterministic policy gradient) algorithms, with a fully observable environment. Te considered state set involves actual arrival and departure times, delay condition, train speed and position, number of the departing train and its last dwelling time, relative position from the front train, etc. Te action space contains accelerate or decelerate and their magnitude, reordering of the departure sequences, rescheduling timetable, speed and dwelling time of the departing train, halting and departure decisions, etc. Te concerned rewards include delay, frequency of train, driving time, stoppage, negative average of total delay, speed deviation from the target speed, energy consumption, recovery energy, and the negative of traction energy.
For handling the size of the problem of real-time railway trafc management arising from traditional methods (e.g., alternative graph model and mixed integer linear programming model.), the authors in [60] proposed a reinforcement learning-based rescheduling method, i.e., Qlearning, which learns how to reschedule a timetable ofine and then can be used online for immediate optimal dispatching decision. Te online application of the method in the second stage is completed by sensing the current state of the railway environment according to the reinforcement learning mechanism (i.e., map states of an environment to actions and maximize the cumulative rewards of these actions). Prior to this paper [60], some of the existing literatures about reinforcement learning-based train timetable rescheduling consider the trafc controller as the agent (actor) who take actions, and the actions include deciding the departure sequence of trains from stations [61], deciding the sequence of trains passing through a junction [62] or multiple junctions [63], retiming and reordering trains for single-track railway line [64], and controlling the train movements in single-track or double-track railway lines [56].
Regarding the train delay as input to the neural network, the authors in [65] simulated train dispatching using graph theory approach (i.e., the train schedule was expressed in a program evaluation and review technique graph) and proposed a deep Q-network (DQN)) for train rescheduling, which presented positive results for over 50% of test cases for small-scale train delays, and the obtained train rescheduling Discrete Dynamics in Nature and Society could decrease passengers' dissatisfaction to a certain extent. For the capacity management in the railway sector, AI is supposed to be able to act as a game changer [4]. For intelligent train manipulation and prediction, the authors in [66] proposed a step-by-step long-short-term memory (LSTM), dealing with the issues of falling into local optimum when optimizing the structure, weight, and threshold by the gradient descent approach.
To overcome the possible poor performance due to changeable environments and unrealistic assumptions of parametric based models for train dynamic model FCF-Net, the authors in [67] developed data-driven long-short term memory network based algorithm in a nonparametric way. In addition, the proposed method was extended to predict train velocity for multiple steps ahead. By formulating the train unit shunting problem (TUSP) as a Markov decision process, the authors in [68] developed an image-like state space representation which can be approached by a deep reinforcement learning solution, i.e., the deep Q-network. Te results indicate that the deep Q-network can develop an online consistent strategy and solution for TUSP capable of handling uncertainty. For the frst time, the authors in [69] solved the periodic timetable optimization problem with a novel approach based on reinforcement learning, multiagents, and Boolean satisfability problem (SAT) for public transportation scheduling with respect to the travel time. Te authors in [70] investigated the machine learning method, i.e., two diferent deep Q-learning methods, for solving the real-time online single-track train scheduling problem, and they presented both centralized architectures and decentralized architectures. In the centralized approach, the line coordinator is taken as the agent (actor), and the entire line is taken as the environment. While in the decentralized approach, the train is considered as the agent, i.e., the train itself is capable of decision-making, and the observable line is considered as environment. Te main diference between them lies in the topology of the state.

Synthesis of Intersection between AI Techniques and Rail TPM.
Referring with the AI taxonomy to the rail sector [2,6], AI techniques cover evolutionary computing (evolutionary algorithm and swarm intelligence), machine learning, and logic programming. Based on the distributions of the publications in the feld of rail TPM, we mainly focus on evolutionary algorithm, swarm intelligence, and machine learning (RL, DRL, etc.). Te use and adaptation of ML for rail trafc planning and management have been multifold, spanning many types of problems from train delay management to NP-hard train scheduling, rescheduling problem, and capacity management. Te conventional methods for train timetabling problem can be categorized into two main categories, i.e., mathematical programming, e.g., integer programming (solved with Lagrangian relaxation algorithm) and mixed-integer programming model (solved with branch-and-bound algorithm), and system simulation modelling the train timetabling problem (TTP) as a Markov decision process (MDP). For railway trafc scheduling and rescheduling problems, certain exact mathematical programming methods are available, e.g., branch and bound algorithm for mixed integer linear programming but usually are not practical for the realistic larger scale problem. Besides, heuristics are alternatives to exact approaches, but they need difcult balances among development efort, computational performance, and solution quality. Beyond these, reinforcement learning of artifcial intelligence can be regarded as a third approach which can tackle the limitations of the two aforementioned methods [56]. Te intersections between RL and rail TPM are the most prosperous section. Article [64] appears to be the frst to use reinforcement learning for train rescheduling by leveraging Q-learning approach. Table 4 summarises the specifc methods of AI techniques applied in the rail trafc planning and management.

Paper Reviews by AI Applications.
Train timetabling is a type of ofine (done month in advance) and static railway trafc planning and management. While on the online perspective (done during operations) [11], there are three types of rail trafc management, i.e., online and static trafc rescheduling (open loop control and optimize only once), reactive dynamic trafc rescheduling (closed loop control and optimize solutions when updated information is available), and proactive dynamic trafc rescheduling (closed loop control and optimize solutions when updated information is available). For the latter two dynamic methods, short computational time of few seconds/minutes, i.e., efciency, is a prerequisite. Tus, AI applications in this feld are indispensable. In both practice and theory, train trafc rescheduling in network is a challenging work. Te authors in [14] presented a comprehensive survey on this rescheduling problem by a clear classifcation, including the frequently used models (e.g., integer programming model, mixed-integer programming model, constraint programming model, and alternative graph model) and their variables and constraints, as well as the solution approaches (e.g., heuristics and meta-heuristics). With operations research (OR) methods, the problems can be split into components and solved with mathematical analysis, e.g., linear or nonlinear programming and dynamic programming. Operations research involves the scientifc decision-making by using mathematical models to represent real issues under specifc conditions, and AI is a powerful instrument for fnding the best available solutions towards the complex mathematical optimization models with the characteristics of NP-hard combinatorial search problems, e.g., constraint programming, evolutionary algorithms, swarm intelligence, and reinforcement learning. Te vehicle rescheduling problem (VRSP) for railway networks has been a major focus of operations research for many decades. Traditional approaches, e.g., simulation, to solve large scale VRSP are usually time consuming and sufer a huge computational overhead. Te authors in [55] proposed a two-dimensional simplifed grid environment Flatland, which provided an interface to explore novel  Discrete Dynamics in Nature and Society solutions for VRSP from machine learning, e.g., multiagent reinforcement learning (MARL) on trains and combinations of OR and RL, and further to reduce the complexity of the simulation process and allow for faster experimentations. In Flatland, the railway network is represented as 2D grid environments with transition constraints between adjacent cells according to basic transition maps, and the trains are described as multiple collaborative agents with various scheduling objectives to minimize the global travel time or delay on the network for a long-term reward. Te discrete action space of Flatland consists of go forward (or turn to opposite direction and continue forward if the agent is facing a dead end), left turn, right turn, halt on current cell, and noop (let the agent continue what it was doing previously). Tere are three types of observation space in Flatland environment, i.e., global observation (the whole scene is observed), local grid observation (a local grid around the agent is observed), and tree observation (the agent can observe its navigable path to some predefned depth). Te experimental results of Flatland demonstrate that RL has the potential to efciently and efectively solve vehicle rescheduling problems for railway network.
As train movement planning at a railway station is complexed, for the frst time, the authors in [71] developed a reinforcement learning-based approach to learn heuristics for generating the operational train movement plan at the station, under diferent interarrival times between trains. In congested scenario, the heuristics learned by the reinforcement learning approach outperformed that developed using operational heuristics being used in practice. From the machine learning and operations research communities, the authors in [72] surveyed the recent attempts at using machine learning to solve combinatorial optimization problems. Tey advocated for further integrating machine learning and combinatorial optimization, e.g., the machine learning model can be used to augment or boost an operation research algorithm with valuable pieces of information, machine learning can provide a parametrization of the combinatorial optimization algorithm. Also, they detailed the theoretical learning framework and methodology to do so.
During the past few years, various approaches on machine learning-supported metaheuristics have been proposed; the authors in [73] conducted a comprehensive survey and taxonomy on this research in applying machine learning (ML) to design well-performed (e.g., efective, effcient, and robust) metaheuristics, so as to motivate scholars in operational research optimization to include ideas from ML into metaheuristics. According to the concerned search component, there are three hierarchical ways to use ML in metaheuristics, i.e., problem-level ML-supported metaheuristics, low-level ML-supported metaheuristics, and high-level ML-supported metaheuristics. It has been proven that incorporating machine learning into metaheuristics is advantageous in convergence speed, solution quality, and robustness.
Bilevel optimization has widespread applications in transportation. Machine learning can be used as an auxiliary tool assisting an original heuristic solution method. For tackling the NP-hard bilevel transportation problem efciently, the authors in [74] developed a hybrid machinelearning and optimization method, which transformed the original problem, i.e., the nonlinear discrete bilevel transportation network design problem with equilibrium constraints, to an integer linear programing problem using a supervised learning technique and a tractable nonlinear problem. Tey employed MATLAB to solve the machine learning tasks and GAMS (with CPLEX solver) to solve the optimization problems. Te authors in [75] designed a reinforcement learning system for generating marshaling plan of freight cars in a train, applying Q-learning to minimize both the total transfer distance and the number of movements of a locomotive for the desired layout of freight cars for an outbound train. Te state of marshaling yard was described by the layout and movements of freight cars, while the state transitions were defned based on Markov decision process.

Synthesis of Intersection between AI Applications and
Rail TPM. In summary, AI applications in rail TPM mainly focus on operations research, combinations of OR and RL, machine learning-supported metaheuristics, MARL, bilevel optimization, etc. Also, reinforcement learning has demonstrated efectiveness and advantages in many applications of scheduling and rescheduling problems, e.g., train scheduling and rescheduling [61]. Referring with the AI taxonomy to the rail sector [2,6], AI applications cover scheduling and planning, operations research, natural language processing, speech recognition, image processing, compute vision, autonomous systems, and robotics. Based on how critical the role that AI technique plays in the system, AI applications can be categorized into system level applications (mainly autonomous system and the whole system is controlled by AI technique to a large extent), and module/ component level applications (the problems can be solved with innovative approaches from AI techniques and modules in many areas) [76]. In nature, the TPM problems mainly involve scheduling and planning-related decisionmaking process, which can be addressed using various AI applications, e.g., evolutionary algorithms [77] and operations research. Te typical intersections between AI applications and rail TPM are summarized in Table 5.

Intersection Analysis between AI Related Disciplines and
Rail TPM 3.4.1. Paper Reviews by Big Data Analytics. Before the implementation of the AI model, data collection is the core of the data-driven AI model and need to be handled accordingly. Most of the publicly available datasets related to the railway domain of trafc planning and management are the type of numerical data, followed by the type of label data [45]. Te authors in [78] provided a comprehensive review of the recent applications of big data in the context of railway engineering and transportation, which has covered three areas of railway transportation, i.e., operations (including train delay analysis and prediction, passenger route choice, passenger demand forecasting, train positioning and confict detection, and disruption management), maintenance and safety, including the level of big data analytics (i.e., descriptive, predictive, and prescriptive), types of big data models and a variety of big data techniques (i.e., association, clustering, simulation, image processing, process mining, statistical analysis, semantic analysis, optimization, and prediction). In order to understand the patterns of train delays and to predict train delay time, the authors in [79] analyzed a three-month dataset of weather, train delay, and train schedule records and developed a machine-learning model to predict the train delay time at each station. Focusing on delay distribution, delay propagation, and timetable rescheduling, the authors in [8] explored the datadriven methods on the train dispatching problem, which include statistical methods (SM), graphical models (GM), and machine learning (ML). Also, they concluded that machine learning methods are the most promising datadriven models for innovative train dispatching solutions with rich data obtained from practical train operations. For addressing the data-driven train dispatching issues, the involved data cover train position data, arrival and departure time, train delay records, train actual timetable, train occupation data, freight data, train operation data, train timetable, etc. Using and implementing big data analytics in railway industry have been one of the key research points [80,81]. Te information about every train movement, i.e., every train arrival and departure timestamp at "checkpoints (e.g., a station and a switch)" monitored by signaling systems, plays a key role in the rail trafc management systems. In [82], datasets composed by train movement records have been used as fundamental data sources for addressing the problem of building a dynamic data-driven train delay prediction system, by exploiting the state-of-the-art powerful tools and techniques in the area of time varying big data analysis. Compared with traditional trafc simulation [83], big data analytics can be taken as a subarea of data mining [7], which is the process of extracting valuable information and identifying patterns from large data sets. As massive amount of data can be generated from sensors and need to be further processed and analyzed, big data analytics is useful in the railway system.

Paper Reviews by Digital Twins.
Prefguring digital twins (DT), the authors in [84] presented some of the various digital modelling activities, including formal methods, for railway design, development, validation, qualifcation, and exploitation. Te authors in [83] discriminated the diference and relationship between conventional trafc simulation and digital twin (DT) in terms of features, functions, input data, modelling, and interaction, and they proposed three-layer technical architecture for DT in intelligent transportation, i.e., data access layer (lowest level), computational simulation layer (middle level), and application management layer (highest level). Te review in [85] showed that most of the digital twin-related publications focused on the railway subdomain maintenance, inspection, and resilience, most of which applied machine learning algorithms and techniques in digital twin to predict failures, detect faults, make automated decisions, supervise train movements, provide information on passenger behavior onboard trains, and monitor health status of railway systems.
With regard to the complex nature of Cyber-Physical Systems (CPS), the authors in [86] adopted an intelligent agent-based approach to deal with the complexity and the challenge that are encountered while building a digital twin for CPS by programming intelligent agents, i.e., agent-based digital twin. Te authors in [87] developed a mobility digital twin (MDT) framework for connected vehicles, which is an artifcial intelligence (AI)-based data-driven cloud-edgedevice framework for mobility services, and consists of three building blocks in the physical space (i.e., human, vehicle, and trafc) and their associated digital twins in the digital space (i.e., human digital twin, vehicle digital twin, and trafc digital twin).

Synthesis of Intersection between AI-Related Disciplines and Rail TPM.
Referring with the AI taxonomy to the rail sector [2,6], AI-related disciplines cover big data analytics, digital twin, and augmented reality. It is believed that datadriven decisions are practical, remarkable, and reasonable. Big data analytics (BDA) has increasingly attracted a strong attention of researchers, analysts, and practitioners in railway industry. Digital twin is a virtual and data-driven representation of the characteristics and behaviors of any real-world object DTs with embedded intelligence, e.g., physical asset, process, or system. A cognitive digital twin enabling technology AI is regarded as the most coupled technology for smart railways, with the potential of learning and adapting to variant situations. Te typical intersections between AI-related disciplines and rail TPM are summarized in Table 6.

Summary and Discussions
Most of the existing publications focused on timetabling, scheduling, rescheduling, delay [88], neglecting capacity more or less in terms of capacity calculation, capacity planning, capacity allocation, capacity expansion, capacity  AI is taken as an enabling technology to achieve the smart railway operation planning and management. In the future, it is possible to partially shift the dispatching activities to the customers with AI assistance for smart operation, e.g., decisions about the train connection maintaining [89]. Followup research should focus more on adopting AI in rail TPM (e.g., capacity assessment [90] in the context of the next generation of signaling system i.e., virtual coupling). As can be seen, AI is a set of methodology system, rather than one unique method. Te application scope of AI in rail TPM could be in greater width and depth. Particularly, using RL in rail TPM is a promising direction, as RL is a kind of approximate dynamic programming and can match closely with the attributes of most of the rail TPM problems for intelligent decision-making. On the other hand, as what has been analyzed in [7], hybrid models, e.g., combining mathematical-based models with machine learning strategies, and AI-aided optimization approach, should be considered more and more. With limited data to address the target task, transfer learning can enable AI in railways [91], by reusing (fne-tuning/freezing) the knowledge resulting from a given source task. Te biggest obstacles for AI applied in railway TPM lie in selecting the most applicable AI techniques and designing a suitable model to represent the operational scenarios and the network. Based on the practical steps for implementing AI models in railway environments [45], i.e., the four stages for the creation of an AI-based railway model, the practical roadmap for application of AI in rail TPM is proposed as Figure 2. AI-enabled techniques provide a multidisciplinary research roadmap, which can facilitate to utilize the Define application scenario Strategic level of rail TPM, e.g., demand analysis, capacity (infrastructure) management, etc.
Operational level of rail TPM, e.g., real-time management, scheduling, rescheduling, etc.  Figure 2: Practical roadmap for application of AI in rail TPM.
Discrete Dynamics in Nature and Society multisource data, learn from examples, and improve with experience. For defnition of application scenarios, much attention should be paid to tackling the large-scale real-life problems and the intelligent networking of rail mobility (e.g., platoon planning of train trafc under virtual coupling) in TPM. For problem formulation, most of the objectives are identical with the traditional methods, i.e., to improve the efciency, efectiveness, capacity, resilience, fexibility, and other positive performances of the demand-aware railway system. However, more constraints closer to the railway operation practice could be taken into account, due to the AI capability of model optimization, especially the optimization of larger and more complex combinatorial models, and algorithm improvement for NP-hard problems. After the defnition of the solution mechanism, i.e., the centralized mechanism or the decentralized mechanism, scenariospecifc AI model/algorithm could be investigated from the AI methodological/technical system, e.g., the promising application of multiagent systems and reinforcement learning. It is necessary to conduct a scenario analysis for the achieved solution, so as to ensure the reliability and trustworthy for intelligent decision-making.

Conclusion
Tis survey examined 95 articles to provide a comprehensive picture on the state-of-the-art of AI applied in railway TPM from the perspective of AI research felds, AI techniques, AI applications, and AI-related disciplines. Also, focused area in railway subdomain of TPM included rescheduling, delay management (delay analysis/prediction), timetabling, railway capacity, confict prediction, train trajectory, train routing, railway disruptions, train shunting, and stop planning. As can be seen, AI has been applied to solve a series of rail TPM problems, e.g., NP-hard train timetable scheduling problem. Particularly, we have developed a practical roadmap for application of AI in rail TPM. For AI applied in railway transportation planning and management, scenario analysis, or a combination of scenario analysis and sensitivity analysis, rather than mere sensitivity analysis should be implemented. All these techniques will pave the way to the development of the new Railway 4.0, which can help tackle the challenges associated to modern smart railways. AI is indispensable for the virtualization and automation of network functions of future smart railways imbedding intelligence [92]. It is promising to use AI as alternative algorithms in fnding good (near-optimal) solutions in practical time for rail TPM problems. Rails [93] have developed the methodological and technological concepts for stimulating further innovation in railways, providing new research directions to improve reliability, maintainability, safety, security, and performance. Before AI can be fully implemented in practical railway operations, the concepts of explainable AI (XAI) and reliable, efcient, and trustworthy AI need to be stressed further. After all, the fnal key decision-making should be left to humans (human-inthe-loop and human-on-the-loop) [94]. Also, the feedback scenario analysis is necessary for validation test of solutions as illustrated in Figure 2. While many AI applications and adaptations have been reported as aforementioned, there does not exist a single and universal rule for AI system design to handle all railway transportation problems. To ensure successful AI use, one needs to have an in-depth understanding of the nature of specifc railway TPM problem investigated as well as AI per se, e.g., federated learning paradigm for NP-hard rail TPM problem [95]. Tough AI is still in its infancy level in the railway sector, it is recognized that AI can play a key role in improving railway performance, by leveraging the application of AI algorithms, methods, and techniques.

Data Availability
Te data used to support this study are included in this paper.

Conflicts of Interest
Te authors declare that there are no conficts of interest.