A Virtual Reality Platform for Safety Training in Coal Mines with AI and Cloud Computing

,


Introduction
Coal mines, with dangerous working environments and complex production systems, have a strong demand for virtual reality (VR).VR technology has its unique advantages in several fields related to coal mining, such as miner safety training and emergency rescue drilling, besides drifting-producing process simulation, disaster scenario simulation, and mechanical operation training [1].e fifthgeneration (5G) cellular, artificial intelligence (AI), data analytics, Internet of ings (IoT), and cloud computing will provide the information infrastructure required for the intelligent mine construction in the next three to five years.With these emerging technologies, the next-generation VR systems for underground coal mining will be achieved, which will improve the professional adaptation process and occupational health and safety situation, especially for the young miners [2].
e United States, Australia, and the UK have used VR as a training environment for mining simulation, accident reconstruction and investigation, education, and safety training in the past two decades.e UK, in particular, has a long history of developing and using VR technology in coal mine safety training.e VR products, such as SafeVR and Vroom, are very famous for open-pit truck operator training [3].e researchers of the National Institute for Occupational Safety and Health (NIOSH) explored how the mining industry could effectively use "serious games" for fire-escape training.Spokane Research Laboratory (SRL) developed fire-escape training software for a mine safety training course.e software allowed four trainees to work together in a virtual environment through a computer network.e trainees responded favorably and showed a marked improvement in locating the proper evacuation routes on subsequent trials [4].e University of New South Wales developed iCinema (an advanced visualization and interaction environment), which included a 360 °immersive theater environment with 18 modules for improving teaching and learning activities in mining engineering [5].
e University of Queensland Experimental Mine (UQEM) laboratory has been used by students for experiments since 1956.
is institution developed several VR applications, including VR-based teaching modules, drill rig, Instron UCS rock testing simulations, and VR-based ventilation simulations [6].
In China, universities, large mining companies, and local governments utilize the VR hardware and software for safety and emergency rescue training [7].
ose VR systems provide the desktop versions of character "Q and A" sessions.e VR hardware is composed of a projection-based panoramic display system, the infrared tracking stereoscopic glasses, the VR headset display, a pad, and the other devices.Furthermore, the 3D visualization system is very popular in coal mines with the functions of the 3D geological model, ventilation simulation, real-time data monitoring, and emergency response.
In recent ten years, many coal mine enterprises established underground industrial networks and tried to apply cloud computing, big data, Internet of ings, and even AI technology into mining, ventilation, safety, and education projects.ere has been a shift from traditional mining to intelligent mining.A three-tier ITframework with automatic control system, production execution system, and enterprise management system is established.Based on cloud computing techniques, big data related to underground coal mining can be analyzed quickly, and critical data associated with user requirements can be extracted precisely.With the established platform, the danger-spotting and subsequent decision-making capabilities of users can be greatly enhanced to ensure underground mining safety [8].
Although VR is an important part of intelligent mine construction, there are still some problems with these applications that need to be tackled.Firstly, most of the existing VR-based training systems lack a good learning experience.Although these systems adopt teaching and learning functions via a single-player mode and an ask-answer pattern, there is no complex AI man-machine interaction.VR training systems always focus on professional computation or disaster 3D simulation or text and voice presentation.e learning process needs to be improved.Furthermore, the advantages of immersion, interaction, and imagination of VR technology have not been fully utilized yet [9].Secondly, traditional VR-based training systems are standalone or client/server versions, without direct browser access.Cloud rendering will become the mainstream to support various light terminals, such as PCs, mobile phones, tablets, and headsets.
ese technologies will enable real-time transmission and enhance cyber-physical and social experiences in a fully immersive environment [10].Hence, VR training software should adopt gamification and online learning, which have become a tendency in education.

Methodology
To address the abovementioned problems and train the coal miners, a VR-based system using the Unreal Engine is designed and developed in this paper.Contrary to the traditional standalone or client/server VR software, this system adopts browser/client architecture and uses cloud rendering and Game AI technologies.It aims to enhance users' learning experience, strengthen their memory, and improve safety awareness.e system uses the Unreal Engine from Epic Games, which includes a rendering engine (renderer), Blueprint scripting language, animation, artificial intelligence (AI), networking, terrain, streaming, and scene graphs.Figure 1 illustrates the system interface, which has eight modules as follows: (i) Geological environments, such as procedure of coal formation, geology condition, coal seam, and geology structures (ii) Geographical environment and surface plants (iii) Underground roadway arrangement (iv) Mining area and working face design (v) Coal mine production system, which includes ventilation, transportation, belt, lifting, power supply, water supply, and drainage (vi) Mining processes, including coal mine blasting and longwall mining processes (vii) Coal mine disaster (gas leakages, roof falls, and fire events) simulation (viii) Underground safety monitoring system (methane, wind speed, temperature, and carbon monoxide), self-rescue, and emergency response In this paper, we use three technologies, namely, cloud rendering architecture, 3D modeling, and AI behavior design to achieve multiuser online operation and improve the interaction and immersion of VR devices.
e browser/client architecture of cloud rendering means that each user connects to the server and accesses 3D streaming over the cloud.e cloud servers handle all events and the state of the virtual world and transfer the VR render streaming to the client.ese servers can be virtual servers or server clusters.ey take full advantage of virtually infinite computing power to create, capture, and compress highresolution rendered streaming, which is sent to the client with low performance and limited computing power [11].
Compared with other popular Web3D technologies, such as HTML5 and WebGL, cloud rendering has incomparable advantages mainly because it has no special or expensive requirements on the client side.It allows users to choose different platforms, including PCs, laptops, tablets, and smartphones.Furthermore, since Wi-Fi 6 and 5G will provide sufficient network bandwidth, online cloud rendering will deliver adequate computing and rendering power.With the advancements in the Internet infrastructure, the users can interact with VR programs through the network on the client with high-quality videos and lowresponse delays [12].

2
Discrete Dynamics in Nature and Society In this B/S architecture of the cloud rendering system, the system is divided into three parts: portal server, render server, and client, as shown in Figure 2.
e portal server is responsible for user login, finding the appropriate server according to the user's choice, and controlling the server to start the virtual machine, i.e., render server.
e client and the render server establish a connection when the portal server notifies the client with the address of the virtual machine.e user can run and interact with the 3D program in the remote server through the network.
e render server is responsible for processing the input events of the client sourced by mouse or keyboard, as well as translating and sending them to the interaction module of the 3D program for further parsing.e 3D scene images are captured, encoded, and sent to the client as a video streaming.
e render server program includes three modules, namely, basic function, Game AI, and advanced function.e VR basic function module provides 3D scene management, sound, particle effect, physical simulation, and other basic functions.e Game AI module uses a scripting language to design a behavior tree for virtual miners.e VR advanced module builds complex computation models and simulates disasters, such as fire numerical simulation and escaping route computation.
e client is responsible for displaying the received streaming of the 3D scene and sending user inputs, such as keyboard and mouse events, to the server.
We use 3ds Max to build 3D scenes of the underground and the surface.en, we import the 3D models into the Unreal Engine editor.We adopt a series of material tools to apply 3D models, such as a normal map, diffusion map, metal map, and texture map.ese materials are like the "paint" applied to the model.en, we set the lights to illuminate our scenes.From the computer-aided design (CAD) or geographic information system (GIS) files, we automatically build 3D geological and underground laneway models.We use the Blueprint scripting language to program the characters' animation, the AI behavior tree, and the user interface (UI).
AI in games or "Game AI" is a module of a 3D engine.Traditional AI typically demonstrates some of the following behaviors associated with human intelligence: planning, learning, reasoning, problem-solving, knowledge representation, perception, motion, and manipulation.e emerging Game AI is becoming a narrow branch differentiated from the traditional AI.Game AI refers to the virtual man, also called non-player characters (NPCs), controlled by the computer, which simulates the intelligent behaviors of human beings or other creatures, providing users with a reasonable challenge to overcome [13].
Figure 3 shows the behaviors of the two NPCs we designed.ese NPCs can simulate the behaviors of miners and guide young miners and/or students to understand the whole mining production, experience the underground scene, and improve the relationship between users and computer programs during the training, conversation, or other tasks.Besides, these AI characters are designed to automatically find the local navigation grid path.
e first NPC is a task-oriented character.When the user meets it somewhere on the ground, he will provide general knowledge about the coal mines and guide the user to descend a well from a cage and visit the underground work sites, such as longwalls or heading faces.Once the user completes the corresponding task and provides feedback, the NPC releases the next task.ose tasks are performed before entering the underground mine, preparing the miners to look for the entrance, enter the cage, and visit the underground working face.e second NPC is a disaster-oriented character which guides the user to experience typical disasters, such as gas explosions, roof falls, and fire accidents.
To realize the NPC's behavior, the blueprint is used to build a behavior tree graph, which adds and connects a series of nodes.A behavior tree, mainly composed of a leaf node and composite node, is essentially an acyclic graph [14,15].We use composite tasks and design nodes of selectors, sequences, parallels, and decorators.Figure 4 shows an example of the second NPC's behavior tree, where selector, sequence, and parallel are used to trigger accidents.A selector is a branch task that runs the behavior of each of its children.It will immediately return a success status code when one of its children runs successfully.Once one of the child nodes is successful, it returns to the selector, and the rest of the nodes are not executed.A sequence executes the node task sequentially instead.A sequence node tries to execute each child in turn until all children have reported that they have executed successfully.Once a task fails, the sequence will return to the upper node.In this case, the remaining tasks are not executed.A parallel is a composite task that handles concurrent behaviors.It is a special branch task that runs all children during execution time.e behavior tree allows parallel node tasks, which might be state machines.Multiple state machines can be executed in parallel.For example, when the second NPC's behavior tree is triggered, it first judges whether it is dangerous or not and then runs the user and sends alerts.Following these, it gives the on-site emergency solution by a series of animation actions.
e sequence executes the disaster animation in turn and tests whether the user's reaction is correct.

System Implementation.
e cloud rendering cluster is composed of two virtual rendering servers with a configuration of 4-core 8G M60 CPU, 32 GB RAM, and 40 Mbps BGP bandwidth.
e experiments have shown that the hardware can support up to seven players at the same time.A client PC should have at least Intel i7 CPU and 16 GB RAM.
e system has been tested by undergraduate students in the VR Teaching and Experiment Laboratory, China University of Mining & Technology, Beijing.e laboratory also has a 120-degree 2-channel VR projector and an infrared laser tracking system as shown in Figures 5 and 6, respectively.
e system can run either on the immersive VR hardware or the PC headset.

AI Guidance.
e virtual roaming function of the NPC will help the user to walk around buildings, enter the mine through a cage, and visit the longwall mining face or the blasting place.It will also introduce the ventilation system, transportation system, water supply, drainage system, and power supply system. is function enables the new miners or students to have an overall understanding of the mine.
When a user is wandering in the roadway, the accident guidance function of the NPC will start a dialogue with them to detect their behavior and provide information on illegal operations.e system will then trigger disaster animation depending on the location, and the NPC will demonstrate the escape routes for self-rescue.

Disaster Simulation.
In our system, mining accidents are limited to gas explosions, roof falls, and fire events.e disaster simulation module integrates various 3D animation, audio, particle effect, and physical-mechanical models of the 3D engine.e 3D model is to establish the models of mines, terrain, tunnels, and NPCs.e audio function is to create sounds as realistic as possible considering the dimension of the source, such as footsteps when walking on the grass and the roadway, wheezing when running, and the mechanical sound of an opening cage.By analyzing the disaster/accident reasons, features, and occurrence conditions, a disaster simulation knowledge library is constructed.
is library contains a brief description of prevention and control measures, emergency plans, site disposal programs, mitigation, and postdisaster evaluation.It is combined with system guidance to show the potential consequences of disasters.

Gas and Coal Dust Explosion.
is simulation is to show the boundary conditions of gas and coal dust explosion as illustrated in Figure 7.It helps users to perceive the disaster from the vision, auditory, and other aspects and understand the movement and destruction of the wind that is mixed with dust.Besides, some emergency measures, such as collaborative gas extraction, are fully displayed.

Roof Fall and Wall Collapse.
rough this simulation, the user will understand the main factors of roof fall and wall collapse as shown in Figure 8.It can simulate various rock burst and ground pressure disasters, introducing the form of disasters, their causes, and the potential methods of disaster prevention and control.

Fire Event.
e user will see the main forms of fires, such as the external fires that are mostly caused by open flames, blasting, and short circuits, as well as the internal ones caused by the combustion of coal or other flammable substances due to their oxidation or burn.e VR system can simulate the conditions of various fires, illustrate their severity, and therefore inform decision making for disaster prevention.e system also shows the ways of treatment, including the closed grouting and nitrogen injection methods.

6
Discrete Dynamics in Nature and Society the complex underground coal mine environment and provide the function of login, roaming, virtual simulation, and operation evaluation.However, due to intellectual property rights, only a small part of the functions is made accessible to the public through the website.To date (August 1, 2020), the system has been visited 20, 853 times and the training system drew perfect user reviews (5.0/5.0) on the National Virtual Simulation Experiment Teaching Project Sharing Platform [16].According to the experiment score data, 1355 people have finished the experiment and the excellent score rate was 46.0%.e students commented that the system could give them a more intuitive of the underground environment and disasters without visiting the real underground coal mines.Despite satisfactory performance and remarkable feedback, the current cloud render hardware needs to be upgraded to a larger cloud rendering cluster to provide a quicker response and hence a better experience.Furthermore, a performance test is needed to estimate the amount of concurrency and runtime efficiency.e system functions also need more research, where a more detailed behavior tree design can improve the user experience.Combined with the numerical computation, the system can be turned into a testbed to support the ventilation design, emergency drilling, and other systems and fields.We expect more innovative applications of VR-based miner training and disaster drilling in the near future.

Conclusions
VR, as a part of intelligence mine, provides the new tool for work procedure management, occupational health and disaster simulation, and visualization.e VR system provides macroscopic and microscopic underground 3D scenes for coal mine and displays information such as personnel positioning, safety monitoring, and equipment management.Combined with cloud rendering, it can achieve more realistic and more detailed 3D visualization effects.Considering the advantages of cloud rendering and AI, this paper designs a cloud-based VR system for mine safety training.AI guidance is adopted to roam underground and simulate accidents, such as gas explosions, roof falls, and fire events.We believe that the results of this study will provide a useful tool for coal mine safety training, avoiding the potential hazards of the extreme underground environment.

Figure 1 :
Figure 1: e interface of the virtual reality system for underground coal mine.

Figure 2 :
Figure 2: e architecture of the cloud rendering system.

Figure 3 :
Figure 3: VR mission design of Game AI NPCs.

Figure 4 :
Figure 4: e behavior tree of the disaster-oriented NPC.

Figure 5 :
Figure 5: e professor introducing the system to the visitors on the 120-degree 2-channel VR projector.

Figure 6 :
Figure 6: e overview of VR Teaching and Experiment Laboratory.
VR system uses cloud rendering and AI behavior trees to enhance training performance.It can realistically simulate e