Differences in Drivers ’ Glance Behavior and Lateral Control Ability during Full-Touch Interaction Mode and Conventional Interaction Mode: A Case Study of Road Experiments

In recent years, the full-touch human – machine interaction (HMI) mode has gained popularity in the automotive market. However, little research has been conducted on how this interaction mode a ﬀ ects drivers ’ glance behavior and lateral control ability. In this study, we evaluated the visual engagement and driving performance of 30 participants while driving two vehicles equipped with either the full-touch interaction mode (FTIM) or the conventional interaction mode (CIM) provided by the original equipment manufacturer (OEM). We found that both air conditioning – related tasks required more visual engagement, longer task completion time, and worse lateral vehicle control under FTIM. Furthermore, the gray correlation analysis demonstrated that FTIM exhibited slightly di ﬀ erent disadvantages in the two secondary tasks. In the temperature adjustment task, the correlations of glance behavior and lateral control ability between the two interactive modes were 0.688 and 0.680, respectively. In the air ﬂ ow adjustment task, the correlations of glance behavior and lateral control ability between the two interactive modes were 0.659 and 0.668, respectively. In addition, this study revealed that driving speed had signi ﬁ cant e ﬀ ects on glance behavior and lateral driving performance in both interaction modes. As speed increased, self-adjusting glance behavior was evident in performing the secondary task; however, this behavior could not compensate for the deterioration in lateral driving performance caused by the increased speed. The ﬁ ndings will help improve drivers ’ perception of FTIM and provide theoretical guidance for the design development of


Introduction
Human-machine interface (HMI) is an important carrier for human-vehicle information interaction. Using HMI, drivers can interact with the vehicle's air conditioning system, infotainment system, and driver assistance system. With the development of the Internet, the full-touch HMI mode has been sweeping the automotive industry and gaining popularity among younger drivers. The full-touch HMI mode refers to the interactive mode that has a large HMI display size and no physical buttons, and all operations are concentrated on the HMI display (e.g., Tesla Model S, Model 3). FTIM allowed completing tasks with the HMI display, whereas conventional interactive mode (CIM) completed tasks with buttons or knobs. In practical terms, full-touch interactive mode (FTIM) offers some advantages over CIM, such as the integration of full-touch HMI with in-vehicle entertainment systems to create more intelligent ways to engage. FTIM, however, also has inherent flaws. First, the HMI display cannot give the driver the same sense of feedback as conventional physical buttons or knobs, which is not conducive to operation solely by manual memory. Second, the completion of many tasks in touch screens requires entry into multiple levels of submenus, which may increase the complexity of the driver's task completion. In addition, the driver may unconsciously devote more attention to the display when performing secondary tasks on the HMI, considering the impact of the brightness on the distribution of attention during driving [1]. Therefore, it is worth exploring whether FTIM increases the visual demand of secondary tasks compared with the CIM.
Driving is a complex task that integrates the cognitive, visual, and manual resources of a driver. According to the multiresource model theory [2,3], conflicts emerge in resource allocation when a driver performs multiple tasks that occupy the same dimensional resources at the same time, which affects task completion performance. The visual channel is the main resource channel occupied by the driving task. Thus, if a driver simultaneously performs a secondary task that occupies more visual resources, it competes with the driving task for the visual channel, thus affecting driving safety. A traffic accident occurred in 2020 with a Tesla Model 3 embedded with a full-touch HMI system; a German owner was involved in a traffic accident while performing a wiper operation in a secondary menu on the HMI display, because the vehicle was not equipped with a physical toggle to adjust wiper frequency. This example demonstrates the necessity of exploring whether performing typical secondary tasks in full-touch HMI mode impair the driver's behavior.

Research on the Impact of Secondary Tasks on Resource
Demands and Driving Performance. Current researchers have classified three types of distractions: cognitive distractions, visual distractions, and motion distractions [4][5][6]. Whether in full-touch interaction mode or conventional button-based interaction mode, the driver is primarily involved in visual-manual distraction when performing HMI secondary tasks [7]. Of all distraction types, however, visual-manual distraction poses the greatest driving risk, and the driver's lane-keeping ability is subject to the additive effect of visual and manual distractions [8]. Therefore, the objective evaluation metrics of in-vehicle interaction patterns mainly include multiresource demand and driving performance. Multiresource demands mainly refer to visual demands and motion demands. Visual demand can be characterized by such metrics as number of glance (NoG), percentage of glance (PoG), mean single glance duration (MSGD), and percentage of glance exceeding 2.0 s [9][10][11]. In order to limit the visual demands on the driver required by the HMI system, NHTSA issued manual-distraction guidelines for in-vehicle electronic devices in 2013 [12]. The guidelines include the following four items: when the driver performs in-vehicle secondary tasks, (1) the total glance duration should not exceed 12 s. (2) The MSGD should be less than or equal to 2 s. (3) The percentage of glance exceeding 2 s should not exceed the total number of in-vehicle gaze. (4) The percentage of gaze exceeding 2 s should not exceed 15% of the total number of glances in the vehicle. While the NHTSA guidelines provide a solid starting point for the field of distraction testing, the guidelines have also been subject to many criticisms. Firstly, the guidance is based on simulator tests and lacks ecological validity [13]. Secondly, the test does not take into account the visual demands of different driving scenarios and their effects on glance [14]. Due to the limitations of the NHTSA guidelines and the fact that they are not mandatory, they are not widely used among original equipment manufacturers. However, many scholars have explored the visual demands of the secondary task using the above NHTSA glance parameters.
Motion demands include task completion time (TCT) as well as the duration of time hands are off the steering wheel. A large number of driving simulator-based experiments have been conducted to show that as a secondary task takes up more visual resources and as TCT increases, lateral driving performance will be worse, and the risk of lane departure will increase significantly [15][16][17].
The characterization of driving behavior is of great value in multiple research fields such as driving risk assessment, driving style recognition, and autonomous vehicle development [18]. Driver behavior tends to be assessed in terms of longitudinal and lateral driving performance. In the current research on the effects of driver distraction on driving behavior, the most studied topic is the longitudinal driving performance of drivers. Researchers have not only investigated the detrimental effects of driving distractions on longitudinal driving performance [19][20][21] but also modeled drivers' longitudinal driving behavior [22,23]. For example, to make the ADAS system more aware of driver behavior, Zou et al. [23] predicted vehicle acceleration based on a machine learning model that takes full account of driver heterogeneity, making the model more application-oriented. Comparatively, there are fewer studies on lateral driving performance in the current research, especially those exploring different interaction modes on lateral driving performance on different roads. Lateral driving performance refers to the driver's ability to control the vehicle laterally and can be measured by steering wheel metrics and vehicle motion metrics. Standard deviation of steering wheel angle (SDSWA) has a significant effect on vehicle trajectory, and a larger steering wheel standard deviation is detrimental to vehicle lane-keeping [24,25]. In previous studies, standard deviation of lateral position (SDLP) was found to be significantly different during secondary tasks compared with baseline driving; thus, lane lateral position standard deviation has been widely used to measure lane-keeping ability [26,27]. Time to lane crossing (TLC), which is widely used in the field of lane departure warning, refers to the minimum time necessary to reach any lane line position if the vehicle continues to operate in its current state. TLC is the most direct parameter used to measure the risk of vehicle departure [28,29].
1.2. Impact of Different Interaction Modes. In recent years, the FTIM design has gradually replaced the conventional physical button interaction mode as the primary vehicle interaction mode design. Based on FTIM, researchers have explored the effects of voice interaction mode (as a complement to FTIM), aerial gestures, touch gestures, and touch screen position on resource requirements and driving performance but have ignored the comparative study of CIM versus FTIM. It has been suggested that even the combination of voice and touchscreen still takes longer to complete a task than CIM [30]. Subsequently, Zeng [31] studied the impact of conventional button-based interaction modes, voice interaction modes, and touch screen interaction modes on driving safety from the perspective of driving safety and found that CIM had the least impact on driving safety, and that the touchscreen interaction mode had the greatest impact on driving safety. Moreover, a number of studies have noted that touchscreen interaction modes without nonvisual feedback have worse task performance than traditional physical button interaction modes [32,33].
These studies have considered only the effects of different interaction modes on driving performance in a single driving environment and have not considered the effects on glance behavior or the effects of speed. In addition, the majority of current investigations on secondary tasks have been conducted on driving simulators, which can effectively control variables as well as ensure participant safety. It is difficult, however, for drivers to drive a simulator with the immersion of a real traffic environment and thus exhibit visual behaviors [11] as well as driving performances [34,35] that are not fully consistent with real driving performance. Furthermore, most of the current research on the impact of FTIM on driving performance is not based on product-level HMI, which lacks ecological validity [13].

Objectives and Approach.
The main objective of this study was to qualitatively and quantitatively investigate the differences in drivers' glance behavior and lateral control ability when performing interactive tasks using CIM and FTIM, as well as the effect of driving speed. To better reflect the realistic glance and driving behavior of the driver while performing the secondary task, we selected vehicles embedded with product-level HMIs provided by original vehicle equipment manufacturers for real-world road testing in this study.
Air conditioning-related tasks, wiper-related tasks, music volume adjustment tasks, etc. are all very frequent interactive tasks in the driver's daily driving process. These tasks are usually performed by knobs or buttons in the CIM, while in the FTIM, they are performed by the HMI display. Therefore, it is necessary to investigate the drivers' visual behavior and the lateral control behavior when performing these interaction tasks in the two interactive modes. The air conditioning task is not only a task directly related to vehicle comfort, but in many cases, it is also a task related to driving safety. For example, in winter, due to the difference in temperature between the inside and outside of the vehicle, fog easily appears on the front windshield, making it impossible for the driver to read the road and thus affecting driving safety. If the driver adjusts air volume and blow towards the front windshield, the problem can be solved. In this study, we chose the air conditioning-related tasks (adjusting the air conditioning temperature and adjusting the air volume), which are the most frequently and classically operated, to investigate the differences between the two different interaction modes.
The participants drove the two test vehicles at different driving speeds while following the instructions of the experimenter to complete the secondary tasks related to air conditioning. We collected the glance behavior parameters, operating behavior parameters, and vehicle lateral motion parameters during the test.

Method
2.1. Participants. Thirty volunteers, including 12 professional drivers (M = 45:8 years, SD = 3:94 years) and 18 nonprofessional drivers (M = 34:3 years, SD = 2:28 years), were recruited through online advertising in this experiment. Professional drivers had more than 10 years of driving experience and more than 1,000,000 km of driving distance, and nonprofessional drivers had 3-5 years of driving experience and 100,000-300,000 km of driving distance. All drivers had a legal driving license and no traffic accidents in the past three years. These drivers all had normal or corrected normal vision and were in good health. In addition, none of them had any experience driving the two test vehicles. Each participant was given 500 RMB as compensation for their time after the experiment. This experiment obtained the approval of the ethics committee of the University.

Vehicle Equipment.
Our market research on the interaction design of air conditioning-related tasks revealed a high level of driver acceptance of the 2020 Audi A4L, and this car is embedded with the CIM that combines a touch screen with physical buttons. Meanwhile, we surveyed several of the top-selling vehicles in China with full-touch HMI design mode and found that air conditioning-related secondary tasks are in a two-step menu. Considering their high market share of 2020 BYD Qin in China, the 2020 Audi A4L and 2020 BYD Qin were selected as the test vehicles to explore the changes of FTIM on driver glance behavior and vehicle control behavior. The Audi A4L embedded the CIM, and the BYD Qin embedded the FTIM. The HMIs of the two vehicles were shown in Figure 1. When participants drove the Audi, the air conditioning-related operations were performed in the conventional mode (knob or button), and in the BYD, the operations were performed through the HMI display.
Each experimental vehicle was equipped with three video cameras (installation location shown in Figure 2) and one vehicle data collector. Video cameras (MiVue 786, sampling frequency: 30 HZ) were intended to capture vehicle lateral position, participant's interaction with the vehicle steering wheel and the center console, and the eye movement. The vehicle data collector (Langren, H6 Pro) had an output frequency of 4 HZ, which met the experimental needs. The data collector was connected to the vehicle OBD interface and could capture the CAN bus data of the vehicle for speed and steering wheel parameters. Before the formal test, the lateral position of the vehicle was calibrated. All the instruments in the experiment could record the commands given by the experimenter. Figure 3, a low-speed road and a highway were chosen as the test routes to investigate the effects of CIM and FTIM on driver glance behavior and lateral control ability at different speeds. Road 1 (lowspeed road) was 16.76 km long and had a speed limit of 70 km/h. Road 2 (highway) was 20.58 km long and had a speed limit of 120 km/h. Both test roads were eight lanes in both directions, with a lane width of 3.75 m and a central 3 Journal of Advanced Transportation barrier separating the opposite lanes. The trials at 40 km/h and 60 km/h were conducted on Road 1, and the trials at 80 km/h and 100 km/h were conducted on Road 2 by the participant driving the experimental vehicle. All trials were conducted on straight sections of the test road.

Interactive Tasks.
Participants were required to interact with the BYD vehicle full-touch HMI display (FTIM) and the relevant buttons (knob) of the Audi vehicle (CIM) during the trial to perform the air conditioning secondary tasks, as shown in Figure 4 and Table 1. In task 1, when driving the Audi, the participants needed to press a button to turn on the temperature adjustment function (step 1), and then turn the knob to adjust the temperature (step 2). For the BYD, they needed to click on a specific location on the HMI display to enter the temperature adjustment menu (step 1), and then click "+" to adjust the temperature to the required temperature (step 2). In task 2, participants driving the Audi needed only to press the relevant button twice to complete task 2 (step 1). Conversely, in the BYD, they had to click on the relevant position on the HMI display (step 1) and enter the secondary menu (step 2) to complete task 2. Secondary tasks selected in the trial were the most frequent secondary tasks in the driver's usual driving behavior.
2.5. Experimental Design and Procedure. Before the formal experiment, a basic information form and an informed consent form were filled out. The participants were then familiarized with the test vehicle and the operation of the test, and trials started after they were able to drive the vehicle and complete the secondary tasks with ease.
During the experiment, each participant was required to drive two test vehicles on actual low-speed and highway to perform the test operations. Each secondary task was performed five times under the same conditions, and each par-ticipant performed 40 trials in total. The order of each condition was counterbalanced. Participants were asked to perform the secondary tasks according to their own driving habits and to follow the instructions given by the experimenters on the basis of ensuring driving safety and maintaining driving speed.
Two experimenters followed the test vehicle at all times: one person sat in the front passenger seat to monitor traffic conditions and immediately alert the participant to any driving risk; and a second person sat in the rear seat to give the driver secondary tasks.
2.6. Data Processing. During the data preprocessing, the time was calibrated by the sound recorded by the device. The first time point at which the participant's eyes or hands moved after the experimenter gave the command was the start time. The later time point at which both movements ended was the end time.
Combined with the calibration of the lane lines before the trial, a computer image processing algorithm was used to identify the distance between the left front wheel of the vehicle and the left lane line during the trial, as shown in Figure 5. The accuracy of the obtained data was 1 cm, which satisfied the research requirements.
2.7. Data Analysis. In this study, each participant was asked to perform two secondary tasks under different conditions. Considering the existence of nonindependence between sampled data, repeated measures analysis of variance (ANOVA) was used to investigate the effects of interaction mode and speed on glance behavior as well as lateral control behavior. During data testing, the test level was set to 0.05. Once the Mauchly's spherical assumption was violated, we applied the Huynh-Feldt correction.
If we found a significant main effect of speed, we employed a linear regression to explore the impact coefficient of vehicle speed on each metric in both interaction modes [36,37].
Gray correlation analysis intends to seek the numerical relationship between the subfactors in the system, and its basic idea is to determine whether the connection is strong based on the similarity of the geometry of the sequence; if the trend of the change of two factors has consistency, that is, the degree of association between the two is high; otherwise, it is low. To quantify the degree of influence of performing secondary tasks in FTIM on the driver's visual behavior and driving performance parameters, we used gray CIM FTIM Figure 1: Two types of interaction modes.  Journal of Advanced Transportation relation analysis to analyze the correlation between the driver's performance of secondary tasks in both interaction modes, as shown in Figure 6.

Results
In this study, we explored the effect of performing secondary tasks with FTIM on task completion time (TCT), driver's glance behavior, and lateral control ability by comparing CIM. In addition, we also explored the effect of speed on these metrics in both interaction modes. Therefore, 2 ðinteraction modeÞ × 4 ðspeedÞ repeated measures ANOVA were conducted on each of the metrics.

Task Completion Time.
The time to complete a secondary task has been used to measure secondary task time demands [38,39]. Secondary task duration has been considered in the field of vehicle secondary task interface Step 1 Step 2 x1 x1 Step 1 x5 Step 2 x5 Step 1 x2 x1 Step 1 x2 Step 2 CIM FTIM Task 2 Figure 4: Operation process of secondary tasks in two interactive modes.

Glance Behavior.
We selected the percentage of glance (PoG), number of glances (NoG), mean single glance duration (MSGD), and percentage of long-duration glances (PoLDG) to the HMI during performing the secondary task as glance behavior metrics. In particular, a long-duration glance was a glance lasting more than 2.0 s on the HMI. A 2 ðinteraction modeÞ × 4 ðspeedÞ repeated measure ANOVA was conducted on each visual metrics.

Percentage of Glance.
PoG refers to the percentage of glance engagement during task completion and can be used to measure the visual demand of the secondary task. Under CIM, drivers are more prone to perform interactive tasks with manual memory and tactile-based feedback. Although FTIM can rely only on visual feedback to complete tasks, the PoG was used as an important metric to measure visual demand during secondary tasks. Figure 8 shows the PoG of the two tasks by interaction mode and speed. PoG for task 1 differed significantly by interaction mode (F ð1, 29Þ = 22:395, P < 0:001) and speed (F ð3, 87Þ = 25:535, P < 0:001). On average, PoG was longer   Journal of Advanced Transportation when performing task 1 with FTIM ðmean = 83:2%, SEM = 1:0%Þ than with CIM ðmean = 75:3%, SEM = 1:6%Þ. As the speed increased, the PoG decreased linearly when performing task 1 (F ð1, 29Þ = 56:447, P < 0:001). We also observed a mode by speed interaction (F ð3, 87Þ = 5:243, P = 0:002).

Gray Correlation Analysis.
We used gray correlation analysis to quantitatively analyze the differences in TCT, visual behavior, and lateral control metrics when performing secondary tasks in two different interaction modes.
When conducting gray correlation analysis, the first step is to determine the reference and comparison series. The data series reflecting the characteristics of the system behavior, called the reference series, and the reference series should be a more desirable comparison standard. The data series composed of factors affecting the system behavior is called the comparison series. According to the qualitative analysis of the two modes under different metrics (given in Sections 3.1-3.3), the visual engagement and lateral driving performance under CIM in both task 1 and task 2 were better. Therefore, we selected the measured metrics under CIM as the reference series and the corresponding metrics under FTIM as the calculated series in the gray correlation analysis. The reference series X CIM k and the comparison series X FTIM k are denoted as follows: where k = fSDLP, TLC, SDSWA, PoG, NoG, MSGD, PoLDG, TCTg and X is a vector consisting of the values of a given indicator for all participants, next normalized for individual participants.
x CIM where i is the participant ID. The absolute difference between the FTIM and CIM indicators is then calculated, and the minimum A and maximum B differences are determined by , i = 1, 2, ::::30: The final correlation coefficients and correlations between FTIM and CIM for the same indicators are calculated by the following formula: The relationship between the metrics in the two modes is shown in Figure 15. We observed that among all of the metrics, MSGD had the highest correlation, and PoG had the lowest correlation under the two modes. Finally, the correlations of TCT in the two modes were 0.693 (task 1) and 0.672 (task 2); the correlations of visual indicators in the two modes were 0.688 (task 1) and 0.659 (task 2), and the correlations of vehicle lateral motion indicators in the two modes were 0.680 (task 1) and 0.668 (task 2).

Discussion and Conclusion
The main objective of this study was to investigate whether performing interactive tasks using the full-touch HMI required more visual demands, thus posing a greater challenge to the lateral control capability of the vehicle.
Compared with CIM, using FTIM to perform secondary tasks had greater PoG, NoG, MSGD, and PoLDG. There is no doubt that there were larger SDLP and SDSWA and smaller TLC when performing secondary tasks under FTIM than under CIM. This finding suggested that more driver attention resources were required to perform the air conditioning-related secondary task using FTIM, resulting in poorer lateral vehicle stability and ride comfort. This finding is similar to that of Wang et al. [37], which showed that visual distraction affected the lateral motion characteristics and comfort of the vehicle.
Compared with using FTIM, the advantages exhibited when using CIM to perform interactive tasks in task 1 and task 2 were different. The gray correlation analysis showed greater difference in glance behavior and lateral control ability between the two interaction modes in task 2. Firstly, more advantage of fewer attentional resources was required for CIM in task 2. This result was reflected mainly in the fact that no glance was longer than 2.0 s when using CIM for task 2, whereas the PoG of more than 2.0 s reached 26.5% when using FTIM. Secondly, the TCT when using CIM to perform task 2 was only 52.8% of that when using FTIM. Finally, similar to attentional resources, in terms of lateral driving performance, a clear advantage was demonstrated when using CIM to perform the secondary task compared with FTIM. This finding was consistent with previous studies [31,32]. The TLC when performing the secondary task using CIM was 1.89 s longer than when using FTIM. TLC was the most directly measurable indicator of vehicle lateral departure, which implied that the risk of lateral departure was higher when performing the secondary task using FTIM than CIM. With the correlation coefficients of each metric in the two modes, we found that PoG had the greatest variability. Therefore, we suggest that measures should be taken in the HMI display design (e.g., setting nonvisual feedback) to reduce the visual involvement during task completion.
In line with previous works [7,37], during the secondary task (either FTIM or CIM), the TCT, MSGD, and PoLDG decreased with increasing driving speed, whereas the NoG on the HMI increased. On average, an increase in speed increases the probability of traffic accidents [42]. Accordingly, to reduce driving risk, drivers may shift their glance more frequently between the road ahead and the HMI during secondary tasks and may minimize visual involvement during secondary tasks and the total time spent on task completion. As driving speed increases, drivers improve their ability to self-regulate when performing secondary tasks, and this study supports previous research by Young [43]. That is, drivers have self-regulatory behaviors to reduce the risk of collision during driving. Another finding of this study was that glance behavior while performing secondary tasks was influenced more by speed in FTIM than in CIM. This seems to indicate that as speed increases, drivers feel that it is more dangerous to perform interactive tasks using FTIM. Therefore, drivers are more cautious and increase the degree of self-regulatory behavior. Interestingly, the study also found that lateral driving performance decreased with increasing speed in both interaction modes, as evidenced by the increase in SDLP and by the decrease in TLC and SDSWA with increasing speed. This finding suggests that although the driver's self-paced secondary task behavior increased with speed, it did not fully compensate for the driving risks associated with increasing speed. It was notable that the normalization coefficients of speed on SDLP and TLC were smaller when using FTIM compared with using CIM. This finding may imply that an increase in speed has less effect on driving stability and comfort in FTIM. We found, however, that the lateral driving risk was greater in FTIM than in CIM at either speed.
There are some limitations associated with this study. First, this test is a real road experiment, in order to ensure driving safety, the traffic flow on the test road we chose in this study is relatively low, so the participant is less affected by the vehicle in front of it. It seems not very rigorous to explore the longitudinal control of the driver under such condition. Therefore, in this study we did not discuss the difference in the driver's longitudinal control ability between the two interaction modes. To overcome this limitation, we will conduct driving simulator experiments on a six-degreeof-freedom driving simulator platform in the future research to further investigate the effect of FTIM on the driver's longitudinal control ability. Second, we only considered low speed 11 Journal of Advanced Transportation roads and highways without considering bottleneck roads in the experimental design, which is another limitation of our study. Considering that different roads can affect driving behavior, we plan to design real-world road and driving simulator tests with different road types, including bottleneck roads, to further explore the effects of different road types on driving behavior under FTIM in future studies. Lastly, connected and autonomous vehicles are the inevitable trend of development; however, before they fully cover the market, there is bound to be a long transition phase: a mixed driving phase of traditional and intelligent vehicles. Therefore, the impact of FTIM on driving behavior should be further considered for platoon strategies in mixed traffic [44,45].

Data Availability
The data sharing is not applicable to this article.

Conflicts of Interest
The authors declare that they have no conflicts of interest.