Mobile Information Systems
Volume 2017 (2017), Article ID 2861827, 15 pages
https://doi.org/10.1155/2017/2861827

Research Article

Empirical Study and Modeling of Vehicular Communications at Intersections in the 5 GHz Band

Seilendria A. Hadiwardoyo, Andrés Tomás, Enrique Hernández-Orallo, Carlos T. Calafate, Juan-Carlos Cano, and Pietro Manzoni

Department of Computer Engineering (DISCA), Universitat Politècnica de València, Camino de Vera, s/n, 46022 Valencia, Spain

Correspondence should be addressed to Seilendria A. Hadiwardoyo; seiha@upv.es

Received 22 December 2016; Accepted 13 March 2017; Published 23 March 2017

Academic Editor: Ilaria Thibault

Copyright © 2017 Seilendria A. Hadiwardoyo et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Event warnings are critical in the context of ITS, being dependent on reliable and low-delay delivery of messages to nearby vehicles. One of the main challenges to address in this context is intersection management. Since buildings will severely hinder signals in the 5 GHz band, it becomes necessary to transmit at the exact moment a vehicle is at the center of an intersection to maximize delivery chances. However, GPS inaccuracy, among other problems, complicates the achievement of this goal. In this paper we study this problem by first analyzing different intersection types, studying the vehicular communications performance in each type of intersection through real scenario experiments. Obtained results show that intersection-related communications depend on the distances to the intersection and line-of-sight (LOS) conditions. Also, depending on the physical characteristics of intersections, the presented blockages introduce different degrees of hampering to message delivery. Based on the modeling of the different intersection types, we then study the expected success ratio when notifying events at intersections. In general, we find that effective propagation of messages at intersections is possible, even in urban canyons and despite GPS errors, as long as rooftop antennas are used to compensate for poor communication conditions.

1. Introduction

Intelligent Transportation System (ITS) is a basic element in future Smart Cities by addressing traffic-related issues. In particular, it is the combination of advanced Information and Communication Technology (ICT) systems and a better transportation infrastructure that paves the way for providing novel services in ITS environments [1]. ITS aims at making traffic more efficient, convenient, and safe [2], addressing noble goals like helping emergency services or changing the way we drive to reduce accidents, fuel consumption, and contaminant emissions. This will eventually minimize transportation problems such as congested roads, will promote road safety, and will in general help at making cities more sustainable [3].

A remarkable issue in ITS is its capability of providing safety applications. In fact, ITS solutions can conveniently provide warning notifications in emergency situations. For example, notifications about dangerous traffic conditions or about emergency breaking can be beneficial in providing a safer traffic flow for the drivers in a city [4]. According to [5], improving traffic safety is currently the second highest strategy priority in ITS and the first priority for future ITS solutions. Thus, we find that safety issues in the context of ITS are indeed essential.

As part of the ITS concept, communications between vehicles play an important role in distributing relevant information. This kind of communication can be enabled through vehicle-to-vehicle (V2V) communications which, combined with the ad hoc networking paradigm, gives rise to Vehicle Ad hoc Networks (VANETs) [4]. For more critical applications in the context of ITS, like real-time safety information, the diffusion of messages should be reliable and time-bounded [6]. Another consideration to bear in mind in the critical safety application domain is the delay of the messaging itself. The performance of distributing the message through V2V is expected to have a low transmission delay, though with a limited reliability. So, vehicles that belong to a certain V2V network facing a particular emergency situation should be alarmed as soon as possible. Otherwise, if the delay is too high, the relevance of the message would be reduced, and it would probably expire [7].

Still in the same context, to have such critical communications, message dissemination should be as effective as possible. In the literature, several dissemination schemes were proposed in order to maximize the effectiveness of message dissemination. An example of a work that proposes reducing the warning message notification time while avoiding the broadcast storm problem was presented in [8]. However, it should be kept in mind that the dissemination process is affected by key factors such as the density of vehicles and the roadmap; in [9] authors address this challenge by proposing a system that adapts to high-density vehicular environments by considering the critical handling of intersections. Taking as reference the survey on this topic by Sanguesa et al. [10], we find that most of the cited schemes rely on GPS information and many of them on intersection-awareness to maximize event dissemination. Thus, the accuracy of the geolocation system becomes critical in the safety dissemination process, especially in urban scenarios to allow determining whether a vehicle is near or even in the middle of the intersection.

In this paper, we perform an empirical study of vehicular communication effectiveness at intersections in the 5 GHz band. To this purpose, we select different types of intersections available in the city of Valencia (Spain) and then perform actual field tests using vehicles to determine the communication restrictions imposed by the different intersections. In addition, based on the empirical results obtained, we model the packet delivery probabilities at different distances to the center of each particular intersection to determine the expected success ratio when delivering event-based messages and to allow integrating the results here presented in simulation platforms.

The paper is organized as follows: in the next section, we provide an overview of the main related works regarding intersection-dependent communications in V2V environments. In Section 3 we describe the methodology adopted for our work, detailing the different software and hardware elements involved. Then, in Section 4, we provide details about the different intersections chosen for our experiments. Experimental results are then presented and discussed in Section 5, followed by modeling of these results in Section 6. The models derived and then used to study the event notification effectiveness at intersections are presented in Section 8. Finally, in Section 9 we conclude this paper and refer to future works.

2. Related Works

In the literature, we can find several works in the scope of safety applications using V2V. A VANET-based emergency vehicle warning system was proposed in [24]. The system was tested in traffic environments including emergency vehicles and traffic lights. The system warns when an emergency vehicle is approaching. Another work, called Intersection Collision Warning, studies warning message dissemination using smartphones with built-in GPS; in particular, it uses the WiFi networks in smartphones to retrieve safety-related information like location, moving direction, and velocity [25]. Other works, like [26], proposed a safety-related Android application to inform nearby vehicles when administrative vehicles, like ambulances, police cars, and fire brigades, are approaching.

Focusing on the issue of signal obstruction, several works have studied its impact on the packet delivery ratio in VANET environments. The work by Böhm et al. [11] investigated the impact of the loss of line-of-sight (LOS) in terms of V2V communication impact, finding that limited LOS between vehicles transmitting and receiving the packets can still enable communications. The researchers in [12] additionally found that obstructing vehicles blocking the LOS significantly attenuate the signal; they also studied the packet delivery ratio under different scenarios, including parking lot areas or urban scenarios under both LOS and non-LOS. Sommer et al. [13] have modeled the IEEE 802.11p/DSRC radio shadowing in urban environments. Their model can estimate the signal attenuation of the wireless radio transmission with buildings as obstacles. The same researchers experimented the two-ray path loss model in both real and simulated environments [14]. In a later work [15], they proposed alternative solutions for signal shadowing caused by neighboring vehicles and buildings based on dynamic beaconing and tested their approach in a simulated environment. Other interesting related works include the deployment of RSUs (Road Side Units) in the vehicular environment [16, 17], studying how urban scenarios with buildings and vegetation affect V2I (Vehicle to Infrastructure) communications.

Regarding the issue of intersection management in the scope of the VANET message transmission process, there have been different proposals in recent years, especially at the network layer. In particular, these protocols used intersection locations as a factor to include within the packet routing strategy. The work by Chou et al. [27] proposed an intersection-based routing protocol that accounts for both the direction of packet transfers and the vehicle moving direction. Through simulation, the research investigated the effect of the number of vehicles on the delivery probability. Geographical conditions can also be taken as an approach for routing messages. In particular, the work by Saleet et al. [28] considers road layouts with intersections for routing. Later, Acarman et al. [29] showed how message routing can also be done by selecting intersections as points of relay using commercial navigation map data and having the connectivity information of road IDs. Similarly, other research works like [30] take advantage of intersections for forwarding messages. It focuses on a large scale urban VANET where the vehicles at intersections are used as virtual gateways that will gather the packets that must be forwarded to all passing vehicles [30].

Focusing instead on the different types of intersection, these can be characterized by the degree of obstruction: whether it is blocked by a building, blocked by plants, or blocked by cars themselves. The work by Schumacher et al. [18] finds that, in an urban scenario where there are buildings blocking the line-of-sight, communications are possible for distances ranging from 85 m to 115 m. By using the 5.9 GHz V2V communications under non-line-of-sight, their results showed that not only do buildings affect the communication but also the width of the street representing the largest street intersection has also a significant impact on the delivery probability. Another research work [19] investigated the effects of vegetation on the performance of V2V communications at intersections. Tests were based on the 5.9 GHz communication band under non-line-of-sight conditions in a rural environment having different types of vegetation for the different seasons; results showed that the packet delivery ratio clearly depended on the type of vegetation and season. When transmitting a message between vehicles, a third vehicle located at the intersection would also affect communications, and the effect of cochannel interference will have an impact under both line-of-sight (open space) and non-line-of-sight (with buildings as obstructions) conditions. The work in [20] addressed this issue through measurements in the 5.9 GHz band. Experiments showed that a single vehicle would interfere and decrease the delivery ratio, no matter if the vehicle is placed near the receiver, near the sender, or between the sender and the receiver.

Finally, Table 1 summarizes the aforementioned related works for the sake of clarity. We categorized previous works taking relevant experiment characteristics into consideration, like the environment details and the performance metrics used.

Table 1: Comparison of related works.

Related works
Frequency band used
Experiment type
Environment used
Types of blockages
Street details
Performance metrics

Böhm et al., 2010 [11]
5.9 GHz
Real world
Urban,
highway, rural
Vegetation,
buildings,
parked vehicles
Not specified
Distance versus packet reception ratio

Meireles et al., 2010 [12]
5.85–5.925 GHz,
2.412 GHz
Real world
Urban and
parking lot
Buildings, trees,
vehicles
(truck, van)
Various lanes
street
Distance versus received signal strength indicator,
distance versus packet delivery ratio

Sommer et al., 2011 [13]
5.89 GHz
Real world and
simulation
Urban and
rural
Residential and
commercial
buildings
Intersection
Distance versus received signal strength,
length of intersection versus received signal strength,
index versus received signal strength

Sommer et al., 2012 [14]
5.89 GHz
Real world and
simulation
Urban and
rural
Vehicles
One-lane and two-lane street
Received signal strength versus distance

Sommer et al., 2015 [15]
5.89 GHz
Simulation
Highway and
suburban
Building and
vehicles
Two-lane
street
Empirical cumulative density function versus
beacon interval,
channel busy ratio,
time after encounter versus beacon interval

Lin et al., 2012 [16]
5.9 GHz
Real world and
field trial
Not specified
Cochannel
interference
Not specified
Distance versus packet loss,
distance versus latency

Gozalvez et al., 2012 [17]
5.895–5.905 GHz
Real world
Urban and
highway
Buildings,
trees, vehicles
Various lanes
street
Distance versus packet delivery ratio

Schumacher et al., 2012 [18]
5.9 GHz
Real world
Urban
Buildings
Intersection
Distance versus signal power,
distance versus packet delivery ratio

Tchouankem et al., 2013 [19]
5.9 GHz
Real world and
simulation
Rural
Vegetations
Intersection
Distance versus signal power,
distance versus packet delivery ratio

Tchouankem and Lorenzen, 2015 [20]
5.9 GHz
Real world
Urban and rural
Buildings and vehicle
Intersection
Distance versus signal power,
distance versus packet delivery ratio

Barcelos et al., 2014 [21]
5.9 GHz
Real world
Urban/rural
(campus environment)
Buildings
Two-lane
street
Distance versus average packet loss,
distance versus average latency,
distance versus average delay,
distance versus bitrate

Viriyasitavat et al., 2016 [22]
2.4 GHz
Real world and
simulation
Urban
(open environment)
Vehicle
One-lane
street
Distance versus packet delivery ratio,
time versus penetration distance,
time versus reachability

Our work
5.8 GHz
Real world
Rural, urban,
mixed
Buildings,
vegetations
Intersection
Distance versus packet delivery ratio
heat maps for packet delivery

In this work, our aim is to study transmission effectiveness on different types of intersections and at different distances. Referring again to Table 1, our work differs from other related works in terms of the frequency band and the performance metrics used, as well as the variety of intersections tested. Also, we present heat maps to characterize transmission effectiveness by showing the locations of the sender when having packets successfully delivered in each scenario. Based on our findings, we develop channel models that are applicable to different simulation environments. In addition, by using our models, we then perform an analytic study of the expected success ratio when attempting to deliver event-related messages at intersections for different degrees of positioning accuracy.

3. Methodology

In this section, we describe the methodology of the experiments performed. The goal is to measure the packet delivery ratio depending on the distance to the center of the intersection for different types of intersections and antenna locations. The expected results in different types of intersection will then be modeled. Later, in Section 7, we will also discuss the applicability of the obtained model and discuss its applicability in comparison with other path loss models.

3.1. General Overview

Our experimental work requires the utilization of appropriate hardware/software to measure the packet delivery ratio and also of a proper data analysis methodology, to process the gathered data after experiments are completed.

Two devices are used in the experiments. The first one is the GRCBox [23], which is our on-board unit providing fully functional V2V communications. This GRCBox is equipped with an antenna that will allow VANET communications in the 5.8 GHz band. The transmitting antennas have a 5 dBi gain and 200 mW transmission power. Packet transmission tests using this device will consider two alternative positions for the antenna. In one case we will put the antenna inside the vehicle, specifically on the dashboard. The other alternative considered will be putting the antenna on the rooftop of the vehicle. These variations will allow us to achieve new findings regarding the effect of antenna locations, expecting that having the antenna on the rooftop will provide a better transmission quality than having it on the dashboard.

Another device used in this experiment is an Android mobile phone. Taking into account the trends of using smartphones for vehicular communications and ITS-related researches, deploying an Android mobile phone can be an alternative solution to exploiting the cost of high-end ITS equipment for research purposes [31–33]. The Android phone is equipped with a custom application (the Android tool). This Android tool will allow performing controlled experiments in real environments by generating messages resembling those associated with the European (ETSI) standard, particularly the Decentralized Environmental Notification Messages (DENM) [34].

For the experiment itself, which includes real moving vehicles, at least two vehicles are needed: one acting as data sender and the other one acting as a data receiver. Regarding the positioning of the vehicles, the one acting as a data receiver will be static and stopped a few meters away from the center of the intersection, representing a vehicle stop at a semaphore or stop sign. The other vehicle, acting as a data sender, will be moving along a different street, crossing a common intersection.

Once the sender is moving and the receiver starts to receive packets, the developed test tool will record the location of both sender and receiver vehicles, which will then be saved in a log file stored on the Android device. After we record the location of the sender in the log file, our data analysis consists of calculating the packet delivery ratio along with the distance between sender and receiver. Specifically, by obtaining the packet delivery ratio for each intersection at different distances, we can draw conclusions on how intersection characteristics will impact message dissemination in vehicular scenarios. In addition, the different intersection types can be modeled, and further analysis can be done based on the obtained models.

3.2. GRCBox Overview

To enable V2V communications, we require a device that provides ad hoc network connectivity in the 5.9 GHz band. Since we use Android devices to launch applications, an option would be to enable ad hoc network connectivity in this device. However, in order to do that, a rooted Android phone is required, thus not being very practical for end users. In addition, the communication range achieved would be quite limited. We provide an alternative solution called GRCBox [23], a solution capable of providing ad hoc communications without having to root smartphones. GRCBox is a multi-interface low-cost connectivity device based on Raspberry Pi. Using the GRCBox, V2X communications are supported, and full integration with smartphones is achieved.

A Raspberry Pi 2 device model B1 is the main hardware of our GRCBox, a single board computer that has the size of a credit-card and it costs only 35 USD. This device has enough CPU power to perform low-scale network routing. A Raspbian distribution based on Debian is installed in this device. This Raspbian distribution supports the current networking hardware while avoiding common problems of other embedded operating systems.

Each GRCBox is equipped with several network interfaces: one inner interface acting as an Access Point for the users, allowing them to connect to the GRCBox using smartphones supporting WiFi communications in the 2.4 GHz band. The outer interface offers vehicular communications, where it connects to a vehicular network in the 5.8 GHz band. In addition, one can add other network interfaces that connect to the Internet. For instance, one network interface can connect to a WiFi Access Point, and yet another can be used to connect to a 4G cellular base station. Figure 1 shows a descriptive diagram of our GRCBox connectivity features.

Figure 1: GRCBox Hardware module connected to a VANET with three different nodes [23].

Several services are provided by the GRCBox. GRCBox’s inner interface acts as a soft-AP (Access Point) for smartphones. Once these smartphones are connected to the GRCBox, they can access the services that run on the external networks. Since every connection is forwarded by the GRCBox, any application needing to use an available interface that differs from the default one (providing Internet connectivity) must notify the GRCBox. These steps require rules that are defined by rule type, interface name, protocol, source port, source address, destination port, and destination address.

In this work, we use the GRCBox at intersections to forward packets that are produced by the Android application at the user’s side and placed on one of the vehicles, to the packet destination, which is the Android application placed at the receiver vehicle. Thus, both GRCBoxes act as the entry gate of packets traveling in the VANET and to be delivered to the GRCBox-aware application running on the smartphones. In our case, the packets are forwarded in a broadcast manner. Thus, appropriate rules are defined before transmission starts to match the target ports and interfaces.

Regarding the configuration of the GRCBox, thanks to our specific application, which will be explained in Section 3.3, the user does not need to set up the rules, as it will be done automatically once the application is launched. In terms of hardware, we need to configure the interfaces to indicate whether they are inner or outer interfaces. Since we only need the VANET communication, only two interfaces are needed. One interface (inner) acts as an Access Point (AP) for mobile devices. Another interface (outer) will create a VANET with other GRCBoxes located within nearby vehicles. In this case, a device capable of transmitting in the 5.8 GHz band is needed for the outer interface.

3.3. Android Tool

We have built a specific tool for measuring the data delivery ratio in the target scenarios. The tool is actually an Android-based application that is GRCBox-aware. The Android application contains libraries and plugins able to connect to the GRCBox module so that, at the user side, one does not need to configure the connection to the GRCBox’s outer interface. Hence, once the Android smartphone connects to the Access Point (AP) of the GRCBox (in this case GRCBox’s inner interface), it would instantly be connected to the whole GRCBox environment and the VANET without further settings. Also, based on its functionality, we have different instances running at the sender and at the receiver ends.

At first, the application will check if it is connected to the GRCBox device on the sender’s side. Also, the user can input the log file name, the packet transmission rate, and the size of the packet. In this case, we have chosen the sending parameters similar to those typical of CAM/DENM messages [34, 35], having a size of about 300 bytes, selecting a packet rate of 30 packets per second to allow quick gathering of large amounts of data. Also notice that since packets are being broadcasted, the transmission rate is limited to 6 Mbps. At the receiver’s side, GRCBox connectivity is also tested, and the user can introduce the log file name and select when to start gathering data. The transmission of packets is started when, at the sender’s end, the user presses the start button, triggering the transmission of a packet train at the defined rate and packet size and using the broadcast mode. Similarly, the transmission stops as the user stops the application. This will cause the application to automatically store the whole log file in a local file at both sender and receiver ends. In a nutshell, our Android-based application is used to define parameters such as data rate, packet size, and log file name. It starts the sending process upon receiving the corresponding user command; then, when the user stops the application, the log file is automatically stored.

3.4. Data Analysis

The data collected is saved in a log file located in the Android device’s storage. This log file contains all the data required to analyze the packet delivery ratio at different distances. For this purpose, we are interested in comparing the geographic information of both endpoints, and so the log file contains the coordinates of the sender and the receiver in terms of latitude and longitude (flat terrain is assumed). Based on this geopositioning information, we then calculated the distance in meters between the vehicle localization and the center of the intersection, where the sending vehicle passes through as part of its designated trajectory.

The distance is calculated with the help of GeographicLib [36], a tool providing a straightforward calculation of distance based on latitudes and longitudes. From each endpoint, in this case, the sender and the receiver, we analyzed the log file that is stored on both sender and receiver sides.

As for the delivery ratio, we need to compare the log files containing packets sent at the sender side and the log files containing packets received at the receiver side. This can also be calculated by referring to the geoposition information provided by both sides.

4. Selection of Target Intersections

For our experiments, we selected three different types of intersections, each one with different characteristics. In particular, the types of intersections were chosen to obtain different degrees of obstruction. The geographical location of each intersection is shown in Figures 2, 4, and 6. The yellow line with an arrow indicates the trajectory and the direction of the sending vehicle, while the yellow point indicates the location of the receiving vehicle (static). We now proceed to provide more details about each of them.

Figure 2: Location and the trajectory for Scenario 1 (open).

4.1. Intersection 1 (Open Scenario)

The first intersection selected is an open space. It was taken in a low populated area in the outskirts of the city of Valencia (Latitude 39.483920, Longitude −0.333793). In this intersection, no relevant signal blockages are present. In fact, Figure 2 shows that the only blocking structures are two buildings, one south of the roundabout and the other one north. Thus, the line-of-sight is not blocked along the trajectories of the vehicle, meaning that the degree of obstruction is minimal. As shown in Figure 3(a), the receiving vehicle is located near the intersection, being surrounded by grass fields and facing no significant signal blockage.

(a) Outside View

(b) Inside View

Figure 3: View of the vehicle parked at Intersection 1.

Figure 4: Location and the trajectory for Scenario 2 (buildings).

4.2. Intersection 2 (Buildings Scenario)

The second intersection selected is in a residential block, which is located in a crowded and dense area of the city (latitude 39.473695, longitude −0.332307). In this intersection, buildings are present as blockages to the line-of-sight, meaning that the degree of obstruction is nearly maximum. Based on the aerial view shown in Figure 4, we can see that the environment consists of a dense neighborhood, without additional urban elements separating them except streets themselves, meaning that an urban canyon is formed. In Figure 5(a) we can see that the chosen intersection is surrounded by at least two-floor buildings. So, from the perspective of the receiving vehicle, the line-of-sight is quite limited as the furthest view that one can glimpse from inside the mentioned vehicle is of, at most, 20 meters. By defining this type of intersection, we expect that the transmission of packets in this location would be the worst case scenario.

(a) Outside View

(b) Inside View

Figure 5: View of the vehicle parked at Intersection 2.

Figure 6: Location and the trajectory for Scenario 3 (trees).

4.3. Intersection 3 (Trees Scenario)

The third selected intersection is a mix of the previous two scenarios, as the line-of-sight is blocked by either buildings or trees. The intersection lies in a residential area near a university campus (latitude 39.473848, longitude −0.341330), and the degree of obstruction can be considered moderate. Figure 6 shows that this kind of intersection has some buildings in the surroundings, but also open space and vegetation along the trajectory. A mix between these characteristics causes communication in this kind of environment to produce interesting results as the line-of-sight characteristics are variable, being that sometimes it is blocked by a building or tree, while at other times no obstacle will block sight. Figure 7(a) presents the real view of the intersection. As shown, the receiving vehicle is surrounded by trees and, within meters, there is an open field. However, the street itself is located in a residential area plenty of tall buildings.

(a) Outside View

(b) Inside View

Figure 7: View of the vehicle parked at Intersection 3.

5. Experimental Results

We have done real experiments with vehicles to gather the location (coordinates) of the vehicles when packets are successfully delivered at each intersection. We also gathered data when the antennas are located either inside the vehicle on the dashboard or on the rooftop of the vehicles. The data was gathered from a set of five vehicle runs at each intersection, and the measured coordinates are then validated using real maps.

5.1. Results for Intersection 1

Figure 8(a) shows the percentage of messages received as a heat map for the first intersection, which has the lowest degree of obstruction, based on the locations of the sender associated with successful packet delivery. In this experiment, the two antennas involved were located in the vehicles’ dashboard. As expected, the packets can be delivered successfully having as source nearly any position along the vehicle trajectory. The only gap detected occurs due to the blocking caused by the two available buildings. However, the delivery ratio is much higher when it is at the center of the intersection and gets lower as the vehicle moves away until the maximum tested distance (about 300 meters).

(a) Scenario 1 (open)/dashboard

(b) Scenario 1 (open)/rooftop

(c) Scenario 2 (buildings)/dashboard

(d) Scenario 2 (buildings)/rooftop

(e) Scenario 3 (trees)/dashboard

(f) Scenario 3 (trees)/rooftop

Figure 8: Heat maps of different scenarios. Each plot shows the packet delivery ratio depending on the sender position, scenario (open, building, and trees), and antenna location (dashboard, rooftop).

Figure 8(b) presents the results when the antenna is located on the rooftop instead. We can now observe that the point density is higher and that there are no gaps. In fact, the impact of physical obstructions is still relevant, but it is merely limited to a slight reduction of the packet delivery ratio.

5.2. Results for Intersection 2

This second intersection is quite narrow, having tall buildings on all sides that create an urban canyon. Figure 8(c) shows the locations associated with successful transmission of the packets when the antenna is located in the dashboard. We can see that successful transmissions are basically restricted to a range of just one block (about 50 meters), again experiencing a descending delivery ratio when moving away from the center of the intersection.

If instead the antenna is located on the rooftop, Figure 8(d) shows results similar to those in the previous figure, although now the overall delivery ratio increases, reaching up to two blocks away from the intersection (about 100 meters) while mostly maintaining an acceptable delivery ratio.

5.3. Results for Intersection 3

Since this third type of intersection presents a moderate degree of obstruction, the results in terms of radio range should be in between the results for the first and second types of intersections. Figure 8(e) shows that the locations associated with a successful transmission indeed reach greater distances compared to the second type of intersection, having values resembling more the first intersection. This occurs since the obstacle closer to the vehicle in this situation is mainly vegetation, which does not negatively impact communications the same way as buildings; in addition, the streets are wider than in the second scenario.

When locating the antenna on the rooftop, results become much better. Now, although we cannot associate the entire vehicle path with good transmission conditions, the results of Figure 8(f) clearly show that high delivery ratios can be maintained up to about 100/150 meters. Again, the delivery ratio near the intersection is significantly high, experiencing a constant drop as we move away.

6. Intersection Modeling

Based on the results obtained in our experiments, we now proceed to detail how the different intersections were modeled. Our purpose is to obtain a generic model that allows integrating the different behaviors observed in simulation tools, as well as analytically studying the effectiveness of event-related message delivery at intersections.

6.1. Modeling Procedure

In order to model the different intersections, our procedure was the following: first, we obtained the number of packet transmissions and receptions for each position registered; second, we determined the packet delivery ratio value associated with each distance range; finally, we performed a curve fitting process to derive optimal parameters.

In detail, the results of the first step of the experiment consist of a list of coordinates (i.e., latitudes and longitudes) stored at the sender, with another similar list stored at the receiver’s side. The sender coordinates are the sender’s actual location when a packet is sent. Logically, the list at the sender side has more entries than the one at the receiver side as several packets get lost. So, we must compare the difference between these two lists of coordinates. A packet is successfully received if an entry (coordinate) at the sender’s list is also present in the receiver’s list. The coordinate is then translated into the distance by considering the coordinates relative to the center of the intersection using the haversine formula implemented in the GeographicLib library.

The outcome of the first step is then grouped into small intervals, with the interval width equal to 5 meters. The delivery ratio derived for the different intervals is then plotted using a bar chart, thereby resembling a histogram (although it is not so in a strict sense). By following the same procedure for the different antenna locations (rooftop versus dashboard), we can then obtain a comparative chart for the target intersection that allows checking packet delivery ratio in both cases.

Once these distributions are calculated for the three types of intersection and for both antenna locations, we proceeded to find the best fit for our data. The curve fitting was done using the nonlinear least-squares Marquardt-Levenberg algorithm (implemented in the GNUPLOT software) to derive a general model for the curve, determining which function is more adequate for our purposes, as well as the best parameter values for each distribution.

6.2. Fitting Results

For each curve, we tried to find a common model that would be suitable for both antenna positions (dashboard or rooftop) and the different type of intersections (open, building, and trees). Our intention was that only one parameter would vary from one scenario to another, allowing to seamlessly model different types of intersections having variable degrees of radio visibility. After evaluating several fitting functions (polynomial, power) for the different types of intersection and antenna locations, the best fitting was obtained using a Gaussian function:

Notice that variables , , , and are generic variables, meaning that (1) will be adapted to our specific requirements. In particular, we set as the distance from the intersection and fix the probability of reception to one exactly at the intersection (distance zero), so is one and is zero. Thus, the only parameter to fit is , that is, the standard deviation. So, finally we have the following expression:

This exponential function computes the delivery ratio for a particular distance . As the distance grows, this probability asymptotically becomes 0. The value of the constant (or standard deviation ) will depend on the scenario and the antenna position and reflects the variation or dispersion of the data values.

The resulting bar charts and fitting results are shown in Figures 9(a), 9(b), and 9(c). If we take a look at the experimental results for Intersection 1 (see Figure 9(a)), we can quickly notice that there is a significant difference between the delivery ratio for the dashboard and rooftop location cases. The curve fit for the dashboard antenna location shows that, for low distances, the delivery ratio is still comparable to the one from the rooftop fit. After a distance of about 20 meters from the intersection, the bars show a quick attenuation. Also, we observe that it loses contact after about 200 meters. We can observe how, when the distance is of about 120 meters, the delivery ratio suddenly drops, being followed by a moderate increase. This is an effect of the buildings present in the environment, as we can see in an aerial view of the street shown in Figure 2. Concerning the delivery ratio for the rooftop scenario, high delivery values are sustained for a distance up to 70 meters, after which they experience a 20% drop. In this same scenario for the dashboard case, the drop ratio is significantly higher (50%). Thus, we can conclude that antenna location has a very significant impact on packet delivery success.

(a) Delivery ratio at Intersection 1 with the antenna put on the dashboard or rooftop

(b) Delivery ratio at Intersection 2 with the antenna put on the dashboard or rooftop

(c) Delivery ratio at Intersection 3 with the antenna put on the dashboard or rooftop

Figure 9: Curve fittings of delivery ratio versus distance.

Concerning the second intersection, the bar chart shown in Figure 9(b) clearly shows a significant difference compared to the previous one. In fact, the distance range is now quite reduced, with the fact that the packet delivery ratio drops to only 35% in about 40 meters (for the “dashboard” antenna position). Beyond 50 meters, the delivery ratio becomes near zero. If we focus now on the curve fit for the rooftop scenario, we find that differences towards the dashboard case are quite clear, similarly to what occurred for Intersection 1. However, with respect to that first intersection, we now see that, at a distance of 50 meters, the packet delivery ratio for the rooftop is nearly 0.8, while for the dashboard it is only 0.3, both values much lower than those measured in Intersection 1. This is why we can categorize this second intersection as an urban canyon, which is a worst case scenario associated with the maximum degree of obstruction.

Regarding the results for Intersection 3, Figure 9(c) shows that when the antenna is located in the dashboard, there is a loss of radio connectivity after about 70 meters. Instead, when the antenna is on the rooftop, contact is maintained beyond 250 meters, a much greater distance. Compared to the two previous intersections, the fittings in this scenario are indeed a situation in between intersections 1 and 2.

Overall, we consider that the obtained results are quite reasonable by considering that our experiments were made in scenarios where no interference is hindering our communications band, meaning that the channel only experiences the effect of additive white Gaussian noise. In such situation, the fitting corresponds to a standard AWGN channel model.

We now focus in detail on the outcome of fitting results and the corresponding fitting errors. Notice that (2) introduced parameter , the standard deviation of the Gaussian function, which allows adapting the fitting curve to each type of intersection and antenna location. In Table 2, we now detail the values of this parameter for each case. It is interesting to observe that this parameter decreases for lower radio ranges at intersections, being directly related to the packet delivery ratio. In fact, the higher the parameter, the higher the packet delivery ratio for a certain distance towards the intersection.

Table 2: parameter and error values for each scenario and antenna position.

Antenna on dashboard
Antenna on rooftop

Intersection 1
56.57
15.35
240.72
30.15

Intersection 2
30.92
12.84
71.82
21.67

Intersection 3
38.98
6.18
136.15
20.14

In detail, we can see that values for the first intersection are the highest ones. On the other hand, for Intersection 2, the lowest values are obtained, with Intersection 3 characterized by intermediate values. Also, regarding antenna locations, we find that value for the rooftop results is always more than twice those obtained with the antenna in the dashboard. The largest relative difference is detected for Intersection 1, where value for the rooftop case is more than four times greater than the one for the dashboard case. This occurs because, for this kind of intersection, packet losses are mostly related to signal power dropping due to distance, and the rooftop antenna location thereby emerges as the optimal option to mitigate such power losses.

Table 2 also shows the fitting error expressed as , the sum of the squares of the differences between the model function and the actual delivery ratios obtained from the experiments. Additionally, Figure 10 shows a box and whisker plot of the difference distribution for each scenario and antenna position. The box shows the 2nd, 3rd, and 4th percentiles, and the whisker is the mean value plus/minus the standard deviation. The model fitting is clearly more accurate for the dashboard scenarios than when mounting the antenna on the rooftop. This occurs because, in the latter case, the range is not large enough to reach near-zero values.

Figure 10: Differences among the fitted models and data.

7. Model Applicability to Simulation Environments

In general, a detailed channel characterization between two endpoints requires studying the signal to noise plus interference values at the receiver, which includes modeling in detail the signal propagation conditions in the target environment. In the specific case of vehicular networking environments, this includes the modeling of signal reflections and Doppler spread in the presence of various obstacles, including buildings, trees, and vehicles. However, such a detailed signal propagation analysis is extremely complex, and so it becomes computationally prohibitive to undertake such a detailed analysis when studying traffic communications in a large area, especially for vehicular networking studies where this area can grow up to the size of an entire city or even greater. To address such problem, empirical path loss models for urban environments have emerged (e.g., Nakagami [37] and Durgin et al. [38]). However, these models provide a generalization of the propagation behavior, meaning that they fail to provide a detailed characterization of very specific transmission conditions, such as the intersection propagation conditions addressed in this paper. Yet the problem of how to adapt our model to simulation environments remains, as it requires knowing in advance the actual characteristics of each specific intersection in order to adequately model it.

To achieve the intersection modeling requirements enabling the adoption of our models, we propose automating the intersection classification process by analyzing the street width and the presence of buildings in a preprocessing step before the actual simulation. This way we avoid having to manually tag each intersection manually and benefit from the models hereby derived with little additional complexity.

It is worth highlighting that widely used map providers such as OpenStreetMap [39] already include such street and building information for many relevant cities, which simplifies and makes feasible the adoption of our solution.

8. Event Notification Effectiveness on Intersections

To further validate our research work, we have also analyzed the probability of successful delivery of notifications associated with critical events. As explained earlier, such event notification dissemination typically relies on multihop broadcasting to make sure that the information arrives to all vehicles in a certain target area. However, since such dissemination procedure is prone to cause broadcast storm problems and since urban obstacles will typically hinder dissemination towards vehicles in nearby streets, different proposals consider it optimal to perform timely broadcasts when vehicles are located at intersections to maximize reachability. Such timely broadcasts for moving vehicles, though, rely on mapping GPS coordinates to map details, and the overall effectiveness will highly depend on the GPS error introduced at the time of broadcasting.

Taking the aforementioned issues into consideration, in this section, we will use the models derived in Section 6 for the different intersection types and antenna locations to study the probability of successfully delivering an event-related message at an intersection when considering different GPS error values. We are assuming that the vehicle intends to send a packet when located at the center of the intersection to maximize the packet delivery ratio. However, if we take the GPS error into account, we could expect that the error it introduces could impact the packet delivery ratio, especially in urban canyon scenarios. To this purpose, we define different maximum values for the GPS error (which typically ranges between 5 and 50 meters) and create normal distributions where of the values are inside this maximum distance ( rule). Then, considering this probability distribution for the vehicle location when transmitting a packet, we combined it with the models derived in the previous section to gain awareness about the expected success ratio for the event message delivery.

In Figure 11, we evaluate the impact that the GPS error ranges will have on the delivery ratio for each scenario, with the antenna located either on the dashboard or on the rooftop. The three intersections that have different levels of obstruction are compared. In these plots the delivery ratio from the fitted model is shown for three significative points in the error distribution: the bar corresponds to the interval from to ( of values), the line corresponds to (), and the cross is ().

(a) Dashboard

(b) Rooftop

Figure 11: Delivery ratio at the three intersections for different GPS error ranges.

If we focus on the case where the antenna is located in the dashboard (see Figure 11(a)), a significant difference is detected when we have a GPS error of 50 meters. In the case of Intersection 1, a GPS error of up to 50 meters still shows acceptable packet delivery levels; on the contrary, for Intersection 2 (urban canyon), the delivery ratio is much worse than for the other two cases.

Figure 11(b) shows that when installing the antenna on the rooftop, the impact of GPS error is now reduced as the delivery ratio in these cases, when compared to the previous ones, is much better. This means that, in general, effective propagation of messages at intersections is possible, even in urban canyons and despite GPS errors, as long as rooftop antennas are used so that their extended radio range compensates for the poor radio visibility and positioning error.

In a nutshell, again we find that the different antenna positions and the characteristics of intersections clearly affect the probability of successful packet delivery even with the presence of GPS error. That being said, the most reliable sending process takes place when we put the antenna on the rooftop of the vehicle and the transmission occurs at an open space intersection (with minimum obstructions). The worst case occurs at the urban canyon intersection (maximum obstruction) when the antenna is located within the vehicle, in the dashboard, thereby matching our initial hypothesis.

9. Conclusions and Future Works

Recent efforts to minimize accidents in vehicular environments have led safety issues to represent one of the most important applications in the context of ITS. In this scope, intervehicular communication can play a very relevant role by allowing quickly notifying neighboring vehicles about dangerous events. However, these notifications should be delivered quickly and reliably, which can be a strict requirement in urban environments since buildings and other obstacles are prone to hinder signal propagation. Thus, timely message delivery at the center of intersections emerges as the main solution allowing avoiding urban obstacles in a simple and straightforward manner.

In this paper we have studied the packet delivery effectiveness achieved on different types of intersections (no obstacles, urban canyon, and partial obstruction) and when locating the antennas on either the dashboard or the rooftop. Extensive experimental results using broadcast traffic have shown that the impact of the intersection type is significant, as differences of up to 150 meters in transmission range were detected. Also, we find that having a rooftop antenna is also a critical factor, allowing extending the transmission range between 100 and 250 meters, which may represent more than a 100% increase in some cases.

Additionally, we have modeled all the obtained results by finding the best-fitting function and then applying regression. We find that a Gaussian function offers adequate fits for all cases by just varying one parameter. This way, our model allows seamlessly representing different types of intersections and bringing these results to simulation environments.

Based on our model, we then made an analytic study to determine the probability of a successful event dissemination process at intersections, for the different types of intersection and antenna locations tested, when varying the maximum GPS error. We find that, in general, dissemination is highly effective, even in urban canyons and for high GPS error conditions, as long as rooftop antennas are used, with the more restrictive dashboard solutions being not recommended. This way, using the previous models and assisted by real-time geolocation and maps, we can first determine the type of scenario to use and then, knowing the GPS error, determine the expected delivery ratio.

As future work, we will translate our results to a simulation platform in order to achieve a more realistic simulation model able to better resemble real-life experiments. Also, since we used the standard GPS device embedded in the smartphone for localization purposes, it would be interesting to test with more precise outdoor geolocation devices. Another consideration to be kept in mind is to evaluate the feasibility of V2V communications through the use of LTE-based intervehicle communications.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was partially supported by the “Ministerio de Economía y Competividad, Programa Estatal de Investigación, Desarollo e Innovación Orientada a los Retos de la Sociedad, Proyectos I+D+I 2014,” Spain, under Grants TEC2014-52690-R and BES-2015-075988.

References

J. M. Hernández-Mũnoz, J. B. Vercher, L. Mũnoz et al., “Smart cities at the forefront of the future internet,” in The Future Internet Assembly, pp. 447–462, Springer, 2011.

Z. Xiong, H. Sheng, W. Rong, and D. E. Cooper, “Intelligent transportation systems for smart cities: a progress review,” Science China Information Sciences, vol. 55, no. 12, pp. 2908–2914, 2012.

C. T. Barba, M. Á. Mateos, P. R. Soto, A. M. Mezher, and M. A. Igartua, “Smart city for VANETs using warning messages, traffic statistics and intelligent traffic lights,” in Proceedings of the IEEE Intelligent Vehicles Symposium (IV '12), pp. 902–907, June 2012.

P. Papadimitratos, A. La Fortelle, K. Evenssen, R. Brignolo, and S. Cosenza, “Vehicular communication systems: enabling technologies, applications, and future outlook on intelligent transportation,” IEEE Communications Magazine, vol. 47, no. 11, pp. 84–95, 2009.

S. Grant-Muller and M. Usher, “Intelligent transport systems: the propensity for environmental and economic benefits,” Technological Forecasting and Social Change, vol. 82, no. 1, pp. 149–166, 2014.

M. Sepulcre and J. Gozalvez, “Experimental evaluation of cooperative active safety applications based on V2V communications,” in Proceedings of the 9th ACM International Workshop on Vehicular Inter-NETworking, Systems, and Applications (VANET '12), pp. 13–20, ACM, June 2012.

X. Ma, X. Chen, and H. H. Refai, “Performance and reliability of DSRC vehicular safety communication: a formal analysis,” EURASIP Journal on Wireless Communications and Networking, vol. 2009, Article ID 969164, 13 pages, 2009.

F. J. Martinez, C.-K. Toh, J.-C. Cano, C. T. Calafate, and P. Manzoni, “A Street Broadcast Reduction scheme (SBR) to mitigate the broadcast storm problem in VANETs,” Wireless Personal Communications, vol. 56, no. 3, pp. 559–572, 2011.

M. Fogue, P. Garrido, F. J. Martinez, J.-C. Cano, C. T. Calafate, and P. Manzoni, “PAWDS: a roadmap profile-driven adaptive system for alert dissemination in VANETS,” in Proceedings of the 10th IEEE International Symposium on Network Computing and Applications (NCA '11), pp. 1–8, Cambridge, Mass, USA, August 2011.

J. A. Sanguesa, M. Fogue, P. Garrido, F. J. Martinez, J.-C. Cano, and C. T. Calafate, “A survey and comparative study of broadcast warning message dissemination schemes for VANETs,” Mobile Information Systems, vol. 2016, Article ID 8714142, 2016.

A. Böhm, K. Lidström, M. Jonsson, and T. Larsson, “Evaluating CALM M5-based vehicle-to-vehicle communication in various road settings through field trials,” in Proceedings of the 35th Annual IEEE Conference on Local Computer Networks (LCN '10), pp. 613–620, October 2010.

R. Meireles, M. Boban, P. Steenkiste, O. Tonguz, and J. Barros, “Experimental study on the impact of vehicular obstructions in VANETs,” in Proceedings of the IEEE Vehicular Networking Conference (VNC '10), pp. 338–345, December 2010.

C. Sommer, D. Eckhoff, R. German, and F. Dressler, “A computationally inexpensive empirical model of IEEE 802.11p radio shadowing in urban environments,” in Proceedings of the 8th International Conference on Wireless On-Demand Network Systems and Services (WONS '11), pp. 84–90, January 2011.

C. Sommer, S. Joerer, and F. Dressler, “On the applicability of Two-Ray path loss models for vehicular network simulation,” in Proceedings of the IEEE Vehicular Networking Conference (VNC '12), pp. 64–69, Seoul, Korea, November 2012.

C. Sommer, S. Joerer, M. Segata, O. K. Tonguz, R. L. Cigno, and F. Dressler, “How shadowing hurts vehicular communications and how dynamic beaconing can help,” IEEE Transactions on Mobile Computing, vol. 14, no. 7, pp. 1411–1421, 2015.

J.-C. Lin, C.-S. Lin, C.-N. Liang, and B.-C. Chen, “Wireless communication performance based on IEEE 802.11p R2V field trials,” IEEE Communications Magazine, vol. 50, no. 5, pp. 184–191, 2012.

J. Gozalvez, M. Sepulcre, and R. Bauza, “IEEE 802.11p vehicle to infrastructure communications in urban environments,” IEEE Communications Magazine, vol. 50, no. 5, pp. 176–183, 2012.

H. Schumacher, H. Tchouankem, J. Nuckelt et al., “Vehicle-to-Vehicle IEEE 802.11p performance measurements at urban intersections,” in Proceedings of the IEEE International Conference on Communications (ICC '12), pp. 7131–7135, June 2012.

H. Tchouankem, T. Zinchenko, H. Schumacher, and L. Wolf, “Effects of vegetation on vehicle-to-vehicle communication performance at intersections,” in Proceedings of the IEEE 78th Vehicular Technology Conference (VTC Fall '13), pp. 1–6, September 2013.

H. Tchouankem and T. Lorenzen, “Measurement-based evaluation of interference in Vehicular Ad-Hoc Networks at urban intersections,” in Proceedings of the IEEE International Conference on Communication Workshop (ICCW '15), pp. 2381–2386, IEEE, London ,UK, June 2015.

V. P. Barcelos, T. C. Amarante, C. D. Drury, and L. H. A. Correia, “Vehicle monitoring system using IEEE 802.11p device and Android application,” in Proceedings of the 19th IEEE Symposium on Computers and Communications (ISCC '14), pp. 1–7, Madeira, Portugal, June 2014.

W. Viriyasitavat, S. Midtrapanon, T. Rittirat, and S. Thanumaiweerakun, “Performance analysis of android-based real-time message dissemination in VANETs,” in Proceedings of the International Conference on Computing, Networking and Communications (ICNC '16), pp. 1–5, IEEE, February 2016.

S. M. Tornell, S. Patra, C. T. Calafate, J.-C. Cano, and P. Manzoni, “GRCBox: extending smartphone connectivity in vehicular networks,” International Journal of Distributed Sensor Networks, vol. 2015, Article ID 478064, 13 pages, 2015.

A. Buchenscheit, F. Schaub, F. Kargl, and M. Weber, “A VANET-based emergency vehicle warning system,” in Proceedings of the IEEE Vehicular Networking Conference (VNC '09), pp. 1–8, October 2009.

J. Yang, J. Wang, and B. Liu, “An intersection collision warning system using Wi-Fi smartphones in VANET,” in Proceedings of the 54th Annual IEEE Global Telecommunications Conference: “Energizing Global Communications” (GLOBECOM '11), pp. 1–5, December 2011.

S. Patra, S. Tornell, C. Calafate, J. Cano, and P. Manzoni, “Messiah: an its drive safety application,” in XXV Jornadas Sarteco, Valladolid, Spain, 2014.

L.-D. Chou, J.-Y. Yang, Y.-C. Hsieh, D.-C. Chang, and C.-F. Tung, “Intersection-based routing protocol for VANETs,” Wireless Personal Communications, vol. 60, no. 1, pp. 105–124, 2011.

H. Saleet, R. Langar, K. Naik, R. Boutaba, A. Nayak, and N. Goel, “Intersection-based geographical routing protocol for VANETs: a proposal and analysis,” IEEE Transactions on Vehicular Technology, vol. 60, no. 9, pp. 4560–4574, 2011.

T. Acarman, C. Yaman, Y. Peksen, and A. U. Peker, “Intersection based routing in urban VANETs,” in Proceedings of the 18th IEEE International Conference on Intelligent Transportation Systems (ITSC '15), pp. 1087–1092, IEEE, Las Palmas, Spain, September 2015.

X. Guan, Y. Huang, Z. Cai, and T. Ohtsuki, “Intersection-based forwarding protocol for vehicular ad hoc networks,” Telecommunication Systems, vol. 62, no. 1, pp. 67–76, 2016.

W. Vandenberghe, I. Moerman, and P. Demeester, “On the feasibility of utilizing smartphones for vehicular ad hoc networking,” in Proceedings of the 11th International Conference on ITS Telecommunications (ITST '11), pp. 246–251, IEEE, Saint Petersburg, Russia, August 2011.

S. M. Tornell, C. T. Calafate, J.-C. Cano, P. Manzoni, M. Fogue, and F. J. Martinez, “Evaluating the feasibility of using smartphones for ITS safety applications,” in Proceedings of the IEEE 77th Vehicular Technology Conference (VTC Spring '13), pp. 1–5, Dresden, Germany, June 2013.

R. Frank, W. Bronzi, G. Castignani, and T. Engel, “Bluetooth low energy: an alternative technology for VANET applications,” in Proceedings of the 11th Annual Conference on Wireless On-Demand Network Systems and Services (WONS '14), pp. 104–107, IEEE, Obergurgl, Austria, April 2014.

ETSI, “Intelligent Transport Systems (ITS); vehicular communications; basic set of applications; part 3: specification of decentralized environmental notification basic service,” TS 102 637-3, 2010.

ETSI and Intelligent Transport Systems (ITS), “Vehicular communications; basic set of applications; part 3: specification of decentralized environmental notification basic service,” TS 102 637-2, 2010.

C. F. F. Karney, “Transverse Mercator with an accuracy of a few nanometers,” Journal of Geodesy, vol. 85, no. 8, pp. 475–485, 2011.

M. Nakagami, “The m-distribution—a general formula of intensity distribution of rapid fading,” in Statistical Methods in Radio Wave Propagation, Pergamon Press, 1960.

G. Durgin, T. S. Rappaport, and H. Xu, “Measurements and models for radio path loss and penetration loss in and around homes and trees at 5.85 GHz,” IEEE Transactions on Communications, vol. 46, no. 11, pp. 1484–1496, 1998.

M. Haklay and P. Weber, “OpenStreet map: user-generated street maps,” IEEE Pervasive Computing, vol. 7, no. 4, pp. 12–18, 2008.