International Journal of Navigation and Observation
Volume 2017 (2017), Article ID 6750346, 10 pages
https://doi.org/10.1155/2017/6750346

Research Article

Performance Evaluation of Pedestrian Locations Based on Contemporary Smartphones

Jalal Ibrahim Al-Azizi and Helmi Zulhaidi Mohd Shafri

Department of Civil Engineering and Geospatial Information Science Research Centre (GISRC), Faculty of Engineering, University Putra Malaysia (UPM), 43400 Serdang, Malaysia

Correspondence should be addressed to Helmi Zulhaidi Mohd Shafri; helmi@upm.edu.my

Received 8 August 2016; Revised 8 November 2016; Accepted 29 November 2016; Published 13 February 2017

Academic Editor: Letizia Lo Presti

Copyright © 2017 Jalal Ibrahim Al-Azizi and Helmi Zulhaidi Mohd Shafri. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Nowadays, a Global Navigation Satellite System (GNSS) unit is embedded in nearly every smartphone. This unit allows a smartphone to detect the user’s location and motion, and it makes functions, such as navigation, tracking, and compass applications, available to the user. Therefore, the GNSS unit has become one of the most important features in modern smartphones. However, because most smartphones incorporate relatively low-cost GNSS chips, their localization accuracy varies depending on the number of accessible GNSS satellites, and it is highly dependent on environmental factors that cause interference such as forests and buildings. This research evaluated the performance of the GNSS units inside two different models of smartphones in determining pedestrian locations in different environments. The results indicate that the overall performances of the two devices were related directly to the environment, type of smartphone/GNSS chipset, and the application used to collect the information.

1. Introduction

These days many different brands and models of smartphone are available on the market. Most of these smartphones incorporate different types of embedded sensors such as GNSS receivers, Radio-Frequency Identification, a camera, and an inertial platform [1]. These devices could be used to provide seamless integration of three positioning technologies: the Assisted Global Positioning System (A-GPS), WiFi positioning, and cellular network positioning [2]. Thus, positional information could be obtained at almost every moment and in almost every location, which has led to an increase in the number of available navigation and location based applications.

Numerous studies have been conducted testing smartphones as navigational and/or tracking systems. However, less attention has been paid to pedestrian navigation. Klimaszewski-Patterson [3] compared the GPS capabilities between a smartphone (HTC G1 Dream) and dedicated GPS (Trimble Juno SB) device. Gikas et al. [4] concluded that positioning sensors embodied in modern smartphones, such as the HTC One S and iPhone 5S, can provide accuracy metrics of sufficiently high accuracy for a variety of intelligent transportation system applications. Kos and Brčić [5] considered weather and ionospheric conditions when studying the horizontal positioning performance of two identical smartphone devices and found distinct differences in horizontal positioning accuracy.

Bauer [6] proved that the accuracy of smartphone positioning depends on the application. In her study, nine popular applications were run on a single smartphone and used to take measurements on a running track. Bauer’s results showed different outputs for each application, indicating that GPS accuracy is affected by various influencing factors. Menard et al. [7] used three different smartphones (each with an embedded GPS sensor) and analyzed their positional capability when vehicle tracking. Each of the three smartphones was accurate to within 10 m about 95% of the time. Zhang et al. [8] evaluated high sensitivity GPS and mobile phones (iPhone 3GS, iPhone 4, Nokia N95, and HP iPAQ) in three scenarios: urban canyon, shopping center, and an area under dense tree foliage in kinematic mode. One of their conclusions was that the performance of the receivers from different manufacturers varies, whereby a given receiver may perform very well in one aspect, but not as well based on other criteria.

In this study, for the purposes of research and to evaluate the performance of contemporary smartphones in estimating pedestrian location, two popular smartphones (Samsung Galaxy S3 based on the Android platform [9] and iPhone 5S based on the iOS platform [10]) were selected to obtain kinematic and static observations using four different smartphone applications in two different environments: an open outdoor area to represent ideal conditions and inside a shopping center to represent compromised conditions. Figure 1 shows the selected devices.

Figure 1: The selected devices in this research.

2. Smartphone GNSS Analysis

The technical specifications of the Samsung Galaxy S3 and Apple iPhone 5S have been studied to assess the positioning accuracy and reliability obtained by these devices [1]. Based on data from the GSMArena website, Table 1 shows the most important technical specifications for these two devices.

Table 1: Technical specifications of the devices used in this study (data from http://www.gsmarena.com/).

Name
Galaxy S3
iPhone 5S

Released
May 2012
Sep 2013

 Technology
GSM/HSPA
GSM/CDMA/HSPA/EVDO/LTE

OS/Platform
Android
iOS

CPU
Quad-core 1.4 GHz Cortex-A9
Dual-core 1.3 GHz Cyclone (ARM v8-based)

Memory
1 GB RAM
1 GB RAM DDR3

A-GPS
Yes
Yes

GLONASS (chipset)
Yes
Yes

(Broadcom BCM47511)
(Qualcomm WTR1605L)

 Sensors
Accelerometer, gyro, proximity, compass, barometer
Accelerometer, gyro, proximity, compass

http://www.gadgethelpline.com/samsung-galaxy-iii/.

https://discussions.apple.com/thread/6246837?tstart=0.

To evaluate the availability and accuracy of locations obtained by smartphones, four tests were conducted in the two selected environments. Table 2 shows the four types of the conducted tests.

Table 2: Types of tests conducted.

Test number
Observation
mode
Observation
environment

Test A
Static
Open outdoor area

Test B
Kinematic
Open outdoor area

Test C
Static
Indoor (inside the shopping center)

Test D
Kinematic
Indoor (inside the shopping center)

First, suitable smartphone applications [11, 12] that have the capability of storing navigational data in the National Marine Electronics Association (NMEA) format [14] were selected. Many applications were downloaded and tested, but only two applications were chosen for each device. Table 3 provides detailed information about the selected applications.

Table 3: Technical specifications of the selected smartphone applications [11, 12].

App 1
App 2
App 3
App 4

Application name
NMEA GPS
Tracks Logger Pro
Ultra GPS Logger
NMEA Data Logger

Smartphone
iPhone 5S
iPhone 5S
Samsung S3
Samsung S3

Version
2.0.6
2.6.0
3.118d
1.0

Last update
Aug 17, 2015
Jan 19, 2016
May 23, 2016
Apr 17, 2013

Application rate
4+
4+
4.7
3.7

Observation sampling rate
1 s
1 s
1 s
1 s

NMEA data logged by Samsung Galaxy S3.

Analysis of the NMEA output (Figure 2) shows that the GNSS sensor output depends on the smartphone platform such as iOS or Android. Table 4 shows the difference in outputs [13].

Table 4: Android and iOS GNSS-sensor output [13].

Android
iOS

Positioning (lat., long.)
√
√

Speed
√
√

Altitude
√
√

√

GPS time
√
√

Number of tracked
√

√

iOS devices do not provide HDOP, number of tracked satellites, and CRN values. Hence, some apps provide constant values.

Figure 2: Samples of NMEA data output logged by the smartphones (Samsung Galaxy S3 (Android platform) and iPhone S5 (iOS platform)) using different applications.

During collection of the observations, each smartphone recorded its own GNSS positioning data at a certain sampling rate (Table 3) using two different dedicated applications that stored the NMEA sentences [14] in a text format. The observations (NMEA data) of each test were analyzed using applications written in MATLAB®.

2.1. Smartphone GNSS Availability

Test A (Outdoor Environment; Static Mode). A location in a parking area (Figure 3) was selected to conduct this test. The two devices were placed on the ground and left for almost 10 minutes to collect data synchronously.

Figure 3: Aerial photo shows the observations that were obtained by the two devices during Test A using different smartphone applications. The pink triangle represents the observation location.

The analysis of the NMEA outputs revealed that Samsung’s applications were giving different numbers of tracked satellites. However, the availability of a 3D positional solution (tracked with a minimum of four satellites [8]) using the Samsung S3 smartphone in the outdoor environment was almost 100% during the static test. Figure 4 shows the number of satellites that were tracked during the observations.

Figure 4: Number of satellites tracked by Samsung’s applications during the 10-minute observational period in Test A.

Test B (Outdoor Environment; Kinematic Mode). This test involved holding the two devices by hand and then walking twice around a loop that started and ended at the same point. Each loop was around 930 m in length and took around 13 minutes to complete. The test track and observational results are shown in Figure 5.

Figure 5: Aerial photo showing the observations that were obtained by the two devices during Test B using different smartphone applications along the same test track. It is noted that App #1 and App #2 (on iPhone) provided the same results and that App #3 and App #4 (on Samsung) provided the same results.

The number of satellites tracked during Test B is shown in Figure 6 in order to assess the availability of 3D positioning when using smartphones in a kinematic mode in an outdoor environment. It is noted that the two Samsung applications were providing different numbers of tracked satellites. However, the availability of 3D positioning [8] using the Samsung S3 in the outdoor environment was 100% during the kinematic test.

Figure 6: Number of satellites tracked by Samsung’s applications during Test B.

Test C (Indoor Environment; Static Mode). The test started outside the shopping center to allow the devices to acquire and receive the GNSS signals. The devices were then taken inside the shopping center to obtain the static measurements. Both devices were placed on a table under the center of the concrete dome and left for around 10 minutes. Figure 7 shows the test location and the observations for each device. Figure 8 displays the number of satellites tracked during this test. Samsung’s applications tracked differing numbers of satellites inside the shopping center.

Figure 7: Aerial photo showing the observations that were obtained by the two devices during Test C using different smartphone applications. The green triangle represents the observation location inside the mall beneath the concrete dome.

Figure 8: Number of satellites tracked by Samsung’s applications during Test C. The observations from 0 to 115 sec reflect the initialization stage of the devices, following which the devices were taken inside the shopping center and placed in static mode.

Test D (Indoor Environment; Kinematic Mode). The same shopping center used in Test C was selected to perform Test D. The acquisition of observations began outside the shopping center under an open sky environment, which allowed the devices to acquire and receive the GNSS signals. After 100 sec, the devices were taken indoors and observations were collected within the shopping center.

The test track inside the mall comprised a rectangular shape that was traced anticlockwise starting from the entrance in three loops. Each loop was around 180 m in length and took 2-3 minutes to complete. Figure 9 shows the test track and the observations that were obtained by the applications on each device.

Figure 9: Aerial photo showing the observations that were acquired by the two devices during Test D using different smartphone applications. The magenta line represents the test track.

Samsung’s applications could track some satellites some of the time (Figure 10), which provided 3D positioning for 30% of the observation time (142 observation records out of a total of 440).

Figure 10: Number of satellites tracked by Samsung’s applications during Test D. The initialization period lasted from 0 to 100 sec and started from a point outside the shopping center, following which the devices were taken inside the shopping center and placed in static mode.

2.2. Smartphone GNSS Accuracy

According to NovAtel’s GPS Position Accuracy Measures report [15], there are various statistical methods that can be used to describe the specifications for GPS receivers. Here, the NMEA outputs of the selected smartphones are analyzed using the most commonly used accuracy measures, that is, 2D and Dilution of Precision (DOP).

2.2.1. Accuracy Measures—2D

Precision and accuracy are often used to describe the position acquired by a GPS receiver [15]. Accuracy is the degree of closeness of an estimate to its true, but unknown value, whereas precision is the degree of closeness of the observations to their arithmetic means [15]. To establish the positional accuracy of the horizontal component of the selected smartphones in static mode (Tests A and C), the following procedures were followed:

(1)The same observation locations for Tests A and C were measured and centimeter-level positional accuracy was obtained as follows:

(i)Test A: obtained using a 7 cm spatial resolution aerial photo (the pink triangle in Figure 3).(ii)Test C: obtained using the shopping center architecture as-built drawing within 3 cm accuracy (the green triangle in Figure 7).

(2)All measurements logged by the two devices were converted from geographical coordinates (latitude and longitude) to local projected coordinates (north and east).(3)The differences in the north and east components between the coordinates of the observational point and the observations were calculated.(4)An application was developed in MATLAB to display the outputs shown in Figures 11 and 12.

Figure 11: Differences in east and north components/horizontal error results from Test A. iPhone was providing an almost constant error value, whereas Samsung was providing differing error values.

Figure 12: Differences in east and north components/horizontal error results from Test C. The observations from 0 to 115 sec correspond to the initialization stage of the devices, following which the devices were taken inside the shopping center and placed in static mode.

In general, the performances of the smartphones in the outdoor area (Test A) were good; the maximum differences in the east and north components were 7.48 and −1.78 m and 6.67 and 3.30 m for the Samsung Galaxy S3 and iPhone 5S, respectively. Table 5 shows the detailed statistic values.

Table 5: Statistics of differences in east and north components from Test A.

App #1
App #2
App #3
App #4

Mean (m)
E
7.48
7.48
2.19
2.19

N
−1.69
−1.78
1.11
1.11

Std dev (m)
E
0.02
0.02
2.65
2.65

N
0.04
0.04
1.23
1.23

Max diff (m)
E
7.48
7.48
6.67
6.67

N
−1.69
−1.78
3.30
3.30

RMSE
E
7.48
7.48
3.44
3.44

N
1.69
1.78
1.66
1.66

The results obtained inside the shopping center during Test C (Figure 12) indicate that the sensors of the Samsung device were affected by missing satellites, and the logged observations were intermittent. However, the statistics that were obtained show that the maximum differences in the east and north components were −56.98 m and −26.95 m, respectively. The iPhone device performed in a different way, and it showed almost fixed positioning error values. However, the maximum differences in the east and north components were 4.40 m and 14.36 m, respectively. Table 6 summarize the most significant statistical parameters.

Table 6: Statistics of differences in east and north components from Test C.

App #1
App #2
App
App

Mean (m)
E
3.21
3.42
−2.43
−2.35

N
14.12
14.15
−1.25
−1.27

Std dev (m)
E
2.37
1.99
4.84
4.43

N
0.72
0.09
2.67
2.65

Max diff (m)
E
4.40
4.40
−56.97
−56.97

N
14.35
14.35
−26.94
−26.94

RMSE
E
4.00
3.96
5.41
5.02

N
14.14
14.15
2.95
2.94

the test, Samsung’s applications provided zero records when it was not possible to obtain observations, which were removed from the statistics.

To estimate the positional accuracy of the horizontal component of the selected smartphones in kinematic mode (Tests B and D), the reference track was determined as the centerline of the observation path according to the following steps, and then the estimated horizontal positioning accuracy was calculated.

(1)The observation track centerline for Tests B and D were obtained as follows:

(i)Test B: digitizing a polyline in ArcGIS (ESRI®) software using a 7 cm spatial resolution aerial photo (the yellow line in Figure 5).(ii)Test D: obtained using the shopping center architecture as-built drawing within 3 cm accuracy (the pink line in Figure 9).

(2)All measurements logged by the two devices were converted from geographical coordinates (latitude and longitude) to local coordinates (north and east).(3)For both tests, the horizontal accuracy was defined as the shortest distance in meters between the observed position and the track centerline, using the mathematical model in Figure 13.(4)An application was developed in MATLAB to display the outputs shown in Figures 14 and 15.

Figure 13: Mathematical model used to estimate the horizontal positioning error, where A and B represent the coordinates for the closest line feature. C is the observation location. is the estimated horizontal error in meter.

Figure 14: Estimated horizontal error from Test B.

Figure 15: Estimated horizontal error from Test D.

The Smartphones Performances in the Outdoor Environment. Kinematic mode (Test B) was good (Figure 14); the maximum estimated horizontal errors were 7.48 m and 6.48 m for the Samsung Galaxy S3 and iPhone 5S, respectively. Table 7 shows the detailed statistic values.

Table 7: Statistics of estimated horizontal error from Test B and Test D.

Test
App #1
App #2
App #
App #

Mean (m)
B
1.85
1.85
2.27
2.27

D
7.92
7.94
8.01
8.01

Std dev (m)
B
1.35
1.34
1.85
1.85

D
5.60
5.57
5.19
5.19

Max diff (m)
B
6.48
6.46
10.67
10.67

D
23.16
23.19
17.74
17.74

RMSE
B
2.29
2.29
2.93
2.93

D
9.69
9.68
9.54
9.54

Test D, Samsung’s applications provided zero records when it was not possible to obtain observations, which were removed from the statistics.

The Results Obtained inside the Shopping Center. Kinematic mode during Test D (Figure 15) indicates that the sensors of both devices were affected by missing satellites. However, the maximum estimated horizontal errors were 17.74 m and 23.19 m for the Samsung Galaxy S3 and iPhone 5S, respectively. The iPhone device performed in a different way, and it showed almost fixed positioning error values. Table 7 summarizes the most significant statistical parameters.

2.2.2. Dilution of Precision (DOP)

DOP is a term used to describe the strength of the satellite configuration on the accuracy of the data collected by the GPS receivers [3, 14]. It represents the volume formed by the intersection points of the user-satellite vectors with the unit sphere centered on the user. Larger volumes give smaller values of DOP, which represent better positional accuracy. However, a lower DOP value does not automatically mean a low positional error. Positional accuracy is affected by other factors such as multipath and ionospheric effects.

According to NovAtel’s report [15], DOP values can be classified as follows: <1 ideal; 1-2 excellent; 2–5 good; 5–10 position could be provided, but the fix quality could be improved; 10–20 positional measurements should be discarded or used only to indicate a very rough estimate of the current location; and >20 poor positional quality.

DOP can be expressed as a number of separate measurements: horizontal DOP (HDOP), vertical DOP, position (3D) DOP, and time DOP (TDOP). The NMEA sentences include the HDOP records [14], and hence the iOS device does not provide HDOP values [13], whilst the HDOP values for the Android device were analyzed for all four tests. Figure 16 shows that the Android smartphone (Samsung Galaxy S3) displays differing HDOP behaviors over time, which are dependent on the environment. Table 8 summarizes the most significant statistical measurements.

Table 8: Statistics of HDOP results from the four tests using the Android device.

Samsung S3

Test
App #3
App #4

Test
App #3
App #4

Mean (m)
A
0.90
0.90
Max diff (m)
A
2.4
2.40

B
0.89
0.89
B
4.8
4.80

C
39.75
39.76
C
300
300

D
3.73
3.73
D
9.6
9.6

Std dev (m)
A
0.30
0.28
RMSE

A
0.95
0.95

B
0.29
0.29
B
0.94
0.94

C
98.12
98.13
C
99.98
99.98

D
1.42
1.42
D
3.99
3.99

Figure 16: HDOP results from the four applications in each test. App #1 and App #2 on the iOS device did not provide HDOP values.

3. Conclusions

This research studied the availability and accuracy of GNSS receivers embedded inside two of the most popular smartphones by analyzing the NMEA output logged by different mobile applications in two different environments. The results show that both devices have the capability of providing positioning to within accepted accuracy [1, 3] in both outdoor and indoor environments. Detailed analysis indicates that the overall performances of the two devices are related directly to the environment, type of smartphone/GNSS chipset, and the application used to collect the information. Thus, smartphone devices with embedded GNSS receivers could help users to define their locations, which can be used in different applications.

In the static tests, the iPhone device provided constant positioning values. After investigation we found that the iOS application developers have the option to either use the “precise standard location service” resulting in high energy consumption or use the “significant update feature” with less precision [16, 17]. Hence, we tested more than 13 applications and all of these, including those that were used in this research to log the NMEA data, were developed based on the “significant update feature” method. This method, proposed by Apple to conserve the device battery, means that the applications can be suspended and then awoken automatically should a significant change in location occur [17].

Further tests and research will be conducted to study the positioning performance obtained by

(1)other smart devices such as smart glasses and smart watches;(2)an iOS application which will be developed based on “standard location service” method.

Competing Interests

The authors have no conflict of interests to declare in respect of this paper.

References

P. Dabove and M. Petovello, “What are the actual performances of GNSS positioning using smartphone technology?” Inside GNSS, pp. 34–37, 2014.

P. A. Zandbergen, “Accuracy of iPhone locations: a comparison of assisted GPS, WiFi and cellular positioning,” Transactions in GIS, vol. 13, no. 1, pp. 5–25, 2009.

A. Klimaszewski-Patterson, “Smartphones in the field: preliminary study comparing GPS capabilities between a smartphone and dedicated GPS device,” in Proceedings of the Applied Geography Conferences, vol. 33, pp. 270–279, Fort Worth, Tex, USA, 2010.

V. Gikas, C. Antoniou, T. Mpimis, H. Perakis, and C. Danezis, “Performance evaluation of vehicle location estimation in heavily obscured environments based on contemporary smartphones,” in Proceedings of the International Conference on Indoor Positioning and Indoor Navigation (IPIN '15), p. 2, Banff, Canada, October 2015.

S. Kos and D. Brčić, “Smartphone application GPS performance during various space weather conditions: a preliminary study,” in Proceedings of the 21st International Symposium on Electronics in Transport (ISEP '13), p. 4, Ljubljana, Slovenia, March 2013.

C. Bauer, “On the (in-)accuracy of GPS measures of smartphones: a study of running tracking applications,” in Proceedings of the 11th International Conference on Advances in Mobile Computing and Multimedia (MoMM '13), pp. 335–340, Vienna, Austria, December 2013.

T. Menard, J. Miller, M. Nowak, and D. Norris, “Comparing the GPS capabilities of the Samsung Galaxy S, Motorola Droid X, and the Apple iPhone for vehicle tracking using FreeSim_Mobile,” in Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems (ITSC '11), pp. 985–990, IEEE, Washington, DC, USA, October 2011.

J.-H. Zhang, B.-H. Li, A. G. Dempster, and C. Rizos, “Evaluation of high sensitivity GPS receivers,” in Proceedings of the International Symposium on GPS/GNSS, p. 6, Taipei, Taiwan, October 2010.

Android, Welcome to the Android Open Source Project!, 2016, http://source.android.com.

Apple Inc, “Welcome to Apple Support, iOS,” 2016, https://support.apple.com.

Apple Inc, Apple Store, 2016, https://itunes.apple.com/app/apple-store/id375380948?mt=8.

Google Inc, “Google Play,” 2016, https://play.google.com/store/apps?hl=en.

P. Handel, I. Skog, M. Ohlsson, and J. Ohlsson, “Smartphone instrumentation for insurance telematics,” in Proceedings of the IEEE International Instrumentation and Measurement Technology Conference (I2MTC '15), pp. 1613–1616, IEEE, Pisa, Italy, May 2015.

National Marine Electronics Association (NMEA), http://www.nmea.org/.

NovAtel, “GPS position accuracy measures,” APN-029 Rev 1, 6 pages, 2003, http://www.novatel.com/assets/Documents/Bulletins/apn029.pdf.

D. Huber, “Background positioning for mobile devices—Android vs. iPhone,” in Proceedings of the Joint Conference of IEEE Computer & Communication Societies, p. 7, 2011.

Apple Inc, “Getting the User's Location, iOS Developer Library,” 2016, https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/LocationAwarenessPG/CoreLocation/CoreLocation.html#//apple_ref/doc/uid/TP40009497.