Heart rate variability (HRV) is a useful clinical tool for autonomic function assessment and cardiovascular diseases diagnosis. It is traditionally calculated from a dedicated medical electrocardiograph (ECG). In this paper, we demonstrate that HRV can also be extracted from photoplethysmograms (PPG) obtained by the camera of a smartphone. Sixteen HRV parameters, including timedomain, frequencydomain, and nonlinear parameters, were calculated from PPG captured by a smartphone for 30 healthy subjects and were compared with those derived from ECG. The statistical results showed that 14 parameters (AVNN, SDNN, CV, RMSSD, SDSD, TP, VLF, LF, HF, LF/HF, nLF, nHF, SD1, and SD2) from PPG were highly correlated (
The heart rate (HR) of human is not constant but varies from one heartbeat to the next. Heart rate variability (HRV) is the physiological phenomenon of tiny fluctuations in the time intervals between heartbeats. It reflects the tenseness and the balance of the sympathetic and the vagus nerve activities and their effects on cardiovascular motion [
HRV is traditionally determined by digital processing of electrocardiograms (ECG). The Rwave peaks of QRS complexes in ECGs are detected by computer algorithms and RtoR intervals (RRI) are calculated. Then, HRV parameters are computed using timedomain, frequencydomain, and nonlinear methods [
Mobile phones have already shown promising applications in healthcare service [
However, to our best knowledge, the extraction of HRV from the smartphone PPG signals has not been well investigated, especially compared with ECG—the golden standard. Therefore, we comprehensively studied the extraction of HRV from smartphone PPG signals and compared the results with an ECG in order to assess the accuracy. Specifically, we used five algorithms to detect the characteristic points of the smartphone PPG signals: peak point (PP), valley point (VP), maximum first derivative (M1D), maximum second derivative (M2D), and tangent intersection (TI). The performances of these algorithms were also compared.
The experiment was approved by the Institutional Review Board of Shenzhen Institutes of Advanced Technology (registration number: SIATIRB140215H0040). Thirty subjects participated in the experiment (20 males and 10 females, age: 20–32 years, height: 150–183 cm, and weight: 40–90 kg). All the subjects were healthy and provided their informed consent. They were asked to refrain from caffeine, alcohol, cigarette, or strenuous exercise for 2 hours prior to the study.
In the experiment, all the subjects were instructed to lie in the supine position on a mattress and place their right index finger on the camera lens of an HTC S510e smart phone with the builtin LED flash turned on. A camera application (APP) in the smart phone was used to record the video of the fingertip with a resolution of 320 × 240 pixels at an unfixed sampling rate of 20–30 frames per second (fps). The sampling rate is unfixed due to the CPU processing load. ECG electrodes in the standard configuration were attached to the subjects to measure the ECG signals with a Finometer MIDI (Model II, Finapres Medical Systems B.V., The Netherlands). The ECG signals were digitalized at 200 Hz and automatically stored in the computer by BeatScope Easy software (Finapres Medical Systems B.V., The Netherlands). The experiment lasted at least 5 minutes for each subject and the subject was asked to keep still during this period.
All the data were processed offline. The 3GP format videos recorded by the HTC S510e smart phone were converted into AVI format using Pazera Free 3GP to AVI Converter 1.3 (). All further analysis was performed on the AVI videos in MatLab 7.0 (The Mathworks Inc., USA).
First, an 80 × 80 pixel region in the center of the video image was selected as the region of interesting (ROI). Then, the average intensity of the red channel in the ROI for each individual frame was calculated to generate a timeseries waveform (the raw PPG signal). The red channel was chosen because the intensity values of the green and blue channels were often tending to zero and contained no valuable information in most situations. As the smartphone PPG worked in the reflection mode, the generated waveform should be inverted to “normal” mode for further processing [
The raw PPG signals were often corrupted by random noise, baseline drifting, and baseline abrupt changes (increase/decrease). Baseline abrupt changes were possibly caused by sudden moves of the finger or sudden changes of the light illumination, or by other unknown reasons. They could not be completely removed by general digital filters as they contained wideband frequency components. We used a statistics method to solve this problem. First, we calculated the difference of the raw signal and then removed the outliers out of the range mean ± 5 × standard deviation (SD) and interpolated new values using cubic spline interpolation. At last, we reconstructed the new PPG signal by summation, the inverse of the difference. The range mentioned above was determined empirically, which meant that the probability of the outliers was 5.7330 × 10^{−7} if the difference of the PPG was normally distributed. It was the best range according to our data and could be adjusted if required.
The random noise and baseline drifting were reduced by a zerophase Butterworth lowpass filter with cutoff frequency of 10 Hz and a zerophase Butterworth highpass filter with cutoff frequency of 0.5 Hz, respectively. Zero phase filters were implemented by filtering the signal both forward and backward to eliminate phase distortion.
The PPG signals were then resampled to 800 Hz with cubic spline interpolation to increase the temporal resolution. For each cardiac circle, five algorithms were used to obtain the pulsetopulse interval (PPI) by detection of five different characteristic points, as illustrated in Figure
The ECG signals were first passed through a finite impulse response (FIR) lowpass filter with cutoff frequency of 11 Hz and then a FIR highpass filter with cutoff frequency of 5 Hz to reduce most of the noise and interference [
Table
Commonly used HRV parameters.
Parameter  Description 



AVNN  Average of all NN intervals 
CV  Coefficient of variation of NN intervals. The ratio of the standard deviation to the mean. 
SDNN  Standard deviation of all NN intervals. 
SDANN  Standard deviation of the averages of NN intervals in all 5minute segments of the entire recording. 
RMSSD  Root mean square of successive differences between adjacent NN intervals. 
SDSD  Standard deviation of successive differences between adjacent NN intervals. 
NN50  Number of pairs of successive NN intervals that differ by more than 50 minutes. 
pNN50  Proportion of NN50 divided by total number of NN intervals. 


TP  Total power (≤0.4 Hz) 
VLF  Very low frequency power (≤0.04 Hz) 
LF  Low frequency power (0.04–0.15 Hz) 
HF  High frequency power (0.15–0.4 Hz) 
LF/HF  Ratio of LF to HF 
nLF  Normalized LF = LF/(TP − VLF) 
nHF  Normalized HF = HF/(TP − VLF) 


SD1  Standard deviation of short diagonal axis in Poincaré plot 
SD2  Standard deviation of long diagonal axis in Poincaré plot 
HRV parameters derived from smartphone PPG were compared with the corresponding parameters derived from ECG. The Pearson correlation coefficients were calculated and the linear regression equations were obtained. A
The agreement between the two devices (smartphone and ECG) was assessed using BlandAltman method [
Table
Pearson’s correlation coefficients and linear regression equations between HRV parameters derived from the smartphone and the electrocardiograph.
Parameter  PP  VP  M1D  M2D  TI 

AVNN (ms)  1.000 ( 
1.000 ( 
1.000 ( 
1.000 ( 
1.000 ( 


SDNN (ms)  0.722 ( 
0.902 ( 
0.933 ( 
0.859 ( 
0.916 ( 


CV (%)  0.703 ( 
0.881 ( 
0.920 ( 
0.826 ( 
0.900 ( 


RMSSD (ms)  0.596 ( 
0.713 ( 
0.780 ( 
0.629 ( 
0.731 ( 


SDSD (ms)  0.596 ( 
0.713 ( 
0.780 ( 
0.630 ( 
0.732 ( 


NN50  0.254 ( 
0.285 ( 
0.292 ( 
0.081 ( 
0.391 ( 


pNN50 (%)  0.415 ( 
0.508 ( 
0.513 ( 
0.513 ( 
0.513 ( 


TP (ms2)  1.000 ( 
0.999 ( 
1.000 ( 
1.000 ( 
1.000 ( 


VLF (ms2)  0.996 ( 
0.995 ( 
0.998 ( 
0.998 ( 
0.998 ( 


LF (ms2)  0.992 ( 
0.989 ( 
0.996 ( 
0.996 ( 
0.996 ( 


HF (ms2)  0.993 ( 
0.990 ( 
0.996 ( 
0.997 ( 
0.996 ( 


LF/HF  0.963 ( 
0.967 ( 
0.982 ( 
0.984 ( 
0.981 ( 


nLF (%)  0.968 ( 
0.967 ( 
0.982 ( 
0.988 ( 
0.981 ( 


nHF (%)  0.977 ( 
0.985 ( 
0.986 ( 
0.992 ( 
0.988 ( 


SD1 (ms)  0.596 ( 
0.713 ( 
0.780 ( 
0.630 ( 
0.732 ( 


SD2 (ms)  0.920 ( 
0.986 ( 
0.989 ( 
0.978 ( 
0.988 ( 
Table
BlandAltman analysis of HRV parameters derived from the smartphone and the electrocardiograph.
Parameter  PP  VP  M1D  M2D  TI 

AVNN (ms)  −0.05 ± 0.68^{*} 
−0.12 ± 0.54^{*} 
−0.06 ± 0.55^{*} 
−0.05 ± 0.55^{*} 
−0.09 ± 0.51^{*} 


SDNN (ms)  28.39 ± 31.26 
18.40 ± 15.48 
13.03 ± 12.85 
22.65 ± 12.85 
12.76 ± 14.37 


CV (%)  2.95 ± 2.96 
1.99 ± 1.81 
1.41 ± 1.48 
2.45 ± 1.48 
1.39 ± 1.66 


RMSSD (ms)  67.84 ± 61.35 
46.77 ± 35.77 
33.96 ± 29.77 
57.44 ± 29.77 
32.95 ± 33.85 


SDSD (ms)  67.95 ± 61.47 
46.84 ± 35.83 
34.01 ± 29.82 
57.53 ± 29.82 
33.00 ± 33.91 


NN50  57.57 ± 57.65 
45.53 ± 53.24 
41.20 ± 54.79 
57.87 ± 54.79 
37.67 ± 52.02 


pNN50 (%)  17.88 ± 17.79 
13.87 ± 15.82 
12.47 ± 15.82 
12.47 ± 15.82 
12.47 ± 15.82 


TP (ms2)  23.82 ± 63.56 
14.55 ± 75.59 
20.67 ± 56.93^{*} 
0.52 ± 56.93^{*} 
17.10 ± 53.88^{*} 


VLF (ms2)  6.30 ± 81.19 
8.08 ± 93.97 
4.25 ± 67.45^{*} 
4.36 ± 67.45^{*} 
4.10 ± 67.97^{*} 


LF (ms2)  −1.22 ± 113.36 
−7.05 ± 128.70 
−1.77 ± 81.75^{*} 
−5.67 ± 81.75^{*} 
−3.39 ± 78.84^{*} 


HF (ms2)  18.69 ± 75.18^{*} 
16.59 ± 88.85^{*} 
17.98 ± 55.56^{*} 
2.71 ± 55.56^{*} 
17.12 ± 60.00^{*} 


LF/HF  −0.09 ± 0.34 
−0.10 ± 0.33 
−0.09 ± 0.26 
−0.06 ± 0.26 
−0.09 ± 0.26 


nLF (%)  −1.50 ± 5.75^{*} 
−1.30 ± 6.56^{*} 
−1.36 ± 4.89^{*} 
−0.60 ± 4.89^{*} 
−1.28 ± 4.93^{*} 


nHF (%)  1.37 ± 5.68^{*} 
1.25 ± 5.10^{*} 
1.23 ± 4.53^{*} 
0.62 ± 4.53^{*} 
1.20 ± 4.28^{*} 


SD1 (ms)  48.05 ± 43.47 
33.12 ± 25.33 
24.05 ± 21.08 
40.68 ± 21.08 
23.33 ± 23.97 


SD2 (ms)  14.35 ± 17.58 
8.98 ± 7.49 
6.54 ± 6.51^{*} 
10.50 ± 6.51 
6.47 ± 6.74^{*} 
Data are presented as bias ± 1.96 standard deviation (SD). ^{*}Bias ± 1.96 SD within the acceptable limits. BAR: BlandAltman ratio, PP: peak point, VP: valley point, M1D: maximum first derivative, M2D: maximum second derivative, and TI: tangent intersection. HRV parameters are explained in Table
An example of outlier removal. (a) A raw smartphone photoplethysmogram with abrupt change. (b) The difference of the signal in panel (a). The circle shows the location of the outlier. (c) The outlier was removed and replaced with a new value using cubic spline interpolation. (d) The new smartphone photoplethysmogram without abrupt change.
Illustration of five characteristic points including A, the peak point; B, the valley point; C, the maximum first derivative; D, the maximum second derivative; and E, the tangent intersection.
Comparison of HRV derived from the smartphone and the electrocardiograph for one subject. (a) RtoR intervals (RRI) derived from the electrocardiogram. (b)–(f) Pulsetopulse intervals (PPI) derived from the smartphone photoplethysmogram, using the characteristic points determined by (b) peak point, (c) valley point, (d) maximum first derivative, (e) maximum second derivative, and (f) tangent intersection.
As shown in Table
BlandAltman plots of HRV parameters derived from the smartphone and the electrocardiograph. For each plot, the horizontal axis represents the mean of HRV parameters derived from smartphone and electrocardiograph, while the vertical axis represents the difference between HRV parameters derived from smartphone and electrocardiograph. The five columns correspond to five different algorithms: PP, peak point; VP, valley point; M1D, maximum first derivative; M2D, maximum second derivative; and TI, tangent intersection. LF, low frequency power; HF, high frequency power; LF/HF, ratio of LF to HF; nLF, normalized LF = LF/(TP − VLF); and nHF, normalized HF = HF/(TP − VLF).
In terms of both bias and SD, we analyzed these data satisfying the condition BAR < 20% in Table
Comparison of five algorithms for detection of characteristic points.
Parameter  PP  VP  M1D  M2D  TI 

AVNN  
Bias  *  —  —  *  — 
SD  —  —  —  *  * 
TP  
Bias  —  *  —  *  — 
SD  —  —  —  *  * 
VLF  
Bias  —  —  *  —  * 
SD  —  —  *  *  — 
LF  
Bias  *  —  *  —  — 
SD  —  —  —  *  * 
HF  
Bias  —  *  —  *  — 
SD  —  —  *  *  — 
LF/HF  
Bias  —  —  —  *  * 
SD  —  —  —  *  * 
nLF  
Bias  —  —  —  *  * 
SD  —  —  *  *  — 
nHF  
Bias  —  —  —  *  * 
SD  —  —  —  *  * 
SD2  
Bias  —  —  *  —  * 
SD  —  —  *  —  * 


Total stars  2  2  7  14  11 
PP: peak point, VP: valley point, M1D: maximum first derivative, M2D: maximum second derivative, and TI: tangent intersection. SD: standard deviation. HRV parameters are explained in Table
As previously mentioned, no researches have been reported to measure HRV with smartphone PPG. Nevertheless, many researches have been reported to measure HRV with traditional PPG (tPPG, i.e., a pulse oximeter), which can provide valuable information for our work. A good review of tPPGderived HRV can be found in [
The detection of the characteristic points is also an impact factor for the accuracy of smartphonederived HRV measurement [
A possible consideration of the smartphonebased HRV analysis is the sampling rate. Our HTC smartphone has a sampling rate of approximately 20–30 Hz that may be considered not suitable for HRV analysis. In fact, the spectrum of the pulse signals has the vast majority power in the range of 0~10 Hz [
The color channel should be also considered. In the processing of the recorded video, most previous studies calculated the intensity of the green channel in RGB color model [
Motion artifacts are another complicated problem and are tough to deal with. To our best knowledge, none of the reported studies have solved this problem very well. In our experiments, the subjects were instructed to lie on a mattress and keep their fingers as still as possible to minimize the motion artifacts. This is not practical in daily life as shorttime HRV testing usually takes 5 minutes that seems so long time for keeping still. Therefore, efficient motionresistant algorithms are required. Several motion artifacts detection algorithms in pulse oximeters could be applied in smartphonebased HRV analysis [
Traditional ECG recordings require electrodes attached to body surface and are operated by specially trained nurses in the hospital. The new smartphonebased technology requires no more than placing a finger on the camera lens of a smartphone. It is lowcost and easytouse and can be used in daily life out of hospital. In the present study, we quantitatively investigated the measurement of HRV based on smartphone technology and compared the results with those derived from a standard ECG to assess the accuracy. The results suggest that the smartphone can be of potential use for HRV measurement at resting and would be applied in lowcost healthcare applications.
The authors declare that they have no conflict of interests.
This work was supported in part by the National Basic Research Program 973 (no. 2010CB732606), the National Natural Science Foundation of China (no. 61401453), the Guangdong Innovation Research Team Fund for Lowcost Healthcare Technologies in China, the External Cooperation Program of the Chinese Academy of Sciences (no. GJHZ1212), the Key Lab for Health Informatics of Chinese Academy of Sciences, the Peacock Program to Attract Overseas HighCaliber Talents to Shenzhen, and the Shenzhen City Government (nos. KFJEWSTS097 and KFJEWSTS095).