Improved data transmission technologies have facilitated data collected from positive airway pressure (PAP) devices in the home environment. Although clinicians’ treatment decisions increasingly rely on autoscoring of respiratory events by the PAP device, few studies have specifically examined the accuracy of autoscored respiratory events in the home environment in ongoing PAP use. “PAP efficacy” studies were conducted in which participants wore PAP simultaneously with an Embletta sleep system (Embla, Inc., Broomfield, CO), which was directly connected to the ResMed AutoSet S8 (ResMed, Inc., San Diego, CA) via a specialized cable. Mean PAP-scored Apnea-Hypopnea Index (AHI) was 14.2 ± 11.8 (median: 11.7; range: 3.9–46.3) and mean manual-scored AHI was 9.4 ± 10.2 (median: 7.7; range: 1.2–39.3). Ratios between the mean indices were calculated. PAP-scored HI was 2.0 times higher than the manual-scored HI. PAP-scored AHI was 1.5 times higher than the manual-scored AHI, and PAP-scored AI was 1.04 of manual-scored AI. In this sample, PAP-scored HI was on average double the manual-scored HI. Given the importance of PAP efficacy data in tracking treatment progress, it is important to recognize the possible bias of PAP algorithms in overreporting hypopneas. The most likely cause of this discrepancy is the use of desaturations in manual hypopnea scoring.
Obstructive sleep apnea (OSA) is a chronic medical condition requiring nightly application of therapy to effectively limit the number of apneas and hypopneas that would occur without intervention. The gold-standard treatment for OSA is continuous positive airway pressure therapy (PAP), which provides a pneumatic splint of the soft tissue in the upper airway [
American Academy of Sleep Medicine practice parameters and clinical guidelines recommend routine monitoring of adherence and efficacy data provided by PAP devices as an indication of treatment progress [
A related but different issue concerns the accuracy of the Apnea-Hypopnea Index (AHI), as measured by the PAP unit in the home environment for the purposes of treatment efficacy (i.e., after a period of use). AHI accuracy is particularly important, given the increasing use of and reliance upon PAP data by providers, patients, and intermediaries (i.e., durable medical equipment staff). Ambulatory models of OSA care are gaining popularity, particularly the use of autotitrating PAP devices in lieu of in-laboratory CPAP titrations. In contrast to fixed pressure devices, which simply count the number of apneas occurring while PAP is applied, autoadjusting devices can make pressure changes based on the identification of these disturbances. With an ever-increasing demand for sleep apnea care, the ability to identify patients who may not be therapeutic on their PAP devices is critical. Efficacy of therapy is also an important factor in patient adherence. New technologies allow for data transmission directly from the PAP device to software accessible to the provider and, more recently, to the patients themselves (e.g., SleepMapper, Philips Respironics, Murrysville, PA). A variety of data transmission methods are possible, including the use of a smartcard, wired modem (via telephone line), wireless modem (via cellular network), and, more recently, Bluetooth modems to connect directly into home computers, tablets, or Smartphones. Remote monitoring is a trend within healthcare that is clearly accelerating, and in the sleep field, it facilitates the evaluation of compliance and efficacy of PAP therapy [
Given the improved PAP data transmission technologies and resultant increased use of these data, we sought to investigate the accuracy of the PAP-measured AHI. We had the opportunity to conduct “PAP efficacy” studies in which participants wore PAP devices simultaneously with Type III cardiopulmonary recording equipment. Therefore, the goal of the present study was to specifically examine the accuracy of the identification of apneas and hypopneas by the PAP device.
Twelve research participants from a larger trial evaluating a PAP adherence intervention were included in this study. The PAP adherence intervention study compared a usual care group to a group that was provided with extra education and clinical support via interactive website, phone calls, and in-person clinic visits [
The Embletta (Embla, Inc., Broomfield, CO) was directly connected to the ResMed AutoSet S8 (ResMed, Inc., San Diego, CA) via a specialized cable that allowed for the direct recording of S8 data. Signals recorded include oximetry, chest effort, and body position. Airflow from the PAP device was used for scoring. RemLogic software was used for manual respiratory scoring. Apneas and hypopneas were manually scored according to the 2007 American Academy of Sleep Medicine guidelines, which included defining a hypopnea as being associated with a ≥4% oxygen desaturation [
Descriptive statistics (mean, median, and standard deviation and range) were calculated for the AHI, HI, and AI data. Scatterplots were generated to show the relationship between PAP-scored and manual-scored indices and included the line of identity. Spearman correlation coefficient was calculated. Wilcoxon signed rank test was used to test mean difference between the indices, and concordance correlation coefficients [
Participants were primarily overweight, middle aged, and sleepy with moderate to severe OSA (see Table
Baseline characteristics.
Mean ± SD | Range | |
---|---|---|
Age | 62.0 ± 12.3 | 46–78 |
Body Mass Index (BMI) | 30.4 ± 5.9 | 21.5–41.7 |
Apnea-Hypopnea Index (AHI) | 42.9 ± 21.5 | 11.3–76.9 |
Epworth Sleepiness Scale (ESS) | 14.2 ± 3.9 | 8.0–19.0 |
Mean PAP-scored AHI was 14.2 ± 11.8 (median: 11.7, range: 3.9–46.3) and mean manual-scored AHI was 9.4 ± 10.2 (median: 7.7, range: 1.2–39.3) (see Table
OSA variables measured on efficacy study.
Manual scoring | Autoscoring | |||
---|---|---|---|---|
Mean ± SD | Range | Mean ± SD | Range | |
Apnea-Hypopnea Index (AHI) | 9.4 ± 10.2 | 1.2–39.3 | 14.2 ± 11.8 | 3.9–46.3 |
Apnea Index (AI) | 5.1 ± 7.9 | 0–28.4 | 5.3 ± 7.7 | 0.20–27.6 |
Central Apnea Index (CAI) | 2.6 ± 6.6 | 0–23.5 | — | — |
Obstructive Apnea Index (OAI) | 1.5 ± 2.3 | 0–7.7 | — | — |
Mixed Apnea Index (MAI) | 0.9 ± 1.3 | 0–4.6 | — | — |
Hypopnea Index (HI) | 4.3 ± 3.0 | 0.5–10.9 | 8.8 ± 4.9 | 3.1–18.7 |
Oxygen Desaturation Index (ODI) | 7.7 ± 7.4 | 0.6–27.2 | — | — |
Concordance correlation coefficient is 0.87 (95% CI: 0.71–0.95) for AHI, 0.994 (95% CI: 0.98–0.998) for AI, and 0.50 (95% CI: 0.23–0.70) for HI. Based on published guidelines, the agreement between PAP and manual scoring is considered “poor” in AHI and HI and is “almost perfect” for AI.
Ratios between the mean indices were calculated. The PAP-scored HI was 2.0 times higher than the manual-scored HI, the PAP-scored AHI was 1.5 times higher than the manual-scored AHI, and the PAP-scored AI was 1.04 of the manual-scored AI. It appears that the PAP device evaluated in this study, relative to manual scoring, only slightly overscored the number of apneas but significantly overscored the number of hypopneas. The difference in scoring of hypopneas seems to be the main contributor to the different AHI values between the PAP device and manual scoring.
Two graphical displays of the data were created. Figures
(a) Scatterplot of PAP-scored versus manual-scored Apnea Index. Each point represents one night. The diagonal line represents the line of identity. (b) Scatterplot of PAP-scored versus manual-scored Hypopnea Index. Each point represents one night. The diagonal line represents the line of identity. Note that all PAP HI values are greater than manual HI values. (c) Scatterplot of PAP-scored versus manual-scored Apnea-Hypopnea Index. Each point represents one night. The diagonal line represents the line of identity. Note that all values of PAP AHI are greater than manual AHI values.
(a) Bland-Altman plot of PAP scoring versus manual scoring AI difference (PAP-scoring minus manual-scoring) by the mean. (b) Bland-Altman plot of PAP scoring versus manual-scoring HI difference (PAP scoring minus manual-scoring) by the mean. Note that all of the difference values were greater than 0, indicating that all PAP HI values were greater than the manual HI values. (c) Bland-Altman plot of PAP-scoring versus manual-scoring AHI difference (PAP scoring minus manual-scoring) by the mean. Note that all of the difference values were greater than 0, indicating that all PAP AHI values were greater than the manual AHI values.
The practice of sleep medicine is evolving, and ambulatory models of sleep apnea management using home sleep testing and APAP therapy are not only noninferior to traditional evaluations but are also gaining wider acceptance by sleep providers [
In this study of home-based PAP efficacy, as measured by the S8 APAP device, the PAP-scored HI was on average more than double the manual-scored HI. Given the importance of PAP efficacy data in tracking treatment progress, it is important to recognize that this particular APAP device may overscore hypopneas. The most likely causes of this discrepancy are (a) the use of a proprietary algorithm and (b) the use of desaturations in manual hypopnea scoring. Because the number of apneas was underscored relative to manual scoring, the overall AHI does not appear to be different from manual scoring. This study and the evolving literature in this area suggest that it is important to understand how a specific PAP device identifies both apneas and hypopneas.
One previous study that used the S8 device also found relatively good apnea measurement but an overscoring of hypopneas [
Other studies in this area have utilized the RemStar autoadjusting PAP device by Philips Respironics. These study results show a different pattern, specifically that respiratory event detection varies based on the number of events. For example, RemStar-measured AHI tended to overestimate the AHI at lower AHI levels but underestimate the AHI at higher AHI levels [
If there are systematic differences between PAP devices, it is important for the field to request that the manufacturers provide clinicians and researchers with clear information regarding what level of adjustment is necessary to allow for the most accurate interpretation of the PAP-scored apneas and hypopneas. The PAP-scored AHI value is a useful data point for gathering information on therapeutic efficacy. Previous studies have examined the percentage of patients that continue to have residual OSA even while using a PAP device. In a study of patients using single-pressure CPAP, nearly 20% continued to have PAP AHI >10 after 3 months [
As per published clinical guidelines, the standard recommendation is that sleep monitoring is indicated for the assessment of treatment results on PAP therapy after (i) substantial weight loss (e.g., 10% of body weight) to ascertain whether PAP therapy is still needed at the prescribed pressure settings, (ii) substantial weight gain with return of symptoms (e.g., 10% of body weight) to ascertain whether pressure adjustments are needed, (iii) clinical response is insufficient (e.g., lack of symptom relief, above normal residual AHI, or poor adherence), or (iv) symptoms return despite a good initial response to CPAP [
There are a number of potential study limitations. First, the number of participants is low relative to other studies in this area. That said, the results are consistent with the only other study that used the S8 APAP device. Second, the study was done in the home environment where there is less control than that in a laboratory environment. Third, there are limitations in scoring hypopneas with Type III home testing (events must meet flow and ≥4% saturation criteria but cannot be scored based on an arousal, as sleep is not measured). Finally, newer generation of PAP devices can distinguish central and obstructive events. Future studies can utilize this newer technology to see how well PAP algorithms can distinguish these events and what role they have in overall AHI values.
In summary, PAP devices have automated, proprietary algorithms for respiratory event detection. When event detection scoring is combined with PAP use duration in the denominator, a proxy AHI value is derived. Given the increased reliance on the PAP-scored events by both providers and patients, it is important to better understand the nuances of specific algorithms and how the PAP-scored AHI, HI, and AI values compare to those same values from manual scoring. Doing so is an important step toward making more informed treatment decisions.
This research was supported in part by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, and the Research Service of the San Diego Veterans Affairs Healthcare System (VA HSRD IIR 02-275 and 07-163 and AHRQ R18 017246).