Validation of Sequential ROX-Index Score Beyond 12 Hours in Predicting Treatment Failure and Mortality in COVID-19 Patients Receiving Oxygen via High-Flow Nasal Cannula

Background High-flow nasal cannula (HFNC) is an oxygen delivery method shown to reduce the risk of intubation and mortality in patients with type 1 respiratory failure. The ROX-index score can predict HFNC failure. This study aims to evaluate sequential ROX-index assessments as predictors of HFNC failure and mortality. Methods Prospective observational single-center study including all adult patients with positive SARS-CoV-2 PCR placed under HFNC from 1st November 2020 to 31st May 2021, and patients with hemodynamic instability or unable to tolerate HFNC were excluded. The primary endpoint was successful HFNC de-escalation. Results In univariate analysis, HFNC de-escalation was associated with younger age (59.2 ± 14 vs. 67.7 ± 10.5 and p < 0.001), lower levels of serum lactate (1.1 vs. 1.5 and p=0.013), and higher ROX-index at 12 hrs (5.09 vs. 4.13 and p < 0.001). ROC curve analysis of ROX-index at 12 hrs yielded a c-statistic of 71.2% (95% CI 61.6–80.9 and p < 0.001). ROX-index at 12 hrs and age retained significance in multivariate analysis. Using an optimal cutoff point of 4.43, we calculated a sensitivity of 64.5% and specificity of 69.6%. In univariate survival analysis, older age (68.8 ± 9.7 vs. 58.9 ± 13.9 and p < 0.001), greater creatinine values (0.96 vs. 0.84 and p=0.022), greater SOFA score (p=0.039), and a lower 12 hrs ROX-index (4.22 vs. 4.95 and p=0.02) were associated with hospital mortality. The SOFA score and age retained significance in multivariate survival analysis. Conclusion ROX-index is proven to be a valuable and easy-to-use tool for clinicians in the assessment of COVID-19 patients under HFNC.


Introduction
Until December 2019, treating patients with type 1 hypoxemic respiratory failure was a sporadic problem involving a limited range of specialties. Te emergence of the novel coronavirus SARS-CoV-2 brought the whole medical society face-to-face with concepts, such as the decision for intubation and forced health professionals to quickly become accustomed to optimal oxygen therapy methods. Te pressure of the Coronavirus disease 2019 (COVID-19) pandemic on even the most advanced health systems has led medical staf to increasingly use various oxygen therapy delivery systems and noninvasive ventilation (NIV) methods to provide optimal patient care amid a massive increase in the number of intubated patients [1].
Te high-fow nasal cannula (HFNC) is a device that delivers high mixtures of oxygen (up to 100%), heated and humifed at a maximum fow of 60-80 liters per minute, through a nasal cannula. Originally used in neonatal units, the HFNC is one of the most promising oxygen therapy methods used to treat adult patients with hypoxemia, due to its benefcial efects on the respiratory system, which have been extensively studied over the past decade [2]. In addition to reducing the work of breathing through generating low levels of positive airway pressure, reducing dead space and airway resistance, HFNC has been shown to be better tolerated by hypoxemic patients, with minimal and quickly reversible adverse efects, compared to face masks use [3]. Its ease of use and the capability of immediate improvement of PaO 2 and respiratory rate upon its application, make it a favorable method of oxygen therapy, especially in lowresources environments as it reduces the risk of intubation, the total days under mechanical ventilation, and the total mortality of hypoxemic patients [4].
However, a main concern of HFNC application is the potential delay of intubation which is associated with increased mortality [5]. Te ROX-index, which efectively refers to the ratio of oxygen saturation measured by pulse oximetry (SpO 2 )/fraction of inspired oxygen (FiO 2 ) to the respiratory rate, is an accepted score that emerged during the recent years and has shown its value as a predictor of HFNC failure in patients with hypoxemia [6,7]. A value greater than or equal to 4.88 at 12 hours after HFNC onset has a sensitivity of 70.1% and a specifcity of 72.4% to predict HFNC failure. Te main advantage of the ROX-index is its clinical score form without the need of lab results nor complex calculation methods. Te ease of use fts the nature of hypoxemic COVID-19 patients care, as it gives the capability of quick and reliable assessment of multiple critically ill patients.
Nevertheless, the physicians' question, when to proceed to endotracheal intubation, remains. Te aim of this study is to assess the ROX-index, along with other laboratory and clinical parameters, as prediction tools for the failure of HFNC in COVID-19 patients with hypoxemic respiratory failure to identify early those who may require invasive mechanical ventilation (IMV). We present the following article/case in accordance with the STROBE reporting checklist.

Study Design and Outcomes.
Tis is an analysis of prospectively collected observational data from consecutive patients placed on HFNC (Airvo ™ 2 by Fisher-Paykel) during hospitalization in the COVID-19 department of our hospital. Our primary endpoint was HFNC de-escalation, defned as HFNC withdrawal with improved oxygenation, no need for NIV and/or IMV. Secondary outcomes were the length of stay under HFNC treatment and overall, inhospital mortality. Initial HFNC settings were for all patients 60 liters/min and 90% FiO 2 with an SpO 2 target of 92-96% and further titrated downwards based on needs after 12 hrs.
Tis study was approved by the Laiko General Hospital Scientifc and Ethics Review Board (protocol number: 376 / 19-5-20). A written informed consent was obtained by all participants in the study.

Study Population.
Te total recruiting period was 7 months (from 1st November 2020 to 31st May 2021) encompassing Greece's 2nd and 3rd pandemic waves. Te inclusion criteria were as follows: (a) age >18 years, (b) positive PCR test for SARS-COV-2 RNA, and (c) treatment with application of HFNC due to respiratory failure type I not responding to 10 L low-fow nasal cannula or up to 15 L and 60% FIO 2 oxygen mask (our hospital did not have the option of noninvasive ventilation since the same devices used for it, were also used for mechanical ventilation and shortage of ventilators was a signifcant problem at the time). Te exclusion criteria were as follows: (a) hemodynamic instability, (b) facial injuries preventing application of HFNC, (c) type II respiratory failure, and (d) patient inability to cooperate with HFNC.

Measurements.
We recorded demographic data, relevant medical history, smoking habits, and laboratory and respiratory values on admission and on HFNC application and on several time-points thereafter (at 12 hours and on days 3, 5, 7, and 14). Sequential organ failure assessment (SOFA) [8] and ROX-index scores were calculated for these time-points. We also recorded information pertaining to treatment administered to these patients, prone positioning, HFNC treatment, and duration of symptoms and hospitalization. Bacterial pneumonia was defned as the presence of lobar pneumonia on x-rays and/or CTs as well as isolation of bacteria from sputum.

Statistical Analysis.
Descriptive statistics are presented as counts (%) for categorical variables and as medians (25th-75th percentile) for non-normally distributed continuous variables or as means ± standard deviation (SD) for normally distributed continuous variables. Te normality of distribution was examined using the Kolmogorov-Smirnov test. Group comparisons were performed using the student's t-test and Mann-Whitney or Wilcoxon signed-rank test for normally and nonnormally distributed variables, respectively, chi-square for categorical variables and Spearman correlation for continuous variables' relationships. Multivariate analyses were performed using logistic regression to assess the probability of HFNC failure and Cox proportional hazards regression to assess hospital mortality. All variables with statistical signifcance, as defned by a p < 0.05 in the univariate analysis, were included. Te results of the Cox model are presented as hazard ratios (HRs), while the results of the logistic regression as odds ratios (ORs), both with 95% confdence intervals (CIs) and with a statistical signifcance for p < 0.05. Finally, the validation of the ROX-index was performed using a received operating characteristic (ROC) curve analysis. Te analysis was performed using SPSS Statistics for Windows, version 25.0 (2017, Armonk, NY, IBM Corp.).

Results
During the study period 1,115 patients were hospitalized in our COVID department for a total of 12,041 patient-days. Of these, 116 patients (10.4%) had critical type I respiratory failure and needed treatment with HFNC giving an incidence of 9.63 HFNC applications per 1000 patient-days. Five patients were unable to tolerate treatment with HFNC and were excluded from the statistical analysis. Our fnal analysis sample included 111 patients with mean age 62.8 ± 13.3 years. Most of our participants were male (70/111 and 63.1%) and 89.2% (99/111) of Greek descent. Baseline characteristics of our study population are presented in Table 1.
Te primary outcome was met in 57.7% (64/111) of the participants. In univariate analysis, the only predictors of HFNC de-escalation were younger age (59.2 ± 14 vs. 67.7 ± 10.5 and p < 0.001), lower levels of serum lactate both on hospital admission (1.1 vs. 1.5 and p � 0.013), and 12 hours after HFNC was applied (1.3 vs. 1.5 and p � 0.007) and higher ROX-index scores on the 12-hour mark (5.09 vs. 4.13 and p < 0.001). A comparison between those who met the primary endpoint of HFNC de-escalation and those who did not is presented in Table 2. In multivariate analysis (Table 3), all three variables retained statistical signifcance.
ROX-index was calculated at 12 hours post-HFNC application and again at days 2 (n � 100), 3 (n � 84) and 7 (n � 35). We performed pair-wise comparisons (12 hr vs. d2, d2 vs. d3, d3 vs. d7, 12 hr vs. d3, and 12 hr vs. d7) of ROXindex scores using the Wilcoxon signed-rank tests. In the subgroup of patients that were not successfully de-escalated, there was no statistically signifcant diference in the median ROX-index scores at these timepoints (4.13 vs. 3.99 vs. 4.25 vs. 4.62). In the subgroup of patients that met the primary endpoint, however, there was signifcant improvement in scores when performing comparisons between the 12 hrmark and day 7 (5.09 (4.05-5.87) vs. 5.67 (4.95-7.08) and p � 0.016), between day 2 and day 7 (5.38 (4.26-6.39) vs. 5.67 (4.95-7.08) and p 0.017) and between day 3 and day 7 (5.16 (4.08-7.11) vs. 5.67 (4.95-7.08) and p � 0.005). Tere was also improvement between the 12 hr-mark and day 2, which marginally did not meet statistical signifcance (5.09 vs. 5.38 and p � 0.073). If we calculate the diference between ROX-index scores between these timepoints, we can derive a measure of ROX-index progression, which we can name DeltaROX. DeltaROX calculated between Days 7 and either day 2 or day 3 was found to have signifcantly diferent values between the subgroup of patients that was deescalated and those that did not (0.68 vs −0.87 and p � 0.016 and 0.92 vs −0.5, p � 0.008, for day 7-day 2 and day 7day 3, respectively). No other calculated deltas had statistical signifcance.

Discussion
Te current prospective study evaluated the ROX index and related factors as prediction tools for HFNC failure in COVID-19 patients with hypoxemic respiratory failure. We tried to include various factors to our analysis, some of which had not been examined as predictors for HNFC failure such as treatment with IL-6 receptor inhibition or complications such as pulmonary embolism and HAP. We also explored the ROX-index value as a daily assessment tool, through its calculation at 12 h and on days 2, 3, and 7 after HFNC initiation. Our fndings indicate that the ROXindex is a valid predictor for HFNC failure and that the upturn of its values while under HFNC can serve as an indicator for successful de-escalation.
Troughout the COVID-19 pandemic, a major struggle for physicians treating COVID-19 patients with hypoxemic respiratory failure is the timing of endotracheal intubation and mechanical ventilation [9]. While the correlation of delaying intubation with higher mortality is evident from studies both preceding and following the COVID-19 outbreak [10,11], shortage of ICU beds, ventilators and trained staf, even in developed countries, and presses physicians outside ICU to prolong patients' stay under HFNC/NIV treatment [9]. On the other hand, a recent meta-analysis comparing the outcomes of late and early intubations, showed no mortality beneft of either option [12]. Furthermore, endotracheal intubation is a procedure with multiple risks for COVID-19 patients and healthcare workers alike, while sedation and mechanical ventilation involve various complications [13]. Te HFNC is proven to be a safe and efective way of oxygen supplementation for patients with severe hypoxemic respiratory failure [4]. Mellado-Artigas et al., demonstrated that HFNC might decrease ventilator days, ICU length of stay, and all-causehospital mortality of COVID-19 patients [14]. However, the implementation of HFNC as an alternative to mechanical ventilation could be a factor delaying escalation of care, resulting in unfavorable outcomes. Tus, the need for reliable and usable predictors of HFNC treatment failure is vital for aiding clinical judgement as well as managing available resources.
In this context, the ROX-index is turning to be one of the main focuses of researchers during the pandemic [6]. According to our results, the ROX value after 12 h in HFNC is a fair predictor of failure (AUC � 0.71) with a cutof point of 4.43. It is also important to point out that a ROX-index value of 4.88 predicts HFNC failure with a specifcity of    80.4%, possibly suggesting a more documented decision to follow a wait-and-see strategy, which is often arbitrarily implemented under the pressure of the pandemic, especially in resource-limited areas.
Te day-to-day assessment of COVID-19 patients with severe hypoxemia under HFNC treatment is crucial to making the right therapeutic decisions. We suggest an alternative use of the ROX-index as a daily assessment tool, based on the signifcance of improvement of the ROX value between the frst two days and the 7th day under HFNC among the success group and the absence of such improvement in the failure group. Early intubation should be considered when a patient's ROX value is not improving.
Serum lactate level increase is a well-known indicator of impaired tissue oxygenation [15]. Te measurement of serum lactate is widely available, calculated directly from the ABGs analyzers that operate in many of the COVID-19 wards. Our study demonstrates that lactate is an independent factor of HFNC outcome, with higher levels suggesting a greater risk for intubation. Similarly, to our results, a small study from Baylor University in the US has also shown an association of lactate with HFNC failure [16].
Typical characteristics of the COVID-19 pneumonia is the sudden deterioration which can lead from mild to grave hypoxemia in a matter of hours, and the long time needed to recover once bilateral pneumonia has settled [17,18]. In our study, there was no diference in total duration of hospitalization between the patients who were intubated and those who were successfully de-escalated from HFNC. Yet, the frst group had a signifcantly shorter duration of stay under HFNC before intubation, in contrast with the second group who stayed longer under HFNC. Tis fnding possibly refects the sudden deterioration in the HFNC failure group and the absence of ROX-index improvement in the same patients. Amongst patients successfully de-escalated, there was a signifcant correlation between the ROX-index and the number of days spent under HFNC, further enhancing the utility of the ROX-index.
In addition to investigating predictors for HFNC treatment outcome, we also examined survival predictors for COVID-19 patients in HFNC. Our analysis included all the previously stated factors as well as the HFNC treatment duration and the SOFA score. Consistent with previous studies' fndings, patients' age and SOFA score were shown to increase along with the mortality as independent risk factors [19,20]. Tis fnding indicates that using the SOFA score could help clinicians make rationalized and safe decisions during the management of hypoxemic COVID-19 patients outside the ICU.
A recent meta-analysis by Prakash et al. [21] shows that the validation of the ROX-index as a predictor of HFNC failure has already been established, yet most of the studies have been conducted in ICU environment and are retrospective. Suliman et al. [22] in a similar manner to our study, calculated the ROX-index over diferent days, as opposed to the frst-few-hours approach of most researchers, but their study was not limited to patients under HFNC, including moderate cases of COVID-19.
Our study is not without limitations. Being a singlecenter study means that the number of patients enrolled cannot reach the high numbers of multicenter studies and involves the risk of poor demographic diferentiation, therefore our fndings might not be applicable to other settings. However, physicians who attend to COVID-19 patients could consider applying our fndings, especially if the context of the study matches their own conditions. Secondly, some important prognostic factors for COVID-19 patients, such as lymphocyte count and D-dimers, were not recorded and thus not included in the analysis. Te number of patients included in days 2, 3, and 7 is diminishing as patients have already met their primary endpoint at that time limiting the available data for these timepoints and any conclusions that could be reached. Finally, we did not calculate values of the ROX-index on days 4, 5, and 6, which could have resulted in more consistent fndings regarding the prognostic value of the ROX-index progress during HFNC treatment.

Conclusions
Te ROX-index is a valuable and easy to use tool in the everyday assessment of COVID-19 patients under HFNC. It could serve as an aide in deciding the optimal timing of intubation in patients with respiratory failure under HFNC and reduce mortality in this population. Larger randomized prospective studies are needed to investigate further its use.

Data Availability
Te data have been uploaded to the University of Athens repository, Pergamos, available at this link: https:// pergamos.lib.uoa.gr/uoa/dl/object/3259142.

Conflicts of Interest
Te authors declare that they have no conficts of interest.