Coronavirus disease 2019 (COVID-19) is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). More than 670 million people suffer from this disease worldwide, of whom nearly seven million are expected to die.1 Intensive care units (ICUs) have been used to manage patients with severe forms of the disease. Initial data from China suggested that between 5% and 32% of infected patients required admission to the ICU,2,3 and 95% received ventilatory support.4

In cases of severe SARS-CoV-2 pneumonia (CAP), the use of non-invasive ventilation (NIV) was initially discouraged in various recommendations.5–8 In contrast, the National Health Service (NHS) of the United Kingdom proposed the use of NIV with continuous positive airway pressure (CPAP) as the first mode of ventilatory support for patients with COVID-19 pneumonia.9 Despite this controversy, several studies on patients with hypoxemia have reported a reduction in orotracheal intubation (OTI) after NIV. These results support the use of NIV during the pandemic.10,11 However, late OTI after NIV failure is associated with increased mortality.12 Therefore, it seems reasonable to identify the correct time at which patients would benefit from safe intubation. However, the parameters to identify patients with high risk of NIV failure is unknown.

Predictive modeling of NIV failure would help in decision-making. This would help avoid excessive exposure to NIV and delay in OTI and its prognostic consequences. The main objective of this study was to develop and validate a predictive model for the risk of NIV failure in patients with severe SARS-CoV-2 CAP admitted to the ICU.

A total of 154 patients were included in the training cohort during the study period. Fig. 1 shows the flowchart of these patients. The NIV failure rate in the study population was 37.6% (N = 58).

Figure 1.

Flowchart of inclusion and exclusion of participants. COPD: chronic obstructive pulmonary disease; DNI: do-not intubate; HFNC: high flow nasal cannula; ICU: intensive care unit; IMV: invasive mechanical ventilation; NIV: non-invasive ventilation.

(0.16MB).

In the training cohort, the failure group presented with higher age, SOFA score, percentage of patients with obesity, days from symptom onset to ICU admission, and days from hospital admission to ICU admission (Table 1). In the NIV setting, no differences were observed between interfaces (Table 2). Upon ICU admission, the failure group required a higher FiO2 and presented more deteriorated SatO2/FiO2, PaO2/FiO2, and iROX values than did the success group (Table 2); moreover, at 24 h, they presented with a higher heart rate (HR), greater need for FiO2, and lower values of SatO2, SatO2/FiO2, PaO2/FiO2, iROX, and HACOR scores (Table 2).

In total, 229 patients who met the inclusion criteria during the study period were included in the validation cohort. NIV failure occurred in 42(18%) patients. A comparison of the two cohorts revealed that the validation cohort had a lower SOFA score, shorter time from hospital admission to the initiation of NIV, less use of non-invasive respiratory therapy (NIRT) prior to ICU admission, and a higher proportion of patients admitted to the emergency department (Table 1). Critical ventilators and helmets were frequently used (Table 2). Regarding oxygenation status, it should be noted that patients in the validation cohort were less hypoxemic, as measured by SatO2, SatO2/FiO2, PaO2/FiO2, iROX, and HACOR both on admission and at 24 h (Table 2). ICU mortality in the training and validation cohorts occurred in 39(25%) and 25(11%) patients, respectively, and was higher in the NIV failure group than in the NIV success group in both cohorts ([68% vs. 0%; p < 0.0001] and [60% vs. 0%; p < 0.0001, respectively]) (Table 2).

Multivariate logistic regression analysis revealed that the factors independently associated with NIV failure were higher age, high SOFA score at admission, obesity, elevated HR at 24 h, and high HACOR score at 24 h (Appendix A; Table A1). The model explained 58.7% (Nagelkerke R-squared) of the variation in failure outcomes. A goodness-of-fit test was performed to compare the observed and expected data with the model, demonstrating that there were no differences, with a Hosmer–Lemeshow test result of chi-square = 1.97 and p = 0.961. Calibration curve shows a good fit between the actual and predicted probability shows the curve calibration of our predictive model (Appendix A, Fig. A2).

Based on the final model, a nomogram was constructed using the weights of each model variable associated with NIV failure (Fig. 2). The probability of NIV failure was calculated using equation (1):

Figure 2.

Nomogram to predict NIV failure in patients with SARS-CoV-2 pneumonia. (HR: heart rate; HACOR: heart rate, acidosis, conscienciousness, oxygenation and respiratory rate score; ICU: intensive care unit; NIV: non-invasive ventilation; SOFA: sequential organ failure assessment score).

(0.21MB).

The Supplementary Material (Appendix A, Fig. A1) describes how the nomogram can be used to determine the probability of NIV failure.

The predictive ability of the model was satisfactory in the training cohort, with an AUC of 0.89 (95% CI: 0.839–0.942, p < 0.0001) (Fig. 3A), which was superior to that of each of the model variables separately (Table 3). However, the AUC of the validation cohort was 0.547 (0.449–0.645, p = 0.346) (Fig. 3B).

Figure 3.

Nomogram ROC curves. 3A Training cohort; 3B Validation cohort.

(0.17MB).

In this study, a nomogram was developed to predict the failure of NIV in patients with COVID-19. It is based on age, obesity, SOFA score at admission, and HR and HACOR scores at 24 h. Patients who failed NIV had higher mortality; therefore, it is important to recognize an early failure of NIV and select those who would benefit from escalating ventilatory support. To simplify and quickly apply the model, analytical variables were not included. The predictive capacity of the model was assessed with the Hosmer–Lemeshow test, the calibration curve and by calculating the AUC on an ROC curve, which reinforces the internal strength of our model.

The training cohort presented an NIV failure rate of 37.6%, which is within the range of 29%–53% reported in different studies.14–16 However, the rate of failure in the validation cohort was 18%, which was lower than that reported for patients with COVID-19 in the ICU. This difference could be attributed to the lower severity level in the validation cohort and patients being admitted at earlier disease stages, thus requiring less prior NIRT. Moreover, majority of patients in the validation cohort were administered NIV via a Helmet, an interface linked to lower NIV failure and mortality rates compared with that of the oronasal mask.17

A retrospective observational study conducted across 23 Chinese hospitals, including non-ICU patients treated with NIV or HFNC, developed a predictive model for NIRT failure risk.18 This model incorporated age, comorbidity count, oxygenation index, the GCS score, and vasopressor usage; however, unlike ours, it only included variables at ICU admission. The value of our model lies in its utility for patients who have completed 24 h of support with NIV, suggesting an evaluation after a longer trial period than initially recommended in the early stages of the pandemic.5 We believe that this aspect fills a gap in the available literature.

Age determines the prognosis of these patients19 and is a risk factor for NIV failure in both patients with COVID-1915,20–22 and acute respiratory distress syndrome (ARDS).23,24

Obesity is associated with NIV failure in patients with COVID-1920 and influenza.25 It is assumed that patients with obesity tolerate higher levels of positive end-expiratory pressure (PEEP), with an improvement in the expiratory flow limitation experienced by these patients.26 Although we could not measure the transpulmonary pressure (Ptp), we managed the undetected elevated Ptp, indicating the need for high levels of PEEP. However, there was likely incorrect adherence to the sitting position, related to reducing the diaphragmatic load on the lungs and airflow limitation.26

The SOFA score on admission is a risk factor for NIV failure in patients with ARDS of any etiology or COVID-19.21,27 The training cohort was conditioned by the respiratory component since most patients had no other organ failure. This parameter allowed us to consider that patients with only respiratory failure would be candidates for initiating support with NIV, as suggested for patients with influenza.25

The fourth risk factor identified was HR at 24 h. The nomogram by Liu et al. showed the same result as ours in the univariate analysis, without confirmation in the multivariate analysis.18 We hypothesized that patients with a higher HR have higher tissue O2 demand and consumption.

The last factor included in our model was the HACOR score, which incorporated HR, GCS, RR, PaO2/FiO2, and Ph.28 Its usefulness has been demonstrated in the prediction of NIV failure (cutoff point of 5) in patients with acute respiratory failure.29–31 The novelty of the results from the training cohort is that the HACOR score was identified as a risk factor at a more advanced stage (24 h). It can be concluded that the HACOR score is a reference scale for the detection of possible NIV failure, extending its usefulness up to 24 h after initiation.

Regarding the equipment used in the training cohort, no risk factors were identified in the univariate analysis. It is important to focus on two issues. First, the small number of patients in whom a helmet was used in our series, despite the results of the study by Grieco et al.14 Considering that patients undergo prolonged periods of NIV and that the helmet is better tolerated with fewer lesions on the bridge of the nose,32 it serves as a viable starting point. In favor of this position is the low failure rate of the validation cohort, in which helmets were widely used. Second, not all data were available to analyze the use of active humidification as a risk factor. This is listed as optional during NIV in the humidification guidelines for IMV and NIV, discouraging the use of heat-humidity exchangers during NIV.33 The use of a heated humidifier (HH) improves adaptation and secretion clearance, compared with a heat and moisture exchanger.34 In addition, Esquinas et al. found that 50% of the patients who were labeled as having difficult OTI after NIV failure did not receive HH, resulting in thick and dry secretions which were a probable reason for failure.35 Although this was not a part of our analysis, it could be suggested that HH use be started at the beginning of NIV support.

The nomogram did not have a good predictive capacity in the validation cohort. This result is striking given that both cohorts were managed in units with a clear preference for the use of NIRT as initial support. There are different reasons why an external validation process can be difficult: differences between participating clinicians, populations included, and interventions performed,36 an issue that seems to be observed in our training and validation cohorts (lower severity and greater use of helmets). This reflection should be more profound, but it highlights the importance of validating the proposed predictive model at a local level.

We think it would be questionable the usefulness of external validation of a predictive model for NIV failure. The decision for OTI is complex, arbitrary, individualized, intuitive, and the criteria for patients with hypoxemia have not been clearly established, with great variability between different studies.37 This leads us to believe that the same patient could receive OTI at different times (and even avoid OTI), depending on the department to which the patient is admitted. Furthermore, in a review of 14 studies, it was observed that the protocol for the initiation of IMV after NIV did not affect the intubation rate,38 questioning the usefulness of establishing criteria for NIV failure in hypoxemic patients. In addition, only a small percentage of patients received OTI within 3 h of meeting the previously established NIV failure criteria,39 and even when these criteria were met, a high percentage of patients avoided OTI.40 The criteria for OTI likely differed between the two cohorts. This supports the idea that, in establishing a predictive model, it is possible that this would not be a representation of patient status but rather reflect the decision of the responsible clinicians.

Héctor Hernández: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Writing-original draft, Writing-review & editing, Visutalization, Project administration. Alberto Belenguer: Conceptualization, Methodology, Investigation, Data curation, Writing-review & editing, Formal analysis. Francisco Bernal: Investigation, Writing-original draft, Data curation. Irina Hermosilla: Investigation, Writing-original draft, Data curation. Lluis Tormo: Investigation, Writing-original draft, Data curation. Estefanía Granero: Investigation, Writing-original draft, Data curation. Clara Viana: Investigation, Writing-original draft, Data curation. Rafael Zaragoza: Conceptualization, Methodology, Writing-review & editing, Visualization, SUupervision.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.