Non-invasive respiratory therapy (NIRT), essentially high-flow nasal cannula (HFNC) oxygen therapy and non-invasive ventilation (NIV), was used during the COVID-19 pandemic due to the strong demand for respiratory support. The main inconvenience of NIRT is the need to determine the moment of failure of the technique, since a delay in orotracheal intubation would imply an increase in patient mortality.1 A number of predictive scales were developed in order to be able to determine NIRT failure, based on cohorts of hypoxemic patients: the IROX (index to predict the risk of intubation) for HFNC, and the HACOR (heart rate, acidosis, consciousness, oxygenation and respiratory rate) for NIV.2–4

Considering the lower invasiveness and time required to determine IROX, the predictive ability of the two scales was compared during the pandemic in relation to both NIV and HFNC failure.5–7 However, a concordance and correlation analysis is required in order for the two scales to be used indistinguishably in clinical practice.8–10 The aim of this study was to analyze the homogeneity between IROX and HACOR in patients with COVID-19 pneumonia 24 h after starting NIV, and their ability to predict IROX failure.

A retrospective observational study was conducted in two Departments of Intensive Care Medicine between March 2020 and August 2021, following approval by the Research Ethics Committee (reference 66/20). Due to the nature of the study, patient informed consent was not considered necessary.

The study included patients ≥ 18 years of age admitted due to acute hypoxemic respiratory failure secondary to COVID-19 pneumonia and requiring NIV. Individuals with exacerbated chronic respiratory disease were excluded. Arterial blood gas testing was performed using an ABL90Flex PLUS analyzer (Radiometer Medical ApS, Brønshøj, Copenhagen, Denmark). Oxygen saturation (SatO2) was recorded by pulse oximetry (IntelliVue MX700, Koninklijke Philips, The Netherlands). The fraction of inspired oxygen (FiO2) delivered was recorded by the ventilation monitor BiPAPV60 (Respironics CA, Carlsbad, USA). The indication for intubation was based on the persistence of hypoxemia and the respiratory manifestations (tachypnea, accessory muscle use, thoracoabdominal asynchrony). The scales were not used as an intubation criterion. We recorded HACOR and IROX after 24 h of ventilation, and compared the NIV success and failure groups according to the types of variables involved: the chi-square test (with Fisher’s exact test) was used for qualitative variables, and the Student t-test or Mann-Whitney U test was used for quantitative variables, as appropriate, after evaluating the normality of the data distribution with the Kolmogorov-Smirnov test. Correlation analysis between HACOR and IROX was performed using the Spearman test (R), followed by logarithmic transformation of the parametric (normal distribution) variables to determine the difference of means with the Student t-test for paired measurements. Lastly, the degree of concordance between the two variables was assessed (Bland-Altman test).8,9

A total of 388 patients were analyzed (Table S1 in Supplementary material [SM]), comparing HACOR and IROX after 24 h of NIV, with the observation of significant differences between success and failure. The correlation between the two measurements (Fig. S1 in SM) was R = −0.547 (p = 0.0001) (corresponding to a moderate negative correlation). The log mean comparison proved significant (mean difference −0.835 ± 0.275 standard deviation [SD], 95%CI −0.865 to −0.805; p = 0.0001). The concordance analysis questions the homogeneity of the two variables (Fig. 1). In fact, several determinations were outside the limits of concordance of the standard deviation, although they should have fallen within these limits. In turn, the mean difference (−0.835) of the two determinations did not reach the value 0 (absence of difference, indicating that the two scales are equivalent). On the other hand, the limits of concordance were wide (between −0.296 and −1.374) for considering good concordance. Likewise, the line of 0 value did not fall within the 95%CI (−0.865 to −0.805) of the mean difference, suggesting a lack of precision of the two measurements and a probable overestimation of IROX compared to HACOR. On the other hand, the degree of concordance was found to be low (Fig. 2), indicating that only about 55% (0.55 in Fig. 2) of the samples would have good concordance for the previously obtained mean difference of 0.835 (e.g., in order to obtain 100% concordance between the two determinations the difference would have to be 1.50, which could lead to a large discrepancy between the two scales).

Figure 1.

Bland-Altman concordance analysis between HACOR and IROX after 24 h of NIV.

The figure shows the logarithm of the mean difference (Y-axis) versus the mean logarithm (X-axis) for those variables. Likewise, the difference has been established using the Student t-test and 95%CI (−0.835 [−0.865 to −0.805]) with its margins of two standard deviations (2SD) (−0.835 ± 1.96 × [0.275]).

SD, standard deviation.

(0.2MB).

Figure 2.

Analysis of the degree of concordance between HACOR and IROX after 24 h of NIV.

A degree of concordance of 55% (0.55 on the Y-axis) between the two scales would be represented by a difference between them of 0.835 (X-axis) (solid lines). To achieve a concordance of 100% (1.0 on the Y-axis) between the two scales, there should be a mean difference of 1.5 (X-axis) (dashed lines). The Kaplan-Meier curve of the differences between the two scales was constructed in inverse order (i.e., IROX-HACOR), since it was impossible to construct it from negative values.

(0.12MB).

A secondary analysis of the study that validated the HACOR assessed the applicability of the IROX to NIV failure in patients with de novo acute respiratory failure. After 24 h of NIV, we found that IROX scores of 4, 6 and 8 would correspond to failure rates of 87%, 59% and 39%, respectively. The predictive ability of the model was moderate, with an area under the ROC curve of 0.77 (95%CI 0.74−0.8).6 In contrast, in our sample the intubation rates for IROX scores of 4, 6 and 8 were 65%, 39.4% and 26.5%, respectively, which would indicate a lack of precision in the predictive ability. These differences could be influenced by the different disease states underlying the study (mainly pneumonia, non-respiratory sepsis and pancreatitis) compared to COVID-19 patients, where respiratory problems predominate. Differences in intubation criteria may also be an influence. The different intubation rates, and the lack of concordance between the two variables, suggest that the predictive capacity of IROX was not accurate in our patient sample.

In conclusion, the correlation and concordance of HACOR and IROX in our series of patients with COVID-19 pneumonia requiring NIV can be questioned. This would imply discarding IROX as a reliable predictor of NIV failure.