Sepsis and septic shock are currently a leading problem among intensive care unit (ICU) patients worldwide.1–5 The ICU as well as in-hospital mortality owing to sepsis and septic shock have also been significantly higher than other critical illness conditions over the past few decades.1–5 Until now, there have been three sepsis definitions endorsed by international societies, according to update knowledge in sepsis pathogenesis and pathophysiology. The first sepsis definition (Sepsis-1) was established in 1992, and focused on sepsis as a syndrome of infection, with systemic inflammatory response syndrome (SIRS concept).6 The second definition of sepsis by International Sepsis Definitions in 2001 (Sepsis-2) was also based on the SIRS concept, but tried to revise and included a list of more signs and symptoms, more so than SIRS.7 Signs and symptoms of sepsis according to Sepsis-2 definition was also classified into four categories which were: Predisposition, Insult/Infection, Response, Organ dysfunction; this in turn introduced the PIRO concept as a staging system of sepsis.7 However, the severity scoring system of sepsis was not unified, and so some authors tried to develop an individual score to predict mortality, based on the PIRO concept for both general sepsis patients,8–14 and a more specific group such as; hospital acquired pneumonia.15,16 Unfortunately, the PIRO scores are not widely used nowadays owing to their complexity.

The sepsis definition was change again in 2016, by the Third International Consensus Definitions for Sepsis and Septic shock (Sepsis-3), the changed sepsis into an organ dysfunction syndrome, rather than SIRS, as the organ failure concept is better in mortality prediction.17 This definition reintroduces Sequential Organ Failure Assessment (SOFA)18 score into clinical use. The more simplified score or quick SOFA (qSOFA) was introduced for clinician alert for sepsis.17 The later score was validated by Seymour et al. and reported better accuracy of discrimination in term of mortality other than SIRS, especially in non-ICU encounter situations.19 The external validity was also conducted by other authors, who also reported better performance.20–24

Proof of accuracy, between the different sepsis concepts on the same population, may be useful for further development of a unified sepsis score. Thus, our study aimed to compare the accuracy between severity scores from 3 different definition concepts (SIRS, PIRO and SOFA-based scores), in term of in-hospital mortality prediction among sepsis patients admitted to an intensive care unit.

We conducted a retrospective cohort study analyzing data over a 10-year period (from January 1, 2007 to December 31, 2016), using electronic records along with the sepsis and severity scores database of a single medical ICU of a teaching hospital of Prince of Songkla University, Thailand. The protocol was approved by the institutional ethics committee on January 7, 2014 (Approval letter 57-366-14-4).

There were 6986 patients in the database during the 10-year study period in our ICU. A total of 2234 patients had been diagnosed with severe sepsis and septic shock, using the Sepsis-2 criteria. After exclusion of 82 entries, which were diagnosed as sepsis without organ failure, 2152 patients were included into the primary analysis (Fig. 1). The mean APACHE-II score was 23.9±9.8. The ICU and hospital mortality was 33.3% and 45.9%, respectively. The baseline characteristics of the study population are presented in Table 1.

Figure 1.

Enrollment and exclusion of study population.

(0.1MB).

Mortality tended to be higher according to the score increasing, in all scores, except SIRS (Figures S1–S3). The Moreno PIRO score had the highest AUC for in-hospital mortality (0.835, 95%CI 0.818–0.852), followed by SOFA score (0.828, 95%CI 0.811–0.846), qSOFA (0.792, 95%CI 0.775–0.809), Rubulotta PIRO score (0.708, 95%CI 0.687–0.730), Howell PIRO score (0.706, 95%CI 0.685–0.728), and SIRS (0.578, 95%CI 0.556–0.600). However, the AUC of SOFA score was not statistically different with Moreno PIRO score (p=0.43). The AUC of SOFA and Moreno PIRO scores were significantly higher than that of other scores (Table 2 and Fig. 2). Nevertheless, the APACHE II score provide the higher AUC (0.883, 95%CI 0.868–0.897) than SOFA and other PIRO scores.

Figure 2.

Receiving operating characteristic curves (ROC) of severity scores for predict in-hospital mortality.

(0.12MB).

The SOFA score present the best discrimination with an AUC with 95%CI of 0.838, 0.820–0.855 to predict ICU mortality. There were no differences of AUCs between Moreno PIRO score and qSOFA (AUC with 95%CI 0.804, 0.785–0.823 and 0.787, 0.770–0.805, respectively), but their performance were statistically higher than other scores to predict ICU mortality (Table S7). The AUCs of each score to predict the ICU mortality are shown in Figure S4 and Table S7.

The discrimination of each score for subgroups analysis is shown in Table 3. There was similar predictive ability of SOFA and Moreno PIRO to predict hospital mortality in all subgroup of sepsis patients, as well as, the AUCs of both scores were statistically higher than SIRS and other PIRO scores to predict hospital mortality in all subgroup of our sepsis patients.

However, there were only 1239 patients having had their serum lactate values, so as to be included into subgroup analysis. In subgroup analysis, the AUC of Howell PIRO score was higher than the whole population (0.724 vs. 0.706). However, the performance of Howell PIRO to predict primary outcome was lower than Moreno PIRO and SOFA score and similarly to Rubulotta PIRO, as same as the entire population (Table S8).

To the best of our knowledge, this is the first comparison of severity scores derived from all sepsis definition concepts. We found both the SOFA and Moreno PIRO score had the highest mortality discrimination, additionally they were also higher than the value reported in original articles of both scores (AUC of SOFA and Moreno PIRO score in original article was 0.7419 and 0.772,8 respectively). This finding may be owing to the higher mortality in our center, compared with worldwide epidemiologic data.2 Although, we hypothesized that both SOFA and Moreno PIRO scores still preserve their good performance, even in a more severe sepsis patients. This statement may not be true for other scores, as their AUC did not result in as good results, as in their original articles.9,10,19 Actually, components of organ dysfunction in the Moreno PIRO score8 was derived from the original SOFA, hence, may explain why the Moreno PIRO score is better than other PIRO scores. Even though these results may not directly conclude that the organ failure concept is better than other concepts for sepsis, we found that scores that, focus around organ failure seem to have better mortality discrimination than other scores.

In this study, we aimed to compare between different sepsis definitions. Thus, we excluded patients without organ dysfunction, in order to unify the study population into the same terminology, which preserves severe sepsis in Sepsis-1 and Sepsis-2 definition is actually the same as sepsis in Sepsis-3 definition. Fortunately, sepsis patients without organ dysfunction were the minority our study population (only 3.6%), and may not have affected the primary outcome.

Previous studies from other investigators had conflicted results. MacDonald et al. compared Howell PIRO, SOFA, and Mortality in ED Sepsis (MEDS) in 240 sepsis patients, who visited an emergency department, and found that the Howell PIRO score performed better than the SOFA score, and similarly to the MEDS score to predict 30-day mortality.28 We hypothesized that the SOFA score may have better performance in an ICU setting, rather than in an emergency department setting, which is too early to detect changes in organ functions during the diseases course. Thus, the PIRO concept that includes predisposition factors in the score should be used in this setting. The study also included infections with no organ dysfunction (18% of study population), which is not considered to be sepsis nowadays. Another study from de Groot et al. also found that the performance of Howell PIRO, qSOFA, MEDS, Modified Early Warning Score (MEWS), National Early Warning Score (NEWS) in an emergency department setting were poor and the AUC ranged from 0.56 to 0.64 in older patients, significantly lower than the good AUC range from 0.72 to 0.86 in younger patients (lower than 70 years). The qSOFA had a lower AUC than that of the Howell PIRO score.29 Although some of the study populations in this study were within the older age group, most of them were younger than 70 years of age. The difference in both the study population and setting may explain the different results. The study of de Groot et al. also reported lower mortality (9.5%)29 compare with ours, and qSOFA discrimination may not as good as our results. Tafelski et al. compared the accuracy between three PIRO scores, based on a model of the critically-ill in an ICU, and they found that Moreno PIRO and Howell PIRO were statistically different compared with Rubulotta PIRO.30 Most of the patients in this study were in the organ dysfunction group, much like our study, but hospital mortality was 16.2% lower than our results. Different PIRO scores may have different accuracy dependant on population settings and sepsis severity.

Our findings also support that among the more severe sepsis patients, use of SOFA score, which focus on organ dysfunction has more advantage in mortality prediction. Moreover, SOFA score is more simplified and easier to use than Moreno PIRO score, even though all of them have a comparable discrimination performance. We support the use of the SOFA score more so, than the Moreno PIRO score for risk prediction in ICU sepsis patients.

The limitation of our study was that it only included sepsis patients admitted into a medical ICU of a university teaching hospital; these results may some limitation in application in other sepsis patients in other ICU or sepsis in wards or emergency department. Secondly, our results had a higher prevalence of septic shock and high mortality rate; these findings may not be generalized to less severely-sepsis patients. More studies are required to clarify prognostic determination among these different phenotypes of sepsis patients. Thirdly, the accuracy of these scores for predicting mortality in sepsis patients diagnosed according to Sepsis-2 criteria may not be the same as those diagnosed using the new Sepsis-3.

Severity scores derived from the Sepsis-3 concept that consists of organ dysfunction and organ failure assessment have better discrimination performance than other concepts. Our findings support SOFA score for risk prediction in medical-based ICU sepsis patients.

The study concept and design was created by all authors. TS and BK collected data and performed the statistical analysis. TS drafted the manuscript. BK has reviewed and revised manuscript. All authors read and approved the final manuscript.

None.