According to the World Health Organization (WHO), the serious complications that can occur during pregnancy, delivery or the puerperium period can be defined as follows: 1) maternal death (MD): a woman’s death due to causes related to her pregnancy or management during pregnancy or within the first 42 days after full term pregnancy;1 2) extreme or near-miss maternal morbidity: a woman who survives life-threatening conditions (for example due to organ dysfunction) following a complication occurred during pregnancy, delivery or the puerperium period;2 and 3) severe maternal outcomes including cases of MD and extreme maternal morbidity.3 The American College of Obstetricians and Gynecologists uses the concept of severe maternal morbidity (SMM) including pregnant women or women in the pregnancy-puerperium period who were admitted to the intensive care unit (UCI) or who received, at least, 4 erythrocitary concentrates.4 Similarly, potentially life-threatening conditions are clinical situations that threaten a woman’s life during her pregnancy, delivery or the puerperium period3 such as: a) direct obstetric morbidity resulting from pregnancy, delivery or puerperium complications due to medical and/or surgical interventions or ommissions or incorrect therapy (eg, obstetric bleeding); or b) indirect obstetric morbidity associated with preexisting conditions before pregnancy or during pregnancy, not due to any obstetric causes, but aggravated by the physiological effects of pregnancy (eg, valvular heart disease).1

A total of 830 women die every day across the world due to pregnancy related causes.5 From 2014 to 2016 the United Kingdom reported a total of 259 MDs (9.8 women for every 100 000 live births (LBs).6 The United States reported a total of 18 MDs/100 000 LBs.7 For every MD there are 118 cases of SMM8 and between 0.5% and 1.3% of pregnancies will be associated with SMM.4 Inappropriate therapies, delayed diagnoses, and referral to a special care level contribute to maternal mortality.4 Between 55% and 70% of the cases of MD received suboptimal medical care.9 That is why the target established by the WHO is to reduce the overall MD rate to < 70/100 000 LBs.10 The United Kingdom will try to cut the number of MD in half by 2025.6

On the other hand, a prognostic model is the combination of multiple predictors to assess the risk of a specific outcome.5 These models are useful to characterize the severity of the disease in clinical trials and assess the performance of ICUs.11 Their validity (capacity to predict a specific result) includes: a) the discrimination or capacity to distinguish different individuals based on their different outcomes, and b) calibration or match between the probabilities observed and the probabilities anticipated of an occurring event.5

Prognostic models in obstetric patients are not accurate because: 1) we are talking about young female patients with few comorbidities; 2) many obstetric complications improve significantly after pregnancy, and 3) due to the normal physiological changes of pregnancy.11 The Acute Physiology and Chronic Health Evaluation II (APACHE II) is the predictive model most commonly used at the ICU setting.12 However, 5 of its variables have different reference values during pregnancy (arterial blood pressure, heart rate, hematocrit, creatinine and leukocyte levels),8,13 and it also overestimates the mortality rate.14 The Sequential Organ Failure Assessment (SOFA) score assesses the degree of organ dysfunction and is less affected by the physiological changes of pregnancy.8 Prognostic models can be improved by altering their equations or changing their variables, a process known as personalization or recalibration5 Blanco et al. propose an obstetric SOFA (O-SOFA) created from 2 modifications of the SOFA model (Appendix B supplementary data 1),15 and Paternina et al. propose a modified APACHE II score (APACHE II-M) by changing the mathematical equation of the original APACHE II to better serve its validity among the obstetric population (Appendix B supplementary data 1).16 The objective of this study was to compare the validity of the APACHE II, the APACHE II-M, the SOFA, and the O-SOFA to predict mortality in a cohort of patients with SMM who were admitted to our ICU.

Retrospective, cohort study conducted at the ICU of the IMSS High Specialty Medical Unit 25 of Monterrey, Nuevo León, Mexico. It was approved back on November 27, 2018 by the Health Research Local Committee 1901with registry #R-2018-1901-105. It recruited patients from December 2015 through July 2018. According to Peduzzi formula17n = (TDF × EPV/LP)1 a minimum of 131 patients are needed for the analysis to be statistically significant. The study used a convenience sampling method. Only women who met the definition of SMM were included (pregnant women or within the first 42 days since full-term pregnancy admitted to the ICU). Women without enough information on the health records to assess their prognostic scales at the ICU admission were excluded. In case of ICU readmission during the same hospitalization only the data from the first admission were included. Patients were followed during hospitalization until their death or admission. The patients’ demographic data (age, number of pregnancies, deliveries, c-sections, miscarriages, gestational age, and comorbidities), main diagnosis at admission, time elapsed since the beginning of the complication until admission to our unit, and the variables needed to estimate the APACHE II, APACHE II-M, SOFA, and O-SOFA scores were included. Also, clinical data on the patients’ clinical progression such as the use of mechanical ventilation, vasopressors, renal replacement therapy, and days at the ICU were included too.

Demographic data were documented with descriptive statistics using measures of central tendency and dispersion: mean and standard deviation were used to expressed variables with normal distribution; median and interquartile range were used to expressed variables without a normal distribution. The inferential analysis was conducted using the Student t-test for independent samples or the Mann-Whitney U test for quantitative variables based on their normal distribution. The chi-square test or Fisher’s exact test were used for the qualitative variables. The degree of discrimination of every prognostic model was estimated using the area under the curve (AUC) of the receiver operating characteristics (ROC) and their 95% confidence intervals (95%CI). Calibration was estimated using the standard mortality ratio (the ratio between the deaths seen and the deaths estimated) and Hosmer-Lemeshow test for goodness of fit. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of every prognostic score was estimated as well. The SPSS v 20.0 software was used for statistical analysis.

During the 56-month follow-up, 6.7% of ICU admissions were obstetric patients (143 admissions due to SMM) all of them referred from other hospitals. Two patients were excluded due to insufficient information to estimate some of the prognostic scales at admission. The final analysis (141 patients) included 99 women in the pregnancy-puerperium period (70.3%) and 42 pregnant women (29.7%). A total of 62.5% of women in the pregnancy-puerperium period and 45% of pregnant women had their complications in the third trimester. Data on the clinical characteristics, prognostic scores, and disease progression are shown on Table 1. Most admissions (54.6%) were due to indirect obstetric morbidity with an average time of 3.1 ± 2.5 days since the beginning of the complication until ICU admission. The main causes of indirect obstetric morbidity were oncological-hematological (Appendix B supplementary data 2). The most common diagnosis at admission was pregnancy-induced hypertensive disease (50 cases). A total of 23 cases (16.3%) were classified as HELLP syndrome, 19 (13.4%) as severe preeclampsia, and 8 (5.6%) as eclampsia. Diagnoses at admission and lethality are shown on Table 2. There were significant differences in the use of mechanical ventilation, vasopressors, and inotropic drugs between survivors and non-survivors. However, no differences were seen in the use of renal replacement therapy and days on intensive therapy (Table 1).

The 4 prognostic scales were associated with the use of mechanical ventilation, vasopressors, renal replacement therapy, and days at the ICU (Appendix B supplementary data 2).

The characteristics of how the scores performed are shown on Table 3. In our cohort, the rate of MD was 7% and the 4 scales discriminated between survivors and non-survivors. The largest ACU-ROC of all was that of the APACHE II and the APACHE II-M models (Fig. 1), although this difference was not statistically significant (P = .105). All prognostic scales had low PPVs and high NPVs. The 4 scales lost their statistical significance when they assessed pregnant patients and septic patients separately (Appendix B supplementary data 2). Based on the mortality ratio, the APACHE II-M model predicted a lower number of deaths than observed. The remaining models overestimated the risk. In the Hosmer-Lemeshow logistic regression model the mortality prediction of both the APACHE II and the O-SOFA model were significantly different compared to the number of deaths reported (Fig. 2).

Fig. 1.

ROC curves of the APACHE II-M, O-SOFA, APACHE, and SOFA models in a cohort of 141 patients with severe maternal morbidity. APACHE II-M, modified version of the Acute Physiology and Chronic Health Evaluation II for obstetric patients; AUC, area under the curve; O-SOFA, modified version of the Sequential Organ Failure Assessment Score for obstetric patients. APACHE II, Acute Physiology and Chronic Health Evaluation II; ROC, receiver operating characteristics; SOFA, Sequential Organ Failure Assessment Score.

Fig. 2.

Calibration chart of the APACHE II-M, O-SOFA, APACHE, and SOFA predictive models in a cohort of 141 patients with severe maternal morbidity: observed mortality (axis Y) vs estimated mortality (axis X). APACHE II, Acute Physiology and Chronic Health Evaluation II; APACHE II-M, modified version of the Acute Physiology and Chronic Health Evaluation II for obstetric patients; O-SOFA, modified version of the Sequential Organ Failure Assessment Score for obstetric patients; SOFA, Sequential Organ Failure Assessment Score.

The 4 models analyzed showed discrimination capabilities. Regarding calibration, both the APACHE II-M and the SOFA models confirmed their superiority. The patients’ age of the non-survivors group (32.8 ± 15.7 years, P = .043) was older compared to the survivors group. Consistent with the MBRRACE-UK report that described a higher risk of MD with age (7/100 000 in the group of 20-to-24 year-old patients), this risk duplicated to 14/100 000 in women between 35 and 39 years-old, and tripled to 22/100 000 in women > 40.6 In this cohort, indirect obstetric morbidity was more common (54.6%) including neurological, oncological, septic processes, and acute pancreatitis in order of frequency. Regarding direct obstetric morbidity, pregnancy-induced hypertensive complications (35%) and obstetric bleeding (5.6%) were the main diagnoses. On the contrary in the cohort of the study conducted by Rojas et al., admissions due to pregnancy-induced hypertensive disease and obstetric bleeding were predominant. Also, these were the 2 main causes of death.18 This difference can be explained by the fact that the ICU of the study conducted by Rojas is obstetric unlike our ICU (polyvalent). In our cohort, the main cause of MD was pregnancy-induced hypertensive disease (30%) followed by cardiovascular diseases (20%) and neurological conditions (20%). This contrasts with the data from the United Kingdom where 56.4% of all MDs were due to indirect causes such as heart disease and the causes of direct MD (43.5% of the cases). Meanwhile, 43.5% of all MDs were due to direct causes, the main one being pulmonary thromboembolism. Pregnancy-induced hypertensive diseases were the fourth cause of death.6 In the United States, the main cause of MD is indirect obstetric morbidity with cardiovascular diseases (15.5%)4 as the leading cause of death followed by pregnancy-induced hypertensive disease (6.8%) (8th place).7 However, in other Mexican reports, most admissions and deaths at the ICU setting are due to direct obstetric morbidity.1

Peripartum cardiomyopathy and acute fatty liver of pregnancy (AFLP) were the 2 major conditions that triggered the highest lethality rates (66.6% and 33.3%, respectively). Unlike what has been reported in the medical literature, the average mortality rate was 10% for peripartum cardiomyopathy19 and 10% to 15% for AFLP.20 On the other hand, pregnancy-induced hypertensive disease had a lower lethality rate (6%) compared to what the study conducted by Curiel et al. reported (a 1.5% lethality rate).21 It is estimated that 0.8% of all cases of SMM end up in MD;8 however, the rate of MD in our cohort was 7%, nearly 10 times higher. This may be explained by 3 situations: a) tertiary referral centers admitting obstetric patients referred from other units that manage complex underlying conditions; b) patients included in the study who did not receive medical attention or did not respond to the early management attempted in other centers, and c) the delays reported when arriving to this unit since the beginning of the complication are up to 3.1 days on average. The aforementioned conditions cause more adverse events in obstetric patients admitted to this unit which, in turn, compromise the course of their disease. The high mortality rate seen in obstetric patients from tertiary referral centers has also been reported by other authors: In their cohort of 232 women with SMM from a tertiary center, Blanco et al. reported a 3.8% mortality rate;15 similarly, Paternina et al. reported a 4.7% mortality rate in 625 patients with SMM.16

The APACHE II, APACHE II-M, SOFA, and O-SOFA scores were correlated with findings like the use of vasoactive amines and mechanical ventilation, which confirmed their possible utility to define the subgroup of patients with poor disease progression although these models are only validated to predict mortality.12 All models discriminated between survivors and non-survivors without any significant differences among their different AUC-ROCs. This is contrary to the best discrimination of the O-SOFA model (AUC-ROC = 0.86) compared to the SOFA (AUC-ROC = 0.79) as reported by Blanco et al.15

Regarding the selected cohort points, the APACHE II and the APACHE II-M scores had better sensitivity and specificity. However, the PPV of all models was low (< 60%) and its best utility was for its high NPV (> 95%). No model discriminated or calibrated subgroups of pregnant women or septic subgroups separately. This is due to the fact that only 1 out of 20 maternal deaths occurred in a pregnant woman and another one due to a septic process unlike the cohorts of Blanco et al.15 and Paternina et al.16 where these subgroups are far better represented.

Using the mortality ratio, the APACHE II-M model obtained the best value (closer to 1). This result is consistent with that reported by Paternina et al. who maintain a high discrimination value (AUC-ROC of 0.86) but improves calibration at a mortality ratio of 0.85 compared to 0.30 of the original APACHE II score.16 The Hosmer-Lemeshow test for goodness of fit proved there is no calibration in the predictions made by the APACHE II and the O-SOFA models (P = .034 and P = .039, respectively). In patients with lower mortality risk (most patients in our cohort) the APACHE II-M model made a right estimate of the mortality rate, but as severity went up it started underestimating mortality rate. On the other hand, the APACHE II model overestimates mortality in less severe patients and underestimates it in more severe patients, which explains its predictions null calibration. Both the SOFA and the O-SOFA models overestimated the risk of death. Still, the overestimation of the O-SOFA score was higher, meaning that its calibration was null in our cohort.

The strengths of the studio are that it compares the validity of its 2 components (discrimination and calibration) in 2 prognostic models for general ICU patients. Also, its personalized versions for obstetric patients, a comparison that had not been drawn together before for the different models in 1 same cohort; its internal validity comes from the use of validated statistical tools and from the fact that the patients who were not included in the study due to the lack of data in the clinical history only accounted for 1.6% of the sample, which is why it was considered an insignificant loss.

The limitations of this study are selection bias and its poor external validity given it is a single-center trial and the cohort analyzed is only representative of a minority of all patients with severe maternal outcomes from the region who, due to the severity or complexity of their clinical state, were referred to our hospital considering that it does not have an obstetric ICU and only 6.7% of all admissions are obstetric patients; however, thanks to its therapeutic resources and extracorporeal support it is a reference centers for critically ill patients. Another weakness of the study is the small number of pregnant (10 patients) and septic women (15 patients) included, which prevented the analysis of the prognostic model validity in these subgroups. Similarly, another study limitation is the fact that only 10 deaths were reported, which may lead to erroneous conclusions on the performance of the scales under analysis. No autopsies were performed in any of the patients either to find out the causes of death.

We found a favorable tendency towards the prognostic validity of the APACHE II-M score to predict MD thanks to its improved calibration compared to the original APACHE II score, and a negative tendency on the validity of the O-SOFA score in the prediction of MD compared to the original SOFA score due to its lower calibration capabilities. According to our results, the use of the APACHE II-M model can be recommended to assess the prognosis of mortality in women with SMM.

Baltazar Jonguitud López designed the research, manuscript, publication, and data mining protocols.

Daniel Álvarez Lara reviewed and provided counseling during the design phase of the research protocol.

Gerardo del Carmen Palacios Saucedo provided counseling during the design phase of the research and manuscript protocols. Statistical analysis.

Miguel Ángel Sosa Medellín provided counseling during the design phase of the research protocol. Statistical analysis.

Fabio Montoya Barajas conducted data mining and provided counseling during the design phase of the manuscript protocol.

None reported whatsoever.