We included patients over 16 years old who have been admitted to ICUs with available data within the first 72 h after ICU admission. After applying these selection criteria, and conducting data cleaning and management procedures, a total of 106,449 patients were included in the analyses. The flowchart of patient selection from the eICU database, including the exclusion criteria, is presented in Fig. 1. Besides, in the Supplementary File the procedure carried out in order to obtain the final datasets used is described.

Figure 1.

Flowchart of patient selection from eICU dataset.

The outcome of interest was ICU mortality within the first 72 h after admission, which presented imbalance classes (mean prevalence of alive patients = 95%; and deceased patients = 5%). Data to predict mortality included: (1) personal and demographic data as patient age, gender, and ethnicity; and (2) clinical data and biomarkers such as the time between hospital admission and ICU admission, differences between analyzed time points of bicarbonate, sodium, blood urea nitrogen (BUN), glucose, potassium, hemoglobin, and calcium serum concentrations, differences in respiratory and heart rates, differences in blood pressures parameters (i.e., diastolic, systolic, and mean values), administration of norepinephrine, underlying conditions as diabetes, congestive heart failure, chronic obstructive pulmonary disease (COPD), cancer, or chronic kidney disease. We also recorded Charlson Comorbidity index, and coronary artery bypass grafting past surgery, or past ICU stay due to cardiac arrest, congestive heart failure, cerebrovascular accident, diabetic ketoacidosis, myocardial infarction, rhythm disturbance, kidney or pulmonary sepsis, and whether the hospital was linked to university teachings or not.

We developed a multi-model approach based on three ML models to predict 72-h ICU mortality, each using a different time frame (data windows A, B, and C) as input data: (1) model A uses data from 24 h before patient admission up to the moment of ICU admission; (2) model B from 24 h pre-admission to 24 h post-admission; and (3) model C from 24 h pre-admission to 48 h post-admission (Fig. 2).

Figure 2.

Schematic of the three data windows that are considered in the multi-model approach.

The rationale for these windows is not only methodological but also clinical, as they correspond to distinct decision points in ICU workflow. Model A supports decisions regarding ICU admission, integrating pre-admission and admission data. Model B enables early reassessment after 24 h in the ICU, when treatment response becomes observable. Model C, although temporally closer to the outcome and therefore yielding the best predictive performance, is designed for continued ICU stay evaluation at 48 h, reflecting a common practice of intensivists to reassess prognosis and adapt resource allocation.

Thus, the three models should not be viewed as alternatives but as a sequential decision-support framework, aligning with the evolving nature of critical care management and capturing different phases of the peri-admission period.

For each data window, several classification ML algorithms have been trained and benchmarked to tackle our research problem. In particular, the following models have been tested: stochastic gradient boosting, Random Forest, eXtreme gradient boosting, boosted logistic regression, (robust) linear discriminant analysis, partial least squares, and boosted classification trees.25 We used 10-fold repeated cross-validation with random search for parameter tuning, and down-sampling method to tackle class imbalance.

The outcome obtained from the different ML algorithms was the probability of 72-h ICU mortality. Once these probabilities have been obtained, it is necessary to adjust the threshold at which the model predicts patient death or survival. We have set clinically optimal thresholds considering (1) the cost of false positive errors (i.e. people who were predicted to die, and they actually do not), (2) performance measures, and (3) the prevalence of the target outcome in our population. Additionally, we plotted the precision-recall curve applying 0.02-increases from 0 to 1 for classification thresholds.

Taking into account all the ML algorithms that have been designed, we selected the best models for each data window based on: (1) biological plausibility of the posterior predictions on the test dataset (i.e., whether predicted responses on variables of interest make sense biologically), (2) performance measures such as F1-score and balanced accuracy obtained on the test and train datasets, and (3) number of true positives (i.e., deaths).

Next, in order to explore variable contributions in a consistent way across models, we implemented a model-agnostic explainability approach based on marginal predicted probabilities. Unlike previous approaches, this method standardizes the interpretation process across any classification model, allowing consistent and temporally stratified visualization of each variable's marginal contribution to mortality risk.26

For each selected model, we identified the three variables with the highest predictive relevance and visualized their marginal associations with the predicted probability of 72-h ICU mortality.

For continuous variables (e.g., respiratory rate difference), predicted probabilities were estimated across the observed range in the test dataset. We fitted a cubic spline to summarize the relationship between the variable and mortality risk, including a 95% confidence interval (CI) based on bootstrap sampling (10 iterations).

For categorical variables (e.g., norepinephrine administration), we estimated the median predicted probability and associated 95% confidence interval for each category using the same bootstrap procedure.

The formulas proposed for the analysis are presented below. First, for the case of categorical variables:

For the case of continuous variables, the equation of the probability of death is:

We studied if the models perform consistently across different potentially vulnerable subgroups of patients. The analyzed subgroups were categorized by age, according to classifications defined by the World Health Organization (i.e., children, adolescence, youth, adults, and older adults)27; sex (i.e., men and women); ethnicity (i.e., Hispanic, African American, Caucasian, Native American, Asian, or other); and hospital financial status assessed by whether the hospital had linked teaching actions (i.e., yes or no).

As we mentioned in Section 2, in our study several models have been trained and analyzed for each data window in order to achieve the best one. Details about models’ performance measures of all the trained models are shown in Table 3.

As can be seen in Table 3, Random Forest (rf) is one of the models that present better performance metrics for the different temporal windows. Besides, eXtreme Gradient Boosting (xgb) and Boosting Classification Trees (ada) also present good results. Taking into account the (1) biological plausibility of the posterior predictions on the test dataset, (2) performance measures, and (3) number of true positives, an optimal threshold of 0.40 (i.e., if the probability of mortality is 60% or less, the prediction was that the patient will live) was determined for patient status classification (see the Supplementary Figs. 1–3). We optimized our predictive models using the F1-score. Random Forest was the trained model that achieved the best trade-off between F1-scores, the capacity to predict true positives (deaths), and biological plausibility of the predictions within all data windows. Other models such as eXtreme Gradient Boosting, boosted classification trees, and stochastic gradient boosting, also performed well. As can be seen in Table 3, comparing F1-scores between selected models, the model C performed better (F1mean = 0.93; 95% CI 0.93 to 0.94) on average than model A (F1mean = 0.83; 95% CI 0.83 to 0.85), and B (F1mean = 0.92; 95% CI 0.92 to 0.93) predicting death probabilities.

We applied the above method to the best-performing Random Forest models for each data window. The most important predictors differed by model. For data window A, the most relevant variables were: time from general hospital admission to ICU admission, age, and history of cardiac arrest. For data windows B and C, the most relevant variables were norepinephrine administration, respiratory rate difference, and temperature difference (see Supplementary Figs. 4–6).

For each of these variables, we plotted the predicted probability of death as a function of the variable’s value (Fig. 3). This representation allows a better understanding of the association between individual predictors and mortality risk, regardless of the underlying model structure.

Figure 3.

Predicted mortality probabilities for each key predictor within models A, B, and C. The point estimate represents the median probability value, with error bars and shades indicating the 95% CI.

The methodology can be applied to any classification model. It aims to provide a reproducible and interpretable summary of variable-level effects without relying on model-specific mechanisms.

For the initial window, previous to ICU admission (model A), the predicted probability of 72-h ICU mortality gradually increased if patients spent more than 10 h in general hospital wards, had more than 45–50 years old (e.g., from 45 to 65 years old there was an increase on mortality probability of 12%), and suffered a cardiac arrest (the probability of death of patients who suffered a cardiac arrest = 0.92 [95% CI 0.82 to 0.97]; the probability of patients who did not = 0.32 [95% CI 0.23 to 0.43]). Although this association is clinically expected, its consistent emergence as a key predictor confirms the biological plausibility of the framework and strengthens confidence in its application to less obvious variables.

Regarding the second window, patients that required norepinephrine administration had associated higher mortality risk (0.66; 95% CI 0.50–0.77) than those who did not (0.24; 95% CI 0.16–0.32). In case of respiratory rate difference, if patients presented high values, the probability of mortality increased (e.g., −25 chest raises doubled mortality probabilities compared to no difference between before and after ICU admission). Patients whose body temperature varied more than 1 degrees Celsius (ºC) had increased mortality probabilities. Similar patterns were found in the last addressed window (model C).

The main objective of this research was to develop a multi-model approach based on ML to predict the early mortality within the first 72 h of ICU admission taking into account three different temporal configurations. Table 4 illustrates how this system can support clinical decision-making by comparing the 72-h mortality probabilities predicted by our multi-model approach with actual patient outcomes, highlighting potential outcomes for different cases. In particular, the multi-model approach developed would have supported the decision to decline ICU admission of 132 patients who ultimately expired within the first 72 h: 73 of these patients, with a 95% CI upper bound above 60% prior to ICU admission, remained in the ICU for more than 24 h, while 32 patients under the same conditions stayed for more than 48 h.

Table 4 presents four different examples: 1) patient ID #3342213 had a very high mortality probability before ICU admission (using data from window A) and expired within 72 h in the ICU; 2) patient ID #923407 had a medium mortality probability value before ICU admission (model A) but a high one after 24 h in ICU (model B) and expired within 72 h in the ICU; 3) patient ID #386538 is an example of system failure where a low and medium probability of mortality value was predicted using models A and B, but the patient finally died after less than 37 h of ICU admission; 4) patient ID #163973 had a very low mortality probability before ICU admission (model A), after 24 and 48 h (models B and C), and was discharged from the ICU.

Conversely, this multi-model approach underestimated the mortality probability in 120 patients who remained in the ICU for more than 48 h, with predictions yielding 95% CI lower bounds below 60%. For instance, patient ID #386538 stayed in the ICU and had a low mortality risk estimated by our model (data window B), but ultimately expired. When assessing time efficiency, the system potentially saved 283.58 h, compared to 276.01 h spent on patients who did not survive, resulting in a 7.54 -h positive trade-off in decision-making efficiency.

This reduction in ICU stay time could potentially translate into lower healthcare costs per patient and increased bed availability for new admissions. However, these implications should be considered exploratory and illustrative, as our study was not designed to directly evaluate economic outcomes. Further validation in prospective and context-specific studies would be required before drawing firm conclusions on resource management.

In order to analyse in depth the results obtained, the population has been separated into different groups. In particular, as we have mentioned in Section 2.5, we have decided to analyse four different classifications separating the population according to age, sex, ethnicity and hospital financial status.

In Table 5 we have presented the different F1-scores obtained for the global test dataset and for each group mentioned before for each data window. This can also be seen in Supplementary Figs. 7–9.

As can be seen in Table 5 there are some differences between the classes defined in the different groups, the most significant one is the observed age group between the class “Older adults” and the rest.