Since ancient times, knowing the patient’s prognosis has been one of the main objectives in Medicine. Risk stratification dates back to the early days of intensive care medicine, with the use of different specific scales or scores validated for each type of disease condition. At present, the most widely used tools in this regard are the APACHE and SOFA scores, while in the concrete setting of the Coronary Unit, the most commonly used scores are the GRACE (Global Registry of Acute Coronary Events) or TIMI (Thrombolysis in Myocardial Infarction).1

Acute coronary syndrome (ACS) is frequently seen in the Department of Intensive Care Medicine (DICM), and by the year 2049, its incidence is expected to increase in Spain as a result of progressive aging of the population, exceeding 175,000 cases/year.2,3

In recent decades, one of the main aims of experts in intensive care has been the development and subsequent validation of predictive models, with a view to adapting treatment to individual risk status, avoiding needless costs, and planning secondary prevention strategies.1

Unfortunately, the existing coronary patient stratification scales are far from perfect.1 They all have considerable limitations that evidence the need to establish an indicator or algorithm capable of adapting to the contemporaneous population in an individualized and effective manner.

Traditionally, the reference scores in ACS have been the GRACE and TIMI, which are very useful and afford good mortality discriminating capacity, with low areas under the receiver operating curve (ROC) of close to 1. Nevertheless, they have important limitations. For example, they make no reference to mortality on admission to the DICM; they have been developed from a highly selected population that is scantly representative of the current population; and they are not adjusted to the therapeutic advances to date (Table 1).1,4

The ARIAM registry is a multicenter (national) observational registry characterized by voluntary participation (Annex 1) and with an annual cross-section (3 months), created by the Spanish Society of Intensive and Critical Care Medicine and Coronary Units (Sociedad Española de Medicina Intensiva, Crítica y Unidades Coronarias [SEMICYUC]). The registry was created in 1994, and in 2010 became known as the ARIAM-SEMICYUC registry. It is integrated within the Cardiological Intensive Care and Cardiopulmonary Resuscitation working group of the SEMICYUC, and seeks to improve patient care in the field of ischemic heart disease. At present, it is the largest Spanish registry in this field, with an average of 2335 new registries a year and a current total of 23,357 registries.5,6

The optimum method for designing adequate scores remains unclear. In their article published in Medicina Intensiva, Nuñez et al. described intensive care medicine as an ideal field for the application of big data analysis (BDA) and machine learning (ML) techniques, which in the future may improve clinical research and allow more precise patient treatment. Artificial neural networks are a clear example of this.7-12

The aim of the present study was to design a mortality indicator for all forms of ACS in the Intensive Care Unit (ICU) (ARIAM indicator [ARIAM’s]), based on the variables that may be available at the time of patient admission to the ICU, employing the data of the ARIAM registry, and using a supervised predictive model (neural network).

The study included a total of 18,123 patients with ACS admitted to the DICM. Of these, 865 were excluded due to missing information or data compilation error. The patients corresponded to 64 Spanish hospitals of all levels and distributed throughout the country. The global mortality rate of the included patients was 3.5% (n = 603).

Thirty variables were analyzed (Table 2), including sociodemographic data, vital signs upon admission, laboratory test parameters, treatments and techniques used, and place of care, etc. In the group of patients that died, 4.6% were females, and the mean age was 73.5 ± 11 years versus 64.8 years among the patients that were discharged. In turn, 4.1% of the patients presented arterial hypertension, 3.6% dyslipidemia, 4.3% had electrocardiographic (ECG) alterations consistent with ST-elevation ACS (STEACS), and 36% presented Killip class IV. The mean systolic blood pressure was 110.9 ± 35 mmHg, with a heart rate of 87 ± 27 bpm, creatinine 1.62 ± 1.3 mg/dl and hemoglobin 12.6 ± 2.3 g/dl. The management strategy consisted of primary percutaneous coronary intervention (PCI) (3.5%) or fibrinolysis (6.1%).

In relation to the study endpoint or event, 19 variables showed very significant differences (P < .001) (Table 3). The type of treatment (primary PCI or fibrinolysis) also proved significant (P < .001).

Supervised predictive model (SPM) (Annex 4)

The new indicator (ARIAM’s) yielded a mean score of 0.0257 (95%CI: 0.0245−0.0267) in the patients discharged from the ICU and of 0.27085 (95%CI: 0.2533−0.2886) among those who died (P < .001). This indicator derived from the softmax function of the output layer can be easily interpreted as a mortality predictor, with a score of 0.0835 indicating a mortality probability of 8.35%.

In the model obtained, all the entered variables showed a certain predictive value, which proved very high (IVn ≥ 20%) for creatinine, Killip class, age, cardiac arrest, systolic blood pressure, vascular disease, heart rate, first medical contact, body weight, hemoglobin, heart failure, ECG and initial management strategy (Fig. 2).

Figure 2.

Ranking of variables according to the Importance Value (IV) of the SPM.

(0.78MB).

Performance measures

Regarding the performance measures for the entire series (with an indicator cut-off point of > 0.04), specificity was 88.19%, with a negative predictive value of 99.19% (Annex 5).

The area under the ROC curve of the model obtained was 0.918 (95%CI: 0.907−0.930). This “c” statistic compared with two other indicators (GRACE and TIMI scores) as predictors of ACS mortality in the DICM showed: ARIAM’s 0.918 (95%CI: 0.907−0.930) versus GRACE 0.889 (95%CI: 0.874−0.903; P < .05) and versus TIMI 0.763 (95%CI: 0.741−0.784; P < .01)(Fig. 3).

Figure 3.

ROC areas ARIAMs vs GRACE and TIMI.

(0.34MB).

Fagan plots were used to determine the clinical relevance of the score. A positive ARIAM’s score indicated an increase in mortality risk to 19% (95%CI: 18%–20%), while a negative score indicated a decrease in mortality risk to 0.9% (95%CI: 0.8%–1.01%) (Annex 6).

Cardiovascular disease is the leading cause of death, morbidity and healthcare costs worldwide, and its frequency is increasing as a result of aging of the population. At present, cardiovascular disease causes 1.8 million deaths each year in Europe (20% of total mortality), with important variations among countries.1,13,14 In Spain, the management of ACS implies a great consumption of resources; correct risk stratification can therefore be regarded as crucial.3

In addition to establishing diagnostic, management and predictive criteria, a prognostic scale should be easy to use and should measure a clinically relevant outcome.15 Both the current clinical practice guides and the quality indicators of the SEMICYUC recommend early risk assessment of all ACS patients, based on the GRACE scale.13,16

Prognostic scales should be applied to the population for which they were designed. Thus, an indicator developed for the general population will not be valid in the DICM setting, and vice versa.15 In this regard, the tools used to date (GRACE and TIMI) have important limitations, since they are not specifically related to mortality on admission to the DICM but to mortality at 6 months and 14 days, respectively.17–21 In addition, the TIMI score exhibits differences concerning the clinical characteristics of the included patients: it does not include individuals with cardiogenic shock (Killip class IV), and furthermore, since it was designed based on a study of fibrinolytic treatments, it does not include patients subjected to primary percutaneous revascularization – which nowadays is the most frequent treatment prescribed in patients of this kind.17–22

In order for a scale to be practical in the clinical setting, it must include those variables which have been shown to be most relevant in predicting the established endpoint or event. In addition, the instrument should be novel, reproducible and easy to upgrade. The ARIAM’s uses 19 variables upon admission, including sociodemographic, laboratory test and clinical parameters.14,16 This new scale has shown good mortality discriminating capacity at patient admission, surpassing the performance of other previously used scales.

Patients with a positive ARIAM’s score have a 19% probability of dying from ACS. The diagnostic accuracy of the model was 96.81% in the training set and 96.79% in the validation set. The mortality risk increases to over 19% when the score is positive, and drops to under 1% when negative. The scale therefore shows good study event discriminating capacity and prioritizes the treatment strategy. In contrast, a negative ARIAM’s score identifies low-risk individuals and can guide patient triage and prevention strategies.

The area under the ROC curve of our indicator is even better than that of GRACE; it is therefore able to discriminate mortality more precisely.18

The design of new synthetic variables created from the times recorded in the ARIAM could improve the performance of the model. An example of this is an analysis of the time interval chest pain onset – arrival in the ICU; although not identified as an independent predictor of mortality in the ICU (since there is great data dispersion in the sample), it does exhibit interesting differences that should be analyzed more in detail in future studies.16

This new model offers improvement with regard to the potential limitations of the existing scales, since it allows for early risk stratification. In recent years, new scores such as the M-CARS have been developed in view of the need to upgrade the existing tools. Nevertheless, these new scales have important drawbacks with respect to the model we propose: they are not specific of patients with ACS, and have been designed based on single-center studies. Thus, the generalization of the M-CARS to other populations may constitute an important limitation.23

The present study has two limitations that should be mentioned. The first limitation, common to all risk scores, is that although the model is good at discriminating risk groups, it does not necessarily correctly predict individual risk. The second limitation is inherent to machine learning methods, which can work very well in our training and validation sets (internal validity), but the model must be extended to new series of data or web registries (external or prospective validity), to thus validate the new scale in future populations.

The ARIAM’s, created from an artificial neural network (ANN), is a clinical management scale better suited to the current population, more accurate and reproducible, and which can be upgraded periodically. In the ICU it is very useful for clinical assessments and may serve as a reference in quality studies. This new scale may be of use in establishing comparisons with other previous predictive scales, to promote the conduction of new investigations.

The authors of this study are the principal investigators of their respective hospitals, and have contributed the largest number of cases to the ARIAM registry data included in the study.

The study design and drafting of the manuscript were carried out by JJAB, ARG and HLG, with due correction by the rest of the authors, who agree with the contents.

The authors declare that they have no conflicts of interest.