It is known that the occurrence of adverse events significantly impacts on mortality rates and health costs.1 Therefore, clinical safety is a priority in care.2–4

The critically ill patient is complex and frequently demands for the implementation of a great number of decisions and procedures in short periods of time. This increases the likelihood of errors, and therefore adverse events.5 Although errors, categorized as those of commission or those of omission,6 can be made anywhere in the hospital, critical care patients are among those least able to withstand the consequences of a mistake.7 Recent studies have highlighted the presence of multiple errors in intensive care. For example, Garrouste-Orgeas et al.8 reported that adverse events in the intensive care unit (ICU) have considerable prognostic significance, with a threefold increase in mortality among patients who experience more than two such events. Furthermore, Valentin et al.9 conducted a cross-sectional, observational study, carried out over 24h in 205 ICUs, in which 39 adverse events were described per 100 patients per day. Organ failure, a greater intensity in level of care and time of exposure were all related to these events. The same group10 reported that more than half of the errors were classified as errors of omission. These errors, usually less visible, may be associated to increased morbidity and mortality and are usually related to a lack of adherence to recommendations made based on scientific evidence. Often, in the critical care setting, there is a discrepancy between these recommendations and clinical practice.11 In Spain, the SYREC12 study also reported an increased risk of incidents in patients admitted to ICU (no-harm events and adverse events). Most incidents were considered avoidable.

Several methods have been described involving critical care patients to detect adverse events, ranging from the use of observers,13 to self-reporting systems or retrospective chart reviews.14 These methods tend to focus on the presence of adverse events and are not sensitive for routine monitoring of the areas of care where errors are most likely to occur.15,16 Some authors have investigated alternative, proactive methods for analysing the safety of critical patients.17 Such a methodology, which is less time consuming and provides rapid feedback, allows for immediate changes in practice where it might be necessary.

The main objectives of this study were to develop a checklist of safety measures (SMs) specifically designed for critically ill patients and based on sound scientific literature, and to apply them in real time (randomizing variables and patients) during routine clinical work (audits), with the aim of minimizing errors of both commission and omission, and evaluating the utility and feasibility of the procedure.

The Ethics Committee of the Hospital Universitari Joan XXIII in Tarragona approved the study, considering also that written informed consent by a patient could be waived for participation in the study.

Two sections can be differentiated: (1) Design of the checklist. (2) Pilot study and evaluation of utility and feasibility.

Twenty-one experts were invited and agreed to participate in the study.

Despite the awareness of patient safety and quality of care issues increasing in both clinical and political arenas, the translation into research and clinical practice has been incomplete although the topic has been found to have significant benefits.26 To our knowledge, this is the first study designed to build a clinically useful tool aimed at improving the care and safety of critical adult patients with the following characteristics: aimed to prevent errors of omission and commission, easily applied by the clinician during routine clinical work, and finally, rendering useful information that can be applied immediately by modifying the care regimen of the critical patient. This study has been conducted within the context of an effort of our institution and our ICU to gradually establish a culture of non-punitive safety at conjunctural moment aimed at reshaping medical practice.27

The Delphi method, used to achieve consensus among experts, has been useful in defining the SMs contained in the checklist. Significant qualitative modifications were made in some cases. In particular, the content of the SMs “Assessing water balance and fluid adjustment” and “Acute renal failure assessment” were modified sequentially in all three rounds of Delphi, which shows the effort it takes to reach consensus on controversial issues in the management of critically ill patients. On the other hand, consensus was simple in aspects that are included in national or international guidelines.28

Ursprung et al.17 conducted an interesting pilot study in a neonatal intensive care unit looking at the feasibility and utility of a real-time safety auditing programme that takes place during routine clinical work. The authors showed that real-time safety audits can flag up a broad range of errors. Their study design, however, did not allow errors of omission that might occur in important areas of critical patient management. In this current pilot study, the detection of errors of omission was paramount. For example, ART-SACC has promoted daily reflection on important aspects of critical care, such as fluid adjustment or antibiotic treatment, since it is well known that an adequate or appropriate balance of fluids and/or an adequate antibiotic treatment impacts on the prognosis of critical patients.29,30 In that way, this pilot study suggests the utility of ART-SACC for promoting changes in patient care using a tool designed to limit errors of omission and commission.

Other authors31 have also conducted random safety studies in neonatal ICUs, although focussing their analysis exclusively on issues related to infection. This approach may not be sufficient in the context of the complexity of critical patients. ART-SACC was designed as a functional tool to assist the clinician in the structuring of essential clinical decisions such as assessing haemodynamics, renal failure, sedoanalgesia and nutrition, in addition to infection. In another study, Simpson et al.32 tried to assess the impact of a checklist of clinical safety in critical patient care. They concluded that they were unable to measure, with certainty, the exact impact that the checklist had on discovering errors and omissions in ICU care. In order to address this issue, in the present study a variable aimed to quantify the impact of daily random evaluations was created: “Improvement Proportion Related to ART-SACC (IPR-ART-SACC)”.

It is important to highlight that those SMs clearly defined by international guidelines had a high IPR-ART-SACC. For example, the “assessing adequate tidal volume” had an IPR-ART-SACC of 18.8%. Of particular concern are SMs whose IPR-ART-SACC was over 25%. SMs with a high IPR-ART-SACC require specific analysis in order to plan strategies that will improve the quality of care. It would be expected that, following the implementation of improvement plans, the IPR-ART-SACC would progressively decrease. In this way, ART-SACC could be considered a tool for monitoring the impact of actions (protocols, clinical guidelines, training, etc.) in clinical processes that require improvement.

The “No” answers are also relevant because they did not allow any instant changes in clinical approach. However, the SMs to which there was a “No” response, such as the glycaemia control protocol and the verbal orders policies had been recently addressed as part of the routine clinical in our ICU and considered for urgent completion in specific protocol recently implemented in our ICU.

It was not a specific objective in this study to evaluate how acceptable ART-SACC would be in our ICU, but the clinicians who were involved were generally responsive and helpful. Unlike the experience reported by Ursprung et al.,17 our research team did not receive any comments about reducing the number of SMs in each assessment. Perhaps the randomness of patients and SMs served to improve collaboration and maintain alertness in the healthcare staff, something that we consider beneficial in places such as the ICU, which handles a large amount of information. Furthermore, in some ways, doing it this way reduces the impact of possible bias associated to any observation. Undoubtedly, another aspect that has helped staff to accept ART-SACC has been the absence of punitive consequences. On the contrary, the exchange of views held at bedside has been instructive in many cases from the educational point of view.

This pilot study has had several limitations. Firstly, there were a small number of audits. Secondly, the SMs included in ART-SACC were designed and adjusted to the needs of a single centre. In future studies, in order to attain external validity, will be important to include other ICUs with different levels of complexity. Thirdly, although the information extracted and the perception of the study was positive, the time spent performing ART-SACC needs to be evaluated because it is a variable that is directly related to its feasibility, particularly if it is considered in the mid and long term. Finally, it probably would have been helpful to know the incidence of errors of commission and omission before and after the study.

In conclusion, the Delphi method was useful for designing and compiling the list of 37 SMs that were scheduled in 10 blocks. The utility of ART-SACC was assessed through the IPR-ART-SACC. The audits helped to direct attention towards minimizing errors (action or omission) and improving clinical care in more than 85% of the selected SMs. Although feasibility was adequate and acceptance seems positive, the quantification of the resources used, essentially time spent on developing ART-SACC, has to be considered if this tool is to be implemented in routine clinical work and more centres with different levels of complexity are to be involved.

Sirgo G, Olona M, Martin MC, Esteban F, Pobo A and Bodí M have made substantial contributions to conception and design, acquisition, analysis and interpretation of data. Sirgo G has drafted the submitted article. Olona M, Martin MC, Esteban F, Pobo A and Bodí M, have revised the manuscript critically for important intellectual content. ART-SACC study Experts: Alonso A, Benito E, Cabré Ll, Calvo E, Chamorro C, Corral E, De Mendoza D, Diaz E, Grau S, Gutiérrez I, León C, Maynar J, Merino P, Ochagavía A, Quesada A, Alba A, Roca J, Rodríguez A, Sagrera C, Sirvent JM, Vallés J have made substantial contributions to conception and design during the Delphi process and have revised the manuscript critically for important intellectual content.

This study was supported by grants from the Fondo de Investigación Sanitaria (ISCIII, FIS grants, project PI11/02311). The funding agency had no role in study design, in collection, analysis or interpretation of data; in writing of the manuscript; or in the decision to submit the paper for publication. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Institute of Health Carlos III (ISC III) from Spain.

The authors have no conflict of interest to disclose.