Real-time random safety audits constitute a tool designed to transfer knowledge from the sources of scientific evidence to the patient bedside. It has proven useful in critically ill patients, improving safety in the process of critical patient care, turning unsafe situations into safe ones in daily practice, and ensuring adherence to scientific evidence. In parallel, the design and methodology involved affords process indicators that will make it possible to know how we provide care for our patients, evolution over time (with regular feedback for professionals), the impact of our interventions, and benchmarking.
Los análisis aleatorios de seguridad en tiempo real son una herramienta que ha sido creada como un método de traslación del conocimiento desde las fuentes de la evidencia científica hasta la cabecera del paciente. Ha demostrado ser útil en el entorno del paciente crítico, en términos de mejora de la seguridad en el proceso de cuidados al paciente crítico, transformando situaciones inseguras en seguras en el día a día, garantizando la adherencia a la evidencia científica. Paralelamente, por su diseño y metodología ha permitido disponer de indicadores de proceso que hará posible conocer cómo realizamos la atención a nuestros pacientes, la evolución en el tiempo (y el feedback periódico a los profesionales), el impacto de nuestras intervenciones y el benchmarking.
Intensive medicine lives immersed in a period of deep transformation that requires new healthcare scenarios, new ways of looking at the setting, and new roles. In these times of change, intensive care units (ICU) are no longer the playing ground of superspecialized professionals to become places beyond the constraints of the physical walls whose destiny is dictated by the economy of knowledge – the actual base of research and teaching.1 However, this renovation does not come without the problems that have been dragging us down for a long time and for which we still have not found a solution. One of them is the huge gap between clinical practice and scientific evidence.2
On the other hand, as professionals of one of the most basic specialties within the healthcare system, our responsibility (both individual and collective) should embrace the quality of healthcare as a paradigm in order to achieve the goals that our society demands. Efforts made to be specific within this setting have allowed us to develop models of excellence that have been exported everywhere, such as the European Foundation for Quality Management (EFQM),3 whose landmarks are based on leadership, process management, professional satisfaction, the value measured by the patients, the results adjusted to the means, the promotion of creativity and innovation, the development of alliances, and the promotion of the system sustainability.4
In this context, sending out an invitation to face such a change without suggesting one position that will help face such a transient period or, most important, without suggesting the appropriate tools to lead such an effort, is purely demagogical.
When it comes to position, teamwork is a concept we should conquer again: one setting where members collaborate, interact, and share knowledge and resources, and where we depend on one another to be able to carry out our tasks.5 This requires training (e.g. drills and simulations), the creation of efficient working stations where situational awareness operates6 and effective communication becomes an essential part of the process. In complex situations like clinical practice, effective communication not only means building up the structure of a team but also sharing the mental models needed that, by the way, are conditioned by knowledge and experience.7,8
Tools can be of two different types. Some are transversal tools like clinical information systems (CIS). There is growing experience with CIS and their healthcare and organizational results are promising9–11 but they will also improve safety and teamwork and, eventually, lay the foundations of new clinical research.12,13 There are other kinds of operative tools that are representative of strategic leverage. Ideally these tools are effective in complex settings, have their origin in adaptative leadership, guarantee the adherence of users to better scientific evidence, accompany process execution, and facilitate effective communication. During the last few years, different tools coming from other industries have been introduced in the healthcare setting aimed at improving teamwork, facilitating effective communication among healthcare providers, and improving the safety of patients. Some of these tools have been modified and even re-designed in order to adapt them to specific settings. Table 1 shows the most widely used tools that share elements among them.14
Tools to improve communication and teamwork.
| Tool | Definition |
|---|---|
| Checklist | Tool to verify if certain standard proceedings have been completed and/or we have all the necessary equipments/resources to be able to carry out an activity in a safe way. |
| Daily goal sheet | Through the use of checklists all tasks to be performed before upgrading the patient to the next level of healthcare are verified, agreeing on the goals established within the team for a short period of time. It allows the identification of the goals of everybody in the team. |
| Briefings/Debrifings | Short meetings held by the team to assign roles, establish expectations and anticipate problems. The identification of risk situations or adverse events may be in their agenda. Debriefing sessions are meetings designed to exchange information after the interventions from the team have been carried out in order to assess the effectiveness of the team and review the interventions. |
| Team huddles | Ad hoc short meetings aimed at re-establishing already ongoing plans, by dynamically adjusting and adapting such plans to the real situation at every moment. |
| Handoff | Exchange of structured information among different healthcare providers and transfer of responsibilities during the continuous management of patients. |
| STEP (S – status; T – team members; E – environment; P – progress) | It facilitates situational awareness and crossed monitoring of other team members so that errors and risk situations can easy and rapidly spotted. It establishes goals and evaluates is the adequate progress is being made to achieve such goals. |
Our group has been working on the design of one tool that, on top of meeting the requirements of operative tools, feels close to healthcare providers since their help will be necessary in one of the top health priority areas – clinical safety.15 This is how the real-time random safety audits (RTRSA) have been born. As discussed below, RTRSA can interact with the CIS to safeguard the safety and quality of data while providing significant clinical information (process indicators).16
Safety. Types of errors. Proactive or reactive measurementsThe meaning of clinical safety is intimidating to us, the healthcare providers. You only need to come close to the most basic terms to start feeling a certain sense of unease: errors, incidents, adverse events.
When someone thinks of clinical safety for more than five minutes, the essence of the problem becomes evident: it is elusive and really hard to measure.17 How do I know that my ICU is safe? How should I know that in my ICU we are coming dangerously close to an unsafe threshold of risk? How can I improve safety in my ICU?
Among these errors, the errors of commission (incorrect implementation of one measure or action from the way it was originally planned or indicated) are more evident (e.g. the administration of inadequate doses of drugs due to prescription errors), easier to see, and usually draw more our attention than the errors of omission do (failure to implement one measure or action from the way it was originally planned or indicated). The latter are particularly disturbing; they are more insidious and hard to see; they may be covered and protected by habits and routines, and the clearest example of this is the lack of adherence to good clinical guidelines. Paradoxically, this happens more frequently in patients with more serious conditions18 and has often been cited as one of the most difficult systemic problems to solve.19 Nevertheless, some of the barriers that may help us plan coping strategies have been identified: lack of leadership or lack of resources, fear of losing one's independence while in the decision-making process, disagreement with the results coming from the guidelines, organizational issues, or being refractory to change working habits.20–23
Reactive tools for analysis and safety improvement, like the notification of incidents or adverse events isolatedly cannot guarantee its effectiveness for two reasons basically24: (1) because the notifications may be biased, (2) because there are times when it is not followed by detailed analysis coming from trained and multidisciplinary teams (root cause analysis) and/or the information obtained does not go all the back to healthcare providers (retro-feedback) through the generation of new knowledge or system improvements; this is why we run the risk of everything repeating itself. In complementary and parallel ways, we have proactive tools (e.g. the modal analysis of failures and effects) that are used to analyze the system and facilitate the provision of safe healthcare while managing critically ill patients.
Our proposal is a mixed proposal. It includes aspects from both types of tools. It tries to detect risk, take the opportunity of learning from errors, and go deep into knowledge, and all into the very nature of errors of omission.25 RTRSA stand as a response to the need for quantifying and measuring what we do, adapting the way of practicing healthcare to scientific evidence, and improving the safety of critically ill patients (above all at the expense of minimizing the errors of omission). This tool has been created as a translational method of the knowledge coming from the sources of scientific evidence to the patient's bedside while playing a double role: being a process indicator (from now on, proportion of related improvement with the RTRSA [PRI-RTRSA]) and, incidentally, being an instrument that has protected our setting by improving the culture of safety. Thus, the effects from this tool will not only improve the patient's prognosis but also make changes in the working team and the organization that will eventually lead to long-term results.
Background impact of safety rounds and checkup listsSafety rounds have been designed to consolidate the culture of safety among healthcare institutions. Despite their positive effects, the fact that in these institutions there are professionals who have nothing to do with these services (e.g. managers and CEOs) make clinicians perceive such safety rounds reluctantly because they anticipate a certain feeling of being audited. Also, when it comes to operational aspects, the frequency of their implementation is burdened by their own structure.26 When it comes to checklists, no evidence has been published on their negative impact.27 In the studies previously published, the effect they have reducing mortality rates (as an indicator of greater relevance) has been objectified almost exclusively in the surgical setting, in relation to the surgical checklist proposed by the World Health Organization (WHO). Haynes et al.28 confirmed the reduction of mortality rates and perioperative complications following the implementation of checklists in eight different centres representing different areas from the WHO, and with different realities (economic circumstances, health systems, organizations). Impact meant a higher adherence to six different safety measures, which could be confirmed through process indicators. However, when they tried to replicate it in over one hundred Canadian hospitals, results were not the same, probably due to methodological issues, but mainly due to inadequate planning, low involvement from healthcare providers, and inadequate leadership.29 It is mistake to think that a technical solution (checklist) will solve what is purely an adaptation problem (sociocultural). Simplifying evidence is necessary but it is not enough if we want to guarantee knowledge translation. It should be accompanied by one strategy aimed at mitigating barriers (technical, sociological, psychological, and emotional) by implementing adequate leadership, education, training, and information feedback to healthcare providers.30
In the setting of critically ill patients, the impact that the use of checklists has on mortality has been confirmed sporadically. Most studies assess the postoperative effect compared to pre-checklist periods. Weiss et al.’s study31 confirmed a reduction of mortality rates and hospital stays, and the improvement of various processes with the implementation of checklists conducted by ICU personnel (the so-called prompters). This effect could not be confirmed in other ICUs of similar characteristics from the same centre where the use of the checklist had been implemented without the drive of a healthcare provider. These results could not be confirmed either in one Brazilian multicentre study that has been published recently.32 In this study, the prompter's role was not at the patient's bedside but for verification purposes of the team's measures only; also, the number of patients included by the hospital was low, and the study period was short. However, it showed that the management of critically ill patients improved, as other studies had already confirmed.22,27,33–35
Assessing the impact that the different tools recently published in the medical literature have should be based on healthcare improvement, teamwork improvement, organizational improvement, and any other types of outcomes in patients.
Idea and design of real-time random safety auditsThis is one knowledge translation planning tool aimed at improving the patients’ safety that allows us to identify topics with room for improvement in order to guarantee adherence to good clinical practice, based on high level scientific research while facilitating the modification of the healthcare process when managing critically ill patients. It includes these features: (1) it is a checklist of science-based safe measures; (2) randomization means that during implementation, 50 per cent of safe measures and 50 per cent of patients admitted in ICUs are selected at random; (3) it is implemented on real time, at bedside, during clinical practice, and requires, at least, the presence of the attending physician, the nurse, and one healthcare provider (prompter) for tool verification purposes; (4) the healthcare provider who verifies the tool needs to be an experienced clinician with basic training using this tool; (5) the tone of this checklist needs to be close to what would be an exchange of clinical impression among the participating providers, without any punitive features; lastly (6) it is not a self-applicable tool or should be used during shift changes.
The first step towards the creation of RTRSA was the meticulous review of scientific evidence. The authors wondered: As a professional in intensive medicine, how many measures related to the management of our patients should be mandatory because they are supported by enough solid scientific evidence? This is how 38 different measures were identified. The measures received their own definition and inclusion criteria and were arranged in ten different blocks: (1) mechanical ventilation, (2) haemodynamics, (3) renal function/continuous renal replacement techniques, (4) analgesia/sedation, (5) therapy (1), (6) therapy (2), (7) devices, catheters and additional tests, (8) nutrition, (9) nursing, and (10) structure, and others. The structuring of the checklist was made following internationally accepted recommendations.36
Once the initial measures could be identified, the Delphi method to achieve consensus among different groups of experts was used.37,38 For the creation of this group, specialists and nurses on intensive medicine with extensive experienced in clinical healthcare and research on each of the blocks were contacted. Finally, 21 experts were involved. It was deemed that three rounds would be enough to be able to reach consensus. Such consensus was looked for by assessing the importance and feasibility of agreement through the opinion of each and every one of the 38 variables.39 We took into consideration not only the experts’ suggestions for a better definition of the variables but also the proposals on new variables until we achieved the checklist final version including 37 safety measures distributed in ten blocks in the 3rd round.40 Let us now take a look at the two basic characteristics of RTRSA.
What does randomization contribute?Random checklists have been adapted from the industry. Conceptually, the interest on this form of application lies in the capacity to verify the critical points of a complex working system, not the whole system, with the peculiarity that the repeated use of these proactive tools improves our attention span, while reducing the chances of making mistakes in critical points previously identified.
This methodology has been used successfully in the intensive care setting22,41 but not exhaustively like we propose. In our experience, we contemplate two different types of randomizations: randomization of measures and randomization of patients. Thanks to this formula, the use of RTRSA was not associated to one routine only: except for the days assigned to the rounds no member of the team knew what patients or measures would be evaluated.
Why aren’t checklists self-applicable?It seems contradictory. If we look at the large amount of bibliographic references about the use of checklists,27,42,43 the most effective solution would have been to turn to one checklist that, for instance, on a daily basis, and right after the prescription of a given therapy in our CIS, would automatically appear in our laptop as a reminder, so we could fill it out for the therapy to be visible. This would have been the easiest thing to do and in tune with new technologies.44
The truth is that the problem is a little more complex than that. The verification of a checklist does not guarantee that the gap between the knowledge that we use in daily healthcare and the knowledge that really exists will be shortened. We should remember all the psychological, social, and cultural barriers (including habits, routines, and mechanisms of defense based on our own experiences) that do not let us lean on scientific evidence and that, probably for the same reason, make collaborative effort inffective.30 For this reason, in our opinion, actions aimed at modifying behaviours or coping strategies of complex situations in settings with a lot of information and where the shared result (co-created) will be seen as an added value will require direct interaction with the healthcare providers that we will need to be trained. On a more specific area, other authors31 have mentioned one self-applicable checklist directed by one healthcare provider that improves, among other results, the days on mechanical ventilation and the use of antibiotics. In our own experience, these kinds of transforming experiences occur when a critically ill patient who is in a hospital cubicle and is being assisted by the attending physician (accompanied by the resident), the prompter, or the healthcare provider in charge of conducting the checklist who also acts as a knowledge translation instrument (but not as an examiner). We have been able to see how the nursing specialist feels part of the essential aspects of patient management, for the resident, it is one educational tool with which he/she can adapt to the philosophy of the unit, for the assisting physician (after the initial shock of having to discuss the essentials of clinical practice with a colleague) it is a reminder of the basic measures of treatment and, lastly, for the checklist conductor, it stands as a unique opportunity to go deep into the know-how of each healthcare provider's clinical practice. In this context, we have confirmed that the use of RTRSA creates a favourable environment for the implementation of other additional tools of proven effectiveness as daily goals.45
Implementation methodology of real-time random safety auditsBoth the safety measures included in the checklists, and the methodology used to develop the RTRSA are equally important.
Description of checklistsTaking into account accumulated experience and previous results, and based on the benefit derived from using RTRSA when it comes to work loads, the use of RTRSA is recommended between 2 and 3 days a week. In each round, 50 per cent of variable blocks and 50 per cent of patients admitted to the hospital who will eventually undergo the verification of safe measures are randomly selected. Thirty-seven measures included in 10 blocks have been included (Fig. 1). Each safe measure includes one definition, various criteria and one methodology already published by our group for evaluation purposes.40 We are talking about eligible patients whenever these patients meet the criteria established for every safe measure to be evaluated. If these criteria are not met so that the measure can be evaluated, such measure shall not be applicable.
Prompters’ role and training
In RTRSA, safety rounds are conducted right after clinical rounds and they require the presence and participation of the professionals responsible for the management of every patient (physician, resident, nurse). The prompter is that professional (not directly in charge of the patient) who has been trained in the methodology of RTRSA, and who, together with the team, is responsible for conducing the verification of safe measures. In this sense, RTRSA have had a good level of acceptance, and they have been easily introduced in the daily routine, unlike other types of checklists.22 The causes that explain the better acceptance of RTRSA may be the professional training of the RTRSA conductor, the randomization of patients and variables that provide a permanent state of expectation and, lastly, the absolute absence of punitive elements.40
SystemIn the RTRSA the answer to the verification of every safe measure may be: (1) “Yes”: if what is described in the content of the variable has already been accomplished prior to the evaluation. (2) “Post-evaluation yes”: if what is described in the variable has not been accomplished prior to the evaluation but, thanks to the evaluation, there is a change in the actions from the working team. (3) “No”: if what is described in the variable has not been accomplished and there is no possibility of modification. (4) “Not applicable”: the variable was not analyzed in patients who did not meet the inclusion criteria described in each measure.
The variable used to quantify the results from the rounds and the effect they have is called PRI-RTRSA: It is usually defined by this formula:
PRI-RTRSA=number of “Post-evaluation yes”*100/number of evaluations accomplished (total-“not applicable”).
This is how every time we get a “post-evaluation yes” we will know that the RTRSA have made a correction of this or that clinical action and turned an unsafe situation into a safe one, while adapting scientific evidence to clinical practice. On the other hand, PRI-RTRSA are defined as process indicators in such a way that there may be one PRI-RTRSA for every measure, one PRI-RTRSA-B (for every block of measures), and one PRI-RTRSA-G (global, for all measures).
All answers are included in a platform-enabled website (best accessed through a tablet) at the patient's bedside so there can be real-time assessments of the results, and their progression can be compared through different time frames and among the different units involved. Fig. 2 shows one image of the RTRSA® website (in particular the block of mechanical ventilation, and all the different possible answers to the measures contained in the block). The tool includes the timer that will let us measure the time elapsed in each safety round.
Real-time random safety audits and impact on the monitorization of safety and quality
Measuring safety is one big challenge for the lack of standard definitions, because the exact incidence of incidents, or adverse events is unknown, and, especially, for the lack of tools capable of measuring the variables associated with the process.16 But there is something else, ICUs are a well-defined model of knowledge-intensive organizations. This term is reserved for all those organizations where knowledge, and the management of knowledge and intangibles are the source for creating value and one of the most solid foundations of innovation.46 Thus, measuring safety would improve clinical practice, increase knowledge, and generate spaces for the development of new opportunities.
Our group has been able to define and confirm that RTRSA are feasible and useful. In order to draw these conclusions, and after the creation of this tool, we conducted a 1-month pilot study40 for the assessment of feasibility and utility. The first variable was defined as the number of times the evaluation was completed in relation to the number of days proposed plus all the days scheduled when the rounds could be conducted. This aspect confirmed the capacity of RTRSA to adapt itself to healthcare routine. This utility was analyzed by looking at how the indicator process PRI-RTRSA behaved. In this pilot study we confirmed that immediately after the rounds, 83.7 per cent of the measures were modified. But the most interesting thing of all was that PRI-RTRSA was ≥25 per cent in some measures. Another 4 month-multicentre study conducted some time later47 confirmed the utility of RTRSA when it comes to modifying clinical practice by transforming unsafe situations into safe ones. The multivariate analysis showed that all those changes occurring during the healthcare process were associated with the nurse/patient ratio, the physician/patient ratio, and with the severity of the patient's condition. It concluded that RTRSA improve adherence to clinical guidelines and that they are more useful when healthcare providers deal with big workloads and patients have more severe conditions. But the most relevant thing of all from this second study was that RTRSA caused a larger amount of changes in the management of critically ill patients in centres that participated in the initial design of the tool compared to the remaining participating centres. This transferred the constant need for education, training, and information feedback to healthcare providers. What this does is guarantee not only the team's commitment but also the proper use of tools.48 The implementation of blocks for the design of specific processes and/or new local protocols will probably contribute to strengthen this commitment made by healthcare providers.
In sum, PRI-RTRSA may be defined as process indicators for the assessment of the processes conducted, to know the degree of adherence to scientific evidence, and ultimately, quantify the gap between the therapies or interventions indicated based on the findings from high quality clinical research and therapies or interventions actually used in clinical practice. PRI-RTRSA may be adapted to local needs and improve communication and teamwork.
Real-time random safety audits and safe informationAfter seeing the impact they have had in the healthcare process and safety of the critically ill patient, the RTRSA seem to be ready to improve the safety of information at ICUs.
CIS may be the answer we are looking for to questions as important as having quality indicators to make the continuous improvement of quality, the professional participation in managerial duties (clinical management), and benchmarking49 possible. Similarly, the future of CIS as sources of data for clinical research through secondary analyses looks bright ahead.12 Due to the immense amount and type of information generated in ICUs, it is considered the ideal setting for the implementation of artificial intelligence in collaboration with computer experts. Predictive models can mean a profound change when compared to clinical hypotheses. If, in the past, it was the clinical trials that were determinant on this regard, now it is the power of these models (in one secondary analysis of databases) that has perspective-changing potential.
All this requires that on top of complex computing models or advanced database management systems that guarantee the acquisition and validation of data, the commitment from healthcare providers managing critically ill patients is guaranteed through safe and quality information.50,51 RTRSA may contribute to the education and training of our working teams by verifying the correct storage of data. This may be accomplished by including one block of variables in order to guarantee: (1) the correct integration of information from other hospital information system departments, (2) the safe and continuous integration of different ICU monitoring devices and life-support interventions, and (3) the filling out of annual forms and registries by healthcare providers managing the patients.
ConclusionsSafety audits are useful tools to improve the safety of critically ill patients and, consequently, to improve healthcare, which in turn contributes to making fewer mistakes, above all, errors of omission associated with the lack of adherence to scientific evidence in daily clinical practice. In addition to promoting knowledge translation through immediate information feedback at the patient's bedside, safety audits provide us with process indicators to have information on the management of patients, the passing of time (and the periodic information feedback to healthcare professionals), the impact of our interventions, and benchmarking. Also, a context of safe information provides a bright future of innovative methodology for clinical research. Fig. 3 shows RTRSA, their main goal (improve quality and safety in critically ill patients), their basic characteristics, the impact they have (on structure, process and outcome indicators) and the role they play in the continuous improvement of quality, and future of clinical research.
Funding
This study is partly funded by the Instituto de Salud Carlos III Health Research Fund - Project PI11/02311.
Conflicts of interestsWe the authors declare that while conducting this paper there were no conflicts of interests linked whatsoever.
Please cite this article as: Bodí M, Oliva I, Martín MC, Sirgo G. Análisis aleatorios de seguridad en tiempo real, una herramienta transformadora adaptada a los nuevos tiempos. Med Intensiva. 2017;41:368–376.
