Knowledge discovery and knowledge validation in intensive care

Abstract

Operational protocols are a valuable means for quality control. However, developing operational protocols is a highly complex and costly task. We present an integrated approach involving both intelligent data analysis and knowledge acquisition from experts that support the development of operational protocols. The aim is to ensure high quality standards for the protocol through empirical validation during the development, as well as lower development cost through the use of machine learning and statistical techniques. We demonstrate our approach of integrating expert knowledge with data driven techniques based on our effort to develop an operational protocol for the hemodynamic system.

Introduction

An abundance of information is generated during the process of critical care. Much of this information can now be captured and stored using clinical information systems (CIS) that have become commercially available for use in intensive care over the last years. These systems provide for a complete medical documentation at the bedside and their clinical usefulness and efficiency has been shown repeatedly [8], [9], [13]. While databases with more than 2000 separate patient-related variables are now available for further analysis [10], the multitude of variables presented at the bedside even without a CIS precludes medical judgement by humans. A physician may be confronted with more than 200 variables in the critically ill during a typical morning round [25]. We know, however, that even an experienced physician is often not able to develop a systematic response to any problem involving more than seven variables [22]. Moreover, humans are limited in their ability to estimate the degree of relatedness between only two variables [15]. This problem is most pronounced in the evaluation of the measurable effect of a therapeutic intervention. Personal bias, experience, and a certain expectation toward the respective intervention may distort an objective judgement [6]. These arguments motivate the use of decision support systems.

Clinical decision support aims at providing health care professionals with therapy guidelines directly at the bed-side. This should enhance the quality of clinical care, since guidelines sort out high value practices from those that have little or no value. The goal of decision support is to supply the best recommendation under all circumstances [26]. The computerized protocol of care can take into account more aspects of the patient than a physician can accommodate. It is not disturbed by circumstances or hospital constraints. It bridges the gap between low-level numerical measurements (the level of the equipment) and high-level qualitative principles (the level of medical reasoning). While knowledge-based systems have mostly been applied for diagnosis and therapy planning (e.g. [19], [28]), some systems also aim at on-line patient monitoring [7], [21], [26]. Methods that have proved their value in handling low-frequency patient data are not applicable for on-line monitoring [21]. Quantitative measurements and qualitative reasoning have to be integrated in a system that recommends interventions in real-time. The numerical measurements of the patients' vital signs have to be abstracted into qualitative terms of high abstraction. The aspect of time has to be handled both at the level of measurements and the level of expert knowledge [4], [17], [21], [28]. In the expert's reasoning, time becomes the relation between time intervals, abstracting from the exact duration of, e.g. an increasing heart rate, and focusing on tendencies of other parameters (e.g. cardiac output) within overlapping time intervals.

One of the big obstacles to the more frequent implementation of decision support systems is the tedious and time-consuming task of developing the knowledge base. The decision support system for respiratory care at the LDS Hospital, Salt Lake City, USA [26], for instance, has been developed in about 25 person years. The method of guideline development itself is not supported by a computer system. Mechanisms of temporal abstraction and reasoning presuppose manually designed models or ontologies [4], [21], [28]. Why not use techniques of knowledge discovery and statistical time series analysis in order to ease the process of guideline generation? Machine learning and statistical analysis have been applied in building-up diagnostical systems successfully (e.g. [18], [20], [30], [36]). We now want to exploit the huge amount of data for the development of guidelines for on-line monitoring. Our task is to build a decision support system for online hemodynamic monitoring in the critically ill. We do not aim at modeling the actual physician’s behavior Imitating the actual interventions made by physicians is not the goal. Actual behavior is influenced by the overall hospital situation, e.g. how long is the physician on duty, how many patients require attention at the same time. Machine learning from patients’ data could lead to a knowledge base that mirrors such disturbing effects. Therefore, the learned decision rules have to be checked by additional rules about effects of drug and fluid administration. Our approach is to combine statistics, knowledge acquisition, and machine learning. Our aim is to develop a method for guideline generation that is faster and more reliable than current methods.

Data for statistical evaluation and learning can be provided by the CIS. However, the nature of the data is different from that gathered in controlled experiments. While a CIS in modern intensive care can take numerous measurements every minute, the values of some vital signs are sometimes recorded only once every hour. Other vital signs are recorded only for a subset of the patients. Hence, the overall high dimensional data space is sparsely populated. Moreover, the average time difference between intervention as charted and estimated hemodynamic effect can show a wide variation [12]. Even the automatic measurements can be noisy due to manipulation of measurement equipment, flushing of pressure transducers, or technical artifacts. In some cases, relevant demographic and diagnostic parameters may even not be recorded at all. In summary, we have a large amount of high dimensional, numerical time series data that contains missing values and noise. Using this data already at the stage of development of the decision support system stave off surprises at the stage of clinical experience as have been reported in [2], p. 5721: ‘the huge number of measurements classified as invalid is quite astonishing although it reflects the real clinical environments’.

In addition to problems of knowledge acquisition, we see a particular need for knowledge validation. It should be noted that many medical guidelines published today are neither evidence-based nor sufficiently validated against real patient data. The current procedure is to first develop the guideline, then represent it in a knowledge-based system, and finally to test it in clinical studies. In this ‘waterfall’ process, unrealistic assumptions, mistakes, and flaws are recognized at a late stage. In contrast, our approach includes validation from the very beginning. Using a knowledge-based system early on supports the validation of the knowledge base at earlier stages. Inconsistencies within the knowledge base as well as a mismatch of rules and patient data are detected while developing the knowledge base. This facilitates and focuses the knowledge-acquisition process.

In order to test our approach to using real clinical data for building and validating a knowledge base for online monitoring, we have constructed a system. Its overall architecture is shown in Fig. 1. The patients’ measurements are used to recommend an intervention and are abstracted with respect to their course over time. The recommendation of interventions constitutes a model of physician behavior. This asks for further validation. Therefore, a recommended intervention is checked by calculating its expected effects on the basis of medical knowledge. In this way, a qualitative assessment of a statistical prediction enhances the model of physician behavior in order to obtain a model of best practice. The medical knowledge constitutes a model of the patients' hemodynamic system. This model is validated with respect to past patients' data. The processes we have implemented are outlined in the following sections.

Given series of measurements of one vital sign of the patient, data abstraction detects and possibly eliminates outliers and finds level changes by good statistical practice. This abstracts the measurements to qualitative propositions with respect to a time interval, e.g. within time point 12 and time point 63 the heart rate remained about equal, from time point 63 to time point 69 it was increasing. Our approach is based on statistical time series analysis. Classical autoregressive moving average (ARMA) modelling [3] is applied with corresponding outlier- and level shift detection procedures using the new tool of a phase space embedding. The statistical method for data abstraction is described in Section 3.

Given the numerical data describing vital signs of the patient and his or her current medication, the task of acquiring state-action rules is to find the appropriate intervention. An intervention is formalized as increasing, decreasing or not changing the dose of a drug. The decision is made every minute. These rules were learned by the support vector machine [37]. Section 4 shows how we applied the support vector machine to learn state-action rules.

Given text book knowledge and explanations by an expert, the task of acquiring medical knowledge is to represent the effects of substances in different dosages, relations between vital signs, and interrelations between different substances, and validate the knowledge on the basis of past patients’ data. The knowledge acquisition and validation was supported by the MOBAL system [24]. Section 5 gives a short introduction to the system and its representation of medical knowledge. It is stressed how MOBAL is capable of checking the knowledge base against patient data.

Interventions — especially those recommended by a learned rule — are to be validated by the action-effect of rules of the knowledge base. This validation task is:

Given

• the state of a patient described in qualitative terms;

• medical knowledge;

• a sequence of interventions, and

• a recommended intervention,

find the effects of the current intervention on the patient. The derivation of effects is made for each intervention as forward inference within MOBAL. The effect should result in a stable state of the patient. The validation is detailed in Section 6.

The outline of this paper is as follows. Throughout the paper we report on the continuous development of a decision support system for intensive care as performed at the City Hospital and the University of Dortmund. We start with a description of the data acquisition process at the hospital and the resulting data set [13] and then report on each of the processes which we have developed. A statistical method for data abstraction is described in Section 3. The next section (Section 4) shows, how we applied the support vector machine (SVM) to learn state-action rules. A short introduction to the MOBAL system [24] and its representation of medical knowledge leads to the issue of validation which is presented in Section 6.