We conducted an observational and prospective study of transcultural adaptation, and validation of a tool in the ICU setting of an 18-bed tertiary pediatric hospital of Barcelona, Spain. The study period was from September 2015 through September 2016.

The study population included the total number of patients admitted in the context of the study (around 1450 a year). In an attempt to obtain relevant results and to eliminate bias as much as possible, it was estimated that a minimum sample of 221 individuals would be required (95% confidence interval and precision estimates of ± 5%). The anticipated population percentage was around 80% and that of necessary replacements around 5%.

The probabilistic stratified random sampling technique by age was used. Taking into consideration the recommendations of the World Health Organization (WHO), the sample was categorized into: infants (from 0 days to 28 days), babies (from 29 days to 2 years), preeschoolers (from 3 years to 5 years), schoolers (from 6 years to 12 years), and teenagers (from 13 years to 18 years).

The inclusion criteria were: 1) pediatric patients (from 0 years to 18 years) admitted to the critical care unit during the study period, and 2) acceptation and signing of the informed consent by the legal representatives and, in the case of mature minors (> 12 years), acceptation/consent of the subject. Patients excluded: 1) in palliative care; 2) with language barriers, and 3) those who had been receiving muscle relaxants in continuous infusion.

As we mentioned before, there are currently 2 valid scales to assess the confort of critically ill patients: the COMFORT Scale (1992) and the COMFORT Behavior Scale (2001). Both scales measure 6 behavioral parameters (vigil, level of agitation/calmness, crying or respiratory response, physical movement, muscle tone, and facial tension) and 2 physiological parameters in the case of the 1992 scale (heart rate and mean arterial blood pressure). Because the heart rate and the mean arterial blood pressure can vary depending on different circumstances (drugs, improved or worsened pathological process, etc.),32 it was decided to accept and validate the CBS for the purpose of the present study.

During the process of validating the COMFORT Behavior Scale, Cronbach’s alpha coefficient was 0.78, but it went up to 0.84 after excluding the heart rate and mean arterial blood pressure from the scale.30 The COMFORT Behavior Scale assesses the correct management of the patient’s postoperative pain with scores ≤ 17. Regarding sedation, the authors define scores ≤ 11 as oversedation and they recommend checking scores from 11 to 22 using the Nurses Interpretation of Sedation Scale (NISS), that still has not been validated in Spain. Also, scores ≥ 23 are indicative of undersedation.

Prior to the study, the research team proceeded with a transcultural adaptation of the COMFORT Behavior Scale. For this purpose, the recommendations established by the International Test Commission36–38 were followed. Also, 4 bilingual professionals collaborated to determine which were the best semantic expressions of the scale using a process of translation-retranslation. Also, the content validity indices were estimated. Scores were between 0.76 and 0.80 in every item and 0.87 in the overall scale.39

A group of 12 nurses with at least 5 years of experience managing critically ill pediatric patients was used to analyze the psychometric properties of the CBS-ES.

The social and demographic variables included were: 1) the unit of admission (PICU/Pediatric Semi-Intensive Care Unit [PSICU]); 2) sex (masculine/femenine); 3) age (in days, months or years); 4) medical diagnosis at admission (respiratory/postoperative/infectious/oncological/other); 5) the administration of sedoanalgesia (yes/no); 6) mechanical ventilation (yes/no), and 7) the overall stay at the critical care unit (in days). The main differences between patients admitted to the PICU and those admitted to the PSICU were the severity of their clinical condition and need to administer mechanical ventilation, complex cardiovascular support or extra-renal depuration techniques.

First, a team of 12 nurses from the critical care unit was assembled. In order to guarantee an adequate process of data mining, the principal investigator (ABA) designed a theoretical/practical training that included: a presentation in PowerPoint® (Microsoft) that explained the concept of comfort, the aspects to take into consideration regarding comfort, the final version of the CBS-ES, and the documents pertaining to the study data mining process. The theoretical session started by explaining what we understand as comfort. Then, the main objectives and procedure to be followed during research were established together with the scale that would be used (CBS-ES). The practical part of the training included an assessment of the level of comfort using the CBS-ES. For this purpose, every nurse screened 5 patients admitted in the context of the study. Finally, all the lingering doubts that emerged during the process of data mining were solved. Training lasted 5 hours. After training the CBS-ES required zero changes.

After performing all the necessary procedures that the patients admitted to the unit required, the patients’ informed written consent to participate in the study was formally requested. After obtaining it, 2 nurses per shift (morning, afternoon, and night) performed simultaneous assessments of the patients’ level of comfort using the CBS-ES. Therefore, data were collected 3 times a day (morning, afternoon, and night) within the first 48 hours after admission.

Data mining spanned throughout 7 months (from March to September 2016).

A descriptive analysis of data was conducted. The derivatives of categorical variables were expressed as frequencies (n) and percentages (%), while the derivatives of quantitative variables were expressed as mean and standard deviation or median and interquartile range.

In order to determine the validity of the construct CBS-ES a factorial analysis of the main components was conducted.

The reliability of the CBS-ES was assessed using Cronbach’s alpha coefficient of the scale and every individual factor (valid with values > 0.7). Also, interobserver concordance was assessed using Cohen’s kappa coefficient (values > 0.60 were considered good values).

In order to determine the validity of the scale and taking into consideration the postulates posed by Satorra and Bentler back in 2001,40 a factorial analysis was conducted using the Robust Unweighted Least Squares (RULS) as the estimation method and the polychoric correlation matrix as the baseline matrix. The number of tool factors was determined through the parallel analysis method using the Comparative Factor Index (CFI > 0.90), the Tucker-Lewis Index (TLI > 0.90), and the Root Mean Square Error of Approximation index (RMSEA < 0.08). The adjustment indices were considered satisfactory according to the criteria established by Ullman in 2006.41 The comparisons in the models found were drawn using the S-B χ2 test envisioned by Satorra and Bentler back in 2001.40

The discomfort scores obtained in the CBS-ES were established while taking into consideration the mean and percentiles observed in the sample.

A 95% confidence interval was assumed in all the tests. Also, the data obtained were considered statistically significant with P values < 0.05.

The hospital Ethics and Clinical Reseach Committee approved the study before it was conducted. The Bioethics Commission of a public university approved it as well. Also, Dr. Monique van Dijk, the author of the COMFORT Behavior Scale, gave her written informed consent to the study.

This study included a total of 311 pediatric patients of whom 94.9% were hospitalized at the PICU. Fifty-six-point-six per cent were children with a median age of 5.07 years (0.9-11.7). The diagnosis that triggered the highest number of admissions in the critical care unit was postoperative in 60.1% (n = 187), being heart surgery (14.4%, n = 44) and scoliosis (10.9%, n = 34) the most prevalent ones. Of the total of patients admitted to the PICU, 25.3% were on mechanical ventilation and 65.9% (n = 205) on some sort of sedoanalgesia in continuous infusion. Morphine hydrochlorine (22.5%, n = 70) and fentanyl (5.5%, n = 17) were the analgesics most common of all. Pain was assessed using the FLACC scale, the PAIN scale, and the numeric scale and the following parameters were categotized: no pain = 0; mild pain = 1 to 3 points; moderate pain = 3 to 7 points, and severe pain = 7 to 10 points. Most patients reported no pain (n = 240) since they were on analgesia in continuous infusion (fentanyl or morphine hydrochlorine) sometimes combined with sedation (midazolam or propofol) or because they had an indication for some sort of discontinuous analgesia (metamizol, paracetamol or tramadol, mainly). The mean stay at the critical care unit was 2 days (2-110). No differences were seen in the social or demographic variables between patients admitted to the PICU and those admitted to the PSICU. Table 1 shows the social and demographic characteristics of the sample in detail.

The main components were analyzed to determine the validity of the construct. It was confirmed that the factorial load of each item was > 0.40, indicative of an adequate relevance of the items (Table 2). The CBS-ES consists of 3 factors with 2 items per factor: 1) alertness and physical movement; 2) calmness/agitation and respiratory response/crying, and 3) facial tension and muscle tone. The correlation among the 3 factors of the scale was good (Table 3) as well as the adjusted goodness of fit indices of the model (Table 4).

Taking into account the mean and percentiles obtained in the distribution of the sample, it was determined tha the CBS-ES categorized discomfort as: no discomfort (≤ 11 points); medium discomfort (between 12 points and 22 points), and high levels of discomfort (≥ 23 points).

Fig. 1 shows a graphical representation of the structure of the CBS-ES.

Figure 1.

Graphical representation of the correlation between factors and items of the COMFORT Behavior Scale-Spanish version.

The items of the CBS-ES were represented using circles.

A straight line shows the item factor load in the overall of the scale.

A bidirectional arrow unifies the factors of the CBS-ES.

Residual variance is shown next to each item.

The reliability of the CBS-ES was assessed by estimating the overall item correlation, Cronbach’s alpha coefficient, and the interobserver reliability. The internal consistence of the CBS-ES obtained a Cronbach’s alpha coefficient of 0.715. All the overall item correlations were > 0.30 except for respiratory response/crying (0.10) and muscle tone (0.22). Still, when establishing the matrix of item interrelation was established and and confirming whether the reliability of the scale would go up, it was confirmed that Cronbach’s alpha coefficient would have a slight increase (0.718). Still, this piece of information does not justify the elimination of the items of the scale. Taking into consideration the different age groups established the following values were obtained: infants obtained a Cronbach’s alpha coefficient of 0.785; babies, 0,736; preschoolers, 0.720; schoolers, 0.719, and teenagers, 0.632.

Regarding the interobserver correlation, when comparing the 311 pairs of observations, most items obtained Cohen’s kappa coefficients > 0.60 except for calmness/agitation (0.52), muscle tone (0.30), and facial tension (0.29). The kappa indices obtained based on the age group were infants, between 0.30 and 1; babies, between 0.51 and 0.71; preschoolers, between 0.52 and 0.68; schoolers, between 0.60-0.83, and teenagers, between 0.30 and 0.72.

One of the main limitations of this study is the possibility of reproducing the results in other settings without previous specific training of the health professional on the CBS-ES. At the same time, future studies will be required to determine the predictive capabilities of the CBS-ES to detect situations of discomfort associated with pain and sedation, especially in patients on mechanical ventilation.

The COMFORT B scale was adapted to Spanish language and proved to be a valid tool to determine and quantify the level of comfort in a group of children admitted to a Spanish PICU.

This CBS-ES will allow health professionals to assist and assess pediatric critically ill patients. Also, treat the factors leading to their discomfort and analyze the impact of specific interventions to provide comfort. This will make us be more efficient when having to prioritize activities especially when providing comprehensive care as part of the process and healthcare setting of the pediatric critically ill patient.