A study was made of the psychometric characteristics of the modified Freedman questionnaire to assess sleep in critical patients.
DesignA psychometric study was carried out, with content validity being explored by a group of experts, and internal consistency based on Cronbach's alpha coefficient. Factor analysis was performed to explore construct validity, and stability was assessed by test–retest analysis.
SettingThe Department of Intensive Care Medicine of a reference hospital.
ParticipantsPatients admitted between 23 February 2016 and 20 December 2017.
InterventionsQuestionnaire administration.
VariablesItems of the modified Freedman questionnaire.
ResultsItem relevance and definition yielded scores >3 (Likert scale maximum=4). Cronbach's alpha showed a global value of 0.933. The intraclass correlation index was >0.75 for most of the items of the questionnaire. Factor analysis allowed the detection of specific associations between the studied variables and the four factors.
ConclusionsThe modified Freedman questionnaire showed good psychometric characteristics. It may be a reliable instrument for assessing the quality of sleep in critically ill patients, as well as the environmental factors.
Evaluar las características psicométricas del cuestionario Freedman modificado para la valoración del sueño en el paciente crítico.
DiseñoEstudio psicométrico. Se exploró la validez de contenido mediante un grupo de expertos y la consistencia interna fue evaluada mediante el coeficiente alfa de Cronbach. Se realizó un análisis factorial para evaluar la validez de constructo; la estabilidad fue medida con análisis test-retest mediante el índice de correlación intraclases.
ÁmbitoServicio de Medicina Intensiva de un hospital de referencia.
ParticipantesPacientes ingresados entre el 23 de febrero de 2016 y el 20 de diciembre de 2017.
IntervencionesAdministración de un cuestionario.
VariablesÍtems del cuestionario Freedman modificado.
ResultadosLa pertinencia de los ítems y su definición obtuvieron valores superiores a 3, en una escala tipo Likert con valor máximo de 4 puntos. El alfa de Cronbach indicó un valor global de 0,933. El índice de correlación intraclases obtuvo valores superiores a 0,75 en la mayoría de los ítems del cuestionario. El análisis factorial obtuvo asociación entre las variables analizadas y los 4 factores.
ConclusionesEl cuestionario Freedman modificado presentó unas buenas características psicométricas. Puede resultar un instrumento fiable para evaluar la calidad del sueño en el paciente crítico, así como los factores ambientales relacionados.
Sleep is a basic necessity that modulates the immune system, regulates homeostasis, and improves some cognitive functions; moreover, it contributes to the adjustment of physiological functions through hormonal secretion and anabolic stimulation.1,2
At intensive care units (ICU), due to their characteristics and the high severity of their patients’ conditions, there are more sleep alterations, and patients suffer from both qualitative and quantitative deprivations.3 Studies conducted on the incidence of sleep alteration in the critically ill patient report a prevalence of 22% to 61%. Sleep pattern alterations are observed in the critically ill patient and they consist in a predominance of the stage N1 – light sleep and stage N2 – muscle tone reduction of the sleep cycle with a decrease or absence of stages N3-deep, restorative sleep, and REM sleep. The patients often experience awakenings, a high index of arousal, and periods of daytime sleep (40%–50% of the sleep can occur during the day)4–6; patients rarely complete a full sleep cycle.7,8
Assessing the sleep of patients admitted at the ICU is a complex process,9–11 and this situation requires tools to evaluate it objectively; but these tools like polysomnography and actigraphy are not available in all ICUs. They also require trained researchers for their interpretation. That is why, in order to assess sleep and the factors that interrupt it, there is the alternative of assessment through subjective methods based on questionnaires. This process is less expensive. In contrast, questionnaires, even when they have been validated through comparison with objective evaluation methods, they have not been submitted for psychometric studies.12–14
These psychometric studies assess the characteristics that every measuring tool should have, which necessarily includes controlling its precision by studying its reliability and validity. Validity refers to the degree to which the tool measures what it intends to measure; reliability refers to the trustworthiness conferred to the data obtained from it and it is associated with the internal coherence or consistency and precision of the measures collected.15 The objective of the present study is to assess whether the modified Freedman questionnaire is suitable, due to its psychometric characteristics, for sleep assessment in critically ill patients.
Patients and methodDesign of the studyPsychometric study conducted at the Intensive Medicine Service (IMS) of the Hospital General Universitario de Castellón, Spain. It is a multipurpose ICU including 15 beds and 1 intermediate care unit with 6 beds. The reliability, stability, and validity of the construct and content were assessed.
ParticipantsPatients admitted to the IMS during the study period from February 23, 2016 through December 20, 2017. Work was conducted with a convenience sample that included the patients that met the inclusion criteria selected for the study who had signed the informed consent. Following the criteria established by Walter et al. (1998)16 for 2 observers, a 129 patient sample was estimated accepting at a significance level of α=0.05 for a ρ value 0=0 and a ρ value 1=0.2.
Inclusion criteria- ∘
Critically ill patients admitted to the ICU>18 years.
- ∘
Patients who speak Spanish.
- ∘
Patients undergoing neuromuscular blockade.
- ∘
Patients with hearing or speech difficulties.
- ∘
Patients with a previous diagnosis of dementia.
- ∘
Patients with substance abuse.
- ∘
Patients with Glasgow Coma Scale values <12.
- ∘
Patients with values on the Richmond Agitation Sedation Scale (RASS) out of range (+1 and −1).
Prior to the beginning of the study, the project was presented to the IMS and approval from the Clinical Research Ethics Committee (CEIC) of the Hospital General Universitario de Castellón was obtained.
The tool studied “modified Freedman questionnaire” assesses the quality of sleep in critically ill patients subjectively by taking into account environmental factors. This questionnaire was designed by Freedman et al. back in 1999.17 In this study, the modified version has been used as translated and published in Spanish by Gómez Sanz.18 The items are measured using the Likert scales whose value ranges from 1 to 10. A value of 1 for sleep quality represents “poor quality” while a value 10 represents “excellent quality”. For sleepiness, a value of 1 represents “incapable of keeping awake” and a value 10 means “completely alert and awake”. Regarding environmental factors, a value of 1 is indicative of “no interruption” and a value of 10 means “significant interruption”. The following variables have not been taken into account: room temperature, location of the emergency bay area, home medication, and ICU medication to sleep because in the Gómez Sanz study no significant results were obtained regarding the variables that defined sleep quality.
A group of experts was put together including 5 intensivists and 4 intensive care nurses with over 5 years of experience at the IMS, and a university professor. Data mining was conducted by the members of the work group. The data were collected during the early hours of the morning between 8AM and 10AM. The questionnaire was hetero-administered giving a 1-hour interval between interviewer A and interviewer B. Data were collected during the first day of admission, half-way into the stay, and the day the patient was discharged from the ICU.
Statistical analysisA descriptive analysis of the study variables was conducted; qualitative variables were expressed as absolute frequencies and percentages. Continuous variables were expressed as means and standard deviation.
Content validity was explored through expert consensus. They were asked about the definition of the variables, using a Likert scale (1= incorrect definition, 2=not quite correct definition, 3= correct definition, and 4= very correct definition). They were also asked about the pertinence of every variable for the evaluation it was intended for, which was assessing using the Likert scale (1=not pertinent at all, 2=not too pertinent, 3=pertinent, and 4=very pertinent).19
The questionnaire reliability was assessed by analyzing the internal consistency through Cronbach's alpha coefficient,20 and its stability was assessed through test–retest using an inter-observer level of concordance with the intraclass correlation index (ICC).21
The validity of the construct to explore the dimensionality of the questionnaire was assessed through factor analysis using varimax orthogonal rotation method, accepting a cut-off value of ≥0.4 and defining 4 factors: sleep quality, sleepiness, sleep interruption due to environmental causes, and sleep interruption due to human factor. Factor analysis is a useful technique for finding homogeneous groups or factors from a numerous set of variables. Factor rotation allows generating matrices that are easier to interpret. This method requires assessing previously whether the sample is suitable to conduct this analysis with the Kaiser–Meyer–Olkin (KMO) test (that should take P values ≥0.5 to be acceptable and ≥0.75 to be good? and Bartlett's test of sphericity (that should reject the null hypothesis with P values <.05).22 Statistical analysis was conducted using the Statistical Package for the Social Sciences 23.0. (SPSS) software package. P levels ≤.05 were considered statistically significant.
ResultsA sample of n=129 patients was analyzed. The mean age was 60.61±13.88 years, range (25-85); 62% were males. The mean stay of the period were 10.31±14.41 days, range (3–117). Regarding the reason for ICU admission, 46.5% was for medical reasons, 23.3% for postoperative reasons, 18.6% due to coronary disease, and 11.6% for traumatic reasons.
The results of exploring the validity of the content of the questionnaire from the answers given by members of the expert group to the questions of variable pertinence and its definition, in most of the questions are values >3, indicative of a correct or very correct definition, except for the correct definition in nursing activities, drug administration, and pulse oximeter (Table 1).
Validity of the questionnaire content. Results of expert consensus.
Variables of the modified Freedman questionnaire | Mean (DE) | |
---|---|---|
Rate sleep quality at home | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Rate sleep quality at the ICU | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Rate sleep quality at the ICU during the following days (1st day, mean stay, discharge) | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Assess the general degree of daytime sleepiness during your stay at the ICU | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.37 (0.91) | |
Assess the general degree of daytime sleepiness during your stay at the ICU during the following days (1st day, mean stay, discharge) | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.37 (0.91) | |
Rate to what extent the following environmental activities/factors interrupted your sleep during your stay at the ICU | ||
Noise | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Light | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Nursing care | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Diagnostic test | Correct definition | 3 (0.75) |
Pertinence of question | 3 (0.75) | |
Nursing activities | Correct definition | 2.75 (0.70) |
Pertinence of question | 3 (0.92) | |
Blood sample extraction | Correct definition | 3.25 (0.46) |
Pertinence of question | 3.37 (0.51) | |
Administration of medication | Correct definition | 2.87 (0.83) |
Pertinence of question | 3 (0.92) | |
Rate to what extent the following noises interrupted your sleep during your stay at the ICU | ||
Alarms | Correct definition | 3.62 (0.51) |
Pertinence of question | 3.62 (0.51) | |
Pulse oximeter | Correct definition | 2.87 (0.83) |
Pertinence of question | 3 (0.92) | |
Hearing people talk | Correct definition | 3.62 (0.51) |
Pertinence of question | 4 (0) | |
Aspiration of secretions | Correct definition | 3.5 (0.53) |
Pertinence of question | 3 (0.75) | |
Nebulizations – oxygen therapy | Correct definition | 3.12 (0.35) |
Pertinence of question | 3.12 (0.35) | |
Telephone | Correct definition | 3.25 (0.46) |
Pertinence of question | 3.25 (0.46) |
Likert Scale “Formulation of question”: 1=incorrect definition, 2=not quite correct definition, 3=correct definition, and 4=very correct definition.
Likert Scale “Pertinence of the question”: 1=not pertinent at all, 2=not quite pertinent, 3=pertinent, and 4=very pertinent.
Before conducting the factor analysis, sampling adequacy was explored. The results of the KMO sampling adequacy test were 0.751 (result close to the value of 1 considered as an excellent value) and those of Bartlett's test were a significant value (P<.001), which allowed conducting the factor analysis. Subsequently, we assessed the validity of the construct through factor analysis using the varimax rotation method (results are shown on Table 2). The table shows the correlation between the variables analyzed and the 4 factors extracted (sleep quality, sleepiness, sleep interruption due to environmental causes, and sleep interruption caused by the human factor).
Results of del factor analysis.
Variable | Quality of sleep | Sleepiness | Sleep interruption due to environmental cause | Sleep interruption due to human factor |
---|---|---|---|---|
Sleep quality at home | 0.232 | 0.075 | 0.197 | 0.121 |
Overall general quality of sleep at the ICU setting | 0.931 | 0.049 | −0.115 | −0.119 |
Overall quality of sleep at the ICU setting (daily) | 0.932 | 0.047 | −0.138 | −0.113 |
Overall general degree of sleepiness | 0.152 | 0.948 | 0.013 | −0.011 |
Overall degree of sleepiness daily | 0.149 | 0.936 | 0.061 | −0.015 |
Overall noise | −0.131 | 0.042 | 0.754 | 0.323 |
Overall light | −0.105 | 0.044 | 0.615 | 0.260 |
Overall nursing care | −0.209 | 0.056 | 0.333 | 0.757 |
Overall diagnostic tests | −0.213 | 0.054 | 0.357 | 0.656 |
Overall nursing activities | −0.042 | −0.019 | 0.310 | 0.860 |
Overall extraction of blood samples | −0.011 | −0.036 | 0.179 | 0.900 |
Overall administration of medication | 0.032 | −0.061 | 0.276 | 0.882 |
Overall alarms | −0.185 | −0.081 | 0.749 | 0.211 |
Overall pulse oximeter | −0.058 | 0.052 | 0.677 | 0.296 |
Overall hearing people talk | −0.097 | −0.127 | 0.741 | 0.262 |
Overall telephone | 0.093 | −0.027 | 0.565 | 0.160 |
Analysis of main components. Varimax rotation method with Kaiser. The totals of the 3 times evaluated are analyzed (1st day, mean stay, and discharge). Accepted cut-off value (≥0.4).
The results that exceed the accepted cut-off value are shown in bold face type.
The internal consistency of the questionnaire was assessed using Cronbach's α coefficient that indicated an overall P value of .933. Instrument stability was assessed using an ICC test–retest; the coefficients obtained an excellent correlation in all the items analyzed. Table 3 shows the results associated with the items that study the quality of sleep, daytime sleepiness, environmental activities, and factors that interfere with the 3 phases of sleep.
Results of the inter-observer level of concordance (ICC).
Freedman questionnaire items | ICC (range) | F | p |
---|---|---|---|
Overall sleep quality at home | 0.96 (0.95–0.97) | 54.98 | <0.001 |
Overall sleep quality in general during the ICU stay | 0.92 (0.90–0.94) | 27.08 | <0.001 |
Overall sleep quality at the ICU setting (daily) | 0.93 (0.90–0.95) | 28.71 | <0.001 |
Overall general degree of daytime sleepiness during stay at the ICU | 0.88 (0.84–0.91) | 16.48 | <0.001 |
Overall degree of sleepiness daily | 0.88 (0.83–0.91) | 15.83 | <0.001 |
Environmental activities/factors | |||
---|---|---|---|
Overall noise | 0.99 (0.96–0.98) | 88.43 | <0.001 |
Overall light | 0.96 (0.94–0.97) | 50.88 | <0.001 |
Overall nursing care | 0.82 (0.76–0.87) | 10.47 | <0.001 |
Overall diagnostic tests | 0.79 (0.71–0.84) | 8.53 | <0.001 |
Overall nursing activities | 0.89 (0.85–0.92) | 18.03 | <0.001 |
Extraction of blood samples | 0.91 (0.88–0.94) | 23.27 | <0.001 |
Administration of medication | 0.89 (0.85–0.92) | 17.55 | <0.001 |
Overall (observers A/B) | 0.93 (0.90–0.95) | 27.99 | <0.001 |
Noise | |||
---|---|---|---|
Overall alarms | 0.95 (0.93–0.96) | 43.03 | <0.001 |
Overall pulse oximeter | 0.92 (0.85–0.92) | 17.80 | <0.001 |
Overall hearing people talk | 0.94 (0.92–0.96) | 34.47 | <0.001 |
Overall aspiration of secretions | 0.93 (0.91–0.95) | 30.62 | <0.001 |
Overall nebulizations – oxygen therapy | 0.89 (0.86–0.92) | 18.77 | <0.001 |
Overall telephone | 0.93 (0.90–0.95) | 29.19 | <0.001 |
Overall (observers A/B) | 0.95 (0.94–0.97) | 45.38 | <0.001 |
ICC coefficient: <0.4 low; 0.4–0.75 good; >0.75 excellent.
The Spanish Society of Intensive and Critical Medicine and Coronary Units (SEMICYUC) proposes a series of recommendations to improve safety and quality of clinical practice in critically ill patients. We should mention here that, regarding the issue at hand, the Sedation, Analgesia, and Delirium Working Group at SEMICYUC gives great importance to facilitating sleep and controlling environmental stimuli during the night. Sleep control and evaluation, together with other recommendations, are supposed to be preventive measure to adjust, on the one hand, the administration of sedative drugs and, on the other hand, to minimize states of delirium providing the necessary rest for the critically ill patient.23,24
This fact makes us consider the need to evaluate the quality of sleep as perceived by the patients during their stay at the ICU and figure out what tool would be the most suitable one to provide us with the necessary information on both the patient and the environment. We started on the thought that sending a questionnaire is not a easy process because it requires the patient's collaboration and correct cognitive function, and the systematized performance of interviewers25,26; that is why the first step this study took was to train the interviewers in how to implement the scale.
The initial study conducted by Freedman et al.17 did not assess the psychometric characteristics of the questionnaire, but used factor analysis to study the correlation between each item and the 4 factors set out by the authors (factor 1, sleep interruption secondary to interruptions by the staff and diagnostic tests; factor 2, quality of sleep; factor 3, daytime sleepiness; and factor 4, sleep interruption by environmental factors [light and noise]). They found that the variables that showed the greatest correlation were the ones associated with noise, light, and interventions performed by researchers. The results of our study show a similar correlation between the variables analyzed and the dimensions that correspond with the factors described by Freedman.
Regarding content validity, the members of the expert group assessed practically all the items in the questionnaire with scores ≥3, both to the question that referred to the formulation of the item and to the one that referred to its pertinence. It should be pointed out that the items “nursing activities”, “administration of drugs” and “pulse oximeter” were assessed more poorly since they did not reach the value of 3 regarding their formulation. This leads us to consider that the variables “nursing activities” and “administration of drugs” could be included in the variable “nursing care”. A variable that was better evaluated and includes all the nursing competences. The variable “pulse oximeter” is in a similar situation, that could be considered as “alarms”.
The evaluation of internal consistency conducted by Cronbach's alpha coefficient obtained a global result of 0.933. Based on the study conducted by Streiner,27 who claims that the value to accept a correct internal consistency is a Cronbach's alpha coefficient of ≥0.7, it can be said that the modified Freedman questionnaire shows results that guarantee their homogeneity and reliability.
Regarding the evaluation of the level of concordance, the ICC results showed an excellent level of concordance; all the items showed ICC values >0.75 with statistical significance (P<.001). It is true that in this study intra-observer concordance was not evaluated; the time scheduled between the assessment of observer A and that of observer B did not make the intra-observer evaluation process easier.
LimitationsWe believe that despite the results obtained in this study, we should not obviate the fact that assessments through questionnaires are somehow subjective and that they can only replace objective evaluation methods when these cannot be used.
ConclusionsThe results of this study indicate that the Freedman questionnaire modified by Gómez Sanz shows good psychometric characteristics, which leads us to think that it can turn out to be a reliable tool to evaluate the quality of sleep in the ICU patient with a RASS between −1 and +1, as well as the environmental and human factors that can interfere with sleep.
FundingThis study has received no public or private funding whatsoever.
Conflicts of interestNone reported.
Please cite this article as: Bernat Adell MD, Bisbal Andrés E, Galarza Barrachina L, Cebrián Graullera G, Pages Aznar G, Melgarejo Urendez A, et al. Evaluación psicométrica del cuestionario Freedman para la valoración del sueño en el paciente crítico. Med Intensiva. 2020;44:344–350.