Abstract

Aim

The aim of this study was to perform a Turkish validation of the Stanford proxy test for delirium (S-PTDTV).

Materials and Methods

The original English version of the S-PTD was translated into Turkish using forward-backward translation methods. The Turkish version S-PTDTV was then administered by experienced nurses to elderly patients in an intensive care unit (ICU). The validation process involved assessing the sensitivity and specificity of the S-PTDTV by comparing its results with delirium diagnoses based on the diagnostic and statistical manual of mental disorders, fifth edition (DSM-5) and the confusion assessment method for the ICU (CAM-ICU). Reliability was assessed using internal consistency, intra-rater, and inter-rater reliability analyses.

Results

A total of 102 patients (50% female, mean age 74±9 years) participated in the study. When the cut-off score for the Turkish S-PTD was set at 7 points, the test showed a sensitivity of 96.6% and a specificity of 94.4% for the detection of delirium (area under the curve=0.985, p<0.001). High agreement was observed between S-PTD scores and both DSM-5 (κ=0.885, p<0.001) and CAM-ICU (κ=0.932, p<0.001). In addition, reliability analyses showed high consistency for both inter-rater [intraclass correlation coefficient (ICC=0.993, p<0.001)] and intra-rater (ICC=0.996, p<0.001) ratings. Internal consistency was also high, with a Cronbach’s alpha of 0.914.

Conclusion

The results of this study indicate that the Turkish version of the S-PTD is a valid and reliable tool for the diagnosis of delirium in elderly ICU patients.

Introduction

Delirium and confusion are among the most common mental disorders in the elderly and medically ill patients. They are associated with many complex underlying medical conditions (1). Improving the recognition of delirium has the potential to reduce healthcare costs as well as patient morbidity and mortality, and minimise long-term adverse complications.

The diagnostic and statistical manual of mental disorders, fifth edition (DSM-5) criteria (2) are considered the gold standard for diagnosing delirium. However, many clinicians have difficulty with applying the DSM-5 criteria to diagnose delirium in clinical cases (1). Therefore, several diagnostic tools have been developed to help clinicians diagnose delirium, especially for non-psychiatric physicians (3). The most commonly used tools are the confusion evaluation method in the intensive care unit (CAM-ICU) and the delirium rating scale-revised-98 (DRS-R-98) (4, 5).

The Stanford proxy test for delirium (S-PTD) is a new delirium screening tool developed by Maldonado et al. (6), which is based on the recently published criteria of the DSM-5 and the 10^th revision of the international classification of diseases and is designed specifically for use by nurses who follow patients throughout shifts (7-9). Compared with the data reported in the current literature, the S-PTD is more effective than other screening tools and is quicker and easier to use when administered under similar conditions. In addition, unlike other delirium screening tools, the use of the S-PTD is not hindered by the patient’s unwillingness or inability to cooperate. This diagnostic tool is an important adjunct in the detection of delirium, improving patient care and allowing assessment by nurses, who spend the most time with patients and know them best (6).

The S-PTD consists of a total of 12 questions, which are scored as never (worth 0 points), sometimes (worth 1 point) and most of the time (worth 2 points). These parameters are attention, awareness/orientation, memory, communication, learning new information, reasoning and decision making, visuospatial difficulties, perception, disorganised thinking, behaviour and psychomotor activity, and sleep patterns. In addition, the time of development of all these changes, the fluctuation during the day and the age of the study participants were assessed. There is a 13-item, age, which is scored based on its actual numerical value (i.e, <55 y/o = “0”, 56-70 y/o = “1”, >70 y/o = “2”). To the best of our knowledge, the validity and reliability of the Turkish version of the S-PTD have not been investigated. Therefore, the aim of this study is to investigate the validity and reliability of the Turkish version of the S-PTD.

Materials and Methods

This study was conducted in the Internal Medicine and Nephrology Intensive Care Units at Konya City Hospital. Ethics Committee approval was obtained (decision number: 38/17, date: 17.12.2021).

The Study Protocol and Its Universe

Prior to commencing the study, Prof, Jose R. Maldonado was contacted by email to obtain copyright permission for the use of S-PTD, and the study commenced once permission had been obtained.

Stages of the Study

Forward-backward Translation Processes

First, the S-PTD was translated from its original language, English, into Turkish by professional native translators (forward translation). Then, the final version of the Turkish S-PTD was analysed by a team consisting of intensive care specialists, psychiatrists, internal medicine, and geriatrics specialists, and nurses, and compared with the original version in terms of meaning. The consensus version of the Turkish S-PTD was then reviewed by a linguist and the final version was produced.

The Turkish version of the S-PTD was then translated from Turkish back into English by another professional team, scientifically proficient in both Turkish and English, who had never read the original S-PTD before (back translation). The same team described above checked this version again. Finally, the backward translated form was compared with the original S-PTD for integrity of meaning. A linguist then reviewed the final version of the S-PTD to make any necessary adjustments. The final version of the S-PTD translated into Turkish can be found in Appendix 1.

Validity and Reliability Steps

Location and Population of the Study

This study was conducted at the Konya City Hospital. Patients aged 60 years and older who were treated in the internal medicine and nephrology intensive care units (ICUs) at the hospital with 45 beds were included in the study. Before enrolling, volunteers were given detailed information about the study. Subsequently, participants signed the informed consent form, by the patient or their relatives, and were enrolled in the study consecutively.

Sample Size and Statistical Power

ICUs are where delirium is most common. Some studies have reported that the incidence of delirium in elderly patients hospitalised in ICUs can be as high as 87% (4). Taking this into account, the sample size was calculated. The analysis was performed using OpenEpi version 3.01 (Andrew G. Dean and Kevin M. Sullivan, Atlanta, GA, USA). Our hospital has an ICU with 45 beds. It is predicted that the number of patients over the age of 60 who can be admitted to this ICU within three months will be around 200. Therefore, to achieve a 5% alpha error and a 95% confidence interval, with a design effect of 1, the minimum number of patients needed to reach power was determined to be 94.

Internal Consistency

The same nurse administered the Turkish version of the S-PTD to all patients included in the study. Cronbach’s alpha coefficient was used to determine the internal consistency of the parameters of this test, with a value of 0.70 and above interpreted as indicating a strong level of consistency. As a result of the correlation analysis (Spearman’s rank correlation test) between each parameter, the correlation coefficient of 0.81 and above was considered excellent, 0.61-0.80, was considered very good, 0.41-0.60, was considered adequate, correlations with a correlation coefficient between 0.21-0.40 were interpreted as having an acceptable correlation, and values of 0.20 and below were accepted as an insufficient correlation.

Construct Validity

The diagnosis of delirium was assessed in three ways, and its validity was tested by examining the compatibility of these assessments. First, all patients in the study underwent a neuropsychiatric evaluation according to the DSM-5 criteria, which is considered the gold standard for the diagnosis of delirium. The DSM-5-based clinical assessment was performed by a team consisting of an internist, a psychiatrist, an intensive care physician, and a geriatrician. As a result of this assessment, all patients were categorized as having or not having delirium. In addition, patients diagnosed with delirium were further subdivided into hypoactive, hyperactive, or mixed delirium. This assessment was considered the gold standard. In addition, all patients were assessed by an internal medicine specialist using the CAM-ICU (10) as an objective test in this context. As previously described, patients were assessed with the S-PTD by the study nurse. The S-PTD and the gold standard assessment were administered within 60 minutes of each other. Members of the DSM-5-based diagnostic team and the study nurse were blinded to each other’s diagnostic results and findings.

Patients with or without delirium were independently categorised according to the DSM-5 criteria, and according to the S-PTD. Agreement between the two assessments was analyzed using Kappa statistics. Kappa coefficient was considered: <0, no agreement; 0.0-0.20, insignificant agreement; 0.21-0.40, fair agreement; 0.41-0.60, moderate agreement; 0.61-0.80, significant agreement; 0.81-1.00, almost perfect agreement. In addition, receiver operating characteristic curve (ROC) analysis was performed to determine the best S-PTD estimate that could diagnose delirium using the gold standard method (DSM-5). An area under the curve (AUC) of 0.6 and above, and a p value of <0.05 were accepted as strong results.

Inter-rater Reliability

To test interrater reliability, two nurses were asked to perform the S-PTD on the same patients at different times (within 60 minutes). A total of 40 patients underwent this assessment. The S-PTD scores obtained from these assessments were tested by intraclass correlation coefficient (ICC) analysis. An ICC value of 0.70 or higher is regarded as indicating acceptable reliability.

Intra-rater Reliability

To test intra-practitioner reliability, a nurse was asked to perform the S-PTD on the same patients at two different times, with the second assessment occurring at least 30 minutes and no more than 60 minutes after the first assessment. A total of 40 patients underwent this assessment. ICC analysis was performed to test intra-rater reliability, and a value of 0.70 and above was considered acceptable.

Inclusion Criteria

All patients aged 60 years and older who were hospitalised in the ICU for at least 24 hours were consecutively included.

Exclusion Criteria

Patients under 60 years of age with a diagnosis of severe dementia, suspected delirium tremens, stupor/coma, intubation, acute cerebrovascular accident with intracranial haemorrhage, and unable to communicate were excluded from the study.

General Characteristics of The Patients

General demographic characteristics, education level, comorbidities, medications, reasons for ICU admission, date of ICU admission, and Acute Physiology and Chronic Health Evaluation-II (APACHE II) score were recorded. Chronic diseases were assessed using the Charlson comorbidity index (CCI). In addition, risk factors for delirium were assessed, including chronic diseases; presence of sepsis; acute vascular events; central nervous system pathologies; electrolyte imbalance; hypoxia; malnutrition risk (determined by the Nutritional risk screening-2002 score, which was routinely completed by the service nurses, with a score of three and above being accepted as a risk of malnutrition); dehydration (decided by physical examination); trauma; cancer; history of alcohol consumption; central venous catheter and urinary catheterisation.

Statistical Analysis

IBM SPSS version 21.0 package (Armonk, NY, USA) was used for statistical analysis. The Kolmogorov-Smirnov test, histogram, and coefficient of variation were used to test whether numerical variables were normally distributed. Normally distributed numerical variables were expressed as mean ± standard deviation, and non-normally distributed numerical variables were expressed as median (minimum-maximum). Categorical variables were presented as numbers and percentages.

When comparing numerical variables between independent groups, the Student’s t-test was used for those with a normal distribution and the Mann-Whitney U test for those without a normal distribution. Chi-square or Fisher’s exact tests were used to compare categorical data between independent groups. For reliability analyses, Cronbach’s alpha was used for internal consistency, and ICC analysis was used to assess intra- and inter-practitioner reliabilities. Kappa and ROC analyses were used for validity. A p value of <0.05 was accepted as statistically significant.

Results

General Clinical Features and Delirium Status

The mean age of the patients was 74±9 years (50% female). The incidence of delirium was 29.4% and 27.5% according to DSM-5 and CAM-ICU criteria, respectively. The median values for the CCI, body mass index, and APACHE II score were 7, 25, and 21, respectively, while the number of medications used in the ICU was 5. The demographic data and general characteristics of the patients are summarised in Table 1.

When the S-PTDTV sub-parameters were evaluated in detail, it was found that all three categories were adequately scored in 11 out of 13 parameters. However, it was noted that item 9 (rating of disorganised thinking) was not adequately rated when the dimensions were explored. Therefore, the third column of this item was not scored after the assessments. On the other hand, it was noted that item-13, evaluation of age, was given 1 or 2 points. This is because we only conducted the study with older patients. The numbers and percentages of patients who responded to the S-PTDTV sub-parameters in detail are shown in Table 2. According to the DSM-5 criteria, the patients were divided into two groups with present or absent delirium and compared in terms of clinical characteristics (Table 3). As expected, the number of risk factors for delirium was higher in patients with delirium than in those without (p=0.045).

Validity and Reliability Analysis Results

ROC analyses showed that in our sample, the S-PTDTV had an overall sensitivity of 96.6% and specificity of 94.4% (p<0.001) for detecting delirium (compared with the DSM-5-based assessment/gold standard) when the cut-off was considered to be >6 (AUC=0.985). A difference was observed between hyperactive and hypoactive cases. In fact, these rates were 100% sensitivity and 97.22% specificity to detect hyperactive delirium (AUC=0.997, S-PTDTV cut-off >8, and p<0.001), while these rates had 87.5% sensitivity and 95.83% specificity in patients with hypoactive delirium (AUC=0.957, S-PTDTV cut-off >7, and p<0.001), compared to the gold standard DSM-5-based clinical assessment.

As the correlation coefficients of item-9 (disorganized thinking) and item-13 (age) were below 0.2 (as shown in the internal consistency analysis), the ROC analyses were repeated without the mentioned parameters. The cut-off for delirium were then found to be >5 when excluding item-13 only (AUC=0.984, p<0.001, 96.67% sensitivity and 95.83% specificity) or a different value when excluding both item-9 and 13 from the S-PTD (AUC=0.981, p<0.001, 96.7% sensitivity and 97.2% specificity). The results of the ROC curve analysis are shown in detail in Table 4.

In addition, inter- and intra-practitioner reliability analyses were evaluated using the ICC, which showed high reliability. The results are summarized in Table 5.

S-PTDTV scores were compared with DSM-5 and CAM-ICU scores and evaluated using Kappa concordance analyses. S-PTDTV cut-off were considered separately. The grading system proposed by Maldonado et al. (6), and according to the cut-off we found in our study (by removing age and disorganised items), was accepted. As a result of the Kappa analyses for each score, it was found that there was excellent agreement (Kappa values were in the range of 0.761-0.932) (Table 6). The internal consistency analysis showed a high Cronbach’s alpha coefficient of 0.914 (Table 7). The correlation coefficients of item 9 (disorganised thinking) and item 13 (age) were below 0.2, indicating poor correlations.

Discussion

This study has shown that the Turkish version of the S-PTD is valid and reliable in detecting delirium when used in elderly patients admitted to ICU.

The incidence of delirium in ICUs is about 20 percent and the cumulative prevalence is almost 40 percent (11). More scales are currently being used to assess delirium in these patients (4). However, simple methods that can be used by both clinicians and nurses are needed. As a result, the diagnosis of delirium is often missed by both doctors and nurses (12).

Our study evaluated the validity and reliability of the Turkish version of the S-PTD. It showed that, when the cut-off score was considered >6, the instrument had high sensitivity and specificity rates for the diagnosis of delirium (sensitivity 96%, specificity 94%, AUC=0.985). When items 9 and 13 were removed from the parameters, as the correlation coefficients were <0.2 in the internal consistency analysis, and the cut-off score was considered >5 in the diagnosis of delirium, the model was again found to have high sensitivity and specificity rates (sensitivity 96%, specificity 97%, and AUC=0.984).

In the previous study by Maldonado et al. (6), the sensitivity was 80.72% and the specificity was 90.37% when the cut-off score was >3 in the diagnosis of delirium; in the study by Alosaimi et al. (7), the sensitivity was 82.7% and the specificity was 95.3% when the cut-off score was >5. The sensitivity and specificity of the CAM-ICU test, another commonly used scale for the diagnosis of delirium, were 76-84% and 95%, respectively (13). Another commonly used tool for the diagnosis of delirium, the intensive care delirium screening checklist (ICDSC), has a reported sensitivity of 74-83% and specificity of 75-83%. Similarly, the DRS-R-98 has a reported sensitivity of 56-93% and specificity of 82-92%, while the Nursing delirium screening scale (Nu-DESC) has a reported sensitivity of 32-96% and specificity of 69-82% (5).

Our current study found that using a cut-off score of >5 on the S-PTDTV scale (excluding parameters 9 and 13) gave the best results for diagnosing delirium. In addition, a higher sensitivity rate (96%) was observed than in the two previous studies (7). However, further studies may confirm this result, as the current study was conducted in a single centre with a small sample size and mostly older patients.

Although CAM-ICU, ICDSC, DRS-R-98, and other scales can be widely used to detect ICU delirium (6-10), the limited patient interaction due to ventilator dependence, especially in ICU patients with hypoactive delirium, may limit their use (14-17). Nevertheless, our findings from the current study, supported by the literature, suggest that the Turkish version of the S-PTD can be used quickly and safely in ICUs to assess delirium and its subtypes in hypoactive or hyperactive forms.

The S-PTDTV can be applied by a nurse in as little as one minute (10). On the other hand, the Nu-DESC scale, one of the other scales used, has been reported to take approximately 1-2 minutes; the ICDSC scale approximately 3 minutes; and the CAM scale approximately 5 minutes (10, 18, 19). Therefore, the superiority of the S-PTDTV over other tests is due to its speed of use and because it can be administered by nurses; moreover, nursing practice skills and patient-nurse interaction may influence the results.

Age is known to be one of the most important risk factors for delirium (20), and the risk of delirium increases with age (21). In our study, the mean age of patients with delirium was higher than that of patients without delirium. However, this finding did not reach statistical significance (p=0.082) because only older patients and a relatively small sample size were included in the study. There is insufficient evidence on the effect of gender on the development of delirium (22). However, some studies have suggested that the risk of delirium may be higher in men (23). In our study, 60% of the delirium group was male, and there was no statistical difference compared with the non-delirium group.

When the S-PTDTV sub-parameters were assessed in detail in our study, it was found that all three categories were assessed and scored appropriately in 11 out of 13 parameters. However, in the question assessing item 9 (disorganised thinking), most patients scored 0 and 1. This situation made us think that item 9 might be difficult for nurses to understand in Turkish. Item-13 (age) was scored only 1 or 2 points because we conducted our study on patients over the age of 60. For these reasons, the correlation coefficients of item 9 and item 13 in the internal consistency analysis were <0.2.

Therefore, subtracting these two parameters from one another may be an alternative way to evaluate older patients. However, the ROC and Kappa analyses, both with and without the two mentioned parameters, show that the S-PTD can be used in both ways. In the case of 13 parameters, the sensitivity and specificity for the diagnosis of delirium were 96% (AUC=0.985), when the cut-off was above 6 points. When 9 and 13 were subtracted from the parameters and the cut-off was considered >5 points, the sensitivity was 96% and the specificity was 97% (AUC=0.984).

Study Limitations

The main limitations include the relatively small sample size, that the study was conducted in a single centre, and that it only included patients aged 60 years and older. Replication studies with a larger sample, conducted in several medical centres, might show more accurate results. However, as our findings are supported by strong statistical results in the current study, we believe that our study will demonstrate the potential of S-PTDTV and stimulate further interest in this line of work.

Conclusion

The S-PTDTV is an efficient and easy-to-use delirium screening tool that is not affected by the fluctuating clinical variability of delirium, especially in clinical settings where patient cooperation is limited, such as ICUs, and other similar settings. In addition, the short time required to administer the S-PTDTV is likely to encourage obtaining information from the nurse rather than direct patient involvement in scoring. This study clearly demonstrates that the Turkish S-PTD is valid and reliable in assessing delirium in elderly patients hospitalized in ICUs.