ISRN StrokeVolume 2013 (2013), Article ID 620186, 7 pageshttp://dx.doi.org/10.1155/2013/620186

Research Article

Delirium in Acute Stroke: A Survey of Screening and Diagnostic Practice in Scotland

Gail Carin-Levy,¹ Kath Nicol,¹ Frederike van Wijck,² and Gillian E. Mead³

¹School of Health Sciences, Queen Margaret University, Queen Margaret University Drive, Edinburgh EH21 6UU, UK²Institute for Applied Health Research and School of Health and Life Sciences, Glasgow Caledonian University, Cowcaddens Road, Glasgow G0 4BA, UK³Geriatric Medicine, Clinical and Surgical Sciences, The University of Edinburgh, Room S1642, Royal Infirmary of Edinburgh, Little France Crescent, Edinburgh EH16 4SA, UK

Received 5 June 2013; Accepted 16 July 2013

Academic Editors: R. P. Kessels, P. A. Nyquist, and J. Van Der Grond

Copyright © 2013 Gail Carin-Levy et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Aims. To survey the use of delirium screening and diagnostic tools in patients with acute stroke across Scotland and to establish whether doctors and nurses felt the tools used were suitable for stroke patients. Methods. An invitation to participate in a web-based survey was e-mailed to 217 doctors and nurses working in acute stroke across Scotland. Descriptive statistics were used to report nominal data, and content analysis was used to interpret free text responses. Results. Sixty-five responses were logged (30% return rate). 48% of the respondents reported that they routinely screened newly admitted patients for delirium. Following initial screening, 38% reported that they screened for delirium as the need arises. 43% reported using clinical judgment to diagnose delirium, and 32% stated that they combined clinical judgment with a standardised tool. 28% of the clinicians reported that they used the Confusion Assessment Method; however, only 13.5% felt that it was suitable for stroke patients. Conclusions. Screening for delirium is inconsistent in Scottish stroke services, and there is uncertainty regarding the suitability of screening tools with stroke patients. As the importance of early identification of delirium on stroke outcomes is articulated in recent publications, validating a screening tool to detect delirium in acute stroke is recommended.

1. Introduction

Delirium is a common neuropsychiatric condition affecting 20–30% of elderly patients across most hospital settings [1]. In acute stroke, the incidence of delirium reported by individual studies ranges from 10% [2] to 48% [3], and meta analyses recently performed placed the incidence around 26%–28% [4, 5]. Delirium is associated with increased mortality, morbidity, and length of hospital stay [5–8], and it has been strongly associated with development of cognitive impairment in the long term in the general medical setting [9, 10]. In acute stroke, recent studies have clearly demonstrated that patients who develop delirium are more likely to die within 12 months, have poorer functional outcomes, and are at higher risk of developing dementia [4, 5, 11, 12]. There are calls in the literature for clinicians to place an emphasis on early identification of delirium in stroke patients, using a tool validated specifically for this population, as early intervention may minimise the aforementioned unfavourable outcomes [5, 13, 14]. The most recent guidance published in the United Kingdom (UK) by both the National Institute for Health and Clinical Excellence (NICE) and the Royal College of Physicians (RCP) do not mention delirium as a specific complication of stroke [15, 16]; however, both refer to cognitive impairment and inattention. The word “confusion” is mentioned in the Scottish Intercollegiate Guidelines Network document “Management of Patients with Stroke” (SIGN 118), but there is no specific guidance about how to screen for or manage this “confusion” [17]. Clinical guidelines from other English speaking countries were examined for comparison: Australian guidelines [18] do not mention delirium in stroke patients; American Heart Association (AHA) guidelines mention delirium in the context of screening for psychiatric sequelae to stroke in the end of life care [19]; Canadian guidelines were the most detailed, and they contained a clear message about the importance of delirium as a complication in acute stroke. This was discussed in relation to screening for cognitive impairment or a change in cognitive function, and there is a clear call to screen patients at risk, using a validated screening tool [20]. Screening for delirium in other clinical settings is considered important across several countries. Clinical guidelines published in the UK [7], United States of America [21], Australia [22], and Canada [23] all guide clinicians to screen for delirium in services which are known to have a high prevalence of the condition. This is in order to ensure that delirium is not missed or misdiagnosed and thus to decrease the length of hospital stay and the unfavourable outcomes and ultimately generate cost savings for the organization [7, 23]. As for the method of diagnosis, the UK and Canadian documents specifically recommend the use of the Confusion Assessment Method (CAM) [24] as the diagnostic tool of choice.

2. Description of Screening/Diagnostic Tools

Our systematic review [4] identified a number of tools commonly mentioned in the literature; however, the most frequently cited are the Delirium Rating Scale (DRS) [25] and the CAM [24], both of which are based on the American Psychiatric Association diagnostic and statistical manual (DSM) criteria and designed to identify delirium across a variety of medical settings. The DRS was designed to be used by medical staff with specific training [25]. It comprises of 10 items, the highest possible score is 32, with a cut-off score of ten indicating the presence of delirium, thus it is a useful tool to rate the severity of delirium [13]. The CAM comprised four features (acute onset, fluctuating course, and inattention with either disorganised thinking or altered level of consciousness); it was originally developed for use by any health professional, and it has high sensitivity, specificity, and reliability and is easy to administer [24]. Other tools mentioned in the literature are the mini mental state examination (MMSE) [26]: it is a screening test for cognitive impairment and not specifically designed for the detection of delirium [27]; nonetheless, it seems to be used in some studies as a means of identifying delirium in a stroke patient [4]. Levkoff et al. [27] provide a useful review of instruments available for the detection of delirium in hospital patients.

In summary, a variety of screening and diagnostic tools for the detection of delirium exist; screening for delirium is important in a variety of settings, but there is no clear guidance about how, when, and how often to screen patients for delirium after stroke. Although studies of delirium in acute stroke describe how delirium is identified [4], it is unclear what happens in clinical practice; namely, how delirium is identified and diagnosed and by whom. The literature from the general medical/geriatric settings gives an indication that, in practice, delirium is under-recognised, and staff do not routinely use screening tools in daily practice [28, 29].

The aims of this web-based survey were to investigate the use of delirium screening and diagnostic tools in patients with acute stroke. We sought to identify whether and if so, how doctors and nurses across Scotland screen for and diagnose delirium in acute stroke.

The survey explored the following questions: (i)Is delirium screened for in routine clinical practice? (ii)How often does screening for delirium in acute stroke take place and what is the method of screening and or diagnosis in clinical practice? (iii)Who is most likely to identify delirium in acute stroke? (iv)Which delirium identification tools (if any) are used?(v)What are clinicians’ views about the suitability of screening tools as they are used within acute stroke care?

3. Methods

3.1. Survey Questionnaire

The Bristol Online Survey Tool was used to set up, collect, and subsequently analyse the survey data. This tool is widely used by universities and other public bodies in the UK [30]. Web surveys are inexpensive; they increase the ease of administration for the research team and allow data to be analysed as soon as it is logged on the online survey tool [31]. Web-based surveys yield the same findings as paper surveys in terms of content [32, 33] although online surveys may yield a slightly lower response rate [34]. We attempted to maximise response by keeping the length of the survey as short as possible, maintaining a clear structure, and using clear language [34]. A scrolling design (rather than the questions set over several webpages) was chosen to maximise ease of use and minimise potential technical difficulties. This design is reputed to increase response rate as it reduces the time taken to complete the survey [35]. Survey questions were constructed based on published guidance on effective question writing [36] and effective design for web-based response options such as minimising “drop down boxes” as they are burdensome to respondents [37]. Following questionnaire development, the survey tool was distributed to three clinicians: a stroke physician, a stroke nurse specialist, and a psychiatrist. This process was used to check for language, structure, and sequence of the questions presented [38], but no data were collected during this process. Two minor difficulties related to the ambiguity of questions were identified and rectified prior to the survey being distributed among stroke clinicians practising throughout Scotland.

3.2. Sample and Recruitment

The survey was distributed to 217 clinicians (doctors and nurses) working in the acute stroke setting in Scotland by the administrators of the British Association of Stroke Physicians (BASP) and the Scottish Stroke Research Network (SSRN), and the first author contacted all () members of the Scottish Stroke Nurses Forum (SSNF) directly. The first author cross checked the complete distribution lists of SSRN and SSNF and removed duplicate names and email addresses. The BASP database was not shared with the first author; therefore, it was not possible to check for duplicates with other databases, and we were informed by the administrator that the approximate number of BASP members in Scotland is 60. The initial invitations were sent by email in July 2012. Two further email reminders were sent two weeks apart, in August 2012. In order to be able to calculate response rate as accurately as possible, respondents were asked not to disseminate the email invitation among their colleagues.

3.3. Data Analysis

Descriptive statistics were used to report nominal data. Free text comments were analysed by the first author using qualitative content analysis methodology: the first author read and reread the words used in the responses and then classified them into small sets of categories or codes of shared meaning. The codes were counted to determine how frequently they appear within the text responses and patterns relating to the key themes emerged [39–41]. Data regarding size of stroke unit and number of stroke beds of all hospitals across Scotland were obtained via Information Service Division Scotland [42]; these are presented in Table 1 to categorise respondent characteristics.

Table 1: Respondent characteristics.

Characteristics
(%)

Profession

    Doctors
36 (53.7)

    Nurses
29 (43.3)

Grade

    Consultant
24 (36.9)

    Senior trainees (doctors)
12 (18.4)

    Senior nurse (band 7 and above)
14 (21.5)

    Main grade nurse (band 6 and below)
15 (23.0)

Main practice area

    Specialist stroke unit
47 (72.3)

    General hospital ward
8 (12.3)

    Both of the aforementioned
10 (15.4)

No. of patients admitted to respondents’ workplace each year [39]

    >500
15 (23)

    250–500
39 (60)

    100–250
7 (10.7)

    <100
4 (6.1)

3.4. Ethics

This study did not require ethical approval as it an opinion survey seeking the views of NHS staff on service delivery. A letter of confirmation was obtained from the South East Scotland Research Ethics Service. Ethical approval was gained from Queen Margaret University.

4. Results

Sixty-five (30%) responses were received following an initial email and two reminders. A total of 36/90 (40%) of doctors replied; 29/127 (23%) of nurses replied. The characteristics of the respondents are summarised in Table 1.

4.1. Screening for Delirium

In response to the question: “Does your ward have a policy on screening new patients for delirium?”, 21/65 (32%) respondents selected “yes,” 35 respondents (53.5%) replied “no,” and 9 respondents (14%) responded “unsure.” In response to the question: “Do you routinely screen for delirium on admitting new patients to the ward?,” 31(48%) selected “yes” and 34 (52%) selected “no.” The following question: “Do you screen patients for delirium on a regular basis during admission?” yielded the same result, with 31 (48%) selecting “yes” and 34 (52%) selecting “no.” Of the 31 respondents who selected “yes,” 25 (81%) reported screening “as the need arises”; two (6.5%) selected “once weekly”; and four (13%) selected “other” and provided a short text explanation: two respondents stated that screening occurred during ward rounds or if a concern is raised by a staff member. One respondent stated that they screened daily, and, one respondent stated that they screened on admission (which answers the original question: “Do you routinely screen for delirium on admitting new patients to the ward?”).

4.2. Diagnostic Methods

In response to the question: “How do you normally diagnose delirium in stroke patients?,” 28 respondents (43%) reported applying their clinical judgement, two respondents (3%) reported using a standardised tool, and the remaining respondents reported combining clinical judgement with the application of a standardised tool (, 32.3%). Two respondents selected “other”: one reported using “AMT (abbreviated mental test) and urine testing and observations” and the other reported using the CAM [24] to diagnose delirium. Twelve respondents (18.5%), all of whom were nurses, stated that they do not diagnose delirium in their practice and selected the option “I have not been trained to use a standardised tool”. Table 2 summarises these results.

Table 2: Questions regarding diagnostic practices and tools utilised.

Question
Doctors
Nurses

How do you normally diagnose delirium in stroke patients?

Standardised tool
1 (2.7%)
1 (3.4%)

Clinical judgement
22 (61%)
6 (20.6%)

Both the aforementioned
13 (36.1%)
8 (27.5%)

I do not diagnose delirium in my practice
0
12 (41.3%)

Other
0
2 (6.8%)

If you use a tool to diagnose or screen for delirium in stroke patients, please indicate which tool you use

CAM
11 (30%)
7 (24.1%)

CAM-ICU
2 (5.5%)
0

DRS
0
0

Delirium symptom review
1 (2.7%)
0

Organic brain syndrome scale
0
0

Other
8 (22.2%)
4 (13.7%)

No response
14 (38.8%)
18 (62%)

CAM: confusion assessment method.

CAM-ICU: confusion assessment method for intensive care unit.

DRS: delirium rating scale.

4.3. Clinicians’ Choice of Diagnostic Tool

Table 2 outlines the structure of the questions relating to the choice of diagnostic tool. Free text comments made in response to the question on clinicians’ choice of diagnostic tool revealed that six (9%) respondents used a tool developed by a local collaboration between Liaison Psychiatry and Geriatrics known as “4AT” [43]. Four respondents reported using either the abbreviated mental test (AMT) [44] or the MMSE [26].

4.4. Suitability of the Diagnostic Tool in a Stroke Population

Respondents were asked “Do you think the tool you use is suitable for a stroke population?.” A total of 52 (80%) of the 65 respondents answered this question. Seven respondents selected “yes” (13.5%), 16 respondents selected “no” (31%), and the remaining 29 selected “not sure” (56%). Figure 1 cross-references those who selected their tool of choice with clinicians’ opinion regarding suitability for stroke patients. 15 (23%) participants gave free text comments. The majority (; 53%) of comments related to the difficulty using a generic screening tool with persons who experience communication difficulties such as receptive or expressive aphasia. Four respondents questioned the validity of the tool in a stroke population and discussed in particular cognitive or “neurological abnormalities” arising from the stroke. One respondent felt that the tool they used had “reasonable face validity,” and one further respondent advocated the use of the CAM [24].

Figure 1: Is the tool you use suitable for use in stroke patients?

5. Discussion

Our survey results highlight a number of key findings that reveal current delirium diagnostic and screening practice in Scottish stroke services. Most stroke units either did not have a screening policy for the identification of delirium in acute stroke, or the clinicians were unaware of such policy. Almost half of respondents to this survey stated that they did not routinely screen for delirium in acute stroke. The diagnosis of delirium was reportedly made mainly by doctors, in most cases by means of clinical judgement and in some cases combined with the use of a standardised tool. Interestingly, the majority (41%) of nurses who responded to this survey () claimed that they do not diagnose delirium in their practice, citing lack of training to use a standardised tool as the main reason for this. This finding supports the findings of a survey of nurses across intensive care and general medical/surgical units which highlights that nurses have only modest confidence levels in identifying delirium in clinical practice [45]. Other authors have reported infrequent use of standardised tools for the screening and/or diagnosis of delirium: nurses reportedly rely largely on clinical judgement when it comes to diagnosing delirium. In these studies, the clinicians surveyed had recognised the importance of delirium as an underdiagnosed condition of potentially serious consequences; however, routine screening and utilisation of standardised observation tools were still the exceptions in a variety of studies [45–47]. Surveys of doctors highlight similar concerns. A survey of Brazilian critical care physicians found that less than 15% of respondents used validated delirium assessment tools [48]. An American survey of ICU clinicians found that despite the belief that the literature supported routine screening for delirium, only 40% of respondents did so, and of those, only a small number used specific delirium screening tools [49]. Furthermore, a finding from a survey of junior doctors working in a variety of medical settings in the UK revealed that the fundamental cause of under-recognition and undertreatment of delirium lies in the lack of knowledge of the diagnostic criteria and standardised screening tools [50].

Within our own survey, a small number of respondents reported using a variety of tools to diagnose delirium in their practice, citing tools which have not been validated for the use in acute stroke [26, 43]. Some studies found a degree of usefulness in detecting cognitive changes using the AMT [44] and the MMSE [26] which might be due to delirium [51–53]; however, these tools are not specifically designed to detect delirium [27, 49, 54]. Some of our respondents reported difficulties in using diagnostic tools in stroke patients because of aphasia. Our systematic review highlights that previous studies have excluded patients with aphasia from their cohorts for the same reason [4]. In our survey, only two respondents reported using the CAM-ICU, which might increase the proportion of patients with language difficulties who may be assessable [55] as the CAM-ICU does not rely on language for the diagnosis of delirium [49, 56]. This tool has recently been validated for use in stroke patients, demonstrating high sensitivity, specificity, overall accuracy, and inter-rater reliability [14, 57]. Various authors, in both nursing and medical literature are calling for clinicians to take a key role in the identification of delirium in practice, advocating the use of validated instruments to facilitate accurate and timely recognition, leading to prompt treatment and better outcomes for patients [5, 14, 28, 58].

Our response rate was 30%, a rate lower than a number of surveys (both online and traditional) of delirium identification published within the last five years [46–48, 50]. Our response rate seems to be influenced by the notable difference between doctors and nurses response to our invitation to participate: only 23% of nurses approached actually completed the survey. Eley et al. [59] identified the main barrier to nurses’ access to computers in the ward environment as lack of time due to other demands of the job. This may be a reason why the response rate from the nurses in this survey was comparatively low.

6. Strengths, Limitations, and Future Research

We were keen to explore practice within Scotland only at this stage, and we would plan to roll out the same survey throughout the UK. Our response rate was moderate but consistent with the literature on online surveys return rates [34, 60]. Other surveys examining delirium identification utilised a variety of methods of survey distribution which yielded better response rates, for example, using a combined approach of both paper and online options [47] or using the traditional postal questionnaire design [46, 50]. We were keen to be able to calculate our response rate; therefore, we used convenience sampling and approached specific individuals in the clinical field and avoided snowballing, but this may have introduced a selection bias. Nevertheless, our data are of interest because this is, to the best of our knowledge, the first survey of diagnostic and screening practice in relation to delirium in acute stroke services in the UK. Our survey contributes to a growing body of knowledge on delirium in acute stroke. This field of research is steadily growing as more publications are generated on the various aspects of identification [14, 52] and potential treatment [61, 62] of the condition.

It was interesting to note the inconsistent screening and diagnostic practice identified by this survey, which is perhaps related to the lack of guidance or policy regarding screening and diagnosis of delirium in stroke. It would be beneficial for UK best practice guidelines in stroke care [15, 17] to incorporate information on delirium and perhaps consider establishing a standardised way of identifying the condition in this population. This would require further research to be conducted, not only to validate a tool to detect delirium in stroke patients, but also to establish the most effective time intervals for screening patients. Another avenue for further research is to identify the barriers to regular and effective screening for delirium across all members of the multidisciplinary team. In light of the fact that both this survey and others have identified the need for training and increasing awareness of delirium among staff working with stroke patients, we would like to reiterate the importance of this and call for more staff to become familiar with the risk factors and outcomes associated with delirium. Increasing the amount of correctly identified cases of delirium may lead to better outcomes for these patients and may yield cost benefits to the organisation [7].

Acknowledgment

The authors declare that they have no conflict of interests. This project received no external funding; however, it forms part of Ms Carin-Levy’s Doctoral studies which are funded by Queen Margaret University, Edinburgh.

References

N. Siddiqi, A. O. House, and J. D. Holmes, “Occurrence and outcome of delirium in medical in-patients: a systematic literature review,” Age and Ageing, vol. 35, no. 4, pp. 350–364, 2006.

M. H. Dahl, O. M. Rønning, and B. Thommessen, “Delirium in acute stroke—prevalence and risk factors,” Acta Neurologica Scandinavica, vol. 122, no. 190, pp. 39–43, 2010.

Y. Gustafson, T. Olsson, K. Asplund, and E. Hagg, “Acute confusional state (delirium) soon after stroke is associated with hypercortisolism,” Cerebrovascular Diseases, vol. 3, pp. 33–38, 1993.

G. Carin-Levy, G. E. Mead, K. Nicol, R. Rush, and F. van Wijck, “Delirium in acute stroke: screening tools, incidence rates and predictors: a systematic review,” Journal of Neurology, vol. 259, no. 8, pp. 1590–1599, 2012.

Q. Shi, R. Presutti, D. Selchen, and G. Saposnik, “Delirium in acute stroke: a sysaematic review and meta-analysis,” Stroke, vol. 43, no. 3, pp. 645–649, 2012.

J. McCusker, M. G. Cole, N. Dendukuri, and E. Belzile, “Does delirium increase hospital stay?” Journal of the American Geriatrics Society, vol. 51, no. 11, pp. 1539–1546, 2003.

National Institute for Clinical Excellence, Delirium: Diagnosis Prevention and Management, NICE, 2010.

J. Young and S. K. Inouye, “Delirium in older people,” British Medical Journal, vol. 334, no. 7598, pp. 842–846, 2007.

J. C. Jackson, S. M. Gordon, R. P. Hart, R. O. Hopkins, and E. W. Ely, “The association between delirium and cognitive decline: a review of the empirical literature,” Neuropsychology Review, vol. 14, no. 2, pp. 87–98, 2004.

D. H. J. Davis, G. M. Terrera, H. Keage et al., “delirium is a strong risk factor for dementia in the oldest old: a population based cohort study,” Brain, vol. 135, no. 9, pp. 2809–2816, 2012.

M. W. A. Van Rijsbergen, A. W. Oldenbeuving, R. E. Nieuwenhuis-Mark et al., “Delirium in acute stroke: a predictor of subsequent cognitive impairment? A two-year follow-up study,” Journal of the Neurological Sciences, vol. 306, no. 1-2, pp. 138–142, 2011.

S. Melkas, J. V. Laurila, R. Vataja et al., “Post-stroke delirium in relation to dementia and long-term mortality,” International Journal of Geriatric Psychiatry, vol. 27, no. 4, pp. 401–408, 2012.

J. Mcmanus, R. Pathansali, R. Stewart, A. Macdonald, and S. Jackson, “Delirium post-stroke,” Age and Ageing, vol. 36, no. 6, pp. 613–618, 2007.

A. Mitasova, M. Kostalova, J. Bednarik et al., “Poststroke delirium incidence and outcomes: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU),” Critical Care Medicine, vol. 40, no. 2, pp. 484–490, 2012.

National Institute for Health and Clinical Excellence, Stroke: National Clinical Guidelines for Treatment and Initial Management of Acute Stroke and Transient Ischaemic Attack (TIA), NICE, 2008.

Intercollegiate Stroke Wrking Party, National Clinical Guidelines for Stroke, Royal College of Physicians, London, UK, 4th edition, 2012.

Scottish Intercollegiate Guidelines Network, SIGN 118: Management of Patients with Stroke: Rehabilitation, Prevention and Management of Complications, and Discharge Planning. A National Clinical Guideline, 2010.

National Stroke Foundation, “Clinical Guidelines for Stroke Management,” Melbourne, Australia, 2010, http://strokefoundation.com.au/site/media/Clinical_Guidelines_Acute_Management_Recommendations_2010.pdf

E. L. Miller, L. Murray, L. Richards et al., “Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: a scientific statement from the American heart association,” Stroke, vol. 41, no. 10, pp. 2402–2448, 2010.

M. P. Lindsay, G. Gubitz, M. Bayley et al., Canadian Best Practice Recommendations for Stroke Care on Behalf of the Canadian Stroke Strategy Best Practices and Standards Writing Group, Canadian Stroke Network, Ottawa, Canada, 2010, http://www.strokebestpractices.ca/wp-content/uploads/2011/04/2010BPR_ENG.pdf.

J. W. Devlin, J. J. Fong, E. P. Howard et al., “Assessment of delirium in the intensive care unit: nursing practices and perceptions,” American Journal of Critical Care, vol. 17, no. 6, pp. 555–565, 2008.

Australian Health Ministers Advisory Council, Clinical Practice Guidelines for the Management of Delirium in Older People, Victorian Government Department of Human Services, Melbourne, Australia, 2006.

Canadian Coalition for Seniors' Mental Health, The Assessment and Treatment of Delirium, Canadian Coalition of Seniors' Mental Health, Toronto, Canada, 2006.

S. K. Inouye, C. H. Van Dyck, C. A. Alessi, S. Balkin, A. P. Siegal, and R. I. Horwitz, “Clarifying confusion: the confusion assessment method: a new method for detection of delirium,” Annals of Internal Medicine, vol. 113, no. 12, pp. 941–948, 1990.

P. T. Trzepacz, R. W. Baker, and J. Greenhouse, “A symptom rating scale for delirium,” Psychiatry Research, vol. 23, no. 1, pp. 89–97, 1988.

M. F. Folstein, S. E. Folstein, and P. R. McHugh, “Mini-mental state: a practical method for grading the cognitive state of patients for the clinician,” Journal of Psychiatric Research, vol. 12, pp. 189–198, 1975.

S. Levkoff, B. Liptzin, P. Cleary, C. H. Reilly, and D. Evans, “Review of research instruments and techniques used to detect delirium,” International Psychogeriatrics, vol. 3, no. 2, pp. 253–271, 1991.

L. G. Wells, “Why don't intensive care nurses perform routine delirium assessment? A discussion of the literature,” Australian Critical Care, vol. 25, no. 3, pp. 157–161, 2012.

A. Teodorczuk, E. Reynish, and K. Milisen, “Improving recognition of delirium in clinical practice: a call for action,” BMC Geriatrics, vol. 2012, article 55, 2012.

Bristol Online Surveys, About BOS, http://www.survey.bris.ac.uk/support/about

P. B. Doherty, “The context and culture of web as a research environment,” in Online Assessment Measurement and Evaluation, D. D. Williams, S. L. Howell, and M. Hericko, Eds., pp. 10–27, Information Science Publishing, London, UK, 2006.

H. M. Huang, “Do print and Web surveys provide the same results?” Computers in Human Behavior, vol. 22, no. 3, pp. 334–350, 2006.

M. D. Kaplowitz, T. D. Hadlock, and R. Levine, “A comparison of web and mail survey response rates,” Public Opinion Quarterly, vol. 68, no. 1, pp. 94–101, 2004.

W. Fan and Z. Yan, “Factors affecting response rates of the web survey: a systematic review,” Computers in Human Behavior, vol. 26, no. 2, pp. 132–139, 2010.

D. A. Dillman and J. D. Smyth, “Design effects in the transition to web-based surveys,” American Journal of Preventive Medicine, vol. 32, no. 5, pp. S90–S96, 2007.

F. J. Fowler and C. Cosenza, “Writing effective questions,” in International Handbook of Survey Methodology, E. D. de Leeuw, D. Hox, and D. D. A, Eds., chapter 8, pp. 136–159, Psychology Press, New York, NY, USA, 2008.

K. L. Manfreda and V. Vehovar, “Internet surveys,” in International Handbook of Survey Methodology, E. D. de Leeuw, J. J. Hox, and D. A. Dillman, Eds., chapter 14, pp. 264–284, Psychology Press, New York, NY, USA, 2008.

P. Naithani, “Guidelines for developing a robust web survey,” Advances in Information Technology and Management, vol. 1, pp. 20–23, 2012.

S. Elo and H. Kyngas, “The qualitative content analysis process,” Journal of Advanced Nursing, vol. 62, no. 1, pp. 107–115, 2008.

D. L. Morgan, “Qualitative content analysis: a guide to paths not taken,” Qualitative Health Research, vol. 3, no. 1, pp. 112–121, 1993.

K. Krippendorf, Content Analysis: An Introduction to Its Methodology, Sage Publications, Thousand Oaks, Calif, USA, 2004.

ISD Scotland, “Scottish stroke care audit 2010,” National Report, 2010.

A. MacLullich, T. Ryan, and H. Cash, “The 4A test: screening instrument for cognitive impairment and delirium,” http://www.the4at.com/

H. M. Hodkinson, “Evaluation of a mental test score for assessment of mental impairment in the elderly,” Age and Ageing, vol. 1, no. 4, pp. 233–238, 1972.

B. Flagg, S. McDowell, and J. M. Beulow, “Nursing identification of delirium,” Clinical Nurse Specialist, vol. 24, no. 25, pp. 260–266, 2010.

L. M. Forsgren and M. Eriksson, “Delirium-Awareness, observation and interventions in intensive care units: a national survey of Swedish ICU head nurses,” Intensive and Critical Care Nursing, vol. 26, no. 5, pp. 296–303, 2010.

R. P. Patel, M. Gambrell, T. Speroff et al., “Delirium and sedation in the intensive care unit: survey of behaviors and attitudes of 1384 healthcare professionals,” Critical Care Medicine, vol. 37, no. 3, pp. 825–832, 2009.

J. I. F. Salluh, F. Dal-Pizzol, P. V. C. Mello et al., “Delirium recognition and sedation practices in critically ill patients: a survey on the attitudes of 1015 Brazilian critical care physicians,” Journal of Critical Care, vol. 24, no. 4, pp. 556–562, 2009.

E. W. Ely, R. K. Stephens, J. C. Jackson et al., “Current opinions regarding the importance, diagnosis, and management of delirium in the intensive care unit: a survey of 912 healthcare professionals,” Critical Care Medicine, vol. 32, no. 1, pp. 106–112, 2004.

D. Davis and A. Maclullich, “Understanding barriers to delirium care: a multicentre survey of knowledge and attitudes amongst UK junior doctors,” Age and Ageing, vol. 38, no. 5, pp. 559–563, 2009.

S. T. O'Keeffe, E. C. Mulkerrin, K. Nayeem, M. Varughese, and I. Pillay, “Use of serial mini-mental state examinations to diagnose and monitor delirium in elderly hospital patients,” Journal of the American Geriatrics Society, vol. 53, no. 5, pp. 867–870, 2005.

J. Mc Manus, R. Pathansali, H. Hassan et al., “The evaluation of delirium post-stroke,” International Journal of Geriatric Psychiatry, vol. 24, no. 11, pp. 1251–1256, 2009.

A. Ni Chonchubhair, R. Valacio, J. Kelly, and S. O'Keeffe, “Use of the abbreviated mental test to detect postoperative delirium in elderly people,” British Journal of Anaesthesia, vol. 75, no. 4, pp. 481–482, 1995.

R. J. Hall, D. J. Meagher, and A. M. J. MacLullich, “Delirium detection and monitoring outside the ICU,” Best Practice & Research Clinical Anaesthesiology, vol. 26, no. 3, pp. 367–383, 2012.

E. W. Ely, R. Margolin, J. Francis et al., “Evaluation of delirium in critically ill patients: aalidation of the Confusion Assessment Method for the intensive care unit (CAM-ICU),” Critical Care Medicine, vol. 29, no. 7, pp. 1370–1379, 2001.

G. M. Eastwood, L. Peck, R. Bellomo, I. Baldwin, and M. C. Reade, “A questionnaire survey of critical care nurses' attitudes to delirium assessment before and after introduction of the CAM-ICU,” Australian Critical Care, vol. 25, no. 3, pp. 162–169, 2012.

M. Kostalova, J. Bednarik, A. Mitasova et al., “Towards a predictive model for post stroke delirium,” Brain Injury, vol. 26, pp. 962–971, 2012.

M. J. Schuurmans, “Early recognition of delirium: review of the literature,” Journal of Clinical Nursing, vol. 10, no. 6, pp. 721–729, 2001.

R. Eley, T. Fallon, J. Soar, E. Buikstra, and D. Hegney, “Barriers to use of information and computer technology by Australia's nurses: a national survey,” Journal of Clinical Nursing, vol. 18, no. 8, pp. 1151–1158, 2009.

C. Cook, F. Heath, and R. L. Thompson, “A meta-analysis of response rates in Web- or internet-based surveys,” Educational and Psychological Measurement, vol. 60, no. 6, pp. 821–836, 2000.

A. W. Oldenbeuving, P. L. M. de Kort, B. P. W. Jansen, L. J. Kappelle, and G. Roks, “A pilot study of rivastigmine in the treatment of delirium after stroke: a safe alternative,” BMC Neurology, vol. 8, article 34, 2008.

T. Ohta, K. Murao, K. Miyake, and K. Takemoto, “Melatonin receptor agonists for treating delirium in elderly patients with acute stroke,” Journal of Stroke and Cerebrovascular Diseases, 2012.