Assessments of potential cognitive impairment and diagnostic evaluations of dementia are often initiated and carried out in primary care. However, earlier studies have demonstrated that cognitive impairment remains undetected in many patients [
Other than the MMSE and CDT, few studies have investigated the efficacy of tests that can evaluate dementia in a primary care setting. In some primary care clinical settings in Sweden, occupational therapists (OT) use Cognistat (formerly known as the Neurobehavioral Cognitive Status Examination), a multidomain cognitive test to evaluate cognitive impairment. However, few studies have evaluated the validity and clinical utility of Cognistat for diagnosing dementia. Cognistat assesses several cognitive domains separately but does not sum global cognitive function [
The study has a cross-sectional design. Data were collected over 19 months, between 2007 and 2009, from 4 primary health care (PHC) centres in Linköping, a city in southeast Sweden. The community has around 150 000 inhabitants and the 4 PHC centres serve a population of 49 800 people; 11 200 of these residents were 65 years or older. The 4 PHC centres are socioeconomically characteristic of primary care in Linköping. There is a slightly older demographic profile in the population served by the 4 PHC centres; 22% of the population was over the age of 65 years compared with 16% in the overall primary care population in Linköping. The participants were systematically recruited from the 4 PHC centres during the selection period. All participants were asked to take part in the study during an appointment with a general practitioner (GP) and those who agreed provided written informed consent. The inclusion criteria were as follows: older than 65 years, any complaint or suspicion of cognitive symptoms expressed either by the patient, an informant, or primary care staff. In all, 52 people met these criteria. During the same period, patients visiting the GP for medical reasons other than possible cognitive symptoms were asked to participate in a clinical comparison group. Those participants extended the study group to assure its clinical relevance. Inclusion criteria for the comparison group were as follows: age older than 65 years and no complaint or suspicion of cognitive symptoms expressed either by the patient, an informant, or a GP. Twenty-nine people met the criteria and were willing to participate. These 29 participants were asked to complete a short questionnaire regarding self-estimated memory function and diseases/trauma related to the brain or head (Appendix). Exclusion criteria for all participants were a medical record of recent stroke, brain tumour, brain-related infection, head trauma, ongoing verified psychiatric illness, a previous dementia investigation, or a known dementia diagnosis. A study population of 81 patients were selected to provide an unsorted mixed primary care population with some suspicion of cognitive decline.
Cognistat includes 10 subtests: orientation, attention, language (comprehension, repetition, and naming), constructional ability, memory, calculation, and reasoning (similarities and judgement) [
The MMSE, which takes about 10 minutes to administer, assesses orientation, attention, memory, language, and visual construction [
The CDT, which takes about 5 minutes to administer, is a cognitive instrument that measures visuospatial and executive functions [
The diagnostic evaluation process is presented in Figure
Flowchart of the diagnostic evaluation process. CDT, Clock Drawing Test; CT, computed tomography; GP, general practitioner; MMSE, Mini-Mental State Examination; PHC, primary health care; SC, secondary care.
Diagnoses were made by consensus using the psychologists’ and the geriatricians’ examinations independent of the results from the cognitive test under study (i.e., they were blinded to these results). Diagnoses were based on the patient’s case report, the patient’s physical examination (including a neurologic examination), blood tests, neuroimaging, and the psychologist’s assessment. The blood tests were done to exclude somatic causes for cognitive dysfunction. Etiologic diagnoses for dementia were initially set according to The ICD-10 International Classification (ICD-10) [
Quantitative analysis of the data was performed using SPSS for Windows 19.0 (SPSS, Inc., Chicago, IL). The chi-square and Fisher exact tests were used to compare differences in gender, native language, medical history, and medical drugs between the groups. Age, education, duration of symptoms, and test scores for MMSE, CDT, and Cognistat, and the neuropsychologist test battery were compared using the
Among the group who visited primary care for reasons other than cognitive symptoms (
Descriptive data (mean ± standard deviation (SD)) for the cognitive impairment group and the no cognitive impairment group after the final diagnoses.
Variables | Cognitive impairment |
No cognitive impairment |
|
---|---|---|---|
Females/males | 22/24 | 25/8 | 0.013 |
Age, years ± SD |
|
|
0.004 |
Education, years ± SD |
|
|
0.037 |
Native language, |
|||
Swedish | 42 (91) | 33 (100) |
|
Non-Swedish | 4 (9) | 0 (0) | |
Duration of cognitive symptoms, years ± SD |
|
|
<0.001 |
Range | 1–9 | 0–4 | |
Medical history, |
|||
Cerebrovascular disease | 5 (11) | 1 (3) |
|
Ischemic heart disease | 15 (33) | 5 (15) | 0.078 |
Hypertension | 30 (65) | 17 (52) | 0.221 |
Diabetic disease | 7 (15) | 1 (3) |
|
Anxiety | 4 (9) | 0 (0) |
|
Mild depression | 6 (13) | 1 (3) |
|
Medical drugs, |
|||
Warfarin | 7 (15) | 4 (12) |
|
Salicylic acid (low dose) | 25 (54) | 10 (30) | 0.034 |
Lipid-lowering drug | 25 (54) | 10 (30) | 0.034 |
Antihypertensive | 38 (83) | 18 (54) | 0.007 |
Insulin | 4 (9) | 0 (0) |
|
Antidiabetic | 3 (7) | 1 (3) |
|
Antidepressant | 18 (39) | 7 (21) | 0.091 |
Sleeping drug | 16 (35) | 5 (15) | 0.051 |
Antipsychotic | 4 (9) | 0 (0) |
|
Anxiolytic | 11 (24) | 4 (12) | 0.188 |
Test results (mean ± standard deviation) for the cognitive impairment group and the no cognitive impairment group after final diagnoses: confidence intervals of the mean difference and
Test | Cognitive impairment |
No cognitive impairment |
95% confidence interval |
|
---|---|---|---|---|
MMSE, mean score ± SD |
|
|
4.890–2.513 | <0.001 |
CDT, mean score ± SD |
|
|
0.693–0.004 | 0.048 |
Cognistat subtest (test range), mean score ± SD | ||||
Orientation (0–12) |
|
|
1.328–0.317 | 0.002 |
Attention (0–8) |
|
|
1.784–0.296 | 0.007 |
Comprehension (0–6) |
|
|
0.769–0.223 | 0.001 |
Repetition (0–12) |
|
|
1.133–0.216 | 0.005 |
Naming (0–8) |
|
|
0.872–0.121 | 0.011 |
Construction (0–6) |
|
|
1.824–0.667 | <0.001 |
Memory (0–12) |
|
|
5.113–2.743 | <0.001 |
Calculations (0–4) |
|
|
0.610–0.078 | 0.012 |
Similarities (0–8) |
|
|
1.398–0.340 | 0.002 |
Judgement (0–6) |
|
|
1.227–0.145 | 0.014 |
Neuropsychology test battery, mean score ± SD | ||||
ADAS-cog |
|
|
6.092–9.886 | <0.001 |
Babcock Story Recall test-immediate |
|
|
4.191–1.830 | <0.001 |
Babcock Story Recall test-delayed |
|
|
7.545–4.414 | <0.001 |
Letter fluency test (D-KEFS), raw score |
|
|
19.993–9.639 | <0.001 |
Letter fluency test (D-KEFS), scale score |
|
|
5.765–2.622 | <0.001 |
The Vocabulary Test (WAIS-III), raw score |
|
|
14.158–4.408 | <0.001 |
The Vocabulary Test (WAIS-III), scale score |
|
|
3.064–0.791 | 0.001 |
Trail Making Test A, seconds |
|
|
37.581–73.294 | <0.001 |
Trail Making Test B, seconds |
|
|
63.321–169.413 | <0.001 |
ADAS-cog, Alzheimer Disease Assessment Scale-cognitive subscale; CDT, Clock Drawing Test; D-KEFS, Delis-Kaplan Executive Function System; MMSE, Mini-Mental State Examination; SD, standard deviation; WAIS-III, Wechsler Adult Intelligence Scale-III.
The sensitivity, specificity, PPV, NPV, AUC, CUI+, and the Youden Index are presented in Table
Diagnostic accuracy of the tests.
Test cutoff | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | AUC (95% CI) |
|
CUI+ | Youden Index ( |
---|---|---|---|---|---|---|---|---|
Cognistat ≥1 subtest | 0.85 (0.75–0.92) | 0.79 (0.65–0.88) | 0.85 (0.75–0.92) | 0.79 (0.65–0.88) | 0.82 (0.72–0.92) | <0.001 | 0.72 (good) | 0.64 (0.41–0.78) |
Cognistat ≥2 subtests | 0.57 (0.48–0.59) | 0.97 (0.85–0.99) | 0.96 (0.82–1.00) | 0.62 (0.54–0.63) | 0.77 (0.66–0.87) | <0.001 | 0.55 (satisfactory) | 0.54 (0.33–0.58) |
MMSE ≤26 | 0.59 (0.49–0.63) | 0.91 (0.78–0.98) | 0.90 (0.76–0.97) | 0.61 (0.52–0.66) | 0.75 (0.64–0.86) | <0.001 | 0.53 (satisfactory) | 0.50 (0.27–0.61) |
MMSE ≤23 | 0.26 (0.19–0.26) | 1.00 (0.90–1.00) | 1.00 (0.72–1.00) | 0.49 (0.44–0.49) | 0.63 (0.51–0.75) | <0.049 | 0.26 (poor) | 0.26 (0.09–0.26) |
CDT ≤4 | 0.26 (0.17–0.32) | 0.88 (0.76–0.96) | 0.75 (0.50–0.91) | 0.46 (0.40–0.50) | 0.57 (0.44–0.70) | <0.292 | 0.20 (poor) | 0.14 (−0.07–0.28) |
MMSE + CDT ≤26 and ≤4 | 0.70 (0.60–0.77) | 0.79 (0.65–0.89) | 0.82 (0.70–0.91) | 0.65 (0.53–0.74) | 0.74 (0.63–0.85) | <0.001 | 0.57 (satisfactory) | 0.48 (0.24–0.66) |
MMSE + CDT ≤23 and ≤4 | 0.46 (0.36–0.51) | 0.88 (0.75–0.96) | 0.84 (0.66–0.95) | 0.54 (0.46–0.59) | 0.67 (0.55–0.79) | <0.001 | 0.39 (poor) | 0.34 (0.11–0.47) |
AUC, area under the receiver operating characteristic curve; CDT, Clock Drawing Test; CI, confidence interval; CUI, Clinical Utility Index; MMSE, Mini-Mental State Examination; NPV, negative predictive value; PPV, positive predictive value.
Sensitivity and specificity and AUC of subtests of Cognistat (cognitive impairment 46 versus no cognitive impairment 33).
Subtest of Cognistat (test range) | Cutoff | Sensitivity | Specificity | AUC (95% confidence interval) |
|
---|---|---|---|---|---|
Orientation (0–12) | ≤8 | 0.09 | 1.00 | 0.64 (0.52–0.76) | 0.036 |
Attention (0–8) | ≤4 | 0.35 | 0.94 | 0.66 (0.54–0.78) | 0.016 |
Comprehension (0–6) | ≤4 | 0.09 | 1.00 | 0.68 (0.56–0.79) | 0.008 |
Repetition (0–12) | ≤9 | 0.15 | 1.00 | 0.61 (0.49–0.74) | 0.088 |
Naming (0–8) | ≤5 | 0.13 | 1.00 | 0.58 (0.46–0.71) | 0.216 |
Construction (0–6) | ≤2 | 0.39 | 0.97 | 0.73 (0.62–0.84) | 0.001 |
Memory (0–12) | ≤6 | 0.65 | 0.91 | 0.84 (0.75–0.93) | <0.001 |
Calculations (0–4) | ≤2 | 0.13 | 1.00 | 0.59 (0.47–0.72) | 0.170 |
Similarities (0–8) | ≤3 | 0.15 | 1.00 | 0.66 (0.54–0.78) | 0.018 |
Judgement (0–6) | ≤3 | 0.24 | 0.94 | 0.64 (0.52–0.76) | 0.038 |
AUC, area under the receiver operating characteristic curve.
The ROC curves for Cognistat ≥1 subtest, MMSE ≤ 26, CDT ≤ 4, and combined MMSE and CDT ≤26 or ≤4. The AUC values are presented in Table
The sensitivity, specificity, and the AUC for each subtest of Cognistat are shown in Table
This study found that Cognistat has relatively good sensitivity and PPV for detecting cognitive impairment and an acceptable specificity and NPV in a primary care population. The high sensitivity in combination with acceptable specificity and the high PPV indicates that the test is useful in primary care. The negative aspect of a test with high sensitivity and low specificity is the risk of overdiagnosis. On the other hand, tests with low sensitivity and high specificity increase the risk of underdiagnosis, with the added risk that patients and relatives do not receive adequate social support and medical treatment. This study investigated a population with a high proportion of patients with cognitive impairment; therefore the predictive values should be interpreted with this in mind. To evaluate the test in the whole primary care population, it would have been necessary to include all patients over the age of 65 years at the time of inclusion, a condition that was not feasible. On the other hand, we do not recommend screening all patients over the age of 65 years for cognitive impairment; this study investigated the clinical utility of Cognistat in primary care patients in whom some sort of cognitive decline was suspected.
The results indicate that Cognistat is more sensitive than the MMSE and yet still specific. Earlier research [
Cognistat is appealing as a cognitive instrument for primary care because it includes several cognitive subtests but is still easily administered. The multidomain perspective adds valuable information in a clinically applicable way with the possibility of presenting the results graphically and providing information about the level of impairment. DSM-IV requires memory and another cognitive domain to be impaired for a diagnosis of dementia and a test that includes more cognitive domains than memory might help the GP to determine a diagnosis. The screening and metric procedure in Cognistat is appealing and some qualitative data are provided during the test that are not presented in these figures. However, the test might be improved and even more sensitive with the addition of a short subtest of executive function.
The brief time it takes to perform the tests is important, especially in a clinical primary care setting. The Cognistat takes more time to administer than the MMSE, in some cases twice as long, but compared with other multidomain instruments, 20 minutes is acceptable. The economic cost must also be considered. The Cognistat has to be purchased unlike the MMSE and the CDT, which are free of charge. The extra time it takes to perform Cognistat has to be considered as a limited cost, if the test can lead to an early correct diagnosis and treatment and the benefit of lower social cost. In Sweden, the use of a multiprofessional team is recommended in the investigation and care of people with dementia [
Some limitations need to be discussed. In this study, we used the cutoffs suggested in the Cognistat manual. As a consequence, participants with a high premorbid function could produce false-negative results due to a ceiling effect. A further limitation is the relatively small sample size. A larger study would be of interest to confirm the result. However, a power calculation indicated that a sample size of about 30 patients per group was sufficient. Our study does not analyse different subgroups of cognitive impairment. Van Gorp et al. [
The study results indicate that Cognistat, instead of or as a complement to the MMSE and the CDT, can be used in primary care as an initial instrument to evaluate cognitive impairment. A disadvantage of Cognistat might be that it does not include tests that measure processing speed or more demanding frontal or executive functions (some subtests, however, do incorporate measures of minor executive functions). Earlier studies have shown that this is an early symptom of dementia [
In Sweden, Cognistat is mostly administered by OTs. In addition to evaluating cognitive function, the OT can estimate the consequences of impairment in daily living and plan for further interventions that aim to restore or improve the functioning and activities of patients with cognitive impairment. The results of the test always need to be interpreted in the patient’s daily life situation. None of the cognitive tests used in this study can be used alone to diagnose cognitive impairment; that is, the neuropsychology tests always need to be combined with other parts of the clinical evaluation, such as a medical history, diagnostic radiology, and a physical examination. In clinical use, a GP should consider the test results of each individual and interpret these within the overall status of the patient.
Cognistat shows fairly good diagnostic accuracy and clinical utility for detecting cognitive impairment in this primary care population. Memory and construction were the most prominent findings among the subtests included in Cognistat. Cognistat should be used for basic primary care evaluations to detect cognitive impairment overall, but it could also complement dementia evaluations in primary care facilities when the MMSE and CDT results are questionable. In addition, Cognistat can help a GP to determine which patients need to be referred to SC for further evaluation.
Please, answer by checking the yes or no square for each question. Do you feel that your general memory works in a way that you can easily manage your everyday life?
□ Yes □ No Do you feel that your general thinking works in a way that you can easily manage your everyday life?
□ Yes □ No Do you feel that your general mood is such that it does not affect your everyday life negatively?
□ Yes □ No If you live with a partner or anyone else who knows you well, does he or she agree with your answers?
□ Yes □ No
If the above questions are answered with YES, you can be considered for participation in the study in the reference group.
Have you ever been diagnosed with cerebral infarction or haemorrhage (stroke) or some other types of ischemic brain dysfunction?
□ Yes □ No Have you ever been treated for a brain tumour?
□ Yes □ No Have you ever been treated for any brain infection?
□ Yes □ No Have you suffered any blows to the head in recent weeks?
□ Yes □ No
If the above questions are answered with NO you can be considered for participation in the study in the reference group.
This study was approved by the Ethical Review Board in Sweden (Research Ethics Committee of Linkoping University, DNR M137-07).
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was supported by grants from the Swedish Brain Power, ALF Grants, and the County Council of Östergötland.