Sensitivity and Specificity of the ECAS in Parkinson's Disease and Progressive Supranuclear Palsy

Disentangling Parkinson's disease (PD) and progressive supranuclear palsy (PSP) may be a diagnostic challenge. Cognitive signs may be useful, but existing screens are often insufficiently sensitive or unsuitable for assessing people with motor disorders. We investigated whether the newly developed ECAS, designed to be used with people with even severe motor disability, was sensitive to the cognitive impairment seen in PD and PSP and able to distinguish between these two disorders. Thirty patients with PD, 11 patients with PSP, and 40 healthy controls were assessed using the ECAS, as well as an extensive neuropsychological assessment. The ECAS detected cognitive impairment in 30% of the PD patients, all of whom fulfilled the diagnostic criteria for mild cognitive impairment. The ECAS was also able to detect cognitive impairment in PSP patients, with 81.8% of patients performing in the impaired range. The ECAS total score distinguished between the patients with PSP and healthy controls with high sensitivity (91.0) and specificity (86.8). Importantly, the ECAS was also able to distinguish between the two syndromes, with the measures of verbal fluency offering high sensitivity (82.0) and specificity (80.0). In sum, the ECAS is a quick, simple, and inexpensive test that can be used to support the differential diagnosis of PSP.


Introduction
It has now been over 50 years since progressive supranuclear palsy (PSP) was first described as a progressive neurological disorder with motor, ocular, and cognitive features [1]. Clinically, it remains difficult to distinguish from Parkinson's disease (PD) [2,3], particularly in the early stages [4]. Even when using agreed criteria, the accuracy of diagnosis is not 100% [5]. As it has a significantly worse prognosis than PD, with a more rapid progression [6], early detection is crucial for enabling access to appropriate interventions and support, as well as identifying patients suitable for clinical trials. In the absence of any disease-specific biomarkers, there is a need for a quick, simple, and inexpensive test that can be used for the differential diagnosis of PSP.
Both PSP and PD are characterised by extrapyramidal syndromes, each of which can comprise symptoms of bradykinesia, rigidity, and/or postural instability [7]. Both disorders can feature eye movement abnormalities, and although the presence of the supranuclear vertical gaze palsy in PSP is diagnostically helpful, it is not universal [8,9] and may be absent until quite late in the disease [10]. Although both disorders are thought to feature some similar cognitive signs, there is evidence to suggest that the specific cognitive profile may be a useful distinguishing feature [11].
In contrast, early stages of PD are characterised by only mild deficits in executive functions [24][25][26][27], but with illness progression, there is evolution from a mild cognitive impairment (MCI) to dementia, with greater involvement of posterior-based visual functions [28][29][30].
Existing screens of cognitive functioning can be criticised for being insufficiently sensitive to the cognitive profile of both PD and PSP. For example, the most widely used cognitive screen, the MMSE [31], has no measure of verbal fluency. Both the MMSE and the Addenbrooke's Cognitive Examination [32] have inadequate assessment of executive function. eir ensuing ceiling effects give them a low detection rate for cognitive impairment in Parkinsonian syndromes [33][34][35].
e Frontal Assessment Battery [36] does assess verbal fluency and executive function but has no measure of memory, language, or visuospatial function. Similarly, the Dementia Rating Scale (DRS) [37] has no measure of language or visuospatial function. is reliance upon the executive functions reduces its ability to discriminate PSP from PD [38] or frontotemporal syndromes [39]. e DRS also has a lengthy administration time and requires specialised testing materials, impractical for routine bedside use. e Montreal Cognitive Assessment [40] does have a measure of verbal fluency but does not accommodate for physical disability. Indeed, none of the existing assessments were designed specifically for people with movement disorders, such as Parkinsonian syndromes. Tasks involving speaking, writing, or drawing can be influenced by motor symptoms such as tremor, rigidity, bradykinesia, apraxia, or dysarthria; thus, a genuine cognitive impairment might be sometimes difficult to distinguish from motor dysfunction and performance decrements exaggerated by physical disability.
e ECAS [41] was recently developed as a brief assessment for the identification of cognitive and behavioural changes in disorders characterised by prominent motor symptoms, such as amyotrophic lateral sclerosis (ALS). It was developed to be used with patients with even severe physical disability and thus may be suitable for detecting cognitive impairment in all motor disorders. Many of the subtests can be performed either orally or manually, with some measures corrected for motor speed, reducing the impact that physical disability may have upon performance on cognitive tests [42]. It also allows the clinician to track cognitive impairment throughout the disease course, crucial for any longitudinal studies. e ECAS has been standardised using a sample of healthy controls, providing normative data for clinical use [41]. It has also been validated against other screening tools [43,44] and extensive neuropsychological assessment [45]. It is available in English [41], German [46], Swiss German [46], Italian [44], and Chinese [47]. However, it remains untested whether the ECAS is also sensitive to the cognitive impairment observed in other progressive movement disorders. us, the aims of the present study were to determine firstly whether the ECAS is sensitive to the cognitive impairment seen in PD and PSP and secondly whether it is able to distinguish between these disorders, in order to support the differential diagnosis of PSP.

Participants.
All patients were recruited from the National Hospital for Neurology and Neurosurgery, Queen Square, London. PSP patients (9 males and 2 females) were diagnosed using the NINDS-SPSP criteria [48] and had a mean illness duration of 3.73 years (range 1-11 years). PD patients (24 males and 6 females) fulfilled the Queen Square Brain Bank criteria for PD and had a mean illness duration of 5.67 years (range 0-14 years). All patients with PD-MCI were identified using the Movement Disorder Society Task Force guidelines [49], in which impairment (<2 SD) is present on at least two tests of cognitive functioning, either within or across different cognitive domains. e healthy controls were those reported by Niven et al. [45]. ey (26 males and 14 females) were recruited through the Psychology Department of the University of Edinburgh. No participant had significant neurological or psychiatric history. e research was done in accordance with the Declaration of Helsinki and approved by the NRES Committee London-Queen Square and the University of Edinburgh's Department of Psychology Ethics Committee.

Measures.
e ECAS is a 15-20-minute screen that includes assessment of the following domains: (1) fluency (free: words beginning with "S" and fixed: words beginning with "T" but with only four letters); (2) executive functions, separate from verbal fluency (Reverse Digit Span, Alternation, Inhibitory Sentence Completion, and Social Cognition); (3) language (Naming, Comprehension, and Spelling); (4) memory (Immediate Recall, Delayed Percentage Retention, and Delayed Recognition); and (5) visuospatial (Dot Counting, Cube Counting, and Number Location). Verbal fluency measures take into account the slowing of motor responses, by generating a verbal fluency index corrected for motor speed. Previously published ECAS normative data [41] were used to classify the abnormality of performance on each domain and calculate the total score out of a maximum of 136 (lower score indicating worse performance), with any scores <2 SD considered to be impaired.
Extensive neuropsychological testing was administered to assess the same domains (fluency, executive functions, language, memory, and visuospatial; Table 1). Mood was assessed using the Hospital Anxiety and Depression Scale [50], and patients were also assessed using the Apathy Scale [51].
Scores for the neuropsychological assessments were compared with published normative data. For each measure, patients were judged to be impaired if scores were ≤2 SD. In the case where multiple measures were used, performance was classified as impaired when ≤2 SD on one of the two or two of the three measures was used.

Statistical Analyses.
Data were analysed using SPSS v.19. Between-group comparisons were made using analyses of variance, and Pearson's and Spearman's correlations were used to detail the relationships between measures. Receiver operating characteristic (ROC) curve analyses were used to determine the relative sensitivity and specificity of the ECAS for the two patient groups. Table 2. ere were no significant group differences in age or education, and patients did not differ in symptom duration. ere were significant group effects found for both HADS Anxiety (F (2, 75) � 13.04; p < 0.001) and Depression (F (2, 77) � 19.03; p < 0.001), with post hoc analysis revealing that patients had significantly higher burden of symptoms than healthy controls but no significant group difference between patient groups. ere was no significant group difference in apathy scores between patient groups.

Performance on the ECAS.
ere was a significant effect of diagnosis on ECAS performance (Table 3). PSP patients had significantly lower total scores than PD patients and healthy controls, and PD patients had significantly lower total scores than healthy controls (all p < 0.017). ere was a significant effect of diagnosis on all domains, except visuospatial. PSP patients performed worse than PD patients and healthy controls on fluency, language, executive function, and memory (all p < 0.017). PD patients performed worse than healthy controls on executive function only (p < 0.017).
ere were no significant correlations between duration of symptoms and ECAS scores in either patient groups.
In order to investigate the specific nature of the impairment in both patient groups, individual domains were further investigated. In fluency, post hoc comparisons revealed a significant effect of diagnosis on both free fluency (F (2, 23.16) � 15.19; p < 0.017) and fixed fluency (F (2, 21.63) � 8.30; p < 0.017), with PSP patients performing worse than PD patients and healthy controls (all p < 0.017), but with no significant differences between PD patients and healthy controls. In language, there was a significant group effect on spelling (F (2, 76) � 10.58; p < 0.017), with PSP patients performing worse than PD  Both PSP and PD patients performed significantly worse than healthy controls on reverse digit span and social cognition (all p < 0.017), but with no significant differences between patient groups. PSP patients performed significantly worse than PD patients and healthy controls on alternation (p < 0.017), but with no significant difference between PD patients and healthy controls.

Performance on Full Neuropsychological Assessment.
Upon full neuropsychological assessment, there was a significant effect of diagnosis on fluency, executive function, and visuospatial domains (Table 4). Specifically, there were significant group differences on both measures of fluency, with PSP patients performing worse than PD patients and healthy controls. ere were no significant differences between PD patients and healthy controls. In addition, PSP patients performed worse than healthy controls on the  Reading the Mind in the Eyes Test and Cube Analysis, but with no significant differences between PSP and PD patients, or between PD patients and healthy controls. When scores on each of the neuropsychological assessments were compared with published normative data, there was a significant group difference in incidence of impairment in one domain only: fluency (χ 2 (1) � 7.61; p < 0.001). Nine of the 11 PSP patients (81.8%) were classified as impaired on at least one measure of verbal fluency, in comparison with only 33.3% (n � 10) of the 30 PD patients.
ROC curve analysis revealed that the total score of the ECAS is able to discriminate between PD and PD-MCI with high sensitivity (88.2%) and 100% specificity, when using a threshold score of 112.50/136. e AUC is 0.93 (SE � 0.06; p < 0.001). Confidence intervals are 0.81 (lower bound) and 1.00 (upper bound). Indeed, all PD patients who performed in the impaired range on the ECAS fulfilled the diagnostic criteria for PD-MCI.

Diagnostic
Accuracy of the ECAS for PSP. ROC curve analysis also revealed that the ECAS is highly specific (86.8%) and sensitive (91.0%) when discriminating PSP patients from healthy controls using a threshold score of 113.50/136. e AUC is 0.91 (SE � 0.67; p < 0.001). Confidence intervals are 0.79 (lower bound) and 1.00 (upper bound). All PSP patients who performed in the impaired range on the ECAS demonstrated impairment upon full neuropsychological testing, including impairment on at least one measure of verbal fluency.

Diagnostic Accuracy of the ECAS for Distinguishing between PD and PSP.
e second aim of the study was to determine whether the ECAS is able to distinguish between PD and PSP, in order to support the early and accurate diagnosis of PSP. ROC curve analysis showed that the measure is able to discriminate between PD and PSP (when comparing all patients, irrespective of cognitive performance), with high specificity (76.7%) and sensitivity (72.7%), using a threshold score of 103.50/136 (Figure 1). e AUC is 0.80 (SE � 0.09; p < 0.01). Confidence intervals are 0.62 (lower bound) and 0.98 (upper bound). is generated three false negatives and seven false positives. e false positives were all patients who fulfilled the criteria for PD-MCI.
Within the ECAS, fluency was the best predictor of PSP, with high specificity (80.0%) and sensitivity (82.0%) using a threshold score of 17/24. e AUC is 0.84 (SE � 0.08; p < 0.01). Confidence intervals are 0.69 (lower bound) and 1.00 (upper bound). is generated two false negatives and six false positives (five PD-MCI and one PD).
is is in contrast to when using the raw number of words generated in the two fluency tasks as a predictor, which has lower sensitivity (77.8%) and specificity (79.2%) when using a threshold score of 17.5.

Discussion
Our study has shown that the ECAS is sensitive to the cognitive impairment seen in PD. We found that 30% of PD patients were impaired on the ECAS, all of whom also demonstrated impairments upon full neuropsychological testing and fulfilled the criteria for PD-MCI. Indeed, ROC curve analyses revealed that the ECAS has excellent sensitivity and complete specificity for detection of PD-MCI. On the ECAS, PD patients demonstrated impairments in a number of domains but performed significantly worse than healthy controls on one domain only: executive function. PD-MCI patients also demonstrated deficits in language and visuospatial functioning. ese findings confirm the greater involvement of posterior functions with more advanced Parkinson's disease [30] but also suggest that the pattern of impairment can be fairly heterogeneous, even involving language. is is in accordance with the findings of the MDS Task Force [49], who also report impairments in a range of cognitive domains, including language.
Our data also show that the ECAS is sensitive to the cognitive impairment in PSP. We found that 81.8% of PSP patients were impaired on both the ECAS and full neuropsychological testing, including at least one measure of Parkinson's Disease 5 fluency. Again, ROC curve analyses confirmed that the ECAS total score gave excellent sensitivity and specificity for detection of PSP when compared with healthy controls. On the ECAS, PSP patients demonstrated the expected impairment in fluency, but also executive function, memory, and language. On extensive neuropsychological testing, PSP patients demonstrated impairments in fluency as well as executive function and visuospatial. e prominence of fluency and executive impairment on both the ECAS and full neuropsychological testing is in accordance with previous descriptions [13,15,16,28], confirming that the ECAS is sensitive to the typical profile of cognitive impairment in PSP. Importantly, we also found that the ECAS was able to distinguish between PD and PSP. ROC curve analysis revealed that the ECAS total score was sensitive and specific to PSP, with verbal fluency being the best discriminator. e ECAS was able to identify all PSP cases demonstrating cognitive impairment upon full testing. e few false positives mostly reflected PD patients with advanced cognitive impairment.
e strikingly reduced verbal fluency found on the ECAS and full testing confirms this as the cognitive hallmark of PSP. Importantly, the ECAS revealed this marked deficit even after accounting for the slowed motor speed.
is contrasts with impairments in other cognitive domains, such as memory, which can improve by up to 50% given sufficient extra time [58,59].
is impoverished verbal fluency, alongside the frequent family reports of reduced spontaneous speech and conversation initiation, likely reflects a cognitive adynamia beyond that of simple bradyphrenia but rather a more significant impairment in the generation of a "fluent sequence of novel thought" [19,60]. is may reflect a deficit in novel thought generation and/or its appropriate sequencing [61]. Indeed, it has been argued that the akinesia in motor abilities, reduction of verbal fluency in cognition, and apathy in behaviour are all different manifestations of the same underlying disorder [62].
PSP patients also demonstrated impairments in other domains, which supports the findings of previous studies. In accordance with previous reports of poor delayed recall [17], our PSP patients displayed impaired verbal recall on the ECAS, with three of the seven PSP patients impaired on both the ECAS and full testing. Our patients also demonstrated language impairment, reflecting spelling difficulties, in accordance with previous studies [20,63,64]. Spelling was more impaired on the ECAS, perhaps because its spelling test comprises nouns, verbs, and compounds of low-tomedium frequency, with a longer mean length. In contrast, the spelling test used upon full testing contained mostly nouns of high-to-low frequency, with a shorter mean length. Our patients also demonstrated impaired visuospatial function, which supports previous findings [16,18]. Nearly a third of PSP patients were impaired on the ECAS, with all of these also impaired upon full testing. e PD and PSP patients both performed poorly on measures of social cognition. ese findings echo previous reports of impaired performance on tests of theory of mind and social norms [22,44,[65][66][67].

Conclusions
e ECAS captures the core cognitive deficit of reduced verbal fluency, as well as the wider cognitive profile of PSP.
is may allow longitudinal testing to track progression as verbal fluency reaches floor. It was possible to use the ECAS with all the patients who took part in this study, despite often severe motor symptoms, indicating that it would be well tolerated in those with advanced disease. is suggests that the ECAS is suited for bedside use for detecting cognitive impairment in Parkinsonian syndromes and for distinguishing different cognitive profiles within these, in order to support differential diagnosis. Full neuropsychological assessment can then be used to further elucidate the specific clinical profile of each patient. Future research should examine its sensitivity for detecting cognitive impairment in other progressive movement disorders.

Conflicts of Interest
e authors declare that there are no known conflicts of interest.