Influence of Different Cut-Off Values on the Diagnosis of Mild Cognitive Impairment in Parkinson's Disease

Comparable to Alzheimer's disease, mild cognitive impairment in Parkinson's disease (PD-MCI) is associated with an increased risk for dementia. However different definitions of PD-MCI may have varying predictive accuracy for dementia. In a cohort of 101 nondemented Parkinson patients who underwent neuropsychological testing, the frequency of PD-MCI subjects and PD-MCI subtypes (i.e., amnestic/nonamnestic) was determined by use of varying healthy population-based cut-off values. We also investigated the association between defined PD-MCI groups and ADL scales. Varying cut-off values for the definition of PD-MCI were found to affect frequency of PD-MCI subjects (9.9%–92.1%) and, maybe more important, lead to a “shift” of proportion of detected PD-MCI subtypes especially within the amnestic single-domain subtype. Models using a strict cut-off value were significantly associated with lower ADL scores. Thus, the use of defined cut-off values for the definition of PD-MCI is highly relevant for comparison purposes. Strict cut-off values may have a higher predictive value for dementia.


Introduction
Parkinson's disease (PD) is increasingly recognized as a multidimensional disorder compromising motor but also a wide range of nonmotor features, including cognitive functions [1][2][3]. There is evidence that already a slight deterioration of cognition may enhance the risk of conversion to dementia in PD [4,5]. However, not all PD patients with such a cognitive profile develop dementia (PDD), and early identification of these patients with particularly increased risk is still not possible with sufficient accuracy [5][6][7]. Therefore, a lot of effort has been put on the identification of a clinical risk profile and especially in the characterization of mild cognitive deficits in patients who are later on developed dementia [8].
The current classification of mild cognitive impairment (MCI) in PD (PD-MCI) refers to the classification of Petersen and coworkers [9]. Here, the cognitive profile of MCI is basically defined by (i) the number of domains affected (single-/multiple-domain MCI) and (ii) the involvement of memory function (amnestic/nonamnestic MCI). Thus, for PD-MCI diagnosis, one needs to consider two main aspects. First, assessments covering all relevant cognitive domains associated with PD must be included in a considerable test battery. Therefore, the Movement Disorder Society (MDS) Task Force recommended neuropsychological tasks for the assessment of major areas of subcorticofrontal and cortically mediated functions [10]. Second, a cutoff value to define cognitive impairment leading to the diagnosis of PD-MCI must be defined. In previous studies, dealing with this topic, different cut-off values (−1, −1.5, or −2 standard deviations (SD)) below the mean of a healthy control group have been used to classify a cognitive test performance as relevantly impaired [11][12][13]. In large cohorts, this means that deficits occur in less than 16% (−1 SD), 7% (−1.5 SD), or 2% (−2 SD) of the subjects in the healthy population. Presently, a single test performance of −1.5 SD below the population mean is increasingly accepted as the best cut-off value for PD-MCI [8,14]. So far, it is not known whether this cut-off value has a high predictive value for PDD. In research studies, standard scores could be determined to assess individual neuropsychological performance delineating specific cognitive domains [8]. However, this method is not easy to implement in clinical praxis due to the lack of appropriate reference groups which are necessary for the evaluation of cognitive performance over a repertory of neuropsychological tasks. For clinical purpose, one rather needs to define how many tests characterize one cognitive function and to what degree this function needs to be affected to make the diagnosis of PD-MCI. Therefore, to define a cut-off value for PD-MCI for clinical use, the level of impairment in both a single test as well as within a cognitive domain has to be considered.
The clinical profile of PD-MCI patients who are supposed to be at higher risk for PDD has been investigated in various studies, but these studies differ regarding the cut-off values used to define PD-MCI [12,[15][16][17]. Besides the fact that it is not known yet which PD-MCI subjects have the highest risk for conversion to PDD, it is not evaluated whether the choice of different cut-off values might have an impact on the interpretation of the PD-MCI phenotype.
The aim of this study was twofold: first, to compare the frequency of PD-MCI and subtypes of PD-MCI by use of varying cut-off values, and second, to analyse how this variation of cut-off values might affect the interpretation of the clinical profile investigated in the PD-MCI group.

Subjects.
A nondemented group of 107 patients with idiopathic PD according to the UKPD Brain Bank criteria [18] was recruited from the Outpatient Clinic of the Department of Neurodegenerative Disorders, University of Tuebingen. Only patients older than 50 years, with adequate or corrected hearing/visual abilities and German as mother tongue, were investigated. Exclusion criteria were other neurological diseases affecting the central nervous system, prior surgery for PD and a Mini-Mental State Examination (MMSE [19]) score < 26, to exclude patients with possible dementia [8]. In addition, exclusion of patients with higher MMSE scores but diagnosis of probable dementia based on level II criteria of the MDS Task Force [10] was made. Hence, six patients with a performance (i) below a standard (z) score of −1.5 in at least two of the following cognitive domains: attention, executive functions, praxis and perception, memory or fluency, and naming abilities, (ii) self-report of cognitive decline with insidious onset and slow progression, and (iii) self-reported significant impact on instrumental activities of daily living functions fulfilling the recommended MDS dementia criteria were further excluded from data analysis [14,20]. The data sets of 101 nondemented PD patients were analyzed. The study was approved by the Local Ethical Committee, and all participants gave written informed consent.

Neuropsychological Assessment.
Cognitive and motor assessments were carried out on medication. Neuropsychological testing was conducted within two weeks of motor assessment. A comprehensive test-battery was composed to assess the major areas of subcorticofrontal and cortically mediated functions known to be affected in PD [10]. Different domains were theoretically specified and, composition was evaluated by internal consistency analysis. Cronbachs alpha coefficients indicated moderate to high consistency structure of the scales (0.52 ≤ Cronbachs α-coefficent ≤ 0.86), except for the attention domain (−0.33). For the attention domain, the Go-Nogo condition as well as the Alertness test, which are commonly used paradigms to assess different aspects of attention, were applied [21]. Two memory domains (list learning and memory recall, logical memory) and four nonmemory domains were defined (see below and Table 1 for details).
The domain "Executive function" (5 test scores) was assessed as follows: planning ability was tested by the Tower of London (TL-D) test [22], the trail-making test B [23], and the figure test of the NAI-(Nuernberger Altersinventar-) quantified set-shifting and set-maintenance abilities [23]. Working memory performance was assessed by the digit span part (forward and backward) of the NAI [24]. Ideomotor apraxia (primarily characterized by spatial postural and movement errors) is reported even in nondemented PD patients [25]. This symptom is known to be also caused by frontal lobe dysfunction [26,27]. In corticobasal degeneration, rather frontal lobe damage than parietal or temporal lobe damage is proposed to account for this symptom [28]. In our cohort, performance in the ideomotor part of the Berlin-Apraxia test (BAXT) [29] was found to be predominately associated with tests measuring executive function.
Performance in the domain "attention" (2 test scores) was recorded by using the subtests "Alertness" and "Go-Nogo" of the test for attentional performance (TAP) [30]. The value of phasic alertness expressing the subject's ability to increase attention processes in expectation of important stimuli and the median reaction time for the Go-Nogo test was analyzed.
The domain "Praxis and visual function" was investigated by 3 test scores: two praxis subtests of the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) [23], that is, copying of line drawings and the delayed recall of these drawings, as well as the object decision part of the Visual Object and Space Perception Battery [31].
The trail-making test (TMT) part A [23], the Boston Naming Test, and the semantic verbal fluency part of the CERAD were used to cover the domain "psychomotor speed and naming ability." This test composition showed an acceptable internal consistency structure, and imaging studies support the assumption of a partly overlapping functional brain network activated by the tasks included [32,33].
The memory domain "list learning and memory recall" was evaluated using three test scores of the German version of the CERAD [23], word-list memory, word-list recall after delay, word-list recognition, and the amount of incorrect responses concerning the word list memory recall (word-list intrusion). The memory domain "Logical memory" (2 test scores) was assessed by the logical memory I and II of the Wechsler Memory Scale-Revised [34].
German norm data (percentile rank scores, PR) provided in the test manuals of the neuropsychological assessments, referring to an age-matched healthy population corrected for age or both age and education (CERAD, TAP, TMT, and TL-D), were used to compare test performance, except for the BAXT. For the BAXT tests, information of the PR was not available, but mean and SD of an age-matched healthy control group was available, so that a standard score could be calculated [29].

Neurological Scales.
Neurological assessment included the Hoehn and Yahr stage, the Unified Parkinson Disease Rating Scale motor part (UPDRS-III) [35], and patient's history of medication.

Nonmotor Symptoms and Activities of Daily Living
Function. The Neuropsychiatric Inventory (NPI), which evaluates different behavioural domains (e.g., delusions, hallucinations, depression, and apathy), was used for evaluation of psychiatric disturbances [36]. The NPI total score was applied to investigate the severity of abnormal behaviour. The Parkinson's Disease Questionnaire-PDQ-39 [37] and the Beck Depression Inventory (BDI) [38] served as selfrating scales measuring health-related quality of life and mood.
The Nuernberger-Alters-Alltagsaktivitäten-Skala (NAA), a patient 15-item self-questionnaire focussing on different aspects of the activities of daily living (ADL) functions that is management of the financial situation and social independence, was performed. In addition,its equivalent was also assessed from the caregivers (Nuernberger-Alters-Beobachtungsskala, NAB) [24].

Definition of PD-MCI.
Standard (z) scores of −1, −1.5, and −2 were defined as cut-off values for the definition of PD-MCI. These cut-off values indicate that less than 16% (z < −1), 7% (z < −1.5), or 2% (z < −2) of the healthy population score was below these criteria. We also differentiated whether a subject scored low in at least 1 ( oneT ) or at least 2 test scores ( twoT ) per cognitive domain. The six resulting diagnostic criteria (z < −1 oneT , z < −1.5 oneT , z < −2 oneT , −z < 1 twoT , z < −1.5 twoT , and z < −2 twoT ) were then applied to identify the frequency of PD-MCI subjects within the cohort. We specified cut-off values of a varying continuum, some of which are supposed to be more liberal (z < −1 oneT ), and some being assumed to be more strict (z < −2 twoT ) as diagnostic criteria for PD-MCI. Patients with PD-MCI were further classified into one of the following four subtypes: (1) [8]. Performance in the demographical (e.g., age) data are descriptively shown with median and range. Post hoc explorative analysis of PD-noMCI and PD-MCI groups referring to clinical data (e.g., motor performance) were computed with nonparametric statistics, for example, Mann-Whitney U test or χ 2 test (e.g., gender). Differences were assumed to be significant at P < .05 (two sided). Statistical analyses were done with SPSS 17.0 for Windows (SPSS Inc, Chicago, ILL, USA).

Frequency of PD-MCI
Nearly all patients (92.1%, 95% CI [84.5-96.2]) scored below the most liberal cut-off value (z < −1 oneT , see Table 2 for details). Ten patients (9.9%, 95% CI [5.1-17.9]) scored below the strictest cut-off value (z < −2 twoT ). Frequency of PD-MCI was highly influenced by the selection of the classification criteria (Figure 1). In summary, the association between the number of patients defined as PD-MCI was as follows: the stricter a cut-off was defined (e.g., z < −2 twoT ), the less patients were categorized as PD-MCI. However, there was an overlap between the 95% CI of the defined PD-MCI frequency by application of either of the following cut-offs: "z < −2 oneT " (PD-MCI: 33

Post Hoc Comparison of PD-noMCI with PD-MCI
Results of the explorative post hoc mean group analysis must be interpreted with caution due to variations of sample size and number of comparisons which may lead to compromising power.
The interpretation of the following demographic/clinical variables was independent of the cut-off values applied: male gender, age at evaluation, age at onset, UPDRS-III motor score, and the NPI total score (P > .05).

Discussion
In our nondemented PD cohort, both frequency and clinical profile of PD-MCI patients were relevantly dependent on the cut-off value used for the definition of cognitive impairment. However, disparities in the evaluated number of PD-MCI were not unexpected as they reflect divergent levels of severity of neuropsychological test impairment. It is interesting that, by use of the most liberal cut-off value, nearly all (92%) of our nondemented PD patients were defined as cognitively impaired, compared to less than 16% of subjects in the healthy population (by definition). This result confirms previous findings demonstrating that slight cognitive impairment is frequent in PD, affecting also patients at very early disease stages [13]. Severe cognitive deterioration (reflected by z-scores < −2 in at least two test scores per domain) was found in less than 10% of our patients. In newly diagnosed PD patients, Williams-Gray and colleagues [11] reported a prevalence rate of 62% of PD-MCI after defining a value below 1 SD of the normative means in age and IQ-matched samples as "cognitively impaired." In contrast, Muslimovic and colleagues [13] used a value of 2 SD below the normative mean of healthy controls and found only 23.5% of PD patients to score below these cutoff values in standardized cognitive tasks. Although other variables such as age or education may have contributed to the difference between these two studies, our data make it intriguing that in particular the definition of the threshold for MCI explains this large divergence in prevalence of PD-MCI patients. This effect was also seen in our cohort by using either a more liberal or a more stringent criterion to define PD-MCI. In addition, both authors used different numbers of neuropsychological test scores for the classification of PD-MCI, which has been shown to also affect prevalence of PD-MCI as well as sample size and inclusion criteria [20]. Recently, the authors of a multicentre study including eight centres in Europe [8] evaluating the prevalence of PD-MCI concluded that MCI is a common syndrome affecting about 25% of PD patients. In this study, a standard score for each domain was calculated. Regarding our cohort, a performance below a z-score of −1 in at least two test scores per domain seems to be closest to the population-based prevalence reported in this study. Results from our study lead to the assumption that the application of different diagnostic criteria affects the sensitivity of the detection of PD-MCI. Comparing our data and data from other studies, one may conclude that a unique PD-MCI profile does not exist. Rather, studies using different criteria for the classification of PD-MCI seem to characterize subgroups in varying disease stages of cognitive deterioration. Therefore, varying studies may differ concerning the clinical profile of PD-MCI patients identified [4,12]. Different cut-off values may primarily represent a continuous deterioration of cognitive functions in our study population to which both severity of errors and/or the number of cognitive scores affected may contribute. Therefore, it is intriguing to hypothesize that deterioration within a cognitive domain is paralleled by a spreading of cognitive dysfunction throughout other cognitive domains. This hypothesis may offer an alternative model for the classification of mild to moderate cognitive impairment in PD as cognitive dysfunction is known to be heterogeneous in these patients [40]. According to our findings, we suggest the use of neuropsychological test batteries for future clinical or research application to (i) define a unitary cut-off value and (ii) standardize how many test scores within a cognitive domain need to be below this defined cut-off value for the diagnosis of PD-MCI.
In our cohort, varying cut-off values for the definition of PD-MCI not only affect frequency of PD-MCI subjects but, maybe more important, lead to a "shift" of the proportion of detected MCI subtypes especially within the amnestic single-domain subtype. Amnestic PD-MCI was more frequent when we used more stringent criteria (i.e., if more than one test score was required to be below the defined cut-off value for the diagnosis of PD-MCI). This means that patients with more severe cognitive dysfunction had an increased probability to suffer also from memory dysfunction. Amnestic PD-MCI has already been described to be a frequent syndrome in other studies [4,12]. Based on these finding, we conclude that for investigations dealing with PD-MCI subtypes, rigorous classification standards must be applied. The high frequency of amnestic MCI can not be explained by the number of test scores used per domain (it is suggestive that a higher number of scores assessed per domain increases the probability to detect a deficit), as we used a smaller number of test scores for the diagnosis of amnestic PD-MCI than for the diagnosis of nonamnestic PD-MCI (Table 1).
As limitation of the study presented here, it has to be kept in mind that interpretation of the post hoc comparison of PD-MCI and PD-noMCI patients must be taken with caution as discrepancies of sample size may compromise statistical power. Therefore, we may have not detected differences in other variables known as potential risk markers for dementia in PD such as demographical variables (e.g., age), motor performance and/or behavioural abnormalities between PD-MCI and PD-noMCI patients [1]. However, especially in our PD-MCI groups with smallest sample size, we found the strongest effect concerning to lower ADL functions.
In summary, our data supports the suggestions that a more liberal diagnostic criterion might be helpful for investigating even subtle cognitive impairments or minor changes in the course of PD. Application of a stricter cut-off value might increase specificity and/or the positive predictive value to detect patients at high risk for dementia. This hypothesis, however, needs to be verified in future longitudinal studies. A first corroboration of this hypothesis may be derived from the fact that a relevant association of both reduction of ADL function and lowered self-esteemed health-related quality of life-both parameters are relevantly associated with PDD [41] -was only observable in the PD-MCI group that was classified by the strictest criterion. As, in PD, cognitive dysfunction is most probably progressive [42]; we speculate that patients meeting the strictest criteria for MCI will show a higher conversion rate to dementia within a shorter time period. This hypothesis will be tested in an ongoing longitudinal study.