International Journal of Palliative Care
Volume 2015 (2015), Article ID 623890, 10 pages
http://dx.doi.org/10.1155/2015/623890

Research Article

Concordance between Experiences of Bereaved Relatives, Physicians, and Nurses with Hospital End-of-Life Care: Everyone Has Their “Own Truth”

F. E. Witkamp,1,2 L. van Zuylen,² Y. Vergouwe,¹ C. C. D. van der Rijt,² and A. van der Heide¹

¹Department of Public Health, Erasmus MC, University Medical Centre Rotterdam, 3000 CA Rotterdam, Netherlands
²Department of Medical Oncology, Erasmus MC, Cancer Institute, 3000 CA Rotterdam, Netherlands

Received 13 January 2015; Revised 24 March 2015; Accepted 25 March 2015

Academic Editor: Franco Toscani

Copyright © 2015 F. E. Witkamp et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

When patients die relatives and healthcare professionals may appreciate the quality of the dying phase differently, but comparisons are rare. In a cross-sectional study (June 2009–July 2012) the experiences of bereaved relatives, physicians, and nurses concerning the quality of dying in a large Dutch university hospital were compared, and the relation to communication was explored. Measurements were concordance on the quality of dying (QOD) (0–10 scale), awareness of impending death, and end-of-life communication. Results. Data on all three perspectives were available for 200 patients. Concordance in general was poor. Relatives’ scores for QOD (median 7; IQR 5–8) were lower than physicians and nurses’ (both median 7; IQR 6–8) (). 48% of the relatives, 77% of the physicians, and 73% of the nurses had been aware of impending death. Physicians more often reported to have informed patients and relatives of end-of-life issues than relatives reported. When both physicians and relatives reported about such discussion, relatives’ awareness of impending death and presence at the patient’s deathbed were more likely. Conclusion. Relatives, physicians, and nurses seem to have their “own truth” about the dying phase. Professionals should put more emphasis on the collaboration with relatives and on verification of relative’s understanding.

1. Introduction

As death comes to us all and most people die after a period of declining health and a significant symptom burden, knowledge on how end-of-life care can contribute to a good deathbed is highly needed. Research on patient reported outcomes and the effectiveness of interventions at the end of life is complex, for example, because of methodological and ethical considerations [1–3]. Furthermore, perspectives vary on what good quality of dying is and how this should be achieved, as was recently shown in the debate on the Liverpool Care Pathway for the Dying Patient [4–9]. A major limiting factor is that the main person involved, that is, the dying patient, cannot participate in the debate on outcome measures and in the evaluation of experiences in the dying phase [2]. Proxy assessments are needed, including those of relatives and healthcare professionals (HCPs) who may have various perspectives and values [3, 10–13].

Studies have shown that relatives and patients share important priorities, such as honest and clear communication, involvement in medical decisions, relief of symptom burden, and having the opportunity of completing life and saying goodbye [10, 14–18]; therefore relatives might be able to represent some of the patient’s interests. Most studies on proxy evaluation of end-of-life care compare symptom scores of patients and relatives or HCPs. These studies show a tendency of relatives to overestimate the severity of symptoms in comparison to patients’ self-reports, whereas nurses and physicians tend to underestimate them [10, 12, 16, 19–21]. Only a few small studies have compared the experiences of the dying phase of relatives and HCPs, showing low to moderate concordance between these groups [13, 22]. In the national audit of end-of-life care in Irish hospitals physicians evaluated care more positively than relatives, whereas nurses reported intermediate opinions [23]. More insight is needed into the appreciation of relatives of the care and communication in the last days of life and how this is related to the perspectives of HCPs.

The purpose of this study was to better understand evaluation of the dying experience from different perspectives and how end-of-life communication may affect the quality of dying. Therefore we compared bereaved relatives and physicians and nurses’ assessments of dying patients’ quality of life and quality of dying, their awareness of impending death, and their communication; furthermore, we analyzed how communication was related to relatives’ appreciation of aspects of the quality of dying and the quality of care, such as preparation for death and satisfaction with care.

2. Materials and Methods

The study was embedded in a larger study on the quality of palliative and terminal care in the hospital (the PalTeC-H study). We included cases of patients who died in 18 nonintensive care wards of a 1300-bed university hospital in Netherlands (i.e., 951 consecutive deaths), between June 2009 and July 2012, after a hospitalization of at least 6 hours. The protocol of this study is described elsewhere [24]. Physicians and nurses who had closely been involved in daily care for a dying patient were asked by the team coordinator of the ward to complete a questionnaire within at most two weeks after the patient’s death. Ten to thirteen weeks after the death of a patient a relative was invited by post by the primary investigator (F. E. Witkamp) to complete a questionnaire. When needed, a written reminder was sent after four weeks.

2.1. Ethical Considerations

Approval for this study was given by the Medical Ethical Research Committee of the Erasmus MC. According to Dutch legislation, written informed consent of the patients or respondents was not required because data were gathered after patients’ death and the study involved minimal risk or burden to the respondents.

2.2. Questionnaires

Previous research provided us with a multidimensional framework of quality of life, quality of dying, and quality of care, which are overlapping but distinguishable constructs [25, 26]. Quality of life (at the end of life) involves physical, psychological, social, and spiritual experiences, and quality of dying additionally includes life closure and death preparation, the character of health care, and the circumstances of death [25]. Quality of care at the end of life refers to the extent to which these domains are adequately affected by healthcare [27].

Based on this framework three complementary self-administered questionnaires for relatives, physicians, and nurses were developed by a group of experts in palliative care and research, because previously described instruments met neither our requirements of complementarity of the three perspectives nor those of completeness regarding the last days of life and the dying phase [28–31]. The questionnaire for relatives was critically evaluated by a representative of the hospital patients’ council. All questionnaires were tested on relevance and face validity among members of all targeted groups, including persons who had recently lost a relative, hospital physicians, and nurses. Subsequently, all questionnaires were piloted in the first 30 cases [24]. Some topics were assessed from all perspectives (Table 1), including two summarizing items, in which the respondents were asked to rate the patient’s quality of life during the final three days (further referred to as QOL3) and his/her quality of dying (QOD), both on a 0–10 scale with zero being “very poor” and ten “almost perfect.” Further, all were asked whether HCPs had discussed patient’s impending death with relatives and patients (no or yes) and whether they had been aware of the patient’s impending death (no, more or less, yes). “Being aware” was defined as being aware that death was likely to occur within hours or days, and “being more or less aware” as knowing that the patient would die, but not expecting that death would occur so soon. We also asked when the respondents had become aware of the patient’s impending death (<24 hrs, between 24 and 48, or 48 and 72 hrs and >72 hrs in advance). Finally, we assessed aspects of the quality of dying and the quality of care according to relatives, including opportunities to prepare for death, satisfaction with care, saying goodbye (no, more or less, yes), and presence at the moment of death (yes or no). The validity and reliability of the questionnaire for relatives were previously analyzed with simple and multivariate regression analyses and Principal Component Analyses [32]. In addition, in all groups we analyzed the association between the QOD score and answers to verbal terms, reflecting the way of dying as experienced by the respondents, such as quiet, peaceful, traumatic, sad, shocking, and moving. The QOD score was positive related to “positive” terms, such as quiet, beautiful, with dignity, and peaceful and negative related to “negative” terms, such as shocking, traumatic, and troubled ( values <0.001).

Table 1: Congruent items per response group.

Relatives
Physicians
Nurses

How would you rate the quality of life in the last 3 days of life of the patient? (0 = very poor/10 = almost perfect)
How would you rate the quality of life in the last 3 days of life of the patient? (0 = very poor/10 = almost perfect)
How would you rate the quality of life in the last 3 days of life of the patient? (0 = very poor/10 = almost perfect)

How would you rate the quality of dying of the patient? (0 = very poor/10 = almost perfect)
How would you rate the quality of dying of the patient? (0 = very poor/10 = almost perfect)
How would you rate the quality of dying of the patient? (0 = very poor/10 = almost perfect)

Did you realize that your beloved would die within a few days? 
If so, how long in advance?  
More than 3 days in advance 
72–48 hours in advance 
48–24 hours in advance 
Less than 24 hours in advance
Did you realize that the patient would die within hours to days?
If so, how long in advance?  
More than 3 days in advance 
72–48 hours in advance 
48–24 hours in advance 
12–24 hours in advance 
6–12 hours in advance 
Less than 6 hours in advance
Did you realize that the patient would die within hours to days?
If so, how long in advance?  
More than 3 days in advance 
72–48 hours in advance 
48–24 hours in advance 
12–24 hours in advance 
6–12 hours in advance 
Less than 6 hours in advance

—
Did you or a colleague discuss impending death with the patient?
Did you or a colleague discuss death or dying with the patient?

Did a healthcare professional inform you about the impending death of your beloved?
Did you or a colleague discuss impending death with the relative(s)?
—

2.3. Data Analysis

We analyzed data on all patients for whom respondents from all three groups (relatives, physicians, and nurses) had completed a questionnaire. Linear regression analyses were performed to study the effect of patient characteristics (i.e., age, gender, diagnosis cancer or no cancer, and duration of the final hospitalization) on QOL3 or QOD scores. Differences of QOL3 and QOD scores between the groups were estimated using multilevel regression analysis. The multilevel model included random effects for the intercepts of the regression model to allow for the multiple measurements per patient and fixed effects for the three perspectives. Correlations between participants’ awareness of the patient’s impending death and the moment of their awareness were tested using cross tables with Chi² tests and Spearman correlation coefficients, respectively. Agreement between physicians and relatives on whether physicians had informed relatives of the patient’s impending death was tested using Cohen’s Kappa (κ): values were interpreted as <0.20 = poor; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = good; and 0.81–1.0 = very good concordance [33]. Furthermore we analyzed the relation of the communication of HCPs about a patient’s impending death with patients and relatives’ awareness and death preparation, relatives’ presence at the moment of death, and their satisfaction with care, using Chi² tests. Level of significance was set at for all analyses. All data were analyzed with statistical packages SPSS 21 and R version 2.13.

3. Results

3.1. General Characteristics

During the study period 951 patients died. All invited participants responded independently; the response rates per group were 52% for relatives, 45% for physicians, and 54% for nurses. For 200 patients all three questionnaires were completed. At the time of death, the mean age of these patients was 69 years and the mean duration of the final hospitalization was 14 days; 54% died of cancer; at the time of admission to the hospital the treatment goal was symptom control for 61% and terminal care for 5% (Table 2). Relatives were mostly the spouse (50%) or a child (34%) of the patient, and 88% were involved in patient care during final days; their mean age was 57 years. Participating physicians were mostly interns, and 37% had cared for more than five dying patients during the previous 12 months. Of the participating nurses 28% had cared for more than five dying patients during the previous 12 months. In 61% of the cases, nurses had cared for the patient during at least 2 shifts. Characteristics and outcomes were similar to those of the total study populations [32, 34].

Table 2: General characteristics of patients, relatives, physicians, and nurses.

(%)
Mean (SD)

Patients

Sex

    Male
117 (59)

    Female
83 (41)

Age (years)

69.0 (12.6)

Duration of final hospitalization (days)

13.7 (15.8)

Marital status

    Married/living in partnership
129 (65)

    Widowed/divorced/alone
71 (35)

Diagnosis of cancer
54

Ward

    Acute palliative care (cancer center)
38 (19)

    Neurology
31 (16)

    Ear, nose, & throat diseases
20 (10)

    Lung diseases
21 (11)

    Medical oncology/geriatrics
18 (9)

    Other
72 (36)

Treatment goal(s) at admissiona

    Symptom control
121 (61)

    Recovery or life prolongation
63 (32)

    Diagnostics
7 (4)

    Terminal care
10 (5)

    Other
17 (9)

Relatives

Sex

    Male
74 (37)

    Female
124 (63)

Age (years)

57 (14.1)

Relation to patient

    Spouse
99 (50)

    Child
68 (34)

    Other
28 (14)

Involved in care final days prior to death

    Yes
176 (88)

    No
24 (12)

Time between patient passing away and completion of questionnaire (weeks)

15.5 (4.0)

Physicians

Sex

    Male
86 (37)

    Female
112 (62)

Age (years)

30 (6.0)

Function

    Attending physician on the ward
136 (68)

    Attending physician (out of office hours)
38 (19)

    Other
23 (12)

Number of dying patients cared for during previous 12 months

    ≤2
70 (35)

    3–5
54 (27)

    >5
74 (37)

Nurses

Sex

    Male
18 (9)

    Female
179 (90)

Age (yrs)

35 (12.3)

Number of dying patients cared for during previous 12 months

    ≤2
77 (39)

    3–5
64 (32)

    >5
55 (28)

More than one goal possible, percentage may be added up to >100%.

3.2. Quality of Life and Quality of Dying

Both the QOL3 and the QOD were scored with a range from 0 to 10, in all three groups of participants. The median QOL3 score of the relatives was 3 (Interquartile Range (IQR) 1–6), whereas physicians and nurses scored QOL3 significantly higher (both median 5; IQR 3–6) (;  ) (Table 3). The median QOD score was 7 in all three groups, with a IQR of 5–8 in relatives and 6–8 in HCPs, the latter being significantly higher than relatives’ scores (; ). QOL3 and QOD scores were not related to patients’ diagnosis, age, or gender. Relatives and physicians’ QOL3 scores were slightly lower if the duration of the hospitalization had been longer (, ;  ; , ; , resp.).

Table 3: Quality of life during final 3 days (QOL3) and quality of dying (QOD) ().

Relatives
Physicians
Nurses
Analysis of varianceb

Median (IQR)
Median (IQR)
Median (IQR)
value
value

Quality of life during last 3 days (QOL3)a
3 (1–6)
5 (3–6)
5 (3–6)
12.755
<0.001

Quality of dying (QOD)a
7 (5–8)
7 (6–8)
7 (6–8)
6.47
0.002

IQR = Interquartile Range.

aScale 0–10.

bMultilevel analyses to compare outcomes in 3 groups.

3.3. Awareness of Impending Death

Relatives reported to have been aware of the patient’s impending death in 48% of all cases. Physicians and nurses had been aware in 77% and 73%, respectively (Table 4). There was no correlation in the awareness between relatives and physicians (; ) and poor correlation between relatives and nurses (; ) and nurses and physicians (; ). Of all cases, 29% of relatives had not been aware of the patient’s impending death and in another 29% they became aware during the final 24 hours. In 38% of cases physicians and nurses became aware of the patient’s impending death in the final 24 hours and correlation between physicians and nurses moments of awareness was moderate (; ). Correlation between relatives and HCPs’ moments of awareness was poor and in the opposite direction (; and ; , resp.).

Table 4: Awareness of impending death ().

Physicians
(%)
Nurses
(%)
Relatives
(%)

Awareness of impending death

    Yes
154 (77)
145 (73)
95 (48)

    More or less
27 (14)
22 (11)
40 (20)

    No
16 (8)
30 (15)
57 (29)

    Missing values
3 (1)
3 (1)
8 (3)

Correlation Rho ()

    P-Na
0.20 (0.004)

    N-R
0.26 (<0.001)

    P-R
0.10 (0.17)

Moment of awareness of impending death

    <24 hrs
77 (38)
82 (41)
58 (29)

    24–72 hrs
65 (32)
59 (30)
46 (23)

    >72 hrs
42 (21)
38 (19)
31 (16)

    Missing values
16 (9)
21 (10)
64 (32)

Correlation Rho ()

    P-Na
0.41 (<0.001)

    N-R
−0.26 (0.001)

    P-R
−0.25 (0.003)

Rho = Spearman’s correlation coefficient.

aP-N = physician and nurse; N-R = nurse and relative; P-R = physician and relative.

3.4. Communication

Physicians more often reported to have informed patients and relatives of end-of-life issues than relatives and nurses reported (Table 5). In 57% physicians reported that they (32%) or a colleague (25%) had informed patients about their impending death. In 27%, this had been impossible, due to unconsciousness of the patient. Physicians reported to have informed 90% of relatives of the patient’s impending death, whereas 64% of the relatives reported to have been informed (, ).

Table 5: Communication about end of life, end-of-life decisions, or impending death ().

Perspectives:
With patient
(%)
With relative
(%)

Physician reported
To have discussed the patient’s impending death
114 (57)
180 (90)

Relative reported
That the physician had discussed the patient’s impending death
n.a.
127 (64)

Nurse reported
That the physician had discussed the patient’s impending death
83 (42)
131 (66)

To have discussed the patient’s impending death
72 (36)
n.a.

n.a. = not assessed.

Compared to cases in which only the physician reported that the relative was informed about impending death, relatives who had confirmed this communication were more likely to have been aware of the patient’s impending death (63% versus 27%), to have said goodbye (64% versus 36%), to be satisfied about symptom management (82% versus 69%) and about emotional and social support (76% versus 53%), and to have been present at the moment of death (92% versus 75%) (Table 6). Discussion between relatives and physicians about impending death did not affect the patient’s preparedness. When the physician reported to have discussed impending death with the patient relatives more often reported that the patient had been aware of impending death (41% versus 14%) and had said goodbye (58% versus 27%) and that relatives themselves had said goodbye (59% versus 40%). When nurses had discussed the end of life with the patient this was also associated with increased percentages of patients’ awareness, patients and relatives’ goodbyes, and relatives’ presence at the moment of death.

Table 6: Communication about impending death and its relation to patients and relatives’ awareness, their death preparation, and satisfaction with care ().

Patient (according to relative)
Relative

Was aware of impending death
Was in peace
Said goodbye
Was aware of impending death
Was satisfied about symptom management during final 24 h.
Was satisfied about social and emotional support final 24 h.
Said goodbye
Was present at deathbed

(%)a
(%)a
(%)a
(%)a
(%)a
(%)a
(%)a
(%)a

Physician reported to have informed relative about the patient’s impending death () and relative reported to have been informed

    Yes
119
43 (36)
50 (42)
62 (52)
75 (63)
98 (82)
90 (76)
76 (64)
110 (92)

    No
55
16 (29)
23 (42)
22 (40)
15 (27)
38 (69)
29 (53)
20 (36)
41 (75)

Physician reported to have discussed impending death with the patient

    Yes
114
47 (41)
52 (46)
66 (58)
61 (54)
85 (71)
71 (62)
67 (59)
98 (86)

    No
86
12 (14)
25 (29)
23 (27)
35 (41)
62 (72)
57 (66)
34 (40)
66 (77)

Nurse reported to have discussed impending death with the patient

    Yes
72
35 (49)
39 (54)
50 (69)
39 (54)
56 (77)
52 (72)
48 (67)
68 (94)

    No
128
26 (20)
39 (30)
40 (31)
58 (45)
93 (73)
78 (61)
55 (43)
99 (77)

Row percentages.

Italic = in table (Pearson’s Chi square).

4. Discussion

In the acute hospital setting, bereaved relatives and HCPs rated the QOL3 as poor and the QOD as moderate; HCPs scored QOL3 and QOD significantly higher than relatives. The concordance of their evaluation of QOL3 and QOD, their awareness of the impending death of the patient, and discussions about impending death was poor. However, communication between HCPs, patients, and relatives about impending death was associated with better awareness of and preparedness for the end of life and with more satisfaction with care.

A poor concordance of perspectives on end-of-life care was also found in a few other studies [13, 22, 23]. Different stakeholders seem to experience their “own truth" and probably there is no “one truth.” This finding has important implications regarding the development and use of quality indicators and outcome measures in end-of-life care. The use of such indicators in palliative care is in its infancy and the effects of their implementation have not yet been assessed [35, 36]. Our study shows that outcomes are likely to be affected by the type of assessor.

Relatives had been fully aware of the impending death of the patient in 48% of the cases, whereas physicians and nurses had been aware in 77% and 73% of all cases. However, in four out of ten cases HCPs only became aware of the nearness of death during the final 24 hours. The awareness and moment of awareness of relatives and HCPs were not or even negatively correlated. Awareness of impending death is often seen as an important condition for a good death [25, 37, 38]; however it is known to frequently occur late in the hospital [39–41]. Hospital care is typically aimed at improving patients’ health and subsequently discharging them to go home. Only in 5% the treatment goal at admission was providing care in the dying phase. This practice affects the expectations of patients, relatives, and HCPs. Most participants in our study might have expected the patient to be discharged to go home after the hospitalization, which instead turned out to end with the patient’s death. Recognition of the irreversibility of a worsening health condition and, subsequently, refocusing care to enable the patient and his relatives to have a good death bed, is difficult [39, 42]. Our finding that relatives and HCPs evaluated the dying phase differently and recognized the imminence of death at various moments underlines the need for frequent and adequate communication between patients, physicians, nurses, and relatives, during the final days of life and also after death. Every party involved brings his/her own expertise and jointly they are probably best able to assess the patient’s needs and prospects: physicians have biomedical knowledge and skills, nurses have expertise in personal care and daily observations, and relatives probably best know how to interpret patients’ appearance and behavior.

One-third of the relatively young participating physicians had discussed imminent death with the patient themselves. The provision of clinical care in the learning environment of a teaching hospital is challenging; junior doctors need adequate supervision to learn more about end-of-life care, truth telling, and breaking bad news [43, 44].

This study also showed that being informed about the patient’s impending death did not obviously result in awareness of the nearness of death. Nevertheless, communication of HCPs with relatives and patients clearly affected the awareness of impending death and the preparation for the end of life, especially when the relative confirmed to have been informed by the physician, which was not always the case. Taking into account physicians’ late awareness and the number of cases in which patients could not be informed because they were unconscious, communication about impending death might often have taken place late in the process [45, 46]. Communication at the end of life is one of the most difficult tasks of doctors and concerns deliberations on if bad news should be broken, when, and how [39, 42, 45]. When being confronted with a fatal prognosis, relatives might experience tension between wanting to know what to expect and needing to remain hopeful. This tension might lead to ignoring prognostic information, relying on one’s own beliefs and focusing on positive details [47]. To support relatives in “preparing for the worst and hoping for the best,” sensitive and frequent communication is needed, about the patient’s prognosis and about the physician’s expectations of the effects of treatment [47, 48]. When end-of-life communication is regarded as a continuous process, disclosure of an impending death might be less difficult, for the physician and for the patient and relative, which in turn might yield higher levels of satisfaction with care [48, 49].

Nurses discussed the impending death with patients in only one-third of all cases, but when they did, this was associated with increased awareness of patients of their impending death, with the extent to which patients and relatives had been able to say goodbye and with relatives’ presence at the moment of death. Nurses are known to experience many difficulties in end-of-life communication, such as uncertainty about their role, and their knowledge, and skills. They also need to wait for the physician to disclose bad news and then often hesitate in initiating difficult conversations with patients and relatives [42, 50, 51]. Although nurses are generally dedicated to providing the best end-of-life care, they may in practice often continue to provide care as usual, until the physician explicitly and openly states that the patient is dying [52, 53].

4.1. Strengths and Weaknesses

To our knowledge this is the first study to examine QOL3 and QOD from the perspectives of relatives, physicians, and nurses, in a large number of cases, without restrictions regarding diagnoses or duration of illness. We comprehensively assessed experiences with end-of-life communication and their relation to death preparation and circumstances of death. The response rates were similar to response rates in comparable studies and groups of participants [54–56], and the characteristics and results in the 200 common cases were similar to those in the separate groups. Still, some selection bias cannot be ruled out, which might result in more positive experiences of physicians and relatives [57]. Our findings provide concrete opportunities to improve care, instead of satisfaction rates, which are often high in evaluation studies in healthcare [58].

Due to the cross-sectional design we cannot draw firm conclusions about causality. Although 84% of participating relatives were close relatives of the patient, and 88% were involved in the care of the patient during his/her final days, physicians might have informed other family members, who again might not have informed the participating relative. The moment after death at which relatives and HCPs filled in questionnaires differed. HCPs were asked to fill in a questionnaire as soon as possible after the patient’s death, to restrict recall bias. Relatives were asked later, to diminish disturbance of their bereavement, but their perspectives might have changed in this period, and recall bias cannot be ruled out. However, emotional events, such as the death of a beloved person, are known to be recalled better compared to less moving events. Finally, we used newly developed questionnaires and as a result our findings cannot easily be compared to those of other studies using other instruments.

5. Conclusion

Relatives, physicians, and nurses had different perspectives on the quality of the final days of patients’ lives, on their quality of dying, and on communication about impending death. Everyone seems to experience his/her “own truth.” To achieve improvements in care in the dying phase, more emphasis should be put on the collaboration between HCPs and relatives in the therapeutic alliance with patients. Furthermore, physicians should be more willing to timely discuss a poor prognosis and to check whether the patient and relative’s understanding align with physicians’.

Conflict of Interests

All authors declare that they have no competing interests; they all declare to have full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the analysis.

Authors’ Contribution

F. E. Witkamp made substantial contributions to conception and design, author of the paper. L. van Zuylen made substantial contributions to conception and design, critically revised the paper for important intellectual content, and approved the final version to be published. Y. Vergouwe made substantial contributions to the analysis of data, critically revised the paper for important intellectual content, and approved the final version to be published. C. C. D. van der Rijt made substantial contributions to conception and design, critically revised the paper for important intellectual content, and approved the final version to be published. A. van der Heide made substantial contributions to conception and design, critically revised the paper for important intellectual content, and approved the final version to be published. F. E. Witkamp, Y. Vergouwe, and A. van der Heide have been responsible for the conduction and the analysis of data.

Acknowledgments

This study was financially supported by a grant from the Erasmus MC Medical Research Committee and the Tom and Josephine Rijckes legacy foundation, and both did not have any involvement in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the paper. The authors thank the hospital patient council for their critical advice and support. They thank the Brocher Foundation in Geneva for their hospitality during the process of data analysis.

References

C. J. Evans, R. Harding, I. J. Higginson, and MORECare, “‘Best practice’ in developing and evaluating palliative and end-of-life care services: a meta-synthesis of research methods for the MORECare project,” Palliative Medicine, vol. 27, no. 10, pp. 885–898, 2013.

K. E. Steinhauser, “Measuring end-of-life care outcomes prospectively,” Journal of Palliative Medicine, vol. 8, supplement 1, pp. S30–S41, 2005.

J. M. Teno, “Measuring end-of-life care outcomes retrospectively,” Journal of Palliative Medicine, vol. 8, supplement 1, pp. S42–S49, 2005.

J. Ellershaw, S. Dewar, and D. Murphy, “Achieving a good death for all,” British Medical Journal, vol. 341, article c4861, 2010.

J. Neuberger, C. Guthrie, D. Aaronovitch et al., “More care, less pathway; a review of the liverpool care pathway,” 2013, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/212450/Liverpool_Care_Pathway.pdf

C. Regnard, “The demise of the Liverpool care pathway: should we ban the highway code because of bad drivers?” Age and Ageing, vol. 43, no. 2, Article ID aft195, pp. 171–173, 2014.

K. Chinthapalli, “The birth and death of the Liverpool care pathway,” British Medical Journal, vol. 347, article f4669, 2013.

S. Doughty, “Top doctor's chilling claim: the NHS kills off 130,000 elderly patients every year,” Daily Mail Online, 2012, http://www.dailymail.co.uk/news/article-2161869/Top-doctors-chilling-claim-The-NHS-kills-130-000-elderly-patients-year.html

H. Saul, “Liverpool care pathway: controversial ‘end-of-life’ regime for dying patients ‘to be replaced’,” in The Independent, 2013, http://www.independent.co.uk/life-style/health-and-families/health-news/liverpool-care-pathway-controversial-endoflife-regime-for-dying-patients-to-be-replaced-8707164.html.

D. K. Heyland, P. Dodek, G. Rocker et al., “What matters most in end-of-life care: perceptions of seriously ill patients and their family members,” Canadian Medical Association Journal, vol. 174, no. 5, pp. 627–633, 2006.

C. L. Nekolaichuk, T. O. Maguire, M. Suarez-Almazor, W. T. Rogers, and E. Bruera, “Assessing the reliability of patient, nurse, and family caregiver symptom ratings in hospitalized advanced cancer patients,” Journal of Clinical Oncology, vol. 17, no. 11, pp. 3621–3630, 1999.

C. J. McPherson and J. M. Addington-Hall, “Judging the quality of care at the end of life: can proxies provide reliable information?” Social Science and Medicine, vol. 56, no. 1, pp. 95–109, 2003.

C. R. Levy, E. W. Ely, K. Payne, R. A. Engelberg, D. L. Patrick, and J. R. Curtis, “Quality of dying and death in two medical ICUs: perceptions of family and clinicians,” Chest, vol. 127, no. 5, pp. 1775–1783, 2005.

J. M. Teno, B. R. Clarridge, V. Casey et al., “Family perspectives on end-of-life care at the last place of care,” The Journal of the American Medical Association, vol. 291, no. 1, pp. 88–93, 2004.

I. Higginson, P. Priest, and M. McCarthy, “Are bereaved family members a valid proxy for a patient's assessment of dying?” Social Science and Medicine, vol. 38, no. 4, pp. 553–557, 1994.

R. A. Engelberg, D. L. Patrick, and J. R. Curtis, “Correspondence between patients' preferences and surrogates' understandings for dying and death,” Journal of Pain and Symptom Management, vol. 30, no. 6, pp. 498–509, 2005.

K. E. Steinhauser, E. C. Clipp, M. McNeilly, N. A. Christakis, L. M. McIntyre, and J. A. Tulsky, “In search of a good death: observations of patients, families, and providers,” Annals of Internal Medicine, vol. 132, no. 10, pp. 825–832, 2000.

D. K. Heyland, D. J. Cook, G. M. Rocker et al., “Defining priorities for improving end-of-life care in Canada,” Canadian Medical Association Journal, vol. 182, no. 16, pp. E747–E752, 2010.

C. L. Nekolaichuk, E. Bruera, K. Spachynski, T. MacEachern, J. Hanson, and T. O. Maguire, “A comparison of patient and proxy symptom assessments in advanced cancer patients,” Palliative Medicine, vol. 13, no. 4, pp. 311–323, 1999.

M. Klinkenberg, J. H. Smit, D. J. H. Deeg, D. L. Willems, B. D. Onwuteaka-Philipsen, and G. van der Wal, “Proxy reporting in after-death interviews: The use of proxy respondents in retrospective assessment of chronic diseases and symptom burden in the terminal phase of life,” Palliative Medicine, vol. 17, no. 2, pp. 191–201, 2003.

T. H. M. To, W. Y. Ong, D. Rawlings, A. Greene, and D. C. Currow, “The disparity between patient and nurse symptom rating in a Hospice population,” Journal of Palliative Medicine, vol. 15, no. 5, pp. 542–547, 2012.

S.-Y. Cheng, S. Dy, S.-B. Huang, C.-Y. Chen, and T.-Y. Chiu, “Comparison of proxy ratings of main family caregivers and physicians on the quality of dying of terminally III cancer patients,” Japanese Journal of Clinical Oncology, vol. 43, no. 8, pp. 795–804, 2013.

K. McKeown, T. Haase, J. Pratschke, S. D. H. Twomey, and F. Engling, “Dying in hospital in Ireland: an assessment of the quality of care in the last week of life, report 5, final synthesis report,” Hospice Friendly Hospitals, 2010, http://www.hospicefriendlyhospitals.net

F. E. Witkamp, L. V. Zuylen, P. J. V. D. Maas, H. V. Dijk, C. C. V. D. Rijt, and A. V. D. Heide, “Improving the quality of palliative and terminal care in the hospital by a network of palliative care nurse champions: the study protocol of the PalTeC-H project,” BMC Health Services Research, vol. 13, no. 1, article 115, 2013.

S. Hales, C. Zimmermann, and G. Rodin, “The quality of dying and death,” Archives of Internal Medicine, vol. 168, no. 9, pp. 912–918, 2008.

K. E. Steinhauser, E. C. Clipp, and J. A. Tulsky, “Evolution in measuring the quality of dying,” Journal of Palliative Medicine, vol. 5, no. 3, pp. 407–414, 2002.

A. L. Stewart, J. Teno, D. L. Patrick, and J. Lynn, “The concept of quality of life of dying persons in the context of health care,” Journal of Pain and Symptom Management, vol. 17, no. 2, pp. 93–108, 1999.

J. Addington-Hall, L. Walker, C. Jones, S. Karlsen, and M. McCarthy, “A randomised controlled trial of postal versus interviewer administration of a questionnaire measuring satisfaction with, and use of, services received in the year before death,” Journal of Epidemiology and Community Health, vol. 52, no. 12, pp. 802–807, 1998.

J. R. Curtis, D. L. Patrick, R. A. Engelberg, K. Norris, C. Asp, and I. Byock, “A measure of the quality of dying and death: initial validation using after-death interviews with family members,” Journal of Pain and Symptom Management, vol. 24, no. 1, pp. 17–31, 2002.

C. R. Mayland, E. M. I. Williams, and J. E. Ellershaw, “How well do current instruments using bereaved relatives' views evaluate care for dying patients?” Palliative Medicine, vol. 22, no. 2, pp. 133–144, 2008.

G. Albers, M. A. Echteld, H. C. W. de vet, B. D. Onwuteaka-Philipsen, M. H. M. van der Linden, and L. Deliens, “Content and spiritual items of quality-of-life instruments appropriate for use in palliative care: a review,” Journal of Pain and Symptom Management, vol. 40, no. 2, pp. 290–300, 2010.

F. E. Witkamp, L. van Zuylen, G. Borsboom, C. C. van der Rijt, and A. van der Heide, “Dying in the hospital: what happens and what matters, according to bereaved relatives,” Journal of Pain and Symptom Management, vol. 49, no. 2, pp. 203–213, 2015.

D. G. Altman, Practical Statistics for Medical Research, CRC Press LLC, Boca Raton, Fla, USA, 2000.

D. Houttekier, F. E. Witkamp, L. van Zuylen, C. C. van der Rijt, and A. van der Heide, “Is physician awareness of impending death in hospital related to better communication and medical care?” Journal of Palliative Medicine, vol. 17, no. 11, pp. 1238–1243, 2014.

K. Leemans, J. Cohen, A. L. Francke et al., “Towards a standardized method of developing quality indicators for palliative care: protocol of the quality indicators for palliative care (Q-PAC) study,” BMC Palliative Care, vol. 12, no. 1, article 6, 2013.

K. Woitha, K. van Beek, N. Ahmed et al., “Validation of quality indicators for the organization of palliative care: a modified RAND delphi study in seven European countries (the Europall project),” Palliative Medicine, vol. 28, no. 2, pp. 121–129, 2014.

R. Smith, “A good death. An important aim for health services and for us all,” British Medical Journal, vol. 320, no. 7228, pp. 129–130, 2000.

K. A. Kehl, “Moving toward peace: an analysis of the concept of a good death,” American Journal of Hospice and Palliative Medicine, vol. 23, no. 4, pp. 277–286, 2006.

K. E. Sleeman and E. Collis, “Caring for a dying patient in hospital,” British Medical Journal, vol. 346, no. 7905, Article ID f2174, 2013.

S. J. Goodlin, G. S. Winzelberg, J. M. Teno, M. Whedon, and J. Lynn, “Death in the hospital,” Archives of Internal Medicine, vol. 158, no. 14, pp. 1570–1572, 1998.

M. E. Lokker, L. van Zuylen, L. Veerbeek, C. C. D. van der Rijt, and A. van der Heide, “Awareness of dying: it needs words,” Supportive Care in Cancer, vol. 20, no. 6, pp. 1227–1233, 2012.

M. Gott, C. Ingleton, C. Gardiner et al., “How to improve end of life care in acute hospitals,” Nursing Older People, vol. 21, no. 7, pp. 26–29, 2009.

C. M. Sofka, “Developments and innovations in resident and fellowship education: review article,” HSS Journal, vol. 10, no. 3, pp. 225–229, 2014.

M. A. Pauls and S. Ackroyd-Stolarz, “Identifying bioethics learning needs: a survey of Canadian emergency medicine residents,” Academic Emergency Medicine, vol. 13, no. 6, pp. 645–652, 2006.

K. Almack, K. Cox, N. Moghaddam, K. Pollock, and J. Seymour, “After you: conversations between patients and healthcare professionals in planning for end of life care,” BMC Palliative Care, vol. 11, article 15, 2012.

A. M. Sullivan, M. D. Lakoma, R. K. Matsuyama, L. Rosenblatt, R. M. Arnold, and S. D. Block, “Diagnosing and discussing imminent death in the hospital: a secondary analysis of physician interviews,” Journal of Palliative Medicine, vol. 10, no. 4, pp. 882–893, 2007.

Y. Schenker, D. B. White, M. Crowley-Matoka, D. Dohan, G. A. Tiver, and R. M. Arnold, “‘It hurts to know...And it helps’: exploring how surrogates in the ICU cope with prognostic information,” Journal of Palliative Medicine, vol. 16, no. 3, pp. 243–249, 2013.

A. Walczak, P. N. Butow, P. M. Davidson et al., “Patient perspectives regarding communication about prognosis and end-of-life issues: how can it be optimised?” Patient Education and Counseling, vol. 90, no. 3, pp. 307–314, 2013.

D. K. Heyland, D. E. Allan, G. Rocker, P. Dodek, D. Pichora, and A. Gafni, “Discussing prognosis with patients and their families near the end of life: impact on satisfaction with end-of-life care,” Open Medicine, vol. 3, no. 2, pp. e101–e110, 2009.

S. M. McLennon, M. Uhrich, S. Lasiter, A. R. Chamness, and P. R. Helft, “Oncology nurses' narratives about ethical dilemmas and prognosis-related communication in advanced cancer patients,” Cancer Nursing, vol. 36, no. 2, pp. 114–121, 2013.

F. E. Witkamp, L. van Zuylen, C. C. D. van der Rijt, and A. van der Heide, “Validation of the Rotterdam MOVE2PC Questionnaire for assessment of nurses' knowledge and opinions on palliative care,” Research in Nursing and Health, vol. 36, no. 5, pp. 512–523, 2013.

M. J. Bloomer, R. Endacott, M. O'Connor, and W. Cross, “The ‘dis-ease’ of dying: challenges in nursing care of the dying in the acute hospital setting. A qualitative observational study,” Palliative Medicine, vol. 27, no. 8, pp. 757–764, 2013.

R. L. Beckstrand, J. Collette, L. Callister, and K. E. Luthy, “Oncology nurses' obstacles and supportive behaviors in end-of-life care: providing vital family care,” Oncology Nursing Forum, vol. 39, no. 5, pp. E398–E406, 2012.

C. R. Mayland, E. M. I. Williams, J. Addington-Hall, T. F. Cox, and J. E. Ellershaw, “Does the ‘Liverpool care pathway’ facilitate an improvement in quality of care for dying cancer patients,” British Journal of Cancer, vol. 108, no. 10, pp. 1942–1948, 2013.

E. E. Bolt, A. van der Heide, and B. D. Onwuteaka-Philipsen, “Reducing questionnaire length did not improve physician response rate: a randomized trial,” Journal of Clinical Epidemiology, vol. 67, no. 4, pp. 477–481, 2014.

K. Jors, S. Adami, C. Xander et al., “Dying in cancer centers: do the circumstances allow for a dignified death?” Cancer, vol. 120, no. 20, pp. 3254–3260, 2014.

J. T. van der Steen, L. Deliens, M. W. Ribbe, and B. D. Onwuteaka-Philipsen, “Selection bias in family reports on end of life with dementia in nursing homes,” Journal of Palliative Medicine, vol. 15, no. 12, pp. 1292–1296, 2012.

R. Rozenblum, M. Lisby, P. M. Hockey et al., “Uncovering the blind spot of patient satisfaction: an international survey,” BMJ Quality and Safety, vol. 20, no. 11, pp. 959–965, 2011.